Video analysis-based multi-device compatible alert system and alert method

Abstract

This system provides a multi-device compatible alert system based on video analysis that is easy to use, as it facilitates operations such as updating the settings for detected events and alert conditions. [Solution] An alert system 100 capable of generating a predetermined notification comprises: an imaging unit 32 that acquires imaging data in a predetermined imaging area; a detection unit 36 ​​that uses constructed artificial intelligence to detect a predetermined detection event from the imaging data; a display unit 33 that displays content related to the detected detection event and accepts content selection operations from the user; a setting unit 37 that sets alert conditions based on the content selection operation; a determination unit 38 that determines whether the alert conditions are met based on the detection result of the detection event indicated by the content that was accepted for selection; and an output unit 39 that instructs the output of an alert according to the determination result.

Description

Technical Field

[0001] The present invention relates to an alert system and an alert method, for example, to the alert system and alert method that can be configured by applying a camera or the like.

Background Art

[0002] Conventionally, cameras have been arranged in many places both indoors and outdoors. They are permanently installed in public facilities, medical institutions and the like, and may also be temporarily installed at high-altitude construction sites, event venues and the like. Not only do they play a preventive role in safety management and countermeasures against suspicious persons, but they also play a significant role in troubleshooting verification and cause investigation in case of emergencies, and have virtually become essential infrastructure supporting social foundations.

[0003] Development related to this type of camera and system has also been actively carried out in recent years. For example, a device has been proposed that uses artificial intelligence to detect dangerous acts with bladed tools or the occurrence of disasters such as fires, and the camera itself emits a warning sound as an alert (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0005] By the way, the detection events and alert output conditions expected by users vary depending on their needs, and such needs can naturally change. In this regard, the conventional techniques including Patent Document 1 require much effort for setting and updating detection events and alert conditions, and thus have a problem that there is room for improvement in usability.

[0006] Therefore, the object of the present invention is to provide a multi-device compatible alert system and alert method based on video analysis that allows for easy operation such as updating the settings of detected events and alert conditions, thereby improving usability. [Means for solving the problem]

[0007] One aspect of the present invention is illustrated below. [1] An alert system capable of generating a predetermined notification, An imaging unit that acquires imaging data in a predetermined imaging area, A detection unit that uses constructed artificial intelligence to detect predetermined detection events from the imaging data, A display unit that displays content related to the detected event and accepts user requests to select the content, A setting unit sets alert conditions based on the selection operation of the aforementioned content, A determination unit determines whether the alert condition is met based on the detection result of the detected event indicated by the content that received the selection operation, It includes an output unit that instructs the output of an alert according to the judgment result, A multi-device compatible alert system based on video analysis. [2] The detection unit considers the detected event to be: It detects at least one of a type motion indicating the type of subject and a motion indicating the movement of the subject. The system according to item 1, wherein the content includes at least one selected from the group consisting of content relating to the type of motion, content relating to the motion, and content relating to an imaging region for detecting the detected event. [3] The aforementioned content is The types of subjects include people, clothing, social class, and vehicles. Actions of the subjects include dangerous acts, acts of violence, movement, loitering, and congestion, This indicates at least one selected from the group consisting of the following: The detection unit is a system according to item 1 or 2 that detects the detection event corresponding to the content. [4] It further includes a sound collection unit that acquires audio data corresponding to a predetermined imaging area, A system according to any one of items 1 to 3, wherein the alert conditions can be set based on the audio data collected by the sound collection unit. [5] The detection unit is capable of detecting the content, including the attributes of the detected event, as described in any one of items 1 to 4. [6] The artificial intelligence is pre-built and stored to detect the detected events shown in the content, The detection unit reads the stored artificial intelligence and uses it to detect the detection event, according to any one of items 1 to 5. [7] The display unit is capable of receiving updates to the selection operation of the content, The system according to any one of items 1 to 6, wherein when the display unit receives the update, the setting unit can update the set alert condition to an alert condition that reflects the update on the display unit. [8] The display unit is capable of receiving the selection operation for a plurality of contents, The system according to any one of items 1 to 7, wherein the setting unit can set the alert condition by performing AND and / or OR operations based on the detection results of a plurality of detection events indicated by the content that has received the selection operation. [9] The system described in any one of items 1 to 8, wherein the setting unit is capable of setting multiple different alert conditions.

[10] At least one of the multiple different alert conditions includes a specific alert condition, The system according to any one of items 1 to 9, wherein whether or not the specific alert condition is satisfied overrides whether or not the alert conditions other than the specific alert condition are satisfied.

[11] The system according to any one of items 1 to 10, wherein the determination unit is capable of setting a detection area on the imaging data for determining that the alert condition is satisfied.

[12] The system according to any one of items 1 to 11, wherein the output unit is capable of instructing output of the alert in a plurality of different modes in accordance with a plurality of different determination results from the determination unit.

[13] The alert output mode includes at least one selected from the group consisting of an output type, an output destination, and an output period, The system according to any one of items 1 to 12, wherein the display unit is capable of accepting a selection operation from a user also for the output mode of the alert.

[14] The system according to any one of items 1 to 13, wherein the alert relates to at least one selected from the group consisting of sound, light, presentation, information transmission, and recording.

[15] An alert method capable of generating a predetermined notification, the alert method comprising: a step of acquiring imaging data in a predetermined imaging area; a step of detecting a predetermined detection event from the imaging data using a constructed artificial intelligence; a step of displaying content related to the detected detection event and accepting a selection operation for the content from a user; a setting unit that sets an alert condition based on the selection operation for the content; a step of determining that the alert condition is satisfied based on a detection result of the detection event indicated by the content for which the selection operation has been accepted; a step of instructing output of an alert in accordance with a determination result; A multi-device compatible alert method based on video analysis. Effects of the Invention

[0008] According to the present invention, it is possible to provide a multi-device compatible alert system based on video analysis, which facilitates operations such as setting and updating of detection events and alert conditions, thereby improving usability. Further, according to the present invention, it is possible to provide a multi-device compatible alert method based on video analysis, which facilitates operations such as setting and updating of detection events and alert conditions, thereby improving usability. Brief Description of the Drawings

[0009] [Figure 1] Fig. 1 is a diagram showing an outline of the alert system in the first embodiment. [Figure 2] Fig. 2 is a block diagram showing an outline of the alert device in the first embodiment. [Figure 3] Fig. 3 is a diagram illustrating an example of the display flow in the alert device according to the first embodiment. [Figure 4] Fig. 4 is a diagram illustrating a setting example of a detection area for alert conditions in the first embodiment. [Figure 5] Fig. 5 is a diagram illustrating a setting example for determining "movement" as an alert in the first embodiment. [Figure 6] Fig. 6 is a diagram illustrating an example of a display screen for alert conditions in the first embodiment. [Figure 7] Fig. 7 is a diagram illustrating an example of a display screen in another embodiment. Detailed Description of the Embodiments

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present embodiment is not limited to the aspects described below, and can be implemented with various modifications within the scope of the technical idea of the present invention.

[0011] In this specification, the term "process" is not limited to independent processes; therefore, even if it cannot be clearly distinguished from other processes, it is included in the concept if the function of that process is achieved in that manner. The same applies to the term "step." Furthermore, in the contents shown in the drawings, the scale, shape, and length may be exaggerated for the sake of clarity.

[0012] In various embodiments and designs, components denoted by the same reference numerals can be applied to embodiments that are essentially of the same nature. Their content, including technical considerations and preferred embodiments, may be referenced as appropriate.

[0013] Each device or part constituting this embodiment may consist of a single device or part, or it may span multiple devices or parts, and, if technically possible, may be configured using virtualization, cloud computing, and networks. In this embodiment, imaging data is acquired in a predetermined imaging area, and such imaging area may be set appropriately according to the user's needs, as long as privacy and laws and regulations are taken into consideration.

[0014] This embodiment can be built as a platform that can connect to various different terminals (smartphones, tablets, etc.) via a network. This embodiment makes it easy to achieve multi-device compatibility, allowing various operations based on video analysis (video display, alert setting, confirmation, and settings, etc.) to be used from different user and terminal environments.

[0015] [Embodiment 1] <Video analysis-based multi-device compatible alert system> ≪Overview≫ The alert system according to this embodiment (sometimes simply referred to as "this system") is capable of generating predetermined notifications. This system, An imaging unit that acquires imaging data in a predetermined imaging area, A detection unit that uses constructed artificial intelligence to detect predetermined detection events from the imaging data, A display unit that displays content related to the detected event and accepts user requests to select the content, A setting unit sets alert conditions based on the selection operation of the aforementioned content, A determination unit determines whether the alert condition is met based on the detection result of the detected event indicated by the content that received the selection operation, The system includes an output unit that instructs the output of an alert corresponding to the determination result of the determination unit.

[0016] According to this, for example, detection events and alert conditions can be flexibly and easily set based on video analysis, in accordance with user needs and their changes, resulting in superior usability and improved convenience.

[0017] Figure 1 is a diagram illustrating the schematic of the system 100. The system 100 can be configured to include a camera 10, an output device 20, and a terminal device 30.

[0018] (camera) Camera 10, like known security cameras, is composed of an image sensor and can convert collected light into electrical signals and transmit them to various parts. Here, the signals are transmitted to the terminal device 30 via wireless communication L1 such as Bluetooth® or Wi-Fi®, and image data in the imaging area is acquired.

[0019] Examples of camera 10 include a fisheye (dome-shaped) camera 10A and a pallet-shaped (cylindrical) camera 10B. It can be a fixed-point camera or a PTZ (pan-tilt-zoom) camera, and the imaging method and shape are not limited. The size of the camera is also not limited; it can be a large camera or a small camera (a so-called mini-camera).

[0020] Camera 10 can be placed indoors or outdoors, for example, on the ceiling, interior wall, exterior wall, column, beam, and under the eaves. The imaging area may be set as appropriate according to user needs and their changes, as long as privacy and legal regulations are taken into consideration. The camera may be a fixed type installed on a structure, or a portable type that is intended for movement or wearable (for example, one that can be attached to a helmet, one that can be attached to a body vest, one that can be attached to a small lighting device such as a flashlight or penlight).

[0021] Basic information regarding the placement location, including the placement location of the camera 10 as described above, may be shared with the terminal device 30. This allows for the storage of such basic information in association with imaging data, and its retrieval as needed, facilitating various controls and analyses to improve convenience.

[0022] The basic information mentioned above includes, for example, details related to the location, such as "location and environment," "structure and facilities," "work and processes," and "management and system."

[0023] "Location and environment" can include, for example, the region (city, residential, mountainous, coastal, etc.) and the surrounding environment (presence or absence of schools, hospitals, roads, railways, etc.). "Structure and facilities" can include, for example, altitude (high places, underground, etc.), presence or absence of temporary structures, workspace (narrow, open, etc.), presence or absence of heavy machinery, etc.

[0024] Examples of "work and processes" include the type of work (architecture, civil engineering, demolition, etc.), the construction period (start and end dates), the division of processes (foundation, structure, finishing, etc.), and whether or not there are simultaneous parallel tasks. Examples of "management and systems" include working hours (early morning, daytime, nighttime, weekdays, weekends, etc.) and whether or not there are restricted areas.

[0025] The camera placement locations may be displayed as points on a map screen in the terminal device 30, for example. By storing such basic information linked to the imaging data, users can perform refined searches using such basic information as elements, making it easier to identify the desired camera and, consequently, the imaging area.

[0026] Camera 10 may be integrated with various sensors and output devices. Examples of sensors include temperature sensors, vibration sensors, motion sensors, and GPS (Global Positioning System) sensors. If these are integrated, the sensor signals measured near the imaging area can be easily used for various control purposes.

[0027] Furthermore, known output devices include, for example, those capable of displaying and transmitting voice and messages, and those capable of storing logs. Because the camera 10 possesses these various functions, it may be possible to omit the provision of its own independent output device 20. However, regardless of the configuration of the camera 10, an output device 20 may be provided separately, as in this system 100.

[0028] (Output device / Receiver) The output device (sometimes simply referred to as "output device") 20 may include, for example, an alarm device, a light-emitting device, a display device, an information transmission device, and a recording device. For example, any device that is controlled by turning the power on or off is applicable, and a distribution board or switchboard that centrally manages the above on / off may be used. Similar to the camera 10, it may be connected to a terminal device 30 via wireless communication L1, in which case it can generate an alert upon receiving an output instruction (reception of an output signal) from the terminal device 30.

[0029] An example of an alarm device, which is one form of the output device 20, is a buzzer or speaker, which can generate a warning sound or voice as an alert. An example of a light-emitting device or display device is a light-emitting diode (LED) or an EL (Electro-Luminescence) display, which can generate light emission, flashing, or warning messages as an alert.

[0030] Furthermore, the information transmission device can send (and receive) emails and SNS messages as alerts, and the recording device can store historical information as alerts. The information transmission device and recording device can, for example, be part of the terminal device 30. Emails and SNS information received by these devices can be viewed through the display device in this system and various other devices that cooperate with it.

[0031] In this regard, "alert" as used herein means "notifying" of a predetermined information, in accordance with the meaning of the word. Therefore, it is not limited to issuing warnings to the target person, but also broadly includes electronic notifications (for example, sending emails or SNS messages to the user or other devices) and storage on a storage medium that will retroactively notify the user.

[0032] The output device 20, like the camera 10, may also be equipped with various known sensors, and multiple output devices may be provided. For example, in this system 100, a light-emitting device 20A and an alarm device 20B are used as output devices 20, and since part of the terminal device 30 also serves as an information transmission function and a recording function, different types of output devices 20 can be used in combination as appropriate.

[0033] (Terminal device) Terminal device 30 is a computer device that enables the system 100 to function by installing a dedicated application or running a program. Examples of terminal devices 30 include desktops, smartphones, and tablets. Different types of terminals may be used in combination, as in the system 100 where a mobile terminal (smartphone) 30B is used in addition to a desktop terminal 30A.

[0034] The terminal device 30 is configured to be integrated with a display unit (including a display device). The display unit, for example, displays imaging data from each imaging area in real time as a list and / or as a whole, and switches between them as appropriate. This makes it easy for the user to check the imaging area, is easy to use, and thus facilitates improved convenience.

[0035] Here, the terminal device 30 is capable of detecting predetermined detection events from imaging data using the constructed artificial intelligence (hereinafter sometimes simply referred to as "AI"). Not only the detection events, but also the alert conditions based on the detection results can be flexibly and easily set according to the user's needs and their changes, resulting in excellent usability and improved convenience.

[0036] ≪Alert device≫ Figure 2 is a block diagram showing an example configuration of a terminal device 30, which is one of the alert devices that make up this system 100. The alert device (terminal device 30) is, for example, The system can be configured to include a communication unit 31, an imaging unit 32, a display unit 33, a storage unit 34, and a control unit 35.

[0037] (Communications Department) The communication unit 31 is composed of, for example, a predetermined adapter, and can send and receive signals to and from the camera 10 and the output device 20 via wireless communication L1.

[0038] (Imaging Department) The imaging unit 32 acquires (generates) imaging data in a predetermined imaging area based on the signal transmitted from the camera 10. The imaging data, in one embodiment, may include video data and be transmitted to the display unit 33 and stored in the storage unit 34.

[0039] (Display) The display unit 33 includes a display device 33A, such as a touch panel liquid crystal display, which is incorporated into the terminal device 30. Through this display unit 33, it can receive input from the user and display information from the control unit 35 to the user.

[0040] As described above, the display unit 33 displays, for example, the image data from multiple cameras 10 in real time in a list and / or as a whole, and switches between them as needed. This makes it easy for the user to check the image area, is user-friendly, and thus facilitates improved convenience.

[0041] Although the display unit 33 also functions as an input unit, such an input unit may be provided independently. In this case, the input unit can be connected to various input devices such as a predetermined push-button device, a keyboard, and a mouse, and accepts user operations on these devices.

[0042] (Storage part) The memory unit 34 is composed of, for example, ROM (Read Only Memory), RAM (Random Access Memory), etc., and can store the program (main program) executed in this system 100. By executing this program, the computer can perform various controls executed in this system 100.

[0043] Furthermore, the memory unit 34 can store information acquired from other devices, data and calculation results handled in other units, as well as learning models, alert conditions, content information, thresholds and reference values ​​used for various judgments and decisions.

[0044] ≪Control Unit≫ The control unit 35 is composed of, for example, a CPU (Central Processing Unit) and a microcontroller, and can store and read various information from the storage unit 34, and can also operate each part according to the program stored in the storage unit 34.

[0045] The control unit 35 may further be configured to include a detection unit 36, a setting unit 37, a determination unit 38, and an output unit 39.

[0046] (Detection unit) The detection unit 36 ​​detects predetermined detection events from the imaging data using the constructed AI. A default AI that detects highly versatile detection events that can be roughly set during the manufacturing stage may be pre-stored in the memory unit 34. In this case, the user provided with it is saved the trouble of constructing the AI, resulting in superior usability and easier improvement of convenience.

[0047] (Settings section) The setting unit 37 sets the alert conditions. The set alert conditions are stored in the storage unit 34 and read out as appropriate by the determination unit 38.

[0048] (Judgment Department) The determination unit 38 reads the alert condition from the storage unit 34 and determines whether the alert condition is met based on the detection result from the detection unit 36. The alert condition can be set for each detection result from the detection unit 36, and multiple alert conditions can also be set. In these cases, the system is easy to use and its convenience is easily improved.

[0049] Preferably, the determination unit 38 can set an area on the image data where it is determined whether the alert condition has been met. In particular, when a fisheye camera with a wide-angle lens that has a wide field of view is used, other images are likely to be captured in addition to the intended imaging area. If the determination area can be set, it is easier to eliminate the noise caused by the reflections and to accurately determine whether the alert condition has been met.

[0050] (Output section) The output unit 39 instructs the output of an alert according to the determination result of the determination unit 38. In this system 100, an instruction signal is output to the output device 20 via wireless communication L1, and a predetermined alert is activated.

[0051] Preferably, the output unit 39 can instruct the output of alerts in multiple different ways depending on multiple different judgment results from the judgment unit 38. For example, if an alert is determined to be established based on a detection event that suggests a minor problem, it can be recorded, while if a serious problem is suggested, an email or SNS message can be sent to the user. This makes it easier to output alerts that meet the user's needs and improves usability.

[0052] The alerts that the output unit 39 instructs to output preferably relate to at least one selected from the group consisting of sound, light, presentation, information transmission, and recording. These are preferred embodiments that offer excellent notification functionality and ease of use.

[0053] ≪Example of operation≫ The configuration example of the terminal device 30 will be further explained in accordance with its operation example. Figure 3 is a diagram illustrating an example of the display flow in this system 100.

[0054] (Step S1) First, clicking the "New Alert" button 40 on the screen proceeds to step S1, where the display unit 33 displays, for example, content 41 related to the detected event and accepts the user's selection of content 41.

[0055] Content 41 here is, for example, Content related to type motion indicating the type of subject (type content 41A), Content related to motion showing the subject's movements (motion content 41B), Content related to the imaging area for detecting the detected event (imaging content 42), It may include at least one selected from the group consisting of the following. Here, "person / vehicle" corresponds to type content 41A, and "movement," "staying," and "crowding" correspond to action content 41B. Although not shown in the diagram, action content 41B may include "dangerous behavior" and / or "violent behavior." The imaging content 42 will be described later in step S2.

[0056] Based on the above, in this system 100, the detection unit 36 ​​detects, for example, the following events: It is preferable to detect at least one of the above-mentioned type motion, which indicates the type of subject, and the above-mentioned action motion, which indicates the action of the subject. This allows for a wide variety of detected events to be categorized, making it easier to perform operations such as updating the settings for detected events and alert conditions.

[0057] In this respect, the content is The types of subjects include people, clothing, social class, and vehicles. Actions of the subjects include dangerous acts, acts of violence, movement, loitering, and congestion, This relates to at least one selected from the group consisting of, The detection unit 36 ​​preferably detects detection events corresponding to the content. This makes operations such as updating the settings for detection events and alert conditions even easier, from the same viewpoint as described above.

[0058] In particular, it is preferable that the detection unit 36 ​​is capable of detecting detection events including the attributes of the detection event corresponding to the content. This makes it easier to perform operations such as updating the settings of detection events and alert conditions, including the handling of such attributes. It also makes it possible for users to search based on such attributes, which in turn makes operations such as updating the settings of detection events and alert conditions even easier.

[0059] In this context, "attributes" can refer to, for example, Regarding individuals, the analysis delves into details such as demographics (basic characteristics of the person), geographics (region), and psychographics (values ​​and inclinations). Regarding vehicles, the information includes details such as vehicle type, load capacity, vehicle category, manufacturer / model, size, license plate number, and intended use (personal use, commercial use, rental car, emergency vehicle, public vehicle, EV / HV / gasoline vehicle, etc.). These are some examples.

[0060] Currently, AI is pre-built and memorized to detect the events indicated in the content. The detection unit 36 ​​preferably reads the stored AI and uses it to detect the detected event.

[0061] For example, a default AI that detects highly versatile detection events that can be roughly set during the manufacturing stage can be pre-stored in the memory unit 34. In this case, the user provided with the AI ​​is saved the trouble of building the AI ​​themselves, making operations such as updating the settings for detection events and alert conditions even easier.

[0062] Now, in step S1 shown in Figure 3, if the user selects "People / Vehicles" as the type content 41A, the process proceeds to step S2. Here, the display unit 33 displays content 41 (imaging content 42) related to the imaging area where the detected event is detected, and accepts the user's selection operation for content 41. Here, "All Cameras" and "Individual Cameras" are shown, and these correspond to the imaging content 42.

[0063] (Step S2) In step S2, for example, if the user selects "individual cameras" as the imaging content 42, a list of candidates will be displayed, and once the user selects the target camera (in one example, the fisheye camera 10A shown in Figure 1), the user proceeds to step S3. Note that multiple target cameras may be selected.

[0064] (Step S3) In step S3, the display unit 33 further displays details of type content 41A related to type motion, for example, indicating the type of subject, and accepts a selection operation of content 41 from the user. Here, the options are "person," "vehicle," "person or vehicle," and "person and vehicle," which correspond to the details of type content 41A.

[0065] In step S3, for example, if the user selects "People" as the details for content type 41A, the process proceeds to step S4.

[0066] As before, The detection unit 36 ​​uses the constructed artificial intelligence to detect predetermined detection events from the imaging data, The display unit 33 displays content 41 related to the detected event (for example, at least one selected from the group consisting of type content 41A related to type motion, operation content 41B related to motion, imaging content 42 related to imaging area, and specific content 41C described later), and accepts user input for selecting content 41.

[0067] There are multiple types of content 41 related to the detected events, and the user can select at least one of them, for example, using radio buttons. This operation is easy, and therefore, it is easy to improve usability.

[0068] (Step S4) In step S4, the display unit 33 displays, for example, area selection options 45 related to setting the detection area for determining whether an alert condition has been met. While indicating the target imaging area, the determination unit 38 receives the setting of the detection area for determining whether an alert condition has been met from the user via the display unit 33.

[0069] Figure 4 illustrates an example of setting the detection area for alert conditions in this system 100. From the imaging area (imaging data) displayed on the display unit 33, for example, either the full frame option 45A or the custom frame option 45B can be selected, and a predetermined detection area Ar can be set.

[0070] When selecting custom frame option 45B, the area is defined by the user's actions, such as selecting an area by clicking and dragging with the mouse, or by selecting a pre-defined section. These are also selection operations, making it easier to improve usability.

[0071] The settings unit 37 sets alert conditions based on the user's previous operations, including content selection. In the example shown in Figure 3, the alert condition would be set to detect a "person" in the selected "detection area Ar" using the selected "fisheye camera 10A".

[0072] The determination unit 38 then determines whether the alert condition is met based on various detection results, including the detection result of the detection event indicated by the content that received the selection operation. In the example shown in Figure 3, the determination of whether the alert condition is met is based on the result that the detection unit 36 ​​detected a "person" in the selected "detection area Ar" using the selected "fisheye camera 10A".

[0073] Here, the display unit 33 is capable of receiving updates to the selection operation of the above content. When the display unit 33 receives an update, it is preferable that the setting unit 37 can update the set alert conditions to reflect the update in the display unit 33. This makes it easy to update settings such as detection events and alert output conditions in response to changes in user needs and expectations, thus improving usability.

[0074] (Step S5) When the process proceeds to step S5 shown in Figure 3, the display unit 33 displays, for example, an alert selection 46 indicating the output type of the target alert, and accepts the user's setting of the output method. Here, the options are "alarm sound," "light," "warning display," "email," and "record," which correspond to the alert selection 46. If, for example, the user sets "email" or "record" as the alert selection 46 in step S5, the process proceeds to step S6.

[0075] (Step S6) In step S6, the display unit 33 displays alert options 46, for example, the destination and duration of the alert output, and also accepts settings for the output method from the user. Here, "Destination" and "Duration" are shown, and these correspond to the alert options 46.

[0076] When an information transmission method such as "email" or "SNS posting (not shown)" is selected as the output type, the "output destination" corresponds to, for example, the recipient. Recipients can be stored in advance in the storage unit 34, allowing the user to set the recipient by selecting from a list of recipient candidates.

[0077] Furthermore, "output period" can refer to the time period during which alerts are issued. It is possible to issue alerts targeting specific time periods (e.g., early morning, daytime, nighttime, weekdays, holidays, etc.), and to refrain from issuing alerts. This is also easy to do, thus making it easier to improve usability.

[0078] As described above, the terminal device 30 is, for example, An imaging unit 32 acquires imaging data in a predetermined imaging area, A detection unit 36 ​​uses constructed artificial intelligence to detect predetermined detection events from imaging data, A display unit 33 displays content 41 related to the detected event (for example, at least one selected from the group consisting of type content 41A related to type motion, motion content 41B related to motion, imaging content 42 related to imaging area, and specific content 41C described later), and accepts user input to select that content. Based on the selection operation of the above content, the setting unit 37 sets the alert conditions, A determination unit 38 determines whether the alert condition is met based on the detection result of the detected event indicated by the content that received the selection operation, The system includes an output unit 39 that instructs the output of an alert corresponding to the determination result of the determination unit 38.

[0079] According to this, operations such as updating the settings for detected events and alert conditions are easy, and therefore, it is easier to improve usability.

[0080] In addition to content related to detected events, users can also select various options related to setting and updating alert conditions (for example, area option 45 and alert option 46) using radio buttons. This makes operation easier and thus facilitates improvements in usability.

[0081] Up to this point, we have explained an example where "Person / Vehicle" is selected as content 41 (type content 41A) in step S1, but the basic flow is the same even when selecting other content (action content 41B), such as "Dangerous behavior," "Violent behavior," "Movement," "Staying," or "Congestion."

[0082] Figure 5 illustrates an example of the settings for alert detection of "movement" in the system 100. For example, if "movement" is selected as the action content 41B in step S1, then in step S4, the detection area and the reference line L can be set. At the same time, the crossing direction across the reference line L (represented by arrows in the figure, crossing in one direction and / or crossing in other directions) can also be set, and these operations are also easy to perform.

[0083] Furthermore, although not shown in the diagram, if, for example, "staying" or "crowding" is selected as the action content 41B in step S1, then, for example, in step S4, the time spent in the area and the number of appearances, which serve as criteria for determining stagnation, and the density (amount present in the area), which serve as criteria for determining mixed presence, can be set by selection. Note that "staying" includes states in which the movement speed of objects in a predetermined area is reduced, extremely low, or stopped, and "crowding" includes states in which the density of objects in a predetermined area is above a predetermined value.

[0084] Similarly, although not shown in the diagram, for example, in step S1, "dangerous behavior" and "violent behavior" may be selectable as action content 41B. "Dangerous behavior" includes, for example, abnormal movements based on changes in the posture of the human body (falls, tumbles, etc.) and / or abnormal movements based on changes in the spatial state of machinery and equipment (malfunctions, collapses, etc.), and abnormal behavior based on movement paths or manner of movement (climbing, jumping, etc.). "Violent behavior" includes, for example, repeated or continuous abnormal contact with the body or objects (fights, killing or injuring, etc.). Each of these elements may also be configurable.

[0085] Settings for each element of "dangerous acts" and "violent acts" as action content 41B can also be made via the display unit 33 (display device 33A). In the same way as above, the type content 41A and imaging content 42 can be input and set. For "dangerous acts" and "violent acts," imaging data that actually includes the desired events can be prepared, and an algorithm can be generated using machine learning with that data. By storing the learning model M in the memory unit 34 using such an algorithm and reading it out as needed, a trained AI based on the learning model M can be constructed. In this case, it is easier to detect predetermined dangerous acts and violent acts with high accuracy.

[0086] Regarding updates to various settings related to the operational content 41B, users can also select options using, for example, radio buttons. This makes operation even easier, and thus facilitates improvements in usability.

[0087] The various settings for when "dangerous behavior," "violent behavior," "movement," "staying," or "congestion" are selected as the action content 41B are also stored in the memory unit 34. These are read out as needed and used by the determination unit 38 to determine whether the alert condition has been met.

[0088] Here, the display unit 33 can accept selection operations for multiple contents, Preferably, the setting unit 37 allows setting alert conditions by AND and / or OR operations based on the detection results of multiple detection events indicated by the content that has received the selection operation. Even if detailed settings are desired, operations such as updating the settings are easy, thus improving usability.

[0089] For example, by setting an alert condition that performs an AND operation on the action content 41B related to the detected event "staying" and the action content 41B related to the detected event "movement," the alert condition will be determined to be met only when both events are detected.

[0090] Furthermore, for example, by performing an OR operation on the action content 41B related to the detected event "stagnation" and the action content 41B related to the detected event "congestion," the alert condition will be determined to be met when at least one of them is detected.

[0091] When performing AND and / or OR operations, the detection results of the detected events used are not limited to those relating to the operation content 41B, but can also include those relating to content 41 (type content 41A and image content 42).

[0092] In addition to the above, it is preferable that the setting unit 37 allows for the setting of multiple different alert conditions. This makes it easier to set and update alert conditions to satisfy the changing needs and expectations of the user, while also making the operation easy, thus improving usability.

[0093] Figure 6 illustrates an example of the alert condition display screen in this system 100. The set alert conditions can be displayed in a list and / or all at once in the area enclosed by the thick line B1 in the figure. If an alert condition has actually been output, its output history can be displayed in a list and / or all at once in the area enclosed by the dotted line B2 in the figure. This makes it easy for the user to check the alert conditions and their output status, resulting in superior usability and improved convenience.

[0094] Here, at least one of several different alert conditions includes a specific alert condition. It is preferable that the success or failure of a specific alert condition overrides the success or failure of other alert conditions. This makes it easier to determine whether an alert condition has been met based on specific events such as attention-grabbing, visitor, or VIP, while also being easy to operate, thus improving usability.

[0095] Specific alert conditions can be set, for example, by selecting specific content. In the example shown in Figure 3, specific content 41C corresponds to "Person of Interest" and "Number of Interest (License Plate of Interest)" in step S1.

[0096] However, these specific contents 41C are merely examples. If the detection unit 36 ​​can detect the content, including the attributes of the detection event corresponding to that content, the content corresponding to those attributes may be displayed on the display unit 33 as specific content 41C, and the user may be able to select it.

[0097] For example, although not shown in the illustration, it is possible to detect "age (child)" or "emergency vehicle" as specific content 41C, and by selecting to determine these, it may be possible to announce to the target imaging area that a child is approaching or an emergency vehicle is passing nearby. This can also be applied to other attributes, and in any case, it is easy to operate, and therefore easy to improve usability.

[0098] Furthermore, when using specific content 41C, the default AI, which is built to detect a wide range of commonly configurable detection events that are pre-stored, may not be able to detect all of them. In this case, the user can build a new AI.

[0099] By preparing imaging data that actually contains the desired detection events and their attributes, an algorithm can be generated using machine learning with this data. The learning model M is stored in the memory unit 34 using this algorithm, and by reading it out as needed, a trained AI based on the learning model M can be constructed.

[0100] Newly added content may also be displayed in a list or in a batch on the display unit 33. Conversely, unnecessary content may be deleted from the list or batch display. This provides high flexibility and improved usability.

[0101] <Multi-device compatible alert method based on video analysis> The alert method according to this embodiment (hereinafter sometimes simply referred to as "this method") is capable of generating a predetermined notification. This method, A step of acquiring imaging data in a predetermined imaging area, A step of detecting a predetermined detection event from the imaging data using the constructed artificial intelligence, The steps include displaying content related to the detected event and accepting a user's selection operation for the content, A setting unit sets alert conditions based on the selection operation of the aforementioned content, A step of determining whether the alert condition is met based on the detection result of the detected event indicated by the content that received the selection operation, The system includes the step of instructing the output of an alert corresponding to the determination result in the determination unit.

[0102] This also allows for flexible and easy configuration of detection events and alert conditions according to user needs and their changes, similar to the case of the system 100 described above, resulting in superior usability and improved convenience.

[0103] [Other embodiments] The present invention is not limited to the embodiments described in the first embodiment above, and can be implemented with various modifications within the scope of its technical concept.

[0104] For example, it further includes a sound collection unit that acquires audio data corresponding to a predetermined imaging area, It may be possible to set alert conditions based on the audio data collected by the sound collection unit. In this case, the detection unit can use the constructed artificial intelligence to detect a predetermined event based on the audio data in addition to the image data. By using audio in addition to video, it becomes easier to make more accurate situational judgments, and consequently, it becomes easier to improve usability. Noise reduction measures can also be implemented more easily.

[0105] Of course, the determination unit may determine whether the alert condition is met based on both the imaging data and the audio data, and the output unit may instruct the output of an alert according to the determination result. Similar to imaging data, audio data can also be subject to machine learning and storage in the memory unit. For example, a predetermined threshold can be stored, and in this case, it is possible to detect when the collected audio exceeds that threshold and to provide an appropriate output, making it easier to both generate noise and take countermeasures.

[0106] While microphones are a possible sound-collecting component, their placement and structure are not limited. In fact, any device with sound-collecting capabilities may be applicable.

[0107] In the first embodiment described above, an example was described in which the network is formed on the terminal device side. However, at least a part of the AI-based network may be configured on the camera side. Furthermore, at least a portion of the storage unit 12 may be composed of external storage devices such as flash memory and an HDD (Hard Disk Drive).

[0108] The cameras in this system can be, for example, so-called security cameras, and it is conceivable that they could be used in public facilities, medical institutions, high-altitude construction sites, and event venues, where such security cameras are installed. In this regard, it is also conceivable to position the cameras so that the imaging area includes control panels, distribution boards, and switchboards, where various electrical control systems are concentrated. The on / off status of electrical control systems can be actually checked in real time on a mobile device screen, and by setting alert conditions, it may be possible to prevent serious accidents by controlling the on / off status of these output devices in the event of an emergency. The system is highly advanced because it allows for roles and functions that have not been previously available to cameras.

[0109] Furthermore, the display mode in the display unit 33 is not limited. Figure 7 is a diagram illustrating an example of a display screen in another embodiment. Here, the content 41 related to the detected event is displayed in a list and as a whole. From among the various contents 41 (sub-contents 47), the most frequently used content (main content 48) is duplicated at the top.

[0110] Currently, as Content 41, for example, information relating to a person and their clothing is displayed, more specifically, at least one selected from the group consisting of the presence or absence of hats (here, helmets), the presence or absence of eyeglasses (here, safety glasses), the presence or absence of gloves (here, gloves or medical gloves), the presence or absence of protective clothing (here, gowns or masks), and the presence or absence of high-visibility clothing.

[0111] These are highly versatile detection events that can be set to a general extent. Therefore, as in the first embodiment described above, for example, a default AI may be pre-stored in the memory unit 34 during the manufacturing stage and then provided to the user. Even during the initial setup, the user can appropriately select these default contents 41 to cause the control unit (detection unit) to detect these detection events.

[0112] For example, when a user selects content 41 related to the detection event "person without a helmet," the detection unit uses the constructed artificial intelligence to detect the detection event "person without a helmet" corresponding to the selected content 41 from the image data.

[0113] In a basic configuration, for example, the selection of content 41 displayed on the display unit can also serve as the setting of an alert condition. Therefore, for example, when the user selects content 41 related to the detection event "person without a helmet," the detection of "person without a helmet" becomes the alert condition. In this case, when the detection unit detects the event, the determination unit determines whether the alert condition has been met.

[0114] In more advanced embodiments, similar to the first embodiment described above, the user can appropriately set alert conditions by combining multiple content items 41. In this case, it is preferable that the display unit 33 can accept multiple selection operations for the content 41, and the determination unit can set alert conditions by AND and / or calculations based on multiple detection results corresponding to the multiple content 41.

[0115] By performing an AND operation on content 41 related to the detected event "person without helmet" and content 41 related to the detected event "person without safety glasses," the system aims to determine if both events are detected, thereby determining that the alert condition has been met. By performing an OR operation on content 41 related to the detected event "person without helmet" and content 41 related to the detected event "person without safety glasses," the system aims to determine if the alert condition is met when at least one of the two events is detected.

[0116] Alternatively, a user-created AI may also be applied. By preparing imaging data that actually contains the desired detection events and their attributes, an algorithm can be generated using machine learning with that data. A learning model M can be stored in the memory unit 34 using such an algorithm, and by reading it out as needed, a trained AI based on the learning model M can be constructed.

[0117] Similar to the first embodiment described above, newly added content may also be displayed in a list or in a batch on the display unit 33. Conversely, unnecessary content may be deleted from the list or batch display. This provides high flexibility and improved convenience.

[0118] In recent years, technologies such as VLM (Vision Language Model) have been proposed as machine learning models. These technologies allow for the input of image information into the VLM to obtain corresponding text information, and the technology described in this system can be applied in this way. For example, by inputting images based on captured data into the VLM, it becomes easier to provide users with various alert-related conditions and information as text. [Industrial applicability]

[0119] According to the present invention, it is possible to provide a multi-device compatible alert system and alert method based on video analysis, which allows for easy operation such as updating the settings of detected events and alert conditions, thereby improving usability. [Explanation of symbols]

[0120] 100: Alert System 10, 10A, 10B: Camera 20, 20A, 20B: Output devices 30: Terminal device 31: Communications Department 32: Imaging Unit 33: Display / Input Section 34: Storage section 35: Control Unit 36: Detection unit 37: Settings Section 38: Judgment section 39: Output section 41: Content 41A: Content Type 41B: Actionable Content 42: Image Content 45: Area Selection 45A: Full frame option 45B: Custom frame options 46: Alert Options Ar: Detection area L:Wireless communication

Claims

1. An alert system capable of generating a predetermined notification, An imaging unit that acquires imaging data in a predetermined imaging area, A detection unit that uses constructed artificial intelligence to detect predetermined detection events from the imaging data, A display unit that displays the type of content information related to the detected event and accepts a user's selection operation for the type, A setting unit sets alert conditions based on the selection operation of the aforementioned type, A determination unit determines whether the alert condition is met based on the detection result of the detection event indicated by the type for which the selection operation was received, It includes an output unit that instructs the output of an alert according to the judgment result, A multi-device compatible alert system based on video analysis.

2. The detection unit considers the detected event to be: It detects at least one of a type motion indicating the type of subject and a motion indicating the movement of the subject. The system according to claim 1, wherein the type of content information includes at least one selected from the group consisting of the type of content information relating to the type of motion, the type of content information relating to the motion, and the type of content information relating to the imaging region for detecting the detected event.

3. The aforementioned content information is The types of subjects include people, clothing, social class, and vehicles. Actions of the subjects include dangerous acts, acts of violence, movement, loitering, and congestion, This indicates at least one selected from the group consisting of, The system according to claim 1 or 2, wherein the detection unit detects the detection event corresponding to the type for which the selection operation was received.

4. It further includes a sound collection unit that acquires audio data corresponding to a predetermined imaging area, The system according to claim 1, wherein the alert conditions can be set based on the audio data from the sound collection unit.

5. The system according to claim 1, wherein the detection unit is capable of detecting the attributes of the detected event corresponding to the type.

6. The artificial intelligence is pre-built and stored to detect the detection events indicated by the above-mentioned types, The system according to claim 1 or 2, wherein the detection unit reads the stored artificial intelligence and uses it to detect the detection event.

7. The display unit is capable of receiving updates to the selection operation of the type, The system according to claim 1 or 2, wherein when the display unit receives the update, the setting unit can update the set alert condition to an alert condition that reflects the update on the display unit.

8. The display unit is capable of receiving the selection operation of multiple types, The system according to claim 1 or 2, wherein the setting unit can set the alert condition by AND and / or OR operations based on the detection results of a plurality of detection events indicated by the type for which the selection operation was received.

9. The system according to claim 1 or 2, wherein the setting unit can set a plurality of different alert conditions.

10. At least one of the multiple different alert conditions includes a specific alert condition, The system according to claim 1 or 2, wherein the success or failure of the specified alert condition overrides the success or failure of any other alert condition.

11. The system according to claim 1 or 2, wherein the determination unit can set a detection area on the imaging data for determining whether the alert condition has been met.

12. The system according to claim 1 or 2, wherein the output unit can instruct the output of the alert in multiple different ways according to multiple different determination results from the determination unit.

13. The output mode of the alert includes at least one selected from the group consisting of output type, output destination, and output period. The system according to claim 1 or 2, wherein the display unit can also accept a user selection operation regarding the output mode of the alert.

14. The system according to claim 1 or 2, wherein the alert relates to at least one selected from the group consisting of sound, light, presentation, information transmission, and recording.

15. An alert method capable of generating a predetermined notification, A step of acquiring imaging data in a predetermined imaging area, A step of detecting a predetermined detection event from the imaging data using the constructed artificial intelligence, The steps include displaying the type of content information related to the detected event and accepting a user's selection of the type, A setting unit sets alert conditions based on the selection operation of the aforementioned type, A step of determining whether the alert condition is met based on the detection result of the detection event indicated by the type for which the selection operation was accepted, The system includes a step of instructing the output of an alert according to the judgment result, A multi-device compatible alert method based on video analysis.

Images