Image analysis device, image analysis method, and image analysis program

Abstract

To execute an appropriate reaction to an occurred event by a user setting a combination condition of output results obtained from a plurality of AI logics.SOLUTION: A server device 10 stores therein a plurality of analysis models each of which outputs image analysis targets included in image data, accepts selection of the analysis model from among the plurality of analysis models and also accepts a degree of importance that is associated with a combination of the image analysis targets, acquires the image data, inputs the acquired image data to the analysis model for which the selection has been accepted, and executes a predetermined reaction in accordance with the degree of importance that is associated with the output combination of the image analysis targets.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image analysis apparatus, an image analysis method, and an image analysis program.

Background Art

[0002] There is known a technique for performing parallel image analysis by each of a plurality of AI (Artificial Intelligence) logics on an image captured by a camera.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when implementing facility security and facility management, it is difficult to execute an appropriate reaction to an event that has occurred. For example, the importance indicating the urgency of an event generated by the actor or object of an action, such as "who", "when", "what", "what happened", varies, but it is difficult to determine the urgency of the event based on the combination of the output results of a plurality of AI logics that detect each of them and execute separate reactions.

[0005] The present invention has been made in view of the above, and an object thereof is to enable a user to set combination conditions of output results of a plurality of AI logics and execute an appropriate reaction to an event that has occurred.

Means for Solving the Problems

[0006] The present invention provides an image analysis device comprising: a storage unit that stores a plurality of machine learning models that output objects included in image data; a receiving unit that receives the selection of a machine learning model from the plurality of machine learning models and the importance associated with the combination of objects; an acquisition unit that acquires image data; and an execution unit that inputs the acquired image data into the machine learning model that received the selection and performs a predetermined reaction according to the importance associated with the combination of objects output.

[0007] Furthermore, the present invention provides an image analysis method in which a computer selects a machine learning model from a plurality of machine learning models that output objects contained in image data, accepts the importance assigned to the combination of objects, acquires image data, inputs the acquired image data into the selected machine learning model, and performs a predetermined reaction according to the importance assigned to the combination of objects output.

[0008] Furthermore, the present invention provides an image analysis program that causes a computer to select a machine learning model from a plurality of machine learning models that output objects contained in image data, and to receive importance levels associated with combinations of the objects, acquire image data, input the acquired image data into the selected machine learning model, and perform a predetermined reaction according to the importance levels associated with the combinations of the objects output. [Effects of the Invention]

[0009] According to the present invention, the user can set combination conditions for the output results of multiple AI logics, and it is possible to perform appropriate reactions to events that occur. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example configuration and processing example of the event detection system according to the embodiment. [Figure 2] This block diagram shows an example of the configuration of each device in the event detection system according to the embodiment. [Figure 3] This figure shows an example of a candidate information storage unit for a server device according to the embodiment. [Figure 4] This figure shows an example of a configuration information storage unit for a server device according to an embodiment. [Figure 5] This figure shows an example of an image data storage unit of a server device according to the embodiment. [Figure 6] This figure shows an example of an analysis model storage unit of a server device according to the embodiment. [Figure 7] This figure shows a specific example 1 of the display screen of an operator terminal according to the embodiment. [Figure 8] This figure shows a specific example 2 of the display screen of an operator terminal according to the embodiment. [Figure 9] This flowchart shows an example of the overall flow of the event detection system according to the embodiment. [Figure 10] This figure shows an example configuration and processing example of an event detection system according to a modified example 1 of the embodiment. [Figure 11] This figure shows an example configuration and processing example of the event detection system according to a modified example 2 of the embodiment. [Figure 12] This figure shows an example of a hardware configuration according to the embodiment. [Modes for carrying out the invention]

[0011] An image analysis apparatus, an image analysis method, and an image analysis program according to one embodiment of the present invention will be described in detail below with reference to the drawings. However, the present invention is not limited to the embodiments described below.

[0012] [Embodiment] The configuration and processing of the event detection system 100 according to the embodiment, the configuration and processing of each device of the event detection system 100, and the processing flow of the event detection system 100 will be described in order below, and finally the effects of the embodiment will be described.

[0013] [1. Configuration and Processing of Incident Detection System 100 Using FIG. 1, the configuration and processing of the incident detection system 100 according to the embodiment will be described in detail. FIG. 1 is a diagram showing a configuration example and a processing example of the incident detection system 100 according to the embodiment. Hereinafter, a configuration example of the entire incident detection system 100, a processing example of the incident detection system 100, and the effects of the incident detection system 100 will be described. In the embodiment, the identification of people and the detection of actions in facility security and equipment management will be described as an example, but the application field is not limited, and it can also be applied to the monitoring of parks, roads, rivers, etc., and the objects for detecting actions are not only people but also construction machinery, robots, and animals.

[0014] (1-1. Configuration Example of the Entire Incident Detection System 100) The incident detection system 100 includes a server device 10, an operator terminal 20, and a camera 30. Here, the server device 10, the operator terminal 20, and the camera 30 are communicably connected by wire or wirelessly via a predetermined communication network (not shown). Note that various communication networks such as the Internet and dedicated lines can be adopted for the predetermined communication network.

[0015] (1-1-1. Server Device 10) The server device 10 is an image analysis device that analyzes image data acquired from the camera 30. For example, the server device 10 is realized by a cloud environment, an on-premises environment, an edge environment, etc. Note that the incident detection system 100 shown in FIG. 1 may include a plurality of server devices 10.

[0016] (1-1-2. Operator Terminal 20) The operator terminal 20 is an administrator terminal used by the operator O who is the administrator of the facility or equipment. Note that the incident detection system 100 shown in FIG. 1 may include a plurality of operator terminals 20.

[0017] (1-1-3. Camera 30) Camera 30 is a recording device installed in a facility or equipment, such as a security camera or surveillance camera. Note that the event detection system 100 shown in Figure 1 may include multiple cameras 30.

[0018] (1-2. Example of the overall processing of the event detection system 100) The overall processing of the event detection system 100 described above will now be explained. Note that the processes in steps S1 to S7 below can be executed in a different order. Also, some of the processes in steps S1 to S7 below may be omitted.

[0019] (1-2-1. Candidate Information Transmission Process) Firstly, the server device 10 transmits candidate information to the operator terminal 20 (step S1). For example, the server device 10 transmits candidate information to the operator terminal 20, including identification information of AI logic for each analysis category (or simply "category") that the server device 10 can analyze, a list of output values, etc.

[0020] Here, the analysis category refers to the classification of the image analysis target (or simply "target" as appropriate), such as "person action" which indicates the actions of people included in the image, "person identification" which indicates the attributes of people included in the image (e.g., name, affiliation, authority, gender, age, etc.), and "object identification" which indicates objects included in the image that are involved in the actions of people.

[0021] "Personal actions" is an analysis category that corresponds to the "what happened" aspect of a person's behavior, such as "entering a room," "leaving a room," "taking something out," "leaving something behind," "acting violently," "falling down," "vomiting," or "crying," and is used as the target of image analysis.

[0022] Furthermore, the "human actions" mentioned above are examples of actions detected when the subject of behavioral analysis is a person. However, other examples include actions performed by robots or construction machinery, such as "grabbing," "lifting," "installing," and "excavating." Similarly, when the subject of behavioral analysis is an animal, the animal's actions may be the target of detection.

[0023] "Person identification" is an analysis category that corresponds to "who" in terms of the attributes of a person, such as "pre-registered user with permission A," "pre-registered user with permission B," or "non-registered user," as the target of image analysis.

[0024] Furthermore, the above example of "person identification" is an example where the subject of behavioral analysis is a person. However, other examples include including robots when photographing locations where people and robots are working together, or including construction machinery and other work vehicles. Animals can also be included.

[0025] "Object identification" is an analysis category that corresponds to "what"—objects involved in human actions, such as "desk," "chair," "cardboard box," "locker," "file," "safe," "stamp," "houseplant," and "shredder"—as the target of image analysis.

[0026] Furthermore, AI logic is an analysis method implemented by the analysis model AM, which is a pre-trained machine learning model that outputs the image analysis target. The output value list is a list of analysis items, which are the image analysis targets output by each AI logic. Here, the output value list may also be a list of common analysis items.

[0027] (1-2-2. Candidate Information Display Processing) Secondly, the operator terminal 20 displays candidate information (step S2). For example, the operator terminal 20 displays a settings screen on the monitor that presents candidate information such as a list of identification information for AI logic for each analysis category and a list of output values ​​for each AI logic.

[0028] (1-2-3. Inputting configuration information) Thirdly, operator O inputs configuration information into the operator terminal 20 (step S3). For example, operator O inputs the selection of the AI ​​logic to be used for analysis as configuration information by selecting the identification information of any AI logic from a list of identification information of AI logic for each analysis category.

[0029] At this time, operator O selects, for example, "AI Logic 1" as {Analysis Category Example 1: Human Action}, "AI Logic A" as {Analysis Category Example 2: Person Identification}, and "AI Logic W" as {Analysis Category Example 3: Object Identification}.

[0030] Furthermore, Operator O selects analysis items output by the selected AI logic for each analysis category, and assigns importance levels to the combinations of selected analysis items, thereby inputting the importance levels corresponding to the combinations of analysis items as configuration information. At this time, Operator O may also input the reactions that the server device 10 will execute corresponding to the importance levels as configuration information.

[0031] Here, importance refers to the degree of urgency, priority, and severity of responding to an event, and is expressed, for example, on an 11-point scale from 0 to 10. Alternatively, importance may be expressed as a score from 0 to 100%, or as a rank such as A, B, or C.

[0032] At this point, operator O assigns a "severity level of 8" to the combination of analysis items, for example, {"Who": "Unregistered person", "When": "Weekday business hours", "What": "File", "What happened": "Took it out"}, indicating urgency. Similarly, operator O assigns a severity level to each of the other combinations of analysis items that they want to detect.

[0033] (1-2-4. Setting Information Transmission Process) Fourth, the operator terminal 20 transmits configuration information to the server device 10 (step S4). For example, the operator terminal 20 transmits to the server device 10 as configuration information the selection of AI logic for each analysis category, the importance corresponding to the combination of analysis items, and the reaction corresponding to the importance, which were entered into the operator terminal 20 by operator O. At this time, the server device 10 saves the received configuration information.

[0034] (1-2-5. Image Data Transmission Process) Fifth, the camera 30 transmits image data to the server device 10 (step S5). For example, the camera 30 transmits image data of still images taken every second at the facility where it is installed to the server device 10. At this time, the server device 10 stores the received image data along with the time it was captured. The image data may be image data of a moving image or data that includes audio data.

[0035] (1-2-6. Image Data Analysis Processing) Sixth, the server device 10 analyzes the image data (step S6). For example, the server device 10 refers to the configuration information and analyzes the image data using the AI ​​logic selected by operator O for each analysis category. At this time, the server device 10 inputs the image data into the analysis model AM corresponding to each AI logic, acquires the analysis items that are the target of the output image analysis for each analysis category, and identifies the importance corresponding to the combination of acquired analysis items.

[0036] In other words, to explain the example described above, the server device 10 inputs image data into analysis model AM-A, which corresponds to "AI logic A" as {Analysis Category Example 2: Person Identification}, and obtains an analysis item corresponding to "Who". The server device 10 also identifies the time the image data was taken and obtains an analysis item corresponding to "When". The server device 10 also inputs image data into analysis model AM-W, which corresponds to "AI logic W" as {Analysis Category 3: Object Identification}, and obtains an analysis item corresponding to "What". The server device 10 also inputs image data into analysis model AM-1, which corresponds to "AI logic 1" as {Analysis Category Example 1: Person Action}, and obtains an analysis item corresponding to "What happened". Finally, the server device 10 identifies the importance level indicating urgency assigned to the combination of analysis items "Who", "When", "What", and "What happened".

[0037] (1-2-7. Reaction execution process) Seventh, the server device 10 performs a reaction to the operator terminal 20 (step S7). For example, if the identified severity level is above a predetermined value, the server device 10 notifies the operator terminal 20 of an alarm as a reaction. In this case, if the identified severity level is below the predetermined value, the server device 10 does not notify the operator terminal 20 of an alarm as a reaction, but instead records the event and image data that occurred as a log.

[0038] (1-3. Effects of the Event Detection System 100) The following section will explain the problems with the event detection system 100P related to the reference technology, and then describe the effects of the event detection system 100.

[0039] (1-3-1. Problems with the Event Detection System 100P) The event detection system 100P-1 related to Reference Technology 1 is a technology that tracks a fleeing vehicle using a combination of images from multiple cameras, and performs analysis simultaneously on multiple analysis servers, each with its own separate analysis logic. The event detection system 100P-2 related to Reference Technology 2 is a technology that identifies suspicious persons by combining facial recognition and behavioral analysis. However, the event detection system 100P related to Reference Technology does not take into account that the importance of each combination of AI logic output results, such as "who," "when," "what," and "how," differs. For example, even in the case of taking out a highly confidential file, the importance of "who" differs depending on whether the person had permission to take it out or not. Similarly, even in the case of elementary school children playing in a park, the importance of "when" differs depending on whether it is during weekday school hours or on a holiday.

[0040] (1-3-2. Overview of the Event Detection System 100) The event detection system 100 performs the following processes. First, the server device 10 sends candidate information, including identification information and output value lists for each analysis category of AI logic, to the operator terminal 20. Second, the operator terminal 20 displays a settings screen on the monitor, which presents candidate information such as a list of identification information for each analysis category of AI logic and an output value list for each AI logic. Third, the operator O inputs the selection of AI logic to be used for analysis and the importance corresponding to the combination of analysis items output by the selected AI logic as setting information to the operator terminal 20. Fourth, the operator terminal 20 sends the selection of AI logic for each analysis category and the importance corresponding to the combination of analysis items as setting information to the server device 10. Fifth, the camera 30 sends image data of still images taken every second at the facility where it is installed to the server device 10. Sixth, the server device 10 analyzes the image data and identifies the importance corresponding to the combination of analysis items acquired for each analysis category. Seventh, if the identified severity level exceeds a predetermined value, the server device 10 notifies the operator terminal 20 of an alarm as a reaction.

[0041] (1-3-3. Effects of the Event Detection System 100) The event detection system 100 has the following advantages: Firstly, when performing facility security or equipment management, the event detection system 100 can automatically identify the importance of the situation occurring on-site and notify security personnel, making it possible to monitor many locations with a small number of people. Secondly, because the event detection system 100 is implemented in software on a standardized hardware base, it is possible to efficiently manufacture and install the individual components of the system on-site, and to configure settings according to the specific location and installation environment. Thirdly, the event detection system 100 allows for easy modification of settings even when the installation environment changes.

[0042] As described above, in the event detection system 100, the user can define an appropriate reaction to an event that has occurred based on combination conditions of the output results of separately manufactured AI logics, and the system performs analysis according to that definition and outputs the results, thereby enabling the user to perform an appropriate reaction.

[0043] [2. Configuration and Processing of Each Device in the Event Detection System 100] Using Figure 2, the configuration and processing of each device in the event detection system 100 shown in Figure 1 will be explained. Figure 2 is a block diagram showing an example configuration of each device in the event detection system 100 according to the embodiment. Below, an example configuration of the entire event detection system 100 according to the embodiment will be explained, followed by a detailed explanation of the configuration and processing examples of the server device 10, the operator terminal 20, and the camera 30.

[0044] (2-1. Example of the overall configuration of the event detection system 100) Using Figure 2, an example of the overall configuration of the event detection system 100 shown in Figure 1 will be explained. As shown in Figure 2, the event detection system 100 includes a server device 10, an operator terminal 20, and a camera 30. The server device 10, the operator terminal 20, and the camera 30 are connected to each other via a communication network N, which can be implemented via the internet or a dedicated line.

[0045] The server device 10 is installed in a cloud environment, on-premises environment, edge environment, etc. The operator terminal 20 is installed in a monitoring room of a facility or equipment managed by operator O. The camera 30 is installed at the monitoring site of the facility or equipment.

[0046] (2-2. Example configuration and processing of server device 10) Using Figure 2, an example of the configuration and processing of the server device 10 will be explained. The server device 10 is an image analysis device and includes an input unit 11, an output unit 12, a communication unit 13, a storage unit 14, and a control unit 15.

[0047] (2-2-1. Input section 11) The input unit 11 is responsible for inputting various types of information to the server device 10. For example, the input unit 11 can be implemented using a mouse or keyboard, and it accepts various types of information input to the server device 10.

[0048] (2-2-2. Output section 12) The output unit 12 is responsible for outputting various types of information from the server device 10. For example, the output unit 12 is implemented as a display or the like and displays various types of information stored in the server device 10.

[0049] (2-2-3. Communications Section 13) The communication unit 13 is responsible for data communication with other devices. For example, the communication unit 13 communicates data with each communication device via a router or the like. The communication unit 13 can also communicate data with an operator's terminal (not shown).

[0050] (2-2-4. Storage section 14) The storage unit 14 stores various information that the control unit 15 references when it operates, and various information acquired when the control unit 15 operates. The storage unit 14 includes a candidate information storage unit 14a, a setting information storage unit 14b, an image data storage unit 14c, and an analysis model storage unit 14d. Here, the storage unit 14 can be implemented as, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or optical disc. In the example in Figure 2, the storage unit 14 is installed inside the server device 10, but it may be installed outside the server device 10, or multiple storage units may be installed.

[0051] (2-2-4-1. Candidate information storage unit 14a) The candidate information storage unit 14a stores candidate information. For example, the candidate information storage unit 14a stores candidate information including identification information of AI logic for each analysis category that can be analyzed by the execution unit 15c of the control unit 15, which will be described later, output value list, etc. The analysis category for image analysis targets includes, for example, at least one of the following: person attributes, person actions, and objects involved in person actions. Here, an example of the data stored by the candidate information storage unit 14a will be explained using Figure 3. Figure 3 is a diagram showing an example of the candidate information storage unit 14a of the server device 10 according to the embodiment. In the example in Figure 3, the candidate information storage unit 14a has items such as "analysis category," "AI logic," and "output value list."

[0052] "Analysis Category" indicates identification information for identifying the category of image analysis target, such as an identification number or identification symbol for the category of image analysis target, such as a person's actions, a person's attributes, or objects involved in a person's actions. "AI Logic" indicates identification information for identifying the AI ​​logic that the server device 10 can analyze, such as an identification number or identification symbol for the AI ​​logic. "Output Value List" is a list of analysis items that each AI logic outputs as image analysis targets, such as a list of a person's actions, a list of a person's attributes, or a list of objects involved in a person's actions.

[0053] In other words, in Figure 3, the analysis categories identified by "Analysis Category #1" are {AI Logic: "AI Logic 1", Output Value List: "Output Value List 1", ...}, {AI Logic: "AI Logic 2", Output Value List: "Output Value List 2", ...}, {AI Logic: "AI Logic 3", Output Value List: "Output Value List 3", ...}, {AI Logic: "AI Logic 4", Output Value List: "Output Value List 4", ...}, and the analysis categories identified by "Analysis Category #2" are {AI Logic: "AI Logic A", Output Value List: "Output Value List A", ...}, {AI Logic: "AI Logic B" An example is shown in which data such as {AI Logic: "AI Logic Z", Output Value List: "Output Value List B", ...}, {AI Logic: "AI Logic Y", Output Value List: "Output Value List Y", ...}, {AI Logic: "AI Logic X", Output Value List: "Output Value List X", ...}, and {AI Logic: "AI Logic W", Output Value List: "Output Value List W", ...} is stored in the candidate information storage unit 14a for the analysis category identified by "Analysis Category #3".

[0054] The candidate information storage unit 14a may also include features of AI logic as candidate information. For example, the candidate information storage unit 14a may store data such as {AI Logic: "AI Logic 1", Feature: "Facial Recognition", ...}, {AI Logic: "AI Logic 2", Feature: "Personal Identification by Walking Style", ...}, {AI Logic: "AI Logic 3", Feature: "Gender and Age Estimation", ...}, {AI Logic: "AI Logic 4", Feature: "Clothing and Equipment Detection", ...} as features of AI logic.

[0055] (2-2-4-2. Setting information storage unit 14b) The setting information storage unit 14b stores setting information. For example, the setting information storage unit 14b stores setting information, including the selection of AI logic for each analysis category and the importance corresponding to the combination of analysis items, which is input by the operator O via the setting screen and received by the reception unit 15a of the control unit 15, which will be described later. Here, an example of the data stored by the setting information storage unit 14b will be explained using Figure 4. Figure 4 is a diagram showing an example of the setting information storage unit 14b of the server device 10 according to the embodiment. In the example in Figure 4, the setting information storage unit 14b has items such as "user", "location", "analysis category", "AI logic", "output value list", and "importance information".

[0056] "User" refers to identification information for identifying the manager who manages the facility or equipment, such as the identification number or code of operator O or the management company. "Location" refers to identification information for identifying the facility or equipment managed by the "User," such as the identification number or code of the facility or equipment. "Analysis Category" refers to identification information for identifying the category of the image analysis target, such as the identification number or code of the category of the image analysis target, such as a person's actions, a person's attributes, or objects involved in a person's actions. "AI Logic" refers to identification information for identifying the AI ​​logic received by the reception unit 15a, such as the identification number or code of the AI ​​logic. "Output Value List" is a list of analysis items that are the image analysis target output by each AI logic, such as a list of a person's actions, a list of a person's attributes, a list of objects involved in a person's actions, etc. "Importance Information" is a list of importance levels assigned to combinations of analysis items that are the image analysis target output by each AI logic, such as a list of combinations of analysis items such as "who," "when," "what," and "how," and a list of importance levels corresponding to those combinations.

[0057] In other words, Figure 4 shows an example in which data such as {Analysis Category: "Analysis Category #1", AI Logic: "AI Logic 1", Output Value List: "Output Value List 1"}, {Analysis Category: "Analysis Category #2", AI Logic: "AI Logic A", Output Value List: "Output Value List A"}, and {Analysis Category: "Analysis Category #3", AI Logic: "AI Logic W", Output Value List: "Output Value List W"}, where the importance information is "Importance Information #1", is stored in the setting information storage unit 14b for a user identified as "User #1" and a location identified as "Location #1".

[0058] The setting information storage unit 14b may also include reactions corresponding to importance levels as setting information. For example, the setting information storage unit 14b may store data such as {importance: "0", reaction: "event recording"}, {importance: "1-6", reaction: "image data recording"}, and {importance: "7-10", reaction: "image data recording", "alarm notification"} as reactions corresponding to importance levels.

[0059] (2-2-4-3. Image data storage unit 14c) The image data storage unit 14c stores image data. For example, the image data storage unit 14c stores image data acquired by the acquisition unit 15b of the control unit 15, which will be described later. Here, an example of the data stored by the image data storage unit 14c will be explained using Figure 5. Figure 5 is a diagram showing an example of the image data storage unit 14c of the server device 10 according to this embodiment. In the example in Figure 5, the image data storage unit 14c has items such as "camera", "location", "time", and "image data".

[0060] "Camera" refers to identification information for identifying the imaging device, such as the identification number or code of camera 30. "Location" refers to identification information for identifying the facility or equipment where the imaging device is installed, such as the identification number or code of the facility or equipment. "Time" refers to the time of shooting, such as year, month, day, hour, minute, and second. "Image data" refers to image data acquired during the shooting time, such as image data of still images acquired every second, image data of moving images, image data of moving images including audio data, etc.

[0061] In other words, Figure 5 shows an example in which data such as {time: "time #1", image data: "image data #1"}, {time: "time #2", image data: "image data #2"}, {time: "time #3", image data: "image data #3"}, etc., is stored in the image data storage unit 14c for the camera identified by "camera #1" and the location identified by "location #1".

[0062] (2-2-4-4. Analysis Model Storage Unit 14d) The analysis model storage unit 14d stores analysis models AM. For example, the analysis model storage unit 14d stores multiple analysis models AM, which are machine learning models used by the execution unit 15c of the control unit 15 (described later), and which output image analysis targets included in image data. In this case, the analysis model storage unit 14d stores the multiple analysis models AM for each category of image analysis targets to be output. Here, an example of the data stored by the analysis model storage unit 14d will be explained using Figure 6. Figure 6 is a diagram showing an example of the analysis model storage unit 14d of the server device 10 according to the embodiment. In the example in Figure 6, the analysis model storage unit 14d has items such as "analysis category", "AI logic", and "analysis model".

[0063] "Analysis Category" refers to identification information for identifying the categories of images to be analyzed, such as identification numbers or symbols for categories of images to be analyzed, such as a person's actions, a person's attributes, or objects involved in a person's actions. "AI Logic" refers to identification information for identifying AI logic that the server device 10 can analyze, such as identification numbers or symbols for the AI ​​logic. "Analysis Model" is model data for a machine learning model corresponding to the "AI Logic," and includes data such as execution data for executing the algorithm of the analysis model AM of each AI logic, model parameters which are set values, hyperparameters, etc.

[0064] In other words, in Figure 6, the analysis categories identified by "Analysis Category #1" are {AI Logic: "AI Logic 1", Analysis Model: "Analysis Model 1", ...}, {AI Logic: "AI Logic 2", Analysis Model: "Analysis Model 2", ...}, {AI Logic: "AI Logic 3", Analysis Model: "Analysis Model 3", ...}, {AI Logic: "AI Logic 4", Analysis Model: "Analysis Model 4", ...}, and the analysis categories identified by "Analysis Category #2" are {AI Logic: "AI Logic A", Analysis Model: "Analysis Model A", ...}, {AI Logic: "AI Logic B" An example is shown in which data such as {AI Logic: "AI Logic Z", analysis model: "Analysis Model Z", ...}, {AI Logic: "AI Logic Y", analysis model: "Analysis Model Y", ...}, {AI Logic: "AI Logic X", analysis model: "Analysis Model X", ...}, and {AI Logic: "AI Logic W", analysis model: "Analysis Model W", ...} is stored in the analysis model storage unit 14d for the analysis category identified by "Analysis Category #3".

[0065] (2-2-5. Control Unit 15) The control unit 15 is responsible for controlling the entire server device 10. The control unit 15 includes a receiving unit 15a, an acquisition unit 15b, and an execution unit 15c. Here, the control unit 15 can be implemented by electronic circuits such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or by integrated circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0066] (2-2-5-1. Reception area 15a) The reception unit 15a receives various types of information. The reception unit 15a may also store the received information in the storage unit 14. The following describes the setting screen display process, the AI ​​logic selection reception process, and the importance information reception process.

[0067] (Settings screen display process) The reception unit 15a executes the setting screen display process. For example, the reception unit 15a displays an AI logic selection screen on the user's terminal as a setting screen that presents a list of multiple machine learning models, or analysis models AM, for each analysis category of the image to be analyzed, and accepts input for selecting an analysis model AM for each analysis category.

[0068] To give a specific example of the AI ​​logic selection screen, the reception unit 15a sends candidate information to the operator terminal 20 used by operator O, and displays {"AI Logic 1", "AI Logic 2", "AI Logic 3", "AI Logic 4"} for "Analysis Category #1", {"AI Logic A", "AI Logic B", "AI Logic C"} for "Analysis Category #2", and {"AI Logic Z", "AI Logic Y", "AI Logic X", "AI Logic W"} for "Analysis Category #3" on the input / output unit 21 of the operator terminal 20.

[0069] Furthermore, the reception unit 15a displays an output value list viewing screen on the user's terminal as a settings screen that shows a list of image analysis targets output by each of the multiple analysis models AM.

[0070] To give a specific example of the output value list viewing screen, the reception unit 15a sends candidate information to the operator terminal 20 used by operator O, and displays {"Analysis Item 1-1", "Analysis Item 1-2", "Analysis Item 1-3", "Analysis Item 1-4", "Analysis Item 1-5", "Analysis Item 1-6", "Analysis Item 1-7", "Analysis Item 1-8"} for the "Output Value List 1" that operator O has requested to view on the input / output unit 21 of the operator terminal 20.

[0071] Furthermore, the reception unit 15a displays a list of image analysis targets output by the analysis model AM that has been selected for each analysis category of image analysis targets, and also displays an importance setting screen on the user's terminal as a setting screen that accepts input of combinations of image analysis targets for each analysis category and the importance level for each combination.

[0072] To give a specific example of the importance setting screen, the reception unit 15a sends candidate information to the operator terminal 20 used by operator O, and for "AI Logic 1" selected by operator O, {"Analysis Item 1-1", "Analysis Item 1-2", "Analysis Item 1-3", "Analysis Item 1-4", "Analysis Item 1-5", "Analysis Item 1-6", "Analysis Item 1-7", "Analysis Item 1-8"}, and for "AI Logic A" selected by operator O, {"Analysis Item A-1", "Analysis Item The input / output unit 21 of the operator terminal 20 displays {"Analysis Item A-2", "Analysis Item A-3"}, {"Analysis Item W-1", "Analysis Item W-2", "Analysis Item W-3", "Analysis Item W-4", "Analysis Item W-5", "Analysis Item W-6", "Analysis Item W-7", "Analysis Item W-8", "Analysis Item W-9"} regarding the "AI Logic W" selected by operator O, and {"Analysis Item T-1", "Analysis Item T-2", "Analysis Item T-3"} regarding the occurrence time of the event. Then, the reception unit 15a displays an input screen for the "importance" corresponding to the combination of analysis items selected by operator O {"Analysis Item 1-3", "Analysis Item A-3", "Analysis Item W-5", "Analysis Item T-1"} on the input / output unit 21 of the operator terminal 20.

[0073] (AI logic selection acceptance processing) The reception unit 15a executes the AI ​​logic selection reception process. For example, by accepting the selection of AI logic, the reception unit 15a accepts the selection of an analysis model AM from multiple analysis models AM to be used for the AI ​​logic. At this time, by accepting the selection of AI logic for each analysis category, the reception unit 15a accepts the selection of an analysis model AM from multiple analysis models AM to be used for the AI ​​logic for each analysis category.

[0074] To give a concrete example of the AI ​​logic selection acceptance process, the acceptance unit 15a accepts the following AI logics as input by operator O by operating the AI ​​logic selection screen displayed on the monitor of the operator terminal 20: {Analysis Category: "Analysis Category #1", AI Logic: "AI Logic 1"}, {Analysis Category: "Analysis Category #2", AI Logic: "AI Logic A"}, and {Analysis Category: "Analysis Category #3", AI Logic: "AI Logic W"}, and stores them in the setting information storage unit 14b.

[0075] (Importance information reception processing) The reception unit 15a performs importance information reception processing. For example, the reception unit 15a receives importance levels associated with combinations of image analysis targets. In this case, the reception unit 15a receives importance levels associated with combinations of image analysis targets for each analysis category. The reception unit 15a may also receive reactions corresponding to the importance levels.

[0076] To give a specific example of the importance information reception process, the reception unit 15a receives importance information such as "Importance Information #1," which is a combination of analysis items "Analysis Item 1-3" output by "AI Logic 1" selected in "Analysis Category #1," "Analysis Item A-3" output by "AI Logic A" selected in "Analysis Category #2," "Analysis Item W-5" output by "AI Logic W" selected in "Analysis Category #3," and analysis item "Analysis Item T-1" indicating the time of event occurrence, with an importance level of "8" assigned to it, and stores it in the setting information storage unit 14b. At this time, the reception unit 15a may receive the combination and importance of the analysis items, as well as the reactions that the execution unit 15c will perform according to the importance entered by the operator O, such as {importance: "0", reaction: "event recording"}, {importance: "1~6", reaction: "image data recording"}, {importance: "7~10", reaction: "image data recording", "alarm notification"}, etc., as "importance information #1" and store them in the setting information storage unit 14b.

[0077] (2-2-5-2. Acquisition part 15b) The acquisition unit 15b acquires various types of information. The acquisition unit 15b may also store the acquired information in the storage unit 14. For example, the acquisition unit 15b acquires image data. In this case, the acquisition unit 15b acquires image data captured and transmitted by a camera 30, which is a shooting device installed in a facility managed by the user, operator O.

[0078] To give a specific example, the acquisition unit 15b acquires image data such as {time: "time #1", image data: "image data #1"}, {time: "time #2", image data: "image data #2"}, {time: "time #3", image data: "image data #3"} as image data captured and transmitted by the camera 30, and stores it in the image data storage unit 14c.

[0079] (2-2-5-3. Execution section 15c) The execution unit 15c performs various processes. The execution unit 15c may also acquire various information from the storage unit 14. The importance determination process and the reaction execution process will be described below.

[0080] (Importance determination process) The execution unit 15c performs importance identification processing. For example, the execution unit 15c inputs the image data acquired by the acquisition unit 15b into the selected analysis model AM and identifies the importance associated with the output combination of image analysis targets. Alternatively, the execution unit 15c inputs the image data acquired by the acquisition unit 15b into the selected analysis model AM for each analysis category and identifies the importance associated with the output combination of image analysis targets for each analysis category.

[0081] To give a specific example of the importance determination process, the execution unit 15c inputs image data {time: "time #1", image data: "image data #1"} into "analysis model 1" used in "AI logic 1" and obtains the output result analysis item "analysis item 1-3". The execution unit 15c also inputs image data {time: "time #1", image data: "image data #1"} into "analysis model A" used in "AI logic A" and obtains the output result analysis item "analysis item A-3". The execution unit 15c also inputs image data {time: "time #1", image data: "image data #1"} into "analysis model W" used in "AI logic W" and obtains the output result analysis item "analysis item W-5". Furthermore, the execution unit 15c obtains the analysis item "analysis item T-1", which indicates the occurrence time of the event identified from the image data {time: "time #1", image data: "image data #1"}. Then, the execution unit 15c refers to the setting information storage unit 14b and obtains {AI logic 1: "Analysis item 1-3", AI logic A: "Analysis item A-3", AI logic W: "Analysis item W-5", Occurrence time: "Analysis item T-1", Importance: "8"}, and identifies the importance of the occurrence as "8".

[0082] (Reaction execution process) The execution unit 15c performs reaction execution processing. For example, the execution unit 15c performs a predetermined reaction according to the importance assigned to the combination of image analysis targets output. The execution unit 15c also performs a predetermined reaction according to the importance assigned to the combination of image analysis targets for each analysis category output. At this time, the execution unit 15c performs at least one of the following as a predetermined reaction set by the user, operator O, according to the importance: notifying operator O of an alarm, recording image data, and recording the output image analysis targets.

[0083] To give a specific example of the reaction execution process, if the execution unit 15c determines that the severity of the event that occurred is "8", it refers to the setting information storage unit 14b and obtains {Severity: "0", Reaction: "Event Recording"}, {Severity: "1~6", Reaction: "Image Data Recording"}, and {Severity: "7~10", Reaction: "Image Data Recording" and "Alarm Notification"}. As a reaction, it records image data of the time the event occurred and notifies the operator terminal 20 of the occurrence of an alarm.

[0084] (2-3. Example configuration and processing of operator terminal 20) Using Figure 2 again, an example of the configuration and processing of the operator terminal 20 will be described. The operator terminal 20 is a posting device and a viewing device, and has an input / output unit 21, a transmitting / receiving unit 22, and a communication unit 23.

[0085] (2-3-1. Input / output section 21) The input / output unit 21 is responsible for inputting various types of information to the operator terminal 20. For example, the input / output unit 21 can be implemented as a mouse, keyboard, touch panel, etc., and accepts input such as setting information to the operator terminal 20. The input / output unit 21 is also responsible for displaying various types of information from the operator terminal 20. For example, the input / output unit 21 can be implemented as a display, etc., and displays setting information etc. stored in the operator terminal 20.

[0086] Furthermore, the input / output unit 21 displays candidate information transmitted from the server device 10, which is an image analysis device. For example, the input / output unit 21 displays setting screens based on the candidate information, such as the AI ​​logic selection screen, the output value list viewing screen, and the importance setting screen. Details of the AI ​​logic selection screen will be described later in (2-3-4. Specific Example 1 of the Display Screen of the Operator Terminal 20). Details of the importance setting screen will be described later in (2-3-5. Specific Example 2 of the Display Screen of the Operator Terminal 20).

[0087] (2-3-2. Transceiver Unit 22) The transmitting / receiving unit 22 transmits various types of information. For example, the transmitting / receiving unit 22 transmits setting information entered by operator O via the setting screen to the server device 10.

[0088] The transmitting / receiving unit 22 receives various types of information. For example, the transmitting / receiving unit 22 receives candidate information transmitted from the server device 10. The transmitting / receiving unit 22 also receives alarms transmitted from the server device 10.

[0089] (2-3-3. Communications Section 23) The communication unit 23 is responsible for data communication with other devices. For example, the communication unit 23 communicates data with each communication device via a router or the like. The communication unit 23 can also communicate data with an operator's terminal (not shown).

[0090] (2-3-4. Specific example of the display screen of the operator terminal 20) Here, using Figure 7, we will explain a specific example 1 of the display screen output by the input / output unit 21 of the operator terminal 20. Figure 7 is a diagram showing a specific example 1 of the display screen of the operator terminal 20 according to the embodiment. Below, we will explain the "AI logic selection screen," "human motion," "person identification," "object identification," "AI logic feature viewing screen," and "output value list viewing screen."

[0091] (2-3-4-1. AI Logic Selection Screen) As shown in the example in Figure 7, the operator terminal 20 displays an "AI Logic Selection Screen," which is a settings screen that accepts input for the selection of AI logic (i.e., analysis model AM) for each analysis category. Here, operator O can input the selection of AI logic for each analysis category by clicking on the AI ​​logic displayed for each item: "Example Analysis Category 1: Human Motion," "Example Analysis Category 2: Person Identification," and "Example Analysis Category 3: Object Identification." In the example in Figure 7, the operator terminal 20 displays the AI ​​logic for each of the three analysis categories: "Example Analysis Category 1: Human Motion," "Example Analysis Category 2: Person Identification," and "Example Analysis Category 3: Object Identification," but the analysis categories displayed by the operator terminal 20 are not limited to the above example.

[0092] (2-3-4-2.Human movements) As shown in the example in Figure 7, the operator terminal 20 displays "Human Actions" as one of the analysis categories, indicating human behavior. In the example in Figure 7, "AI Logic 1," "AI Logic 2," "AI Logic 3," and "AI Logic 4" are displayed as AI logics that output "Human Actions." Here, the AI ​​logic that outputs "Human Actions" is an analysis method that realizes motion estimation, skeletal estimation, violent act detection, fall detection, person tracking, etc., by inputting image data, but the analysis method that outputs "Human Actions" is not limited to the example above. Furthermore, the target of the behavior analysis may be not only people, but also robots, construction machinery and other work vehicles, animals, etc., so the actions performed by each of these targets may be the target.

[0093] (2-3-4-3.Person identification) As shown in the example in Figure 7, the operator terminal 20 displays "Person Identification," which indicates the attributes of a person, as one of the analysis categories. In the example in Figure 7, "AI Logic A," "AI Logic B," and "AI Logic C" are displayed as AI logics that output "Person Identification." Here, the AI ​​logic that outputs "Person Identification" is an analysis method that, for example, realizes facial recognition, personal identification by gait, gender and age estimation, clothing and equipment detection, and detection of persons requiring assistance by inputting image data, but the analysis method that outputs "Person Identification" is not limited to the example above.

[0094] Furthermore, "person identification" is just one example of AI logic for identifying subjects; it is also possible to employ AI logic that outputs "animal identification" to identify animals, "working machinery identification" to identify robots, construction machinery and other work vehicles, "natural object identification" to identify natural objects such as river water and fallen rocks, etc.

[0095] (2-3-4-4. Identifying Objects) As shown in the example in Figure 7, the operator terminal 20 displays "Object Identification," which indicates objects involved in human behavior, as one of the analysis categories. In the example in Figure 7, "AI Logic Z," "AI Logic Y," "AI Logic X," and "AI Logic W" are displayed as AI logics that output "Object Identification." Here, the AI ​​logic that outputs "Object Identification" is an analysis method that realizes object type recognition, vehicle detection, animal recognition, plant recognition, hazardous material detection, etc., by inputting image data, but the analysis method that outputs "Object Identification" is not limited to the above example.

[0096] Furthermore, "object identification" is just one example of AI logic that identifies an object; it is also possible to employ AI logic that outputs the aforementioned "person identification."

[0097] (2-3-4-5. AI Logic Feature Viewing Screen) The operator terminal 20 can also display an "AI Logic Feature Viewing Screen," a settings screen that presents the characteristics of AI logic, in order to assist operator O in selecting an AI logic. For example, if operator O clicks the "AI Logic" button for "AI Logic 1" in "Analysis Category Example 1: Human Motion," the operator terminal 20 will display information such as "Facial Recognition," which are characteristics of "AI Logic 1."

[0098] (2-3-4-6. Output Value List Viewing Screen) The operator terminal 20 can also display an "output value list viewing screen," which is a settings screen that presents a list of analysis items output by the AI ​​logic, in order to assist operator O in selecting the AI ​​logic. For example, if operator O clicks the "output value list" button for "AI logic 1" in "Analysis category example 1: Human actions," the operator terminal 20 will display information such as "enter the room," "exit the room," "take something out," "leave something behind," "commit violence," "collapse," "vomit," and "cry" as a list of analysis items output by "AI logic 1."

[0099] (2-3-5. Specific example of the display screen of the operator terminal 20) Here, using Figure 8, we will explain a specific example 2 of the display screen output by the input / output unit 21 of the operator terminal 20. Figure 8 is a diagram showing a specific example 2 of the display screen of the operator terminal 20 according to the embodiment. Below, we will explain the "importance setting screen," "person identification," "occurrence time," "object identification," "person action," "importance," and "reaction."

[0100] (2-3-5-1. Importance setting screen) As shown in the example in Figure 8, the operator terminal 20 displays a "severity setting screen," which is a setting screen that accepts the importance level associated with combinations of analysis items (i.e., events that occur) for each analysis category. Here, operator O can input combinations of analysis items for each analysis category by clicking the checkboxes of the analysis items displayed for each of the items "Who," "When," "What," and "How." Operator O can also input the "severity level," which indicates the urgency, etc., corresponding to the combination of analysis items for each analysis category. Operator O can also input a "reaction," such as an alarm notification, corresponding to the severity level.

[0101] (2-3-5-2.Person identification) As shown in the example in Figure 8, the operator terminal 20 displays "Who" as one of the analysis categories, corresponding to "Analysis Category Example 2: Person Identification" described above. In the example in Figure 8, the AI ​​logic for "Person Identification" selected by operator O on the "AI Logic Selection Screen" displays "Pre-registered User with Authority A," "Pre-registered User with Authority B," and "Non-registered User" as analysis items. Furthermore, "Non-registered User" is selected by operator O's actions. Note that the analysis items output by the AI ​​logic for "Person Identification" are not limited to the example above.

[0102] In this case, the analysis items output by the "person identification" AI logic may be analysis items directly output by the analysis model AM, or analysis items that have been converted and output. For example, the analysis model AM may directly output one of the analysis items "Pre-registered user with permission A," "Pre-registered user with permission B," or "Non-registered user" in response to the input image data, or it may convert and output one of the analysis items "Pre-registered user with permission A," "Pre-registered user with permission B," or "Non-registered user" associated with the personal name output by facial recognition in response to the input image data.

[0103] (2-3-5-3. Occurrence Time) As shown in the example in Figure 8, the operator terminal 20 displays "When" as one of the analysis categories, corresponding to the time the event occurred. In the example in Figure 8, the analysis items that can be identified from the image data capture time are "Weekday Business Hours," "Weekday Outside Business Hours," "Weekday Late Night Hours," "Holiday Daytime," and "Holiday Nighttime / Late Night Hours." Furthermore, "Weekday Business Hours" is selected by operator O. Note that the analysis items that can be identified from the image data capture time are not limited to the example above.

[0104] In this case, the analysis items that can be identified from the image data capture time as the time of the event's occurrence may be classified by the server device 10 based on pre-registered company work schedules, school attendance schedules, etc., or they may be output based on the day of the week and time set by operator O. Furthermore, the analysis items for the event's occurrence time may be output based on the analysis model AM, which directly outputs analysis items in response to image data input.

[0105] (2-3-5-4. Identifying objects) As shown in the example in Figure 8, the operator terminal 20 displays "What" as one of the analysis categories, corresponding to "Example Analysis Category 3: Object Identification" described above. In the example in Figure 8, the AI ​​logic for "Object Identification," selected by operator O on the "AI Logic Selection Screen" described above, displays "Desk," "Chair," "Cardboard Box," "Locker," "File," "Safe," "Stamp," "Houseplant," and "Shredder" as analysis items output. Also, "File" has been selected by operator O. Note that the analysis items output by the AI ​​logic for "Object Identification" are not limited to the above example.

[0106] In this case, the analysis items output by the AI ​​logic for "object identification" may be analysis items directly output by the analysis model AM, or analysis items that have been converted from the outputted analysis items. Furthermore, operator O can skip selecting "object identification" if there are no objects corresponding to "human movement".

[0107] (2-3-5-5.Human movements) As shown in the example in Figure 8, the operator terminal 20 displays "What happened?" as one of the analysis categories, corresponding to "Example Analysis Category 1: Human Action" described above. In the example in Figure 8, the AI ​​logic for "Human Action," selected by operator O on the "AI Logic Selection Screen" described above, displays "Enter room," "Exit room," "Take out," "Leave behind," "Become violent," "Fall down," "Vomit," and "Cry" as analysis items output. Also, "Take out" is selected by operator O's operation. Note that the analysis items output by the AI ​​logic for "Human Action" are not limited to the example above.

[0108] In this case, the analysis items output by the AI ​​logic for "human movement" may be analysis items directly output by the analysis model AM, or they may be analysis items that have been converted from the output items.

[0109] (2-3-5-6.Importance) As shown in the example in Figure 8, the operator terminal 20 can input an "importance level" indicating urgency, etc., corresponding to the combination of analysis items for each analysis category. In the example in Figure 8, "importance level 8" (minimum value 0, maximum value 10) is entered for the combination of analysis items selected by operator O {Who: "Unregistered person", When: "Weekday business hours", What: "File", How: "Take out"}. Note that the input format for "importance level" is not limited to the example above and may be expressed as a score from 0 to 100%, or as ranks such as A, B, C, etc.

[0110] (2-3-5-7. Reactions) The operator terminal 20 can input "reactions" such as alarm notifications corresponding to the "importance level". For example, it can input reactions to be performed by the server device 10, such as {importance level: "0", reaction: "event recording"}, {importance level: "1-6", reaction: "image data recording"}, {importance level: "7-10", reaction: "image data recording" and "alarm notification"}.

[0111] (2-4. Example configuration and processing of camera 30) Using Figure 2 again, we will explain an example of the configuration and processing of the camera 30. For example, the camera 30 is implemented by a security camera, surveillance camera, etc., installed in a facility managed by operator O, and has a shooting unit 31 and a communication unit 32.

[0112] (2-4-1. Photography Section 31) The imaging unit 31 generates image data. For example, the imaging unit 31 may photograph the inside of the facility every second and generate still image data. Alternatively, the imaging unit 31 may photograph the inside of the facility and generate moving image data. At this time, the imaging unit 31 may also record audio inside the facility and generate audio data.

[0113] The imaging unit 31 transmits the generated image data to the server device 10. For example, the imaging unit 31 transmits the generated still image data to the server device 10. The imaging unit 31 also transmits the generated moving image data to the server device 10. At this time, the imaging unit 31 may also transmit the generated audio data to the server device 10.

[0114] (2-4-2. Communications Section 32) The communication unit 32 is responsible for data communication with other devices. For example, the communication unit 32 communicates data with various communication devices via a router or the like. The communication unit 32 can also communicate data with an operator's terminal (not shown).

[0115] [3. Flow of each process in the event detection system 100] The processing flow of the event detection system 100 according to the embodiment will be explained using Figure 9. Note that the processes in steps S101 to S107 below can be executed in a different order. Also, some of the processes in steps S101 to S107 below may be omitted.

[0116] (3-1. Candidate Information Transmission Process) Firstly, the server device 10 performs candidate information transmission processing (step S101). For example, the server device 10 transmits candidate information to the operator terminal 20, including identification information of AI logic for each analysis category that the server device 10 can analyze, output value list, etc.

[0117] (3-2. Candidate Information Display Processing) Secondly, the operator terminal 20 performs candidate information display processing (step S102). For example, the operator terminal 20 displays a settings screen on the monitor that presents candidate information such as a list of identification information for AI logic for each analysis category and a list of output values ​​for each AI logic.

[0118] (3-3. Inputting configuration information) Thirdly, Operator O performs the configuration information input process (step S103). For example, Operator O inputs the selection of AI logic to be used for analysis as configuration information by selecting an AI logic from a list of identification information for AI logic for each analysis category, selects the analysis items output by the selected AI logic for each analysis category, and inputs the importance corresponding to the combination of analysis items as configuration information by assigning importance to the combination of analysis items.

[0119] (3-4. Setting Information Transmission Process) Fourth, the operator terminal 20 performs a configuration information transmission process (step S104). For example, the operator terminal 20 transmits to the server device 10 as configuration information the selection of AI logic for each analysis category, the importance corresponding to the combination of analysis items, etc., which were entered into the operator terminal 20 by operator O.

[0120] (3-5. Candidate Information Display Processing) Fifth, the camera 30 performs image data transmission processing (step S105). For example, the camera 30 transmits image data of still images taken every second at the facility where it is installed to the server device 10.

[0121] (3-6. Image Data Analysis Processing) Sixth, the server device 10 performs image data analysis processing (step S106). For example, the server device 10 refers to the configuration information and uses the AI ​​logic selected by operator O for each analysis category to analyze the image data and identify the importance corresponding to the combination of analysis items that are the analysis results.

[0122] (3-7. Reaction execution process) Seventh, the server device 10 executes a reaction execution process (step S107). For example, if the identified severity level is above a predetermined value, the server device 10 notifies the operator terminal 20 of an alarm as a reaction.

[0123] [4. Effects of the Embodiment] Finally, the effects of the embodiment will be described. Below, effects 1 to 8 corresponding to the processing according to the embodiment will be described.

[0124] (4-1. Effect 1) Firstly, in the process according to the embodiment described above, the server device 10 stores a plurality of analysis models AM that output image analysis targets included in the image data, accepts the selection of an analysis model AM from the plurality of analysis models AM and the importance associated with the combination of image analysis targets, acquires the image data, inputs the acquired image data into the selected analysis model AM, and executes a predetermined reaction according to the importance associated with the output combination of image analysis targets. Therefore, this process can execute an appropriate reaction to the event that occurs.

[0125] (4-2. Effect 2) Secondly, in the processing according to the embodiment described above, the server device 10 stores a plurality of analysis models AM for each analysis category of the image analysis target to be output, accepts the selection of an analysis model AM from the plurality of analysis models AM for each analysis category, and accepts the importance level associated with the combination of image analysis targets for each analysis category, inputs the acquired image data into the analysis model AM selected for each analysis category, and executes a predetermined reaction according to the importance level associated with the combination of image analysis targets for each output analysis category. Therefore, in this processing, the importance level can be set in detail for each event that occurs, so that an appropriate reaction to the event that occurs can be executed.

[0126] (4-3. Effect 3) Thirdly, in the processing according to the embodiment described above, the server device 10 displays a list of multiple analysis models AM for each analysis category of the image to be analyzed, and displays a setting screen on the operator terminal 20 of the operator O that accepts input for the selection of an analysis model AM for each analysis category. Therefore, in this processing, by selecting the AI ​​logic to be used for each analysis category, the operator O can perform an appropriate reaction to the event that occurs.

[0127] (4-4. Effect 4) Fourth, in the processing according to the embodiment described above, the server device 10 displays a list of multiple analysis models AM for each analysis category of the image to be analyzed, and also displays a setting screen on the operator terminal 20 of the operator O that displays a list of the image to be analyzed for each of the multiple analysis models AM. Therefore, in this processing, by providing the information necessary for the operator O to select the AI ​​logic to be used, an appropriate reaction to the event that occurs can be performed.

[0128] (4-5. Effect 5) Fifth, in the processing according to the embodiment described above, the server device 10 displays a list of image analysis targets output by the analysis model AM selected for each category of image analysis target, and displays a setting screen on the operator terminal 20 of operator O that accepts input of combinations of image analysis targets for each analysis category and the importance level for each combination. Therefore, in this processing, by providing a setting screen that operator O uses when setting the importance level, it is possible to perform an appropriate reaction to the event that occurs.

[0129] (4-6. Effect 6) Sixth, in the process according to the embodiment described above, the server device 10 acquires image data captured and transmitted by a camera 30 installed at a facility managed by operator O. In this process, by analyzing the image data of the facility managed by operator O in real time, an appropriate reaction to the event that occurs can be performed.

[0130] (4-7. Effect 7) Seventh, in the processing according to the embodiment described above, the server device 10 performs at least one of the following as a reaction set by operator O, depending on the severity: notifying operator O of an alarm, recording image data, and recording the output image analysis target. In this processing, by flexibly changing the reaction according to the event that occurs, an appropriate reaction to the event that occurs can be performed.

[0131] (4-8. Effect 8) Eighth, in the processing according to the embodiment described above, the analysis category of the image analysis target includes at least one of the following: the attributes of the person, the actions of the person, and the objects involved in the actions of the person. In this processing, an appropriate reaction to an event can be performed by flexibly changing the reaction according to "who," "what," and "how" the event occurred.

[0132] [Modification example 1 of the embodiment] Using Figure 10, the configuration and processing of the event detection system 100M-1 according to the modified embodiment 1 will be described. Note that configurations and processes common to the embodiment will not be explained.

[0133] As shown in Figure 10, the event detection system 100M-1 includes an operator terminal 20M and a camera 30.

[0134] The event detection system 100M-1 performs the following processes: First, the operator terminal 20M displays candidate information (step S11). Second, operator O inputs setting information to the operator terminal 20M (step S12). Third, camera 30 transmits image data to the operator terminal 20M (step S13). Fourth, the operator terminal 20M analyzes the image data (step S14). Fifth, the operator terminal 20M performs a reaction (step S15).

[0135] As described above, in the event detection system 100M-1 according to the modified embodiment 1, the operator terminal 20M performs image analysis processing and reaction execution processing, thereby enabling the execution of appropriate reactions to the events that occur without using the server device 10.

[0136] [Modification of the embodiment 2] Using Figure 11, the configuration and processing of the event detection system 100M-2 according to the modified embodiment 2 will be described. Note that configurations and processes common to the embodiment will not be explained.

[0137] As shown in Figure 11, the event detection system 100M-2 includes an operator terminal 20 and a camera 30M.

[0138] The event detection system 100M-2 performs the following processes: First, camera 30M transmits candidate information to operator terminal 20 (step S21). Second, operator terminal 20 displays the candidate information (step S22). Third, operator O inputs setting information to operator terminal 20 (step S23). Fourth, operator terminal 20 transmits the setting information to camera 30M (step S24). Fifth, camera 30M analyzes the image data (step S25). Sixth, camera 30M performs a reaction to operator terminal 20 (step S26).

[0139] As described above, in the event detection system 100M-2 according to the modified embodiment 2, the camera 30M performs candidate information transmission processing, image analysis processing, and reaction execution processing, thereby enabling the system to perform appropriate reactions to events without using the server device 10.

[0140] 〔system〕 Unless otherwise specified, the processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed at will.

[0141] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown. That is, all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0142] Furthermore, each processing function performed by each device may be implemented, in whole or in part, by a CPU and a program executed for analysis by that CPU, or by hardware using wired logic.

[0143] [Hardware] Next, an example of the hardware configuration of the server device 10, which is an image analysis device, will be described. Note that other devices can also have a similar hardware configuration. Figure 12 is a diagram illustrating an example of the hardware configuration according to the embodiment. As shown in Figure 12, the server device 10 has a communication device 10a, an HDD (Hard Disk Drive) 10b, memory 10c, and a processor 10d. Furthermore, each part shown in Figure 12 is interconnected by a bus or the like.

[0144] The communication device 10a is a network interface card or the like, and communicates with other servers. The HDD 10b stores programs and databases that operate the functions shown in Figure 2.

[0145] The processor 10d operates a process that performs the functions described in Figure 2 by reading a program that performs the same processing as each processing unit shown in Figure 2 from the HDD 10b or the like and loading it into memory 10c. For example, this process performs the same functions as each processing unit of the server device 10. Specifically, the processor 10d reads a program that has the same functions as the reception unit 15a, acquisition unit 15b, execution unit 15c, etc. from the HDD 10b or the like. Then, the processor 10d executes a process that performs the same processing as the reception unit 15a, acquisition unit 15b, execution unit 15c, etc.

[0146] Thus, the server device 10 operates as a device that executes various processing methods by reading and executing a program. Furthermore, the server device 10 can also achieve the same functionality as the embodiment described above by reading the program from a recording medium using a media reader and executing the read program. It should be noted that the program referred to in this other embodiment is not limited to being executed by the server device 10. For example, the present invention can be similarly applied when another computer or server executes a program, or when they cooperate to execute a program.

[0147] This program can be distributed via networks such as the Internet. Furthermore, this program can be recorded on computer-readable storage media such as hard disks, flexible disks (FDs), CD-ROMs, MO (Magneto-Optical disks), and DVDs (Digital Versatile Discs), and executed by reading the program from these media using a computer.

[0148] 〔others〕 Some examples of the combinations of technical features that will be disclosed are listed below.

[0149] (1) An image analysis device comprising: a storage unit that stores multiple machine learning models that output objects included in image data; a receiving unit that receives the selection of a machine learning model from the multiple machine learning models and the importance associated with the combination of objects; an acquisition unit that acquires image data; and an execution unit that inputs the acquired image data into the machine learning model that received the selection and performs a predetermined reaction according to the importance associated with the combination of objects output.

[0150] (2) The image analysis apparatus according to (1), wherein the storage unit stores the plurality of machine learning models for each of the target categories to be output, the receiving unit receives the selection of a machine learning model from the plurality of machine learning models for each of the categories, and the importance associated with the combination of the target for each of the categories, and the execution unit inputs the acquired image data to the machine learning model selected for each of the categories, and performs the predetermined reaction according to the importance associated with the combination of the target for each of the categories to be output.

[0151] (3) The image analysis device according to (1) or (2), wherein the reception unit displays a list of the multiple machine learning models for each of the target categories and displays a settings screen on the user's terminal that accepts input for the selection of a machine learning model for each category.

[0152] (4) The image analysis device according to any one of (1) to (3), wherein the reception unit displays a list of the multiple machine learning models for each of the target categories, and displays a settings screen on the user's terminal that displays a list of the targets output by each of the multiple machine learning models.

[0153] (5) The image analysis device according to any one of (1) to (4), wherein the reception unit presents a list of targets output by the machine learning model that has been selected for each target category, and displays a settings screen on the user's terminal that accepts input of combinations of targets for each category and input of importance for each combination.

[0154] (6) The acquisition unit is an image analysis device according to any one of (1) to (5) that acquires image data captured and transmitted by a camera installed in a facility managed by the user.

[0155] (7) The image analysis device according to any one of (1) to (6), wherein the execution unit performs at least one of the following as a predetermined reaction set by the user, according to the importance level: notifying the user of an alarm, recording image data, and recording the outputted target.

[0156] (8) The image analysis device according to any one of (1) to (7), wherein the target category includes at least one of the attributes of a person, the actions of a person, and objects involved in the actions.

[0157] (9) An image analysis method that performs processing, wherein a computer selects a machine learning model from a plurality of machine learning models that output objects included in image data, and accepts importance levels associated with the combination of objects, acquires image data, inputs the acquired image data into the machine learning model that was selected, and performs a predetermined reaction according to the importance levels associated with the combination of objects output.

[0158] (10) An image analysis program that causes a computer to select a machine learning model from a plurality of machine learning models that output objects contained in image data, and to accept importance levels associated with the combination of objects, to acquire image data, to input the acquired image data into the machine learning model that was selected, and to perform a predetermined reaction according to the importance levels associated with the combination of objects output. [Explanation of Symbols]

[0159] 10 Server devices 11 Input section 12 Output section 13 Communications Department 14 Storage section 14a Candidate Information Storage Unit 14b Configuration Information Storage Unit 14c Image data storage unit 14d Analysis Model Memory Unit 15 Control Unit 15a Reception Desk 15b Acquisition part 15c Execution Unit 20 Operator terminals 21 Input / output section 22 Transmitter / Receiver 23 Communications Department 30 Cameras 31 Photography Department 32 Communications Department 100 Event Detection System

Claims

1. A memory unit that stores multiple machine learning models that output the objects contained in the image data, A receiving unit that receives the selection of a machine learning model from the aforementioned multiple machine learning models and the importance associated with the combination of the aforementioned targets, An acquisition unit that acquires image data, An execution unit inputs the acquired image data into the machine learning model that has received the selection, and performs a predetermined reaction according to the importance assigned to the combination of targets output. Equipped with, The aforementioned reception unit is The system accepts the importance of the combination of the aforementioned target, which corresponds to the attributes of the person, the time of occurrence of the event, the actions of the person, and the objects involved in the actions. The attributes of the person include the time of occurrence, the action, and whether or not they have authority over the combination of the object. Image analysis device.

2. The aforementioned storage unit is The multiple machine learning models are stored for each of the target categories that are output. The aforementioned reception unit is For each category, the selection of a machine learning model from the multiple machine learning models, and the importance level associated with the combination of targets for each category are received. The execution unit is, The acquired image data is input to the machine learning model that has received the selection for each category, and the predetermined reaction is executed according to the importance assigned to the combination of targets for each category output. The image analysis apparatus according to claim 1.

3. The aforementioned reception unit is The system displays a list of the multiple machine learning models for each of the aforementioned target categories, and also displays a settings screen on the user's terminal that accepts input for selecting a machine learning model for each category. The image analysis apparatus according to claim 1.

4. The aforementioned reception unit is The system displays a list of targets output by the machine learning model that has received the selection for each of the target categories, and also displays a settings screen on the user's terminal that accepts input for combinations of targets for each category and the importance level for each combination. The image analysis apparatus according to claim 1.

5. The acquisition unit is, The system acquires image data captured and transmitted by a camera installed in a facility managed by the user. The image analysis apparatus according to claim 1.

6. The execution unit is, As a predetermined reaction set by the user, at least one of the following will be performed, depending on the severity: notifying the user of an alarm, recording image data, and recording the outputted target. The image analysis apparatus according to claim 1.

7. The aforementioned category of subject includes at least one of the following: the attributes of a person, the actions of the person, and the objects involved in the actions. The image analysis apparatus according to any one of claims 1 to 6.

8. Computers The system accepts the selection of a machine learning model from multiple machine learning models that output objects contained in image data, and the importance assigned to the combination of said objects. Acquire image data, The acquired image data is input to the machine learning model that has received the selection, and a predetermined reaction is performed according to the importance assigned to the combination of targets output. Execute the process, The system accepts the importance of the combination of the aforementioned target, which corresponds to the attributes of the person, the time of occurrence of the event, the actions of the person, and the objects involved in the actions. The attributes of the person include the time of occurrence, the action, and whether or not they have authority over the combination of the object. Image analysis methods.

9. On the computer, The system accepts the selection of a machine learning model from multiple machine learning models that output objects contained in image data, and the importance assigned to the combination of said objects. Acquire image data, The acquired image data is input to the machine learning model that has received the selection, and a predetermined reaction is performed according to the importance assigned to the combination of targets output. Execute the process, The system accepts the importance of the combination of the aforementioned target, which corresponds to the attributes of the person, the time of occurrence of the event, the actions of the person, and the objects involved in the actions. The attributes of the person include the time of occurrence, the action, and whether or not they have authority over the combination of the object. Image analysis program.

Images