Image analysis apparatus, image analysis method, and image analysis program

Abstract

To execute an appropriate reaction to an occurred event by a user setting a combination condition of output results obtained from a plurality of AI logics.SOLUTION: A server device 10 stores therein a plurality of analysis models each of which outputs image analysis targets included in image data, accepts selection of the analysis model from among the plurality of analysis models and also accepts a degree of importance that is associated with a combination of the image analysis targets, acquires the image data, inputs the acquired image data to the analysis model for which the selection has been accepted, and executes a predetermined reaction in accordance with the degree of importance that is associated with the output combination of the image analysis targets.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image analysis device, an image analysis method, and an image analysis program. [Background technology]

[0002] There is a known technology that uses multiple AI (Artificial Intelligence) logics to perform image analysis in parallel on an image captured by a camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6989572 [Patent Document 2] JP 2010-211514 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, when implementing facility security or equipment management, it is difficult to execute appropriate reactions to events that occur. For example, the importance of the urgency of an event varies depending on the actor and target of the action, such as "who," "when," "what," and "how," but it is difficult to determine the urgency of an event and execute different reactions by combining the output results of multiple AI logics that detect each of these.

[0005] The present invention has been made in consideration of the above, and aims to enable users to set combination conditions of the output results of multiple AI logics and execute appropriate reactions to events that occur. [Means for solving the problem]

[0006] The present invention provides an image analysis device comprising: a memory unit that stores a plurality of machine learning models that output objects contained in image data; a reception unit that receives a selection of a machine learning model from the plurality of machine learning models and an importance associated with the combination of the objects; an acquisition unit that acquires image data; and an execution unit that inputs the acquired image data to the machine learning model for which the selection was accepted, and executes a predetermined reaction depending on the importance associated with the combination of the output objects.

[0007] The present invention also provides an image analysis method in which a computer executes a process of selecting a machine learning model from a plurality of machine learning models that outputs objects contained in image data and accepting an importance associated with the combination of the objects, acquiring image data, inputting the acquired image data into the machine learning model for which the selection was accepted, and executing a predetermined reaction depending on the importance associated with the combination of the output objects.

[0008] The present invention also provides an image analysis program that causes a computer to execute a process of selecting a machine learning model from a plurality of machine learning models that output objects contained in image data, accepting an importance associated with the combination of the objects, acquiring image data, inputting the acquired image data into the machine learning model for which the selection was accepted, and executing a predetermined reaction depending on the importance associated with the combination of the output objects. Effect of the Invention

[0009] According to the present invention, it has the effect of enabling a user to set combination conditions of the output results of multiple AI logics and execute appropriate reactions to events that occur. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating a configuration example and a processing example of an occurrence event detection system according to an embodiment. [Diagram 2] 1 is a block diagram showing an example of the configuration of each device of an occurrence event detection system according to an embodiment; [Diagram 3] FIG. 4 is a diagram illustrating an example of a candidate information storage unit of the server device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a setting information storage unit of the server device according to the embodiment. [Diagram 5] FIG. 4 is a diagram illustrating an example of an image data storage unit of the server device according to the embodiment. [Figure 6] FIG. 4 is a diagram illustrating an example of an analysis model storage unit of the server device according to the embodiment. [Figure 7] FIG. 4 is a diagram showing a specific example 1 of a display screen of an operator terminal according to the embodiment. [Figure 8] FIG. 11 is a diagram showing a specific example 2 of a display screen of an operator terminal according to the embodiment. [Figure 9] 2 is a flowchart showing an example of the overall flow of an occurrence event detection system according to the embodiment. [Figure 10] 1A and 1B are diagrams illustrating an example of the configuration and processing of an occurrence event detection system according to a first modified example of an embodiment. [Figure 11] 11 is a diagram illustrating a configuration example and a processing example of an occurrence event detection system according to a second modified example of an embodiment. FIG. [Figure 12] FIG. 2 is a diagram illustrating an example of a hardware configuration according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An image analysis device, an image analysis method, and an image analysis program according to an embodiment of the present invention will be described in detail below with reference to the drawings. Note that the present invention is not limited to the embodiment described below.

[0012] [Embodiment] The configuration and processing of the occurring event detection system 100 according to the embodiment, the configuration and processing of each device of the occurring event detection system 100, and the processing flow of the occurring event detection system 100 will be described below in order, and finally the effects of the embodiment will be described.

[0013] 1. Configuration and Processing of Occurrence Event Detection System 100 The configuration and processing of the occurring event detection system 100 according to the embodiment will be described in detail with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration and processing of the occurring event detection system 100 according to the embodiment. Below, an example of the configuration of the entire occurring event detection system 100, an example of the processing of the occurring event detection system 100, and the effect of the occurring event detection system 100 will be described. Note that in the embodiment, the identification of a person and the detection of behavior in facility security and equipment management will be described as an example, but the field of use is not limited, and the invention can be applied to the monitoring of parks, roads, rivers, etc., and the target of behavior detection can be not only a person but also construction machinery, robots, and animals.

[0014] (1-1. Example of the overall configuration of the occurrence event detection system 100) The occurrence event detection system 100 includes a server device 10, an operator terminal 20, and a camera 30. Here, the server device 10, the operator terminal 20, and the camera 30 are connected to each other so as to be able to communicate with each other by wire or wirelessly via a predetermined communication network (not shown). Note that the predetermined communication network may be any of various communication networks such as the Internet or a dedicated line.

[0015] (1-1-1. Server device 10) The server device 10 is an image analysis device that analyzes image data acquired from the camera 30. For example, the server device 10 is realized in a cloud environment, an on-premise environment, an edge environment, or the like. Note that the occurrence event detection system 100 shown in FIG. 1 may include multiple server devices 10.

[0016] (1-1-2. Operator terminal 20) The operator terminal 20 is a manager terminal used by an operator O who is a manager of a facility or equipment. Note that the incident detection system 100 shown in FIG.

[0017] (1-1-3. Camera 30) The camera 30 is an imaging device installed in a facility or equipment, and is, for example, a security camera, a surveillance camera, etc. Incident detection system 100 shown in FIG.

[0018] (1-2. Example of the overall processing of the occurrence event detection system 100) The following describes the overall processing of the occurrence event detection system 100. Note that the processing of steps S1 to S7 below may be executed in a different order. Also, some of the processing of steps S1 to S7 below may be omitted.

[0019] (1-2-1. Candidate information transmission process) First, the server device 10 transmits candidate information to the operator terminal 20 (step S1). For example, the server device 10 transmits candidate information including identification information of the AI ​​logic for each analysis category (simply "category" as appropriate) that the server device 10 can analyze, an output value list, and the like, to the operator terminal 20.

[0020] Here, the analysis category is a classification of the subject of image analysis (or simply "subject" as appropriate), such as "person behavior" indicating the behavior of the person included in the image, "person identification" indicating the attributes of the person included in the image (e.g. name, affiliation, authority, gender, age, etc.), and "object identification" indicating the object included in the image that is involved in the person's behavior.

[0021] "Human behavior" is an analysis category that corresponds to "what happened" as the subject of image analysis, which is the behavior of a person, such as "entering a room," "exiting," "taking out," "leaving," "becoming violent," "falling down," "vomiting," "crying," etc.

[0022] The above "human actions" are examples of actions detected when the subject of behavioral analysis is a person, but other examples include actions performed by a robot or construction machine, such as "grabbing," "lifting," "installing," and "digging." Also, when the subject of behavioral analysis is an animal, the animal's actions may be detected.

[0023] "Person Identification" is an analysis category that corresponds to "who" which is the attribute of a person such as "pre-registered person with authority A," "pre-registered person with authority B," or "non-registered person" as the subject of image analysis.

[0024] In addition, the above "person identification" is an example in which the target of behavior analysis is a person, but as other examples, when a place where a person and a robot are working together is to be photographed, the robot may be included, and a work vehicle such as a construction machine may also be the target. Animals may also be the target.

[0025] "Object identification" is an analysis category that corresponds to "what" as the subject of image analysis, which refers to objects related to a person's behavior, such as a "desk," "chair," "cardboard box," "locker," "file," "safe," "seal," "potted plant," "shredder," etc.

[0026] Moreover, the AI ​​logic is an analysis method realized by an analysis model AM, which is a trained machine learning model that outputs an image analysis target. Moreover, the output value list is a list of analysis items that are image analysis targets output by each AI logic. Here, the output value list may be a list of common analysis items.

[0027] (1-2-2. Candidate information display process) Secondly, the operator terminal 20 displays the candidate information (step S2). For example, the operator terminal 20 displays on the monitor a setting screen presenting a list of identification information of AI logic for each analysis category, a list of output values ​​for each AI logic, etc., as the candidate information.

[0028] (1-2-3. Setting information input process) Thirdly, the operator O inputs setting information to the operator terminal 20 (step S3). For example, the operator O inputs the selection of the AI ​​logic to be used for the analysis as setting information by selecting the identification information of an arbitrary AI logic from a list of the identification information of the AI ​​logic for each analysis category.

[0029] At this time, operator O selects, for example, "AI Logic 1" as {Analysis Category Example 1: Human Behavior}, "AI Logic A" as {Analysis Category Example 2: Person Identification}, and "AI Logic W" as {Analysis Category Example 3: Object Identification}, as an AI logic selection.

[0030] In addition, the operator O selects the analysis items output by the selected AI logic for each analysis category, and assigns importance to the combination of the selected analysis items, thereby inputting the importance corresponding to the combination of the analysis items as setting information. At this time, the operator O may input the reaction executed by the server device 10 corresponding to the importance as setting information.

[0031] Here, importance indicates the degree of urgency of responding to an occurring event, priority, seriousness, etc., and is expressed, for example, on an 11-point scale from 0 to 10. Note that importance may also be expressed as a score from 0 to 100%, or a rank of A, B, C, etc.

[0032] At this time, the operator O assigns an importance level of "importance level 8" indicating urgency to the combination of analysis items {"who": "unregistered user", "when": "weekday business hours", "what": "file", "how": "take out"}. Similarly, the operator O assigns an importance level to each of the other combinations of analysis items as events to be detected.

[0033] (1-2-4. Setting information transmission process) Fourth, the operator terminal 20 transmits setting information to the server device 10 (step S4). For example, the operator terminal 20 transmits to the server device 10, as setting information, the selection of AI logic for each analysis category, the importance corresponding to the combination of analysis items, the reaction corresponding to the importance, and the like, which are input to the operator terminal 20 by the operator O. At this time, the server device 10 stores the received setting information.

[0034] (1-2-5. Image data transmission process) Fifth, camera 30 transmits image data to server device 10 (step S5). For example, camera 30 transmits image data of still images taken every second in the facility where camera 30 is installed to server device 10. At this time, server device 10 stores the received image data together with the shooting time. Note that the image data may be image data of a moving image, or may include audio data.

[0035] (1-2-6. Image data analysis processing) Sixth, the server device 10 analyzes the image data (step S6). For example, the server device 10 refers to the setting information and analyzes the image data using the AI ​​logic selected by the operator O for each analysis category. At this time, the server device 10 inputs the image data to the analysis model AM corresponding to each AI logic, acquires the analysis items that are the targets of the output image analysis for each analysis category, and identifies the importance corresponding to the combination of the acquired analysis items.

[0036] That is, in the above-mentioned example, the server device 10 inputs image data to the analysis model AM-A corresponding to "AI logic A" as {analysis category example 2: person identification}, and acquires an analysis item corresponding to "who". The server device 10 also identifies the shooting time of the image data, and acquires an analysis item corresponding to "when". The server device 10 also inputs image data to the analysis model AM-W corresponding to "AI logic W" as {analysis category 3: object identification}, and acquires an analysis item corresponding to "what". The server device 10 also inputs image data to the analysis model AM-1 corresponding to "AI logic 1" as {analysis category example 1: person action}, and acquires an analysis item corresponding to "how". The server device 10 then identifies the importance level indicating the urgency given to the combination of the analysis items of "who", "when", "what", and "how".

[0037] (1-2-7. Reaction execution process) Seventh, the server device 10 executes a reaction to the operator terminal 20 (step S7). For example, when the identified importance is equal to or greater than a predetermined value, the server device 10 notifies the operator terminal 20 of an alarm as a reaction. At this time, when the identified importance is less than the predetermined value, the server device 10 records the generated event and image data as a log without notifying the operator terminal 20 of an alarm as a reaction.

[0038] (1-3. Effects of the Occurrence Event Detection System 100) In the following, problems with the occurring event detection system 100P according to the reference technology will be described, and then the effects of the occurring event detection system 100 will be described.

[0039] (1-3-1. Problems with the 100P Event Detection System) The occurrence event detection system 100P-1 according to the reference technology 1 is a technology that tracks a fleeing vehicle by combining images from multiple cameras, and performs analysis simultaneously on multiple analysis servers each having a different analysis logic. The occurrence event detection system 100P-2 according to the reference technology 2 is a technology that identifies suspicious individuals by combining face recognition and behavioral analysis. However, the occurrence event detection system 100P according to the reference technology does not take into consideration the fact that the importance of each combination of the output results of the AI ​​logic, such as "who," "when," "what," and "how," differs. For example, even if a highly confidential file is taken out, the importance of "who" differs between a person who has the authority to take out the file and a person who does not. Also, even if elementary school students are playing in a park, the importance of "when" differs between during school hours on weekdays and on holidays.

[0040] (1-3-2. Overview of Occurrence Event Detection System 100) In the occurrence event detection system 100, the following process is executed. First, the server device 10 transmits candidate information including identification information of the AI ​​logic for each analysis category and an output value list to the operator terminal 20. Second, the operator terminal 20 displays a setting screen on the monitor that presents a list of identification information of the AI ​​logic for each analysis category, an output value list for each AI logic, and the like as candidate information. Third, the operator O inputs, to the operator terminal 20, the selection of the AI ​​logic to be used for the analysis and the importance corresponding to the combination of analysis items output by the selected AI logic as setting information. Fourth, the operator terminal 20 transmits, to the server device 10, the selection of the AI ​​logic for each analysis category, the importance corresponding to the combination of analysis items, and the like as setting information. Fifth, the camera 30 transmits image data of still images taken every second in the facility where the camera 30 is installed to the server device 10. Sixth, the server device 10 analyzes the image data and identifies the importance corresponding to the combination of analysis items acquired for each analysis category. Seventh, when the identified importance is equal to or greater than a predetermined value, the server device 10 notifies the operator terminal 20 of an alarm as a reaction.

[0041] (1-3-3. Effects of the Occurrence Event Detection System 100) The occurrence event detection system 100 has the following effects. First, when implementing facility security or equipment management, the occurrence event detection system 100 can automatically identify the importance of the situation occurring at the site and notify the security manager, making it possible for a small number of people to monitor a large number of locations. Second, since the occurrence event detection system 100 is implemented as software on common hardware, it is possible to efficiently manufacture the devices that are the individual elements of the system and install them on site, and then set them according to the installation environment for each individual location. Third, the occurrence event detection system 100 makes it possible to easily change the settings even if the installation environment changes.

[0042] As described above, in the occurrence event detection system 100, the user specifies an appropriate reaction to an occurrence event based on the combination conditions of the output results of separately produced AI logic, and the system then executes analysis in accordance with the specification and outputs the results, allowing the user to execute an appropriate reaction.

[0043] 2. Configuration and Processing of Each Device of Occurrence Event Detection System 100 The configuration and processing of each device included in the occurring event detection system 100 shown in Fig. 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of each device of the occurring event detection system 100 according to the embodiment. Below, an example of the configuration of the entire occurring event detection system 100 according to the embodiment will be described, and then an example of the configuration and processing of the server device 10, the operator terminal 20, and the camera 30 will be described in detail.

[0044] (2-1. Example of the overall configuration of the occurrence event detection system 100) An example of the overall configuration of the occurrence event detection system 100 shown in Fig. 1 will be described with reference to Fig. 2. As shown in Fig. 2, the occurrence event detection system 100 has a server device 10, an operator terminal 20, and a camera 30. The server device 10, the operator terminal 20, and the camera 30 are communicatively connected to each other via a communication network N realized by the Internet, a dedicated line, or the like.

[0045] The server device 10 is installed in a cloud environment, an on-premise environment, an edge environment, etc. The operator terminal 20 is installed in a monitoring room or the like of a facility or equipment managed by an operator O. The camera 30 is installed at the monitoring site of the facility or equipment.

[0046] (2-2. Configuration and Processing Examples of Server Device 10) An example of the configuration and processing of the server device 10 will be described with reference to Fig. 2. The server device 10 is an image analysis device, and includes an input unit 11, an output unit 12, a communication unit 13, a storage unit 14, and a control unit 15.

[0047] (2-2-1. Input section 11) The input unit 11 is responsible for inputting various types of information to the server device 10. For example, the input unit 11 is realized by a mouse, a keyboard, etc., and accepts input of various types of information to the server device 10.

[0048] (2-2-2. Output section 12) The output unit 12 controls the output of various information from the server device 10. For example, the output unit 12 is realized by a display or the like, and displays various information stored in the server device 10.

[0049] (2-2-3. Communications Department 13) The communication unit 13 manages data communication with other devices. For example, the communication unit 13 performs data communication with each communication device via a router, etc. The communication unit 13 can also perform data communication with an operator's terminal (not shown).

[0050] (2-2-4. Storage section 14) The storage unit 14 stores various information referred to when the control unit 15 operates and various information acquired when the control unit 15 operates. The storage unit 14 has a candidate information storage unit 14a, a setting information storage unit 14b, an image data storage unit 14c, and an analysis model storage unit 14d. Here, the storage unit 14 can be realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. In the example of FIG. 2, the storage unit 14 is installed inside the server device 10, but may be installed outside the server device 10, or multiple storage units may be installed.

[0051] (2-2-4-1. Candidate information storage unit 14a) The candidate information storage unit 14a stores candidate information. For example, the candidate information storage unit 14a stores candidate information including identification information of AI logic for each analysis category that can be analyzed by the execution unit 15c of the control unit 15 described later, an output value list, and the like. In addition, the analysis category of the image analysis target includes, for example, at least one of a person's attribute, a person's behavior, and an object involved in the person's behavior. Here, an example of data stored in the candidate information storage unit 14a will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of the candidate information storage unit 14a of the server device 10 according to the embodiment. In the example of FIG. 3, the candidate information storage unit 14a has items such as "analysis category", "AI logic", and "output value list".

[0052] "Analysis category" indicates identification information for identifying the category of the image analysis target, such as a person's behavior, a person's attributes, or an object involved in a person's behavior, and is, for example, an identification number or an identification symbol of the category of the image analysis target. "AI logic" indicates identification information for identifying the AI ​​logic that the server device 10 can analyze, such as an identification number or an identification symbol of the AI ​​logic. "Output value list" is a list of analysis items that are the image analysis targets output by each AI logic, such as a list of a person's behavior, a list of a person's attributes, a list of objects involved in a person's behavior, etc.

[0053] That is, in Figure 3, for the analysis category identified by "Analysis Category #1", the following are true: {AI Logic: "AI Logic 1", Output Value List: "Output Value List 1", ···}, {AI Logic: "AI Logic 2", Output Value List: "Output Value List 2", ···}, {AI Logic: "AI Logic 3", Output Value List: "Output Value List 3", ···}, {AI Logic: "AI Logic 4", Output Value List: "Output Value List 4", ···}, and for the analysis category identified by "Analysis Category #2", the following are true: {AI Logic: "AI Logic A", Output Value List: "Output Value List A", ···}, {AI Logic: "AI Logic B", Output Value List: "Output Value List B", ···} ,output value list: "Output value list B", ···}, {AI logic: "AI Logic C", output value list: "Output value list C", ···}, and for the analysis category identified by "Analysis Category #3", data such as {AI logic: "AI Logic Z", output value list: "Output value list Z", ···}, {AI logic: "AI Logic Y", output value list: "Output value list Y", ···}, {AI logic: "AI Logic X", output value list: "Output value list X", ···}, {AI logic: "AI Logic W", output value list: "Output value list W", ···} is stored in the candidate information storage unit 14a.

[0054] The candidate information storage unit 14a may include features of AI logics as the candidate information. For example, the candidate information storage unit 14a may store data such as {AI logic: "AI logic 1", feature: "face recognition", ...}, {AI logic: "AI logic 2", feature: "personal identification by walking style", ...}, {AI logic: "AI logic 3", feature: "gender and age estimation", ...}, {AI logic: "AI logic 4", feature: "clothing and equipment detection", ...}, etc. as features of AI logics.

[0055] (2-2-4-2. Setting information storage unit 14b) The setting information storage unit 14b stores setting information. For example, the setting information storage unit 14b stores setting information including a selection of an AI logic for each analysis category, an importance corresponding to a combination of analysis items, and the like, which is input by the operator O via a setting screen and accepted by the accepting unit 15a of the control unit 15 described later. Here, an example of data stored in the setting information storage unit 14b will be described with reference to FIG. 4. FIG. 4 is a diagram showing an example of the setting information storage unit 14b of the server device 10 according to the embodiment. In the example of FIG. 4, the setting information storage unit 14b has items such as "user", "location", "analysis category", "AI logic", "output value list", and "importance information".

[0056] The "user" indicates identification information for identifying a manager who manages a facility or equipment, and is, for example, an identification number or identification symbol of an operator O or a management company. The "location" indicates identification information for identifying a facility or equipment managed by the "user", and is, for example, an identification number or identification symbol of the facility or equipment. The "analysis category" indicates identification information for identifying a classification of an image analysis target, and is, for example, an identification number or identification symbol of a classification of an image analysis target such as a person's behavior, a person's attributes, or an object involved in a person's behavior. The "AI logic" indicates identification information for identifying an AI logic accepted by the acceptance unit 15a, and is, for example, an identification number or identification symbol of the AI ​​logic. The "output value list" is a list of analysis items that are image analysis targets output by each AI logic, and is, for example, a list of a person's behavior, a list of a person's attributes, a list of objects involved in a person's behavior, and the like. The "importance information" is a list of importance levels assigned to combinations of analysis items that are image analysis targets output by each AI logic, and is, for example, a list of combinations of analysis items such as "who", "when", "what", and "how", and the importance levels corresponding to the combinations.

[0057] That is, Figure 4 shows an example in which data such as {Analysis category: "Analysis category #1", AI logic: "AI Logic 1", output value list: "Output value list 1"}, {Analysis category: "Analysis category #2", AI logic: "AI Logic A", output value list: "Output value list A"}, {Analysis category: "Analysis category #3", AI logic: "AI Logic W", output value list: "Output value list W"} for a user identified by "User #1" and a location identified by "Location #1" are stored in the setting information storage unit 14b, with importance information being "Importance information #1".

[0058] The setting information storage unit 14b may include, as the setting information, a reaction corresponding to the importance level, etc. For example, the setting information storage unit 14b may store, as a reaction corresponding to the importance level, data such as {importance level: "0", reaction: "event recording"}, {importance level: "1 to 6", reaction: "image data recording"}, {importance level: "7 to 10", reaction: "image data recording", "alarm notification"}, etc.

[0059] (2-2-4-3. Image data storage unit 14c) The image data storage unit 14c stores image data. For example, the image data storage unit 14c stores image data acquired by an acquisition unit 15b of the control unit 15 described later. Here, an example of data stored in the image data storage unit 14c will be described with reference to FIG. 5. FIG. 5 is a diagram showing an example of the image data storage unit 14c of the server device 10 according to the embodiment. In the example of FIG. 5, the image data storage unit 14c has items such as "camera," "location," "time," and "image data."

[0060] "Camera" indicates identification information for identifying the photographing device, such as the identification number or identification symbol of camera 30. "Location" indicates identification information for identifying the facility or equipment in which the photographing device is installed, such as the identification number or identification symbol of the facility or equipment. "Time" indicates the time of photographing, such as expressed in terms of year, month, day, hour, minute, and second. "Image data" indicates image data acquired at the time of photographing, such as image data of still images acquired every second, image data of moving images, image data of moving images including audio data, etc.

[0061] That is, Figure 5 shows an example in which data such as {time: "time #1", image data: "image data #1"}, {time: "time #2", image data: "image data #2"}, {time: "time #3", image data: "image data #3"}, etc. are stored in image data storage unit 14c for a camera identified by "camera #1" and a location identified by "location #1."

[0062] (2-2-4-4. Analysis model storage unit 14d) The analytical model storage unit 14d stores the analytical model AM. For example, the analytical model storage unit 14d stores a plurality of analytical models AM, which are machine learning models used by the execution unit 15c of the control unit 15 described later, and which output an image analysis target included in image data. At this time, the analytical model storage unit 14d stores a plurality of analytical models AM for each category of the image analysis target to be output. Here, an example of data stored in the analytical model storage unit 14d will be described with reference to FIG. 6. FIG. 6 is a diagram showing an example of the analytical model storage unit 14d of the server device 10 according to the embodiment. In the example of FIG. 6, the analytical model storage unit 14d has items such as "analysis category", "AI logic", and "analysis model".

[0063] "Analysis category" indicates identification information for identifying a category of the image analysis target, such as a person's behavior, a person's attributes, an object involved in a person's behavior, or the like, and is an identification number or an identification symbol of the category of the image analysis target. "AI logic" indicates identification information for identifying an AI logic that can be analyzed by the server device 10, such as an identification number or an identification symbol of the AI ​​logic. "Analysis model" is model data of a machine learning model corresponding to "AI logic", such as data including execution data for executing the algorithm of the analysis model AM of each AI logic, model parameters which are setting values, hyperparameters, etc.

[0064] That is, in Figure 6, for the analysis category identified by "Analysis Category #1", the following are true: {AI logic: "AI Logic 1", analysis model: "Analysis Model 1", ···}, {AI logic: "AI Logic 2", analysis model: "Analysis Model 2", ···}, {AI logic: "AI Logic 3", analysis model: "Analysis Model 3", ···}, {AI logic: "AI Logic 4", analysis model: "Analysis Model 4", ···}, and for the analysis category identified by "Analysis Category #2", the following are true: {AI logic: "AI Logic A", analysis model: "Analysis Model A", ···}, {AI logic: "AI Logic B", analysis model: "Analysis Model B", ···} , analysis model: "Analysis Model B", ···}, {AI logic: "AI Logic C", analysis model: "Analysis Model C", ···}, and for the analysis category identified by "Analysis Category #3", data such as {AI logic: "AI Logic Z", analysis model: "Analysis Model Z", ···}, {AI logic: "AI Logic Y", analysis model: "Analysis Model Y", ···}, {AI logic: "AI Logic X", analysis model: "Analysis Model X", ···}, {AI logic: "AI Logic W", analysis model: "Analysis Model W", ···} is stored in the analysis model memory unit 14d.

[0065] (2-2-5. Control unit 15) The control unit 15 controls the entire server device 10. The control unit 15 includes a reception unit 15a, an acquisition unit 15b, and an execution unit 15c. Here, the control unit 15 can be realized by, for example, an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0066] (2-2-5-1. Reception section 15a) The reception unit 15a receives various types of information. The reception unit 15a may store the received various types of information in the storage unit 14. The setting screen display process, the AI ​​logic selection reception process, and the importance information reception process will be described below.

[0067] (Settings screen display process) The reception unit 15a executes a setting screen display process. For example, the reception unit 15a presents a list of analysis models AM, which are multiple machine learning models, for each analysis category of the image analysis target, and displays an AI logic selection screen as a setting screen for receiving input of the selection of the analysis model AM for each analysis category on the user terminal of the user.

[0068] To explain a specific example of the AI ​​logic selection screen, the reception unit 15a transmits candidate information to the operator terminal 20 used by the operator O, and causes the input / output unit 21 of the operator terminal 20 to display {"AI Logic 1", "AI Logic 2", "AI Logic 3", "AI Logic 4"} for "Analysis Category #1", {"AI Logic A", "AI Logic B", "AI Logic C"} for "Analysis Category #2", and {"AI Logic Z", "AI Logic Y", "AI Logic X", "AI Logic W"} for "Analysis Category #3".

[0069] Furthermore, the reception unit 15a causes the user terminal of the user to display an output value list viewing screen as a setting screen presenting a list of image analysis targets output by each of the multiple analysis models AM.

[0070] To explain a specific example of the output value list viewing screen, the reception unit 15a transmits candidate information to the operator terminal 20 used by the operator O, and displays {"Analysis item 1-1", "Analysis item 1-2", "Analysis item 1-3", "Analysis item 1-4", "Analysis item 1-5", "Analysis item 1-6", "Analysis item 1-7", "Analysis item 1-8"} on the input / output unit 21 of the operator terminal 20 for the "Output value list 1" that the operator O wishes to view.

[0071] In addition, the reception unit 15a presents a list of image analysis targets output by the analysis model AM whose selection has been accepted for each analysis category of the image analysis target, and displays an importance setting screen on the user's user terminal as a setting screen that accepts input of combinations of image analysis targets for each analysis category and input of the importance of each combination.

[0072] To explain a specific example of the importance setting screen, the reception unit 15a transmits candidate information to the operator terminal 20 used by the operator O, and sets {"analysis item 1-1", "analysis item 1-2", "analysis item 1-3", "analysis item 1-4", "analysis item 1-5", "analysis item 1-6", "analysis item 1-7", "analysis item 1-8"} for the "AI logic 1" selected by the operator O, and {"analysis item A-1", "analysis item B-2", "analysis item C-3", "analysis item D-4", "analysis item D-5", "analysis item D-6", "analysis item D-7", "analysis item D-8"} for the "AI logic A" selected by the operator O, and sets {"analysis item A-1", "analysis item B-2", "analysis item B-3", "analysis item B-4", "analysis item B-5", "analysis item B-6", "analysis item B-7", "analysis item B-8"} for the "AI logic A" selected by the operator O. The reception unit 15a causes the input / output unit 21 of the operator terminal 20 to display {"analysis item W-1", "analysis item W-2", "analysis item W-3", "analysis item W-4", "analysis item W-5", "analysis item W-6", "analysis item W-7", "analysis item W-8", "analysis item W-9"} for the "AI logic W" selected by the operator O, and {"analysis item T-1", "analysis item T-2", "analysis item T-3"} for the occurrence time of the event. Then, the reception unit 15a causes the input / output unit 21 of the operator terminal 20 to display an input screen for "importance" associated with the combination of analysis items {"analysis item 1-3", "analysis item A-3", "analysis item W-5", "analysis item T-1"} selected by the operator O.

[0073] (AI logic selection reception process) The reception unit 15a executes an AI logic selection reception process. For example, the reception unit 15a receives a selection of an AI logic, thereby receiving a selection of an analysis model AM from a plurality of analysis models AM to be used for the AI ​​logic. At this time, the reception unit 15a receives a selection of an AI logic for each analysis category, thereby receiving a selection of an analysis model AM from a plurality of analysis models AM to be used for the AI ​​logic for each analysis category.

[0074] To explain a specific example of the AI ​​logic selection reception process, the reception unit 15a receives each AI logic input by the operator O by operating the AI ​​logic selection screen displayed on the monitor of the operator terminal 20, such as {analysis category: "Analysis Category #1", AI logic: "AI Logic 1"}, {analysis category: "Analysis Category #2", AI logic: "AI Logic A"}, and {analysis category: "Analysis Category #3", AI logic: "AI Logic W"}, and stores them in the setting information memory unit 14b.

[0075] (Importance information reception process) The reception unit 15a executes an importance information reception process. For example, the reception unit 15a receives the importance associated with the combination of image analysis targets. At this time, the reception unit 15a receives the importance associated with the combination of image analysis targets for each analysis category. The reception unit 15a may also receive a reaction according to the importance.

[0076] To explain a specific example of the importance information reception process, the reception unit 15a receives, as importance information input by the operator O by operating the importance setting screen displayed on the monitor of the operator terminal 20, "importance information #1" in which an importance level of "8" is assigned to a combination of analysis item "analysis item 1-3" output by "AI logic 1" selected in "analysis category #1," "analysis item "A-3" output by "AI logic A" selected in "analysis category #2," "analysis item "analysis item W-5" output by "AI logic W" selected in "analysis category #3," and analysis item "analysis item T-1" indicating the time of occurrence of an event, and stores the information in the setting information memory unit 14b. At this time, the receiving unit 15a may receive, in addition to the combination and importance of analysis items, reactions to be executed by the execution unit 15c according to the importance input by the operator O, such as {importance: "0", reaction: "event recording"}, {importance: "1 to 6", reaction: "image data recording"}, {importance: "7 to 10", reaction: "image data recording", "alarm notification"}, etc., as "importance information #1" and store it in the setting information storage unit 14b.

[0077] (2-2-5-2. Acquisition part 15b) The acquiring unit 15b acquires various kinds of information. The acquiring unit 15b may store the acquired various kinds of information in the storage unit 14. For example, the acquiring unit 15b acquires image data. At this time, the acquiring unit 15b acquires image data captured and transmitted by a camera 30, which is an imaging device installed in a facility managed by an operator O, who is a user.

[0078] To explain a specific example, the acquisition unit 15b acquires image data captured and transmitted by the camera 30, such as {time: "Time #1", image data: "Image data #1"}, {time: "Time #2", image data: "Image data #2"}, {time: "Time #3", image data: "Image data #3"}, etc., and stores the image data in the image data storage unit 14c.

[0079] (2-2-5-3. Executive Unit 15c) The execution unit 15c executes various processes. The execution unit 15c may acquire various information from the storage unit 14. The importance level determination process and the reaction execution process will be described below.

[0080] (Importance determination process) The execution unit 15c executes the importance identification process. For example, the execution unit 15c inputs the image data acquired by the acquisition unit 15b into the analysis model AM whose selection has been accepted, and identifies the importance associated with the combination of the image analysis targets output. The execution unit 15c also inputs the image data acquired by the acquisition unit 15b into the analysis model AM whose selection has been accepted for each analysis category, and identifies the importance associated with the combination of the image analysis targets for each analysis category output.

[0081] A specific example of the importance specification process will be described. The execution unit 15c inputs image data {time: "time #1", image data: "image data #1"} into the "analysis model 1" used in the "AI logic 1" and obtains the analysis item "analysis item 1-3" which is the output result. The execution unit 15c also inputs image data {time: "time #1", image data: "image data #1"} into the "analysis model A" used in the "AI logic A" and obtains the analysis item "analysis item A-3" which is the output result. The execution unit 15c also inputs image data {time: "time #1", image data: "image data #1"} into the "analysis model W" used in the "AI logic W" and obtains the analysis item "analysis item W-5" which is the output result. The execution unit 15c also obtains the analysis item "analysis item T-1" which indicates the occurrence time of an event specified by the image data {time: "time #1", image data: "image data #1"}. Then, the execution unit 15c refers to the setting information memory unit 14b, obtains {AI logic 1: "analysis item 1-3", AI logic A: "analysis item A-3", AI logic W: "analysis item W-5", occurrence time: "analysis item T-1", importance: "8"}, and identifies the importance of the event that occurred as "8".

[0082] (Reaction execution process) The execution unit 15c executes a reaction execution process. For example, the execution unit 15c executes a predetermined reaction according to the importance associated with the output combination of image analysis targets. The execution unit 15c also executes a predetermined reaction according to the importance associated with the output combination of image analysis targets for each analysis category. At this time, the execution unit 15c executes at least one of notifying the operator O of an alarm, recording image data, and recording the output image analysis targets according to the importance as a predetermined reaction set by the operator O who is a user.

[0083] To explain a specific example of the reaction execution process, when the execution unit 15c identifies the importance of the event that has occurred as "8", it refers to the setting information storage unit 14b and obtains {importance: "0", reaction: "event recording"}, {importance: "1 to 6", reaction: "image data recording"}, {importance: "7 to 10", reaction: "image data recording", "alarm notification"}, and records image data at the time the event occurred as a reaction, and notifies the operator terminal 20 of an alarm to notify the occurrence of the event.

[0084] (2-3. Configuration and Processing Examples of Operator Terminal 20) 2 again, a description will be given of a configuration example and a processing example of the operator terminal 20. The operator terminal 20 is a posting device and a viewing device, and includes an input / output unit 21, a transmitting / receiving unit 22, and a communication unit .

[0085] (2-3-1. Input / output section 21) The input / output unit 21 controls input of various information to the operator terminal 20. For example, the input / output unit 21 is realized by a mouse, a keyboard, a touch panel, or the like, and accepts input of setting information and the like to the operator terminal 20. The input / output unit 21 also controls display of various information from the operator terminal 20. For example, the input / output unit 21 is realized by a display or the like, and displays setting information and the like stored in the operator terminal 20.

[0086] Furthermore, the input / output unit 21 displays the candidate information transmitted from the server device 10, which is an image analyzing device. For example, the input / output unit 21 displays an AI logic selection screen, an output value list viewing screen, an importance setting screen, and the like, as setting screens based on the candidate information. Details of the AI ​​logic selection screen will be described later in (2-3-4. Specific example 1 of display screen of operator terminal 20). Details of the importance setting screen will be described later in (2-3-5. Specific example 2 of display screen of operator terminal 20).

[0087] (2-3-2. Transmitter / receiver 22) The transmitting / receiving unit 22 transmits various types of information. For example, the transmitting / receiving unit 22 transmits to the server device 10 setting information input by the operator O via a setting screen.

[0088] The transmitting / receiving unit 22 receives various types of information. For example, the transmitting / receiving unit 22 receives candidate information transmitted from the server device 10. The transmitting / receiving unit 22 also receives an alarm transmitted from the server device 10.

[0089] (2-3-3. Communications Department 23) The communication unit 23 manages data communication with other devices. For example, the communication unit 23 performs data communication with each communication device via a router, etc. The communication unit 23 can also perform data communication with an operator's terminal (not shown).

[0090] (2-3-4. Specific Example 1 of Display Screen of Operator Terminal 20) Here, a specific example 1 of a display screen output by the input / output unit 21 of the operator terminal 20 will be described with reference to Fig. 7. Fig. 7 is a diagram showing a specific example 1 of a display screen of the operator terminal 20 according to the embodiment. The "AI logic selection screen", "person action", "person identification", "object identification", "AI logic feature viewing screen", and "output value list viewing screen" will be described below.

[0091] (2-3-4-1. AI logic selection screen) As shown in the example of FIG. 7, the operator terminal 20 displays an "AI logic selection screen" which is a setting screen that accepts input of a selection of an AI logic (i.e., an analysis model AM) for each analysis category. Here, the operator O can input a selection of an AI logic for each analysis category by clicking an AI logic displayed in each of the items "Analysis category example 1: human action", "Analysis category example 2: human identification", and "Analysis category example 3: object identification". Note that in the example of FIG. 7, the operator terminal 20 displays selectable AI logics for each of the three analysis categories, "Analysis category example 1: human action", "Analysis category example 2: human identification", and "Analysis category example 3: object identification", but the analysis categories displayed by the operator terminal 20 are not limited to the above example.

[0092] (2-3-4-2.Human movements) As shown in the example of FIG. 7, the operator terminal 20 displays "person behavior" indicating the behavior of a person as one of the analysis categories. In the example of FIG. 7, "AI logic 1", "AI logic 2", "AI logic 3", and "AI logic 4" are displayed as AI logics that output "person behavior". Here, the AI ​​logic that outputs "person behavior" is, for example, an analysis method that realizes motion estimation, skeletal estimation, violent act detection, fall detection, person tracking, etc. by inputting image data, but the analysis method that outputs "person behavior" is not limited to the above example. In addition, the target of behavior analysis may be a robot, a construction machine, or other work vehicle, an animal, etc., in addition to a person, so the behavior performed by each target may be the target.

[0093] (2-3-4-3.Person identification) As shown in the example of FIG. 7, the operator terminal 20 displays "person identification" indicating the attributes of a person as one of the analysis categories. In the example of FIG. 7, "AI Logic A", "AI Logic B", and "AI Logic C" are displayed as AI logics that output "person identification". Here, the AI ​​logic that outputs "person identification" is, for example, an analysis method that realizes face recognition, individual identification from walking style, gender and age estimation, clothing and equipment detection, detection of people requiring assistance, etc. by inputting image data, but the analysis method that outputs "person identification" is not limited to the above example.

[0094] In addition, "person identification" is an example of AI logic for identifying a subject, and it is also possible to adopt AI logic that outputs "animal identification" for identifying animals, "work machine identification" for identifying work vehicles such as robots and construction machinery, and "natural object identification" for identifying natural objects such as flowing water in a river or falling rocks.

[0095] (2-3-4-4. Object Identification) As shown in the example of FIG. 7, the operator terminal 20 displays "identification of object" indicating an object involved in a person's behavior as one of the analysis categories. In the example of FIG. 7, "AI logic Z", "AI logic Y", "AI logic X", and "AI logic W" are displayed as AI logics that output "identification of object". Here, the AI ​​logic that outputs "identification of object" is, for example, an analysis method that realizes object type recognition, vehicle detection, animal recognition, plant recognition, dangerous object detection, etc. by inputting image data, but the analysis method that outputs "identification of object" is not limited to the above example.

[0096] In addition, "object identification" is an example of AI logic that identifies an object, and it is also possible to adopt AI logic that outputs the above-mentioned "person identification."

[0097] (2-3-4-5. AI logic feature viewing screen) The operator terminal 20 can also display an "AI logic feature viewing screen", which is a setting screen that presents the features of the AI ​​logic, to support the operator O in inputting the selection of the AI ​​logic. For example, when the operator O clicks the "AI logic" button for "AI logic 1" in "Analysis category example 1: human behavior", the operator terminal 20 displays information such as "face recognition", which is a feature of "AI logic 1".

[0098] (2-3-4-6. Output value list viewing screen) The operator terminal 20 can also display an "output value list viewing screen", which is a setting screen presenting a list of analysis items output by the AI ​​logic, to support the operator O in inputting the selection of the AI ​​logic. For example, when the operator O clicks the "output value list" button of "AI logic 1" in "analysis category example 1: human behavior", the operator terminal 20 displays information such as "enter the room", "leave", "take out", "leave behind", "be violent", "fall", "vomit", and "cry" as a list of analysis items output by "AI logic 1".

[0099] (2-3-5. Specific Example 2 of Display Screen of Operator Terminal 20) Here, a specific example 2 of a display screen output by the input / output unit 21 of the operator terminal 20 will be described with reference to Fig. 8. Fig. 8 is a diagram showing a specific example 2 of a display screen of the operator terminal 20 according to the embodiment. In the following, "Importance setting screen", "Person identification", "Occurrence time", "Object identification", "Person action", "Importance", and "Reaction" will be described.

[0100] (2-3-5-1. Importance setting screen) As shown in the example of FIG. 8, the operator terminal 20 displays an "importance setting screen" which is a setting screen for receiving the importance associated with a combination of analysis items (i.e., an event that occurs) for each analysis category. Here, the operator O can input a combination of analysis items for each analysis category by clicking the check boxes of the analysis items displayed in each of the fields "who," "when," "what," and "how." The operator O can also input an "importance" indicating urgency, etc., corresponding to the combination of analysis items for each analysis category. The operator O can also input a "reaction" such as an alarm notification, corresponding to the importance.

[0101] (2-3-5-2.Person identification) As shown in the example of FIG. 8, the operator terminal 20 displays "Who" corresponding to the above-mentioned "Analysis Category Example 2: Person Identification" as one of the analysis categories. In the example of FIG. 8, "Pre-registered Person Authority A", "Pre-registered Person Authority B", and "Non-registered Person" are displayed as analysis items output by the AI ​​logic of "Person Identification" selected by the operator O on the above-mentioned "AI Logic Selection Screen". In addition, "Non-registered Person" is selected by the operation of the operator O. Note that the analysis items output by the AI ​​logic of "Person Identification" are not limited to the above example.

[0102] In this case, the analysis item output by the AI ​​logic of "person identification" may be an analysis item directly output by the analysis model AM, or may be an analysis item output by converting the output analysis item. For example, the analysis model AM may directly output one of the analysis items "pre-registered person authority A", "pre-registered person authority B", or "non-registered person" according to the input of image data, or may convert it into one of the analysis items "pre-registered person authority A", "pre-registered person authority B", or "non-registered person" associated with the personal name output by face recognition according to the input of image data, and output it.

[0103] (2-3-5-3. Time of occurrence) As shown in the example of FIG. 8, the operator terminal 20 displays "when" corresponding to the time of occurrence of an event as one of the analysis categories. In the example of FIG. 8, "weekday business hours," "outside weekday business hours," "weekday late night hours," "holiday daytime," and "holiday nighttime late night hours" are displayed as analysis items that can be identified from the shooting time of image data. In addition, "weekday business hours" is selected by an operation of the operator O. It should be noted that the analysis items that can be identified from the shooting time of image data are not limited to the above example.

[0104] In this case, the analysis items that can be specified as the occurrence time of the event from the shooting time of the image data may be classified by the server device 10 based on the company work schedule, school attendance schedule, etc. registered in advance, or may be output based on the day of the week, time, etc. set by the operator O. In addition, the analysis items of the occurrence time of the event may be output based on the analysis model AM that directly outputs analysis items in response to the input of image data.

[0105] (2-3-5-4. Object Identification) As shown in the example of FIG. 8, the operator terminal 20 displays "what" corresponding to the above-mentioned "analysis category example 3: identification of objects" as one of the analysis categories. In the example of FIG. 8, "desk", "chair", "cardboard box", "locker", "file", "safe", "seal", "potted plant", and "shredder" are displayed as analysis items output by the AI ​​logic of "identification of objects" selected by the operator O on the above-mentioned "AI logic selection screen". In addition, "file" is selected by the operation of the operator O. Note that the analysis items output by the AI ​​logic of "identification of objects" are not limited to the above example.

[0106] At this time, the analysis item output by the AI ​​logic of "object identification" may be the analysis item directly output by the analysis model AM, or may be the analysis item output by converting the output analysis item. Also, the operator O can skip the selection of "object identification" if there is no target object corresponding to the "human action".

[0107] (2-3-5-5.Human movements) As shown in the example of FIG. 8, the operator terminal 20 displays "What happened" corresponding to the above-mentioned "Analysis Category Example 1: Person's Action" as one of the analysis categories. In the example of FIG. 8, "Enter the room", "Leave", "Take out", "Leave behind", "Be violent", "Collapse", "Vomit", and "Cry" are displayed as analysis items output by the AI ​​logic of "Person's Action" selected by the operator O on the above-mentioned "AI logic selection screen". In addition, "Take out" is selected by the operation of the operator O. Note that the analysis items output by the AI ​​logic of "Person's Action" are not limited to the above example.

[0108] In this case, the analysis item output by the AI ​​logic for "human behavior" may be the analysis item directly output by the analysis model AM, or it may be the analysis item output by converting the output analysis item.

[0109] (2-3-5-6.Importance) As shown in the example of Fig. 8, the operator terminal 20 can input "importance" indicating urgency or the like corresponding to a combination of analysis items for each analysis category. In the example of Fig. 8, "importance 8" (minimum value 0, maximum value 10) is input for the combination of analysis items {who: "unregistered user", when: "weekday business hours", what: "file", how: "take out"} selected by the operator O. Note that the input format of "importance" is not limited to the above example, and may be expressed as a score of 0 to 100% or a rank of A, B, C, etc.

[0110] (2-3-5-7. Reaction) The operator terminal 20 can input a "reaction" such as an alarm notification corresponding to the "importance". For example, as a reaction to be executed by the server device 10, it is possible to input {importance: "0", reaction: "event record"}, {importance: "1 to 6", reaction: "image data record"}, {importance: "7 to 10", reaction: "image data record", "alarm notification"}, etc.

[0111] (2-4. Configuration and Processing Examples of Camera 30) 2 again, a configuration example and a processing example of the camera 30 will be described. For example, the camera 30 is realized by a security camera, a monitoring camera, or the like installed in a facility managed by an operator O, and has an imaging unit 31 and a communication unit 32.

[0112] (2-4-1. Shooting section 31) The photographing unit 31 generates image data. For example, the photographing unit 31 photographs the inside of the facility every second and generates image data of a still image. The photographing unit 31 also photographs the inside of the facility and generates image data of a moving image. At this time, the photographing unit 31 may record the inside of the facility and generate audio data.

[0113] The imaging unit 31 transmits the generated image data to the server device 10. For example, the imaging unit 31 transmits image data of the generated still image to the server device 10. The imaging unit 31 also transmits image data of the generated moving image to the server device 10. At this time, the imaging unit 31 may transmit the generated audio data to the server device 10.

[0114] (2-4-2. Communication section 32) The communication unit 32 is responsible for data communication with other devices. For example, the communication unit 32 performs data communication with each communication device via a router, etc. The communication unit 32 can also perform data communication with an operator's terminal (not shown).

[0115] 3. Flow of each process of the occurrence event detection system 100 The process flow of the occurrence event detection system 100 according to the embodiment will be described with reference to Fig. 9. Note that the processes of steps S101 to S107 below may be executed in a different order. Also, some of the processes of steps S101 to S107 below may be omitted.

[0116] (3-1. Candidate information transmission process) First, the server device 10 executes a candidate information transmission process (step S101). For example, the server device 10 transmits candidate information including identification information of the AI ​​logic for each analysis category that the server device 10 can analyze, an output value list, and the like, to the operator terminal 20.

[0117] (3-2. Candidate information display process) Secondly, the operator terminal 20 executes a candidate information display process (step S102). For example, the operator terminal 20 displays on the monitor a setting screen that presents, as candidate information, a list of identification information of AI logics for each analysis category, a list of output values ​​for each AI logic, and the like.

[0118] (3-3. Setting information input process) Third, the operator O executes a setting information input process (step S103). For example, the operator O inputs the selection of the AI ​​logic to be used for analysis as setting information by selecting an AI logic from a list of identification information of the AI ​​logic for each analysis category, selects analysis items output by the selected AI logic for each analysis category, and inputs the importance corresponding to the combination of analysis items as setting information by assigning importance to the combination of the selected analysis items.

[0119] (3-4. Setting information transmission process) Fourth, the operator terminal 20 executes a setting information transmission process (step S104). For example, the operator terminal 20 transmits the selection of the AI ​​logic for each analysis category, the importance corresponding to the combination of analysis items, and the like, which are input to the operator terminal 20 by the operator O, to the server device 10 as setting information.

[0120] (3-5. Candidate information display process) Fifth, the camera 30 executes an image data transmission process (step S105). For example, the camera 30 transmits to the server device 10 image data of still images captured every second in the facility where the camera 30 is installed.

[0121] (3-6. Image data analysis processing) Sixth, the server device 10 executes image data analysis processing (step S106). For example, the server device 10 refers to the setting information, analyzes the image data using the AI ​​logic selected by the operator O for each analysis category, and identifies the importance corresponding to the combination of analysis items that is the analysis result.

[0122] (3-7. Reaction execution process) Seventh, the server device 10 executes a reaction execution process (step S107). For example, when the identified importance is equal to or greater than a predetermined value, the server device 10 notifies the operator terminal 20 of an alarm as a reaction.

[0123] 4. Effects of the embodiment Finally, effects of the embodiment will be described below: Effects 1 to 8 corresponding to the processing according to the embodiment will be described below.

[0124] (4-1. Effect 1) First, in the process according to the embodiment described above, the server device 10 stores a plurality of analysis models AM that output image analysis targets included in image data, accepts a selection of an analysis model AM from the plurality of analysis models AM and an importance associated with the combination of image analysis targets, acquires image data, inputs the acquired image data into the analysis model AM whose selection was accepted, and executes a predetermined reaction according to the importance associated with the output combination of image analysis targets. Therefore, in this process, an appropriate reaction to an event that has occurred can be executed.

[0125] (4-2. Effect 2) Secondly, in the process according to the embodiment described above, the server device 10 stores a plurality of analysis models AM for each analysis category of the image analysis target to be output, accepts the selection of an analysis model AM from the plurality of analysis models AM for each analysis category and the importance associated with the combination of image analysis targets for each analysis category, inputs the acquired image data to the analysis model AM for which the selection was accepted for each analysis category, and executes a predetermined reaction according to the importance associated with the combination of image analysis targets for each output analysis category. Therefore, in this process, the importance can be set in detail for each event that occurs, so that an appropriate reaction to the event that occurs can be executed.

[0126] (4-3. Effect 3) Thirdly, in the process according to the embodiment described above, the server device 10 presents a list of multiple analysis models AM for each analysis category of the image analysis target, and causes the operator terminal 20 of the operator O to display a setting screen for receiving input of the selection of the analysis model AM for each analysis category. Therefore, in this process, the operator O can select an AI logic to be used for each analysis category, thereby executing an appropriate reaction to an event that has occurred.

[0127] (4-4. Effect 4) Fourth, in the process according to the embodiment described above, the server device 10 presents a list of multiple analysis models AM for each analysis category of the image analysis target, and causes the operator terminal 20 of the operator O to display a setting screen presenting a list of the image analysis targets output by each of the multiple analysis models AM. Therefore, in this process, by providing information necessary for the operator O to select the AI ​​logic to be used, an appropriate reaction to the event that has occurred can be executed.

[0128] (4-5. Effect 5) Fifth, in the process according to the embodiment described above, the server device 10 presents a list of image analysis targets output by the analysis model AM selected for each category of image analysis target, and causes the operator terminal 20 of the operator O to display a setting screen for receiving input of a combination of image analysis targets for each analysis category and an input of the importance of each combination. Therefore, in this process, by providing a setting screen used by the operator O to set the importance, an appropriate reaction to an event that has occurred can be executed.

[0129] (4-6. Effect 6) Sixth, in the process according to the embodiment described above, the server device 10 acquires image data captured and transmitted by the camera 30 installed in the facility managed by the operator O. In this process, the image data of the facility managed by the operator O is analyzed in real time, thereby making it possible to execute an appropriate reaction to an event that has occurred.

[0130] (4-7. Effect 7) Seventh, in the process according to the embodiment described above, the server device 10 executes at least one of notifying the operator O of an alarm, recording the image data, and recording the output image analysis target, as a reaction set by the operator O, depending on the importance. In this process, by flexibly changing the reaction depending on the event that has occurred, it is possible to execute an appropriate reaction to the event that has occurred.

[0131] (4-8. Effect 8) Eighth, in the process according to the embodiment described above, the analysis category of the image analysis target includes at least one of the attributes of a person, the behavior of a person, and an object involved in the behavior of a person. In this process, the reaction can be flexibly changed according to "who", "what", and "how" of the event that occurred, so that an appropriate reaction to the event that occurred can be executed.

[0132] [First Modification of the Embodiment] The configuration and processing of an occurrence event detection system 100M-1 according to the first modified example of the embodiment will be described with reference to Fig. 10. Note that a description of the configuration and processing common to the embodiment will be omitted.

[0133] As shown in FIG. 10, an incident detection system 100M-1 includes an operator terminal 20M and a camera 30. The operator terminal 20M and the camera 30 are connected to the operator terminal 20M.

[0134] The occurring event detection system 100M-1 executes the following processes. First, the operator terminal 20M displays candidate information (step S11). Second, the operator O inputs setting information to the operator terminal 20M (step S12). Third, the camera 30 transmits image data to the operator terminal 20M (step S13). Fourth, the operator terminal 20M analyzes the image data (step S14). Fifth, the operator terminal 20M executes a reaction (step S15).

[0135] As described above, in the occurrence event detection system 100M-1 relating to the embodiment variant 1, the operator terminal 20M executes image analysis processing and reaction execution processing, thereby enabling an appropriate reaction to the occurrence of an event to be executed without using the server device 10.

[0136] [Modification 2 of the embodiment] The configuration and processing of an occurrence event detection system 100M-2 according to the second modification of the embodiment will be described with reference to Fig. 11. Note that a description of the configuration and processing common to the embodiment will be omitted.

[0137] As shown in FIG. 11, an incident detection system 100M-2 includes an operator terminal 20 and a camera 30M.

[0138] The occurring event detection system 100M-2 executes the following processes. First, the camera 30M transmits candidate information to the operator terminal 20 (step S21). Second, the operator terminal 20 displays the candidate information (step S22). Third, the operator O inputs setting information to the operator terminal 20 (step S23). Fourth, the operator terminal 20 transmits the setting information to the camera 30M (step S24). Fifth, the camera 30M analyzes the image data (step S25). Sixth, the camera 30M executes a reaction to the operator terminal 20 (step S26).

[0139] As described above, in the occurrence event detection system 100M-2 relating to the embodiment variant 2, the camera 30M executes the candidate information transmission process, the image analysis process, and the reaction execution process, thereby enabling an appropriate reaction to an occurrence event to be executed without using the server device 10.

[0140] 〔system〕 The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.

[0141] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure. In other words, all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0142] Furthermore, each processing function performed by each device may be realized, in whole or in part, by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0143] [Hardware] Next, a hardware configuration example of the server device 10, which is an image analysis device, will be described. Note that other devices may have a similar hardware configuration. FIG. 12 is a diagram for explaining a hardware configuration example according to an embodiment. As shown in FIG. 12, the server device 10 has a communication device 10a, a HDD (Hard Disk Drive) 10b, a memory 10c, and a processor 10d. In addition, each unit shown in FIG. 12 is connected to each other by a bus or the like.

[0144] The communication device 10a is a network interface card or the like, and communicates with other servers. The HDD 10b stores programs and databases that operate the functions shown in FIG.

[0145] The processor 10d reads out a program that executes the same processes as the respective processing units shown in FIG. 2 from the HDD 10b etc. and loads it in the memory 10c, thereby operating a process that executes the respective functions described in FIG. 2 etc. For example, this process executes the same functions as the respective processing units possessed by the server device 10. Specifically, the processor 10d reads out a program having the same functions as the reception unit 15a, the acquisition unit 15b, the execution unit 15c etc. from the HDD 10b etc. Then, the processor 10d executes a process that executes the same processes as the reception unit 15a, the acquisition unit 15b, the execution unit 15c etc.

[0146] In this way, the server device 10 operates as a device that executes various processing methods by reading and executing the programs. The server device 10 can also realize functions similar to those of the above-mentioned embodiment by reading the programs from a recording medium using a media reading device and executing the read programs. Note that the programs in these other embodiments are not limited to being executed by the server device 10. For example, the present invention can be similarly applied to cases where other computers or servers execute the programs, or where these execute the programs in cooperation with each other.

[0147] This program can be distributed via a network such as the Internet. This program can also be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and can be executed by being read out from the recording medium by a computer.

[0148] 〔others〕 Some examples of combinations of the disclosed technical features are set out below.

[0149] (1) An image analysis device comprising: a memory unit that stores a plurality of machine learning models that output objects contained in image data; a reception unit that receives a selection of a machine learning model from the plurality of machine learning models and an importance associated with the combination of the objects; an acquisition unit that acquires image data; and an execution unit that inputs the acquired image data to the machine learning model for which the selection was accepted, and executes a predetermined reaction depending on the importance associated with the combination of the output objects.

[0150] (2) The image analysis device described in (1), wherein the memory unit stores the multiple machine learning models for each category of the targets to be output, the reception unit receives a selection of the machine learning model from the multiple machine learning models for each category and the importance associated with the combination of the targets for each category, and the execution unit inputs the acquired image data to the machine learning model for which the selection was received for each category, and executes the specified reaction depending on the importance associated with the combination of the targets for each category to be output.

[0151] (3) The image analysis device described in (1) or (2), wherein the reception unit presents a list of the plurality of machine learning models for each target category and displays a setting screen on the user's user terminal that receives input of the selection of the machine learning model for each category.

[0152] (4) The image analyzing device described in any one of (1) to (3), wherein the reception unit presents a list of the plurality of machine learning models for each category of the targets and displays a setting screen on the user terminal of the user presenting a list of the targets output by each of the plurality of machine learning models.

[0153] (5) The image analyzing device described in any one of (1) to (4), wherein the reception unit presents a list of the targets output by the machine learning model for which the selection was accepted for each category of the targets, and displays a setting screen on the user terminal of the user that accepts input of combinations of the targets for each category and input of the importance for each combination.

[0154] (6) The image analyzing device according to any one of (1) to (5), wherein the acquisition unit acquires image data captured and transmitted by an imaging device installed in a facility managed by a user.

[0155] (7) An image analyzing device according to any one of (1) to (6), wherein the execution unit executes at least one of notifying the user of an alarm, recording image data, and recording the outputted target as the specified reaction set by the user, depending on the importance.

[0156] (8) The image analyzing device according to any one of (1) to (7), wherein the category of the target includes at least one of an attribute of a person, an action of the person, and an object involved in the action.

[0157] (9) An image analysis method in which a computer executes a process in which it selects a machine learning model from a plurality of machine learning models that outputs objects contained in image data, accepts an importance associated with the combination of the objects, acquires image data, inputs the acquired image data to the machine learning model for which the selection was accepted, and executes a predetermined reaction depending on the importance associated with the combination of the output objects.

[0158] (10) An image analysis program that causes a computer to execute a process of selecting a machine learning model from a plurality of machine learning models that outputs objects contained in image data and accepting an importance associated with the combination of the objects, acquiring image data, inputting the acquired image data into the machine learning model for which the selection was accepted, and executing a predetermined reaction depending on the importance associated with the combination of the output objects. [Explanation of symbols]

[0159] 10. Server equipment 11 Input section 12 Output section 13. Communications Department 14 Storage section 14a Candidate information storage unit 14b Setting information storage section 14c Image data storage section 14d Analysis model storage section 15 Control section 15a Reception 15b Acquisition part 15c Executive Department 20 Operator terminal 21 Input / output section 22 Transmitter / receiver 23 Communications Department 30 Camera 31 Photography Department 32 Communications Department 100 Incident Detection System

Claims

1. A storage unit that stores a plurality of machine learning models that output objects included in image data; a reception unit that receives a selection of a machine learning model from the plurality of machine learning models and an importance associated with the target combination; An acquisition unit that acquires image data; an execution unit that inputs the acquired image data into the machine learning model for which the selection was accepted, and executes a predetermined reaction according to the importance associated with the output combination of targets; An image analysis device comprising:

2. The storage unit is storing the plurality of machine learning models for each category of the target to be output; The reception unit is Accepting a selection of a machine learning model from the plurality of machine learning models for each of the categories and the importance associated with the combination of the targets for each of the categories; The execution unit is inputting the acquired image data into the machine learning model for which the selection has been accepted for each of the categories, and executing the predetermined reaction in accordance with the importance associated with the combination of the targets for each of the output categories; The image analysis device according to claim 1 .

3. The reception unit is presenting a list of the plurality of machine learning models for each target category and displaying a setting screen on a user terminal of a user for receiving an input of a selection of the machine learning model for each category; The image analysis device according to claim 1 .

4. The reception unit is Displaying a setting screen on a user terminal of a user, the setting screen presenting a list of the plurality of machine learning models for each category of the target and presenting a list of the targets output by each of the plurality of machine learning models; The image analysis device according to claim 1 .

5. The reception unit is presenting a list of the objects output by the machine learning model for which the selection has been accepted for each category of the objects, and displaying a setting screen for accepting input of a combination of the objects for each category and an input of the importance for each combination on a user terminal of a user; The image analysis device according to claim 1 .

6. The acquisition unit is Acquire image data captured and transmitted by a photographing device installed in a facility managed by the user; The image analysis device according to claim 1 .

7. The execution unit is as the predetermined reaction set by the user, at least one of notifying the user of an alarm, recording image data, and recording the output object is executed according to the importance level; The image analysis device according to claim 1 .

8. The object category includes at least one of an attribute of a person, an action of the person, and an object involved in the action. The image analysis device according to any one of claims 1 to 7.

9. The computer Accepting a selection of a machine learning model from a plurality of machine learning models that outputs objects contained in the image data, and an importance associated with the combination of objects; Acquire image data; inputting the acquired image data into the machine learning model for which the selection was accepted, and executing a predetermined reaction according to the importance associated with the output combination of targets; Image analysis methods to carry out processing.

10. On the computer, Accepting a selection of a machine learning model from a plurality of machine learning models that outputs objects contained in the image data, and an importance associated with the combination of objects; Acquire image data; inputting the acquired image data into the machine learning model for which the selection was accepted, and executing a predetermined reaction according to the importance associated with the output combination of targets; The image analysis program that performs the processing.

Images