Analysis device and program

Abstract

To provide an analysis device and program that have high detection accuracy and can be implemented at low cost. [Solution] The analysis device 10 of the embodiment includes an input unit 103, a setting unit 102, and a detection unit 105. The input unit 103 is capable of acquiring at least one image, each associated with identification information. The setting unit 102 sets a prompt set including at least one prompt for each piece of identification information, and sets a common trained image language model for the identification information. The detection unit 105 inputs a first image associated with the first identification information from among the at least one image, and a first prompt set set for the first identification information, into the image language model, and outputs the response of the first prompt set for the first image.

Description

Technical Field

[0001] Embodiments relate to an analysis device and a program.

Background Art

[0002] As equipment used for early detection of events and things for purposes such as disaster prevention, surveillance cameras and the like are effective. By visually checking the images of a surveillance camera by a monitor, when an abnormality is confirmed, responses such as contacting relevant locations or going to the site can be taken. However, visual confirmation of images by a monitor has problems in terms of accuracy and cost.

[0003] In contrast, using image recognition AI (Artificial Intelligence) for early detection of events and things has been considered. Image recognition AI can detect learned individual objects and recognize and discriminate specific shapes included in the input image. For example, there is known a surveillance technology in which image recognition AI detects a certain event or thing shown in a surveillance camera, determines it as an abnormality, and then issues an alarm to notify the monitor. In Patent Document 1, a common general-purpose AI model that can be used for a plurality of cameras is created. Then, by adjusting the AI model for each camera, the accuracy of video analysis is improved.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in Patent Document 1, since the AI ​​model is adjusted for each camera, if there are many cameras to be monitored, it takes time to adjust the AI ​​model for all of them. As a result, the monitoring system may become expensive.

[0006] This invention was made to solve the above problems and aims to provide an analysis device and program that have high detection accuracy and can be implemented at low cost. [Means for solving the problem]

[0007] The analysis device of the embodiment includes an input unit, a setting unit, and a detection unit. The input unit is capable of acquiring at least one image, each associated with identification information. The setting unit sets a prompt set including at least one prompt for each piece of identification information, and sets a common trained image language model for the identification information. The detection unit inputs a first image, which is associated with the first piece of identification information from among the at least one images, and the first prompt set set for the first piece of identification information, into the image language model, and outputs the response of the first prompt set for the first image. [Brief explanation of the drawing]

[0008] [Figure 1] A block diagram showing an example of the overall configuration of the analysis system according to the embodiment. [Figure 2] A block diagram showing an example of the hardware configuration of the analysis device according to the embodiment. [Figure 3] A block diagram showing an example of the functional configuration of the analysis device according to this embodiment. [Figure 4] A block diagram showing an example of the configuration of the data storage unit included in the analysis device according to the embodiment. [Figure 5] A data table showing an example of the configuration of a training data DB stored in the data storage unit of the analysis device according to the present invention. [Figure 6] A data table showing an example of the configuration of a trained model database stored in the data storage unit of the analysis device according to the embodiment. [Figure 7] A data table showing an example of the configuration of a camera information database stored in the data storage unit of the analysis device according to the embodiment. [Figure 8] A data table showing an example of the configuration of a prompt evaluation DB stored in the data storage unit of the analysis device according to the embodiment. [Figure 9] A data table showing an example of the configuration of a prompt DB stored in the data storage unit of the analysis device according to the embodiment. [Figure 10] A data table showing an example of the configuration of a detection result DB stored in the data storage unit of the analysis device according to the present invention. [Figure 11] A data table showing an example of the configuration of the alert issuance history DB stored in the data storage unit of the analysis device according to the embodiment. [Figure 12] A schematic diagram illustrating the overview of a question-answering AI using an analysis device according to the embodiment. [Figure 13] A flowchart illustrating an example of a method for creating an image language model using the analysis device according to the embodiment. [Figure 14] A figure showing an example of a ground truth image used in the analysis device according to the embodiment. [Figure 15] A diagram showing an example of an image containing the object to be detected, used in the analysis device according to the embodiment. [Figure 16] A flowchart showing an example of the analysis process performed by the analysis device according to the embodiment. [Figure 17] A flowchart showing an example of the setting process by the analysis device according to the embodiment. [Figure 18] A schematic diagram showing an overview of the prompt generation process by the analysis device according to the embodiment. [Figure 19] A flowchart showing an example of prompt generation processing by the analysis device according to the embodiment. [Figure 20] A flowchart showing an example of detection processing by the analysis device according to the embodiment. [Figure 21] A flowchart illustrating an example of how to utilize the detection results obtained by the analysis device according to this embodiment.

Best Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described with reference to the drawings. The embodiments illustrate devices and methods for embodying the technical idea of the invention. The drawings referred to below are schematic or conceptual. Components having substantially the same function and configuration are denoted by the same reference numerals. In this specification, "database" is sometimes abbreviated as "DB" as appropriate.

[0010] <1> Configuration First, the configuration of the analysis system 1 according to the embodiment will be described.

[0011] <1-1> Overall Configuration of Analysis System 1 FIG. 1 is a block diagram showing an example of the overall configuration of the analysis system 1 according to the embodiment. As shown in FIG. 1, the analysis system 1 according to the embodiment includes, for example, an analysis device 10, a terminal device 20, a camera set 30, and a reporting device 40. In the analysis system 1, the analysis device 10 is configured to be communicable with each of the terminal device 20, the camera set 30, and the reporting device 40 via, for example, a network NW.

[0012] The analysis device 10 is a computer having a function as a question-and-answer AI (Artificial Intelligence) using an image language model. The question-and-answer AI using an image language model receives an input of a set of an image and at least one prompt (question). Then, the question-and-answer AI can output an answer for each prompt regarding the input image. Further, the analysis device 10 is configured to be able to generate at least one prompt related to the keyword of the detection target. In the analysis device 10, the image acquired by the camera set 30 and at least one prompt generated by itself are input to the question-and-answer AI.

[0013] The terminal device 20 is, for example, an electronic device such as a smartphone or a PC (Personal Computer). Users such as monitors can access the question-answering AI functions of the analysis device 10 using the terminal device 20.

[0014] The camera set 30 is a collection of imaging devices that capture images or videos to be analyzed by the analysis device 10. The camera set 30 includes, for example, a plurality of cameras 31-1 to 31-N (where N is an integer of 2 or more). Each of the plurality of cameras 31-1 to 31-N can transmit the captured images or videos to the analysis device 10, for example, via a network NW.

[0015] The alarm device 40 is a device that notifies users, such as monitoring personnel, of abnormalities, etc., based on instructions from the analysis device 10. The alarm device 40 is, for example, a buzzer configured to emit an alarm. Other devices or methods may be used as the alarm device 40, as long as they can emit an alarm based on instructions from the analysis device 10. For example, the function of the alarm device 40 may be incorporated into an application such as a smartphone. In this case, the analysis device 10 may send a notification to an application such as a smartphone in a remote location via an internet connection, based on the detection result of the detected object. The content notified to the alarm device 40 by the analysis device 10 may include not only sound and messages, but also images of the detection result. The analysis system 1 may be equipped with multiple alarm devices 40. In this case, each of the multiple alarm devices 40 may be associated with at least one camera 31.

[0016] The analysis device 10 may be connected to the terminal device 20, camera set 30, and alarm device 40 without using a network (NW). The analysis device 10 may utilize the functions of a question-answering AI for stored images and videos. Processing by the terminal device 20 may be performed by directly operating the analysis device 10. At least one of the terminal device 20, camera set 30, and alarm device 40 may be omitted from the analysis system 1 depending on the application. The functions of the alarm device 40 may be incorporated into the analysis device 10 and the terminal device 20.

[0017] The following explanation will describe an example where analysis system 1 is used as a monitoring system to detect abnormalities occurring on roads for road safety management.

[0018] <1-2> Hardware configuration of the analysis device 10 Figure 2 is a block diagram showing an example of the hardware configuration of the analysis device 10 according to the embodiment. As shown in Figure 2, the analysis device 10 according to the embodiment includes, for example, a processor 11, a memory device 12, a storage device 13, a display device 14, an input interface 15, and a communication module 16.

[0019] The processor 11 is an integrated circuit capable of executing various programs and controls the overall operation of the analysis device 10. Examples of processors 11 include CPUs (Central Processing Units), MPUs (Micro Processing Units), ASICs (Application Specific Integrated Circuits), FPGAs (field-programmable gate arrays), and GPUs (Graphics Processing Units).

[0020] The memory device 12 is a storage device that stores programs and control data for controlling the analysis device 10. The memory device 12 is used as a workspace for the processor 11. The analysis device 10 may also be equipped with separate memory devices for storing programs and control data and for use as a workspace for the processor 11.

[0021] The storage device 13 is a non-volatile memory device. The storage device 13 stores, for example, application software and system software for the analysis device 10. Examples of storage devices 13 include SSDs (Solid State Drives) and HDDs (Hard Disk Drives).

[0022] The display device 14 is a device configured to display characters, images, and the like. Based on instructions from the processor 11, the display device 14 displays, for example, a GUI (Graphical User Interface) corresponding to application software. Examples of display devices 14 include LCDs (Liquid Crystal Displays) and organic EL (Electro-Luminescence) displays.

[0023] The input interface 15 is an interface used to operate the analysis device 10. The user can operate the analysis device 10 using the input interface 15. Note that the function of the input interface 15 may be built into the display device 14 of the analysis device 10.

[0024] The communication module 16 is a circuit used for sending and receiving data with external devices. Either wired or wireless communication may be used for sending and receiving data by the communication module 16. The communication module 16 may be connected to an external storage device or to an external server via a network (NW).

[0025] The analysis device 10 may have other hardware configurations. For example, if the analysis device 10 is an IoT (Internet of Things) terminal, the display device 14 may be omitted from the analysis device 10. The storage device 13, the display device 14, and the input interface 15 may each be externally connected to the analysis device 10.

[0026] <1-3> Functional configuration of the analysis device 10 Figure 3 is a block diagram showing an example of the functional configuration of the analysis device 10 according to the embodiment. As shown in Figure 3, the analysis device 10 functions as a computer comprising, for example, a learning unit 101, a setting unit 102, an input unit 103, a prompt generation unit 104, a detection unit 105, a data storage unit 106, and a notification unit 107. Each of the learning unit 101, setting unit 102, input unit 103, prompt generation unit 104, detection unit 105, data storage unit 106, and notification unit 107 can be realized by the hardware configuration of the analysis device 10 described with reference to Figure 2.

[0027] The learning unit 101 has the function of training the AI ​​using the training data stored in the data storage unit 106 and creating an image language model used in the question-answering AI. The learning unit 101 then stores the created image language model in the data storage unit 106. Hereafter, the image language model created by the learning unit 101 will be referred to as the "trained model". The learning unit 101 can also perform additional training on the trained model using the trained model and new training data.

[0028] The setting unit 102 has a detection event setting function, a camera setting function, a prompt setting function, and a model setting function.

[0029] The detection event setting function of the setting unit 102 accepts input of information about events or objects (detection targets) to be discovered or detected in monitoring operations from an external terminal device 20. The information input to the detection event setting function may be one or more keywords, text data, or image data. If text data is input, the detection event setting function may extract one or more keywords from the text data. If image data is input, the detection event setting function may extract one or more keywords from the content of the image data. The detection event setting function then transmits the input or extracted keywords to the prompt generation unit 104. The keywords transmitted to the prompt generation unit 104 are used as information for automatically generating prompts to detect the detection targets.

[0030] The camera setting function of the setting unit 102 has the function of setting at least one camera 31 from the camera set 30 to be monitored to the input unit 103. The camera 31 to be monitored may be selected by user operation or may be selected automatically. When the camera 31 to be monitored is selected automatically, the camera setting function receives a keyword input or extracted from the detected event setting function. The camera setting function then compares the received keyword with the keyword of the expected event set for each camera 31 in the camera set 30. Then, the camera setting function extracts at least one camera 31 from the camera set 30 that matches the keyword and sets the extracted camera 31 as the camera to be monitored.

[0031] The prompt setting function of the setting unit 102 has the function of setting prompts generated and extracted by the prompt generation unit 104. The prompt setting function may also have the function of registering prompts created by the user. When a prompt is registered by the user, it may be in the form of direct input or file registration. Prompts set by the prompt generation function are sent to the detection unit 105. The information of the set prompts is stored in the data storage unit 106. The prompt setting function may also have the function of allowing previously used prompts to be selected from the data storage unit 106 and set again.

[0032] The model setting function of the setting unit 102 has the function of selecting an image language model to be used for analysis processing and setting it in the detection unit 105. The analysis processing includes operation as a question-answering AI that analyzes the monitored image and outputs a prompt response. The analysis processing may also be called AI detection. Details of the analysis processing will be described later. The image language model selected by the model setting function is generated by, for example, the learning unit 101 and stored in the data storage unit 106.

[0033] The input unit 103 has the function of acquiring real-time video and other data from the camera 31 for detection by the detection unit 105. Specifically, the input unit 103 is configured to acquire real-time video from the camera 31 designated as the target of monitoring by the camera setting function of the setting unit 102, and transmits the acquired real-time video data to the detection unit 105. Note that the input unit 103 may also receive video data or intermittently captured image data instead of real-time video. Furthermore, the input unit 103 may receive image data or video data acquired outside the analysis system 1.

[0034] The prompt generation unit 104 has a prompt automatic generation function and a prompt selection function.

[0035] The prompt generation function of the prompt generation unit 104 accepts input of a keyword to be detected from, for example, the setting unit 102. When a keyword to be detected is input, the prompt generation function automatically generates at least one prompt to detect events or things related to the input keyword. It is preferable for the prompt generation function to generate as many prompts as possible related to the input keyword. The prompt generation function may use an AI model such as a generation AI to generate prompts based on the input keyword.

[0036] The prompt selection function of the prompt generation unit 104 performs an accuracy evaluation of at least one prompt generated by the automatic prompt generation function. The prompt selection function then automatically selects a prompt or combination of prompts with good accuracy. The prompt selection function then transmits the selected at least one prompt to the setting unit 102 and the data storage unit 106, respectively. The selected at least one prompt is then set in the prompt setting function of the setting unit 102. Hereafter, the selected at least one prompt will also be referred to as a "prompt set".

[0037] The detection unit 105 has a frame division function, a prompt transmission function, and an AI detection function.

[0038] The frame division function of the detection unit 105 has the function of dividing the real-time video or video file received from the input unit 103 into frame units. When dividing the real-time video or video file into frame units, the frame division function may omit some frames. The divided frames (specifically, images with identification information added to the divided frames) are sent to the AI ​​detection function.

[0039] The prompt transmission function of the detection unit 105 transmits the prompt set by the prompt setting function of the setting unit 102 to the AI ​​detection function.

[0040] The AI ​​detection function of the detection unit 105 functions as a question-answering AI using an image language model set by the prompt setting function of the setting unit 102. The AI ​​detection function processes each prompt sent from the prompt transmission function for the frames divided by the frame division function. The AI ​​detection function then sends the detection results, including the answers to each prompt, to the notification unit 107. The AI ​​detection function also stores the detection results in the data storage unit 106. When image data is input to the detection unit 105, the frame division function is not used, and the input image data is sent to the AI ​​detection function.

[0041] The data storage unit 106 stores multiple types of databases. The data storage unit 106 is configured to be accessible by, for example, the learning unit 101, the setting unit 102, the input unit 103, the prompt generation unit 104, the detection unit 105, and the notification unit 107. Details of the multiple databases stored in the data storage unit 106 will be described later. The data storage unit 106 may also be configured to store image or video data input to the input unit 103.

[0042] The notification unit 107 has a detection result determination function and a notification function.

[0043] The detection result determination function of the notification unit 107 checks the score included in the detection result from the AI ​​detection function of the detection unit 105. Based on the score, the detection result determination function determines whether or not to issue an alert. If it is necessary to issue an alert, the detection result determination function sends an alert issuance instruction to the notification unit.

[0044] The notification function of the notification unit 107 has the function of issuing an alert to the alarm device 40 when it receives an alert issuance instruction from the detection result determination function. By issuing an alert to the alarm device 40, the abnormality can be notified, for example, to a monitor or the monitoring room. The notification function also records the content of the alert issued in the data storage unit 106. The notification function may change the destination of the alert depending on the identification information of the camera 31 in which the abnormality was detected.

[0045] <1-4> Configuration of data storage unit 106 Figure 4 is a block diagram showing an example of the configuration of the data storage unit 106 included in the analysis device 10 according to the embodiment. As shown in Figure 4, the data storage unit 106 stores, for example, training data DB111, trained model DB112, camera information DB113, prompt evaluation DB114, prompt DB115, detection result DB116, and alert issuance history DB117.

[0046] The training data DB 111 stores training data for the learning unit 101 to use for training. Specifically, the training data DB 111 stores pairs of videos and images that can be used as training data, along with information about the training data. In the training data DB 111, for example, images associated with identification information are stored in folders with names that correspond to the identification information. When the learning unit 101 performs training, it can use appropriate data from the training data DB 111 or combine data arbitrarily to use as training data.

[0047] The trained model DB112 stores image language models (trained models) generated by the learning unit 101. The trained model DB112 can store multiple trained models. The model setting function of the setting unit 102 selects one trained model from the trained model DB112. When the detection unit 105 performs detection processing, a trained model suitable for the target event is selected from the trained model DB112.

[0048] The camera information DB 113 stores information on all cameras 31 included in the camera set 30. The information on each camera 31 includes, for example, the camera ID, the camera installation location, and the expected events to be detected. The camera setting function of the setting unit 102 can obtain appropriate data from the camera information DB 113 and set the camera 31 to be monitored.

[0049] The prompt evaluation DB 114 stores evaluation data (test data) for each identification information when the prompt selection function of the prompt generation unit 104 performs an evaluation of the accuracy of the prompt. The evaluation data is associated with keyword information of the occurrence event. The prompt selection function of the prompt generation unit 104 can select a more suitable prompt or prompt combination by comparing the keywords entered in the setting unit 102 with the keywords in the prompt evaluation DB 114. The evaluation data may be images or videos.

[0050] The prompt DB 115 stores the prompt set selected by the prompt generation unit 104. In other words, at least one prompt set by the prompt setting function of the setting unit 102 is stored in the prompt DB 115. When the detection process is executed, a prompt set that matches the target event is selected from the prompt DB 115.

[0051] The detection result DB116 stores information on the detection results (inference results) obtained by the AI ​​detection function of the detection unit 105. The detection result information includes information on the time the target was detected and information on the detected prompt. Images may also be stored as part of the detection results.

[0052] The alert history DB 117 stores information notified to the alerting device 40 by the notification function of the notification unit 107. The information stored in the alert history DB 117 is associated with detection results determined by the detection result determination function of the notification unit 107 to be targets for alerting. Based on the detection results stored in the alert history DB 117, the alerting device 40 can receive an alerting instruction from the notification unit 107 and issue an alert.

[0053] (1: Configuration of training data DB111) Figure 5 is a data table showing an example of the configuration of the training data DB 111 stored in the data storage unit 106 of the analysis device 10 according to the embodiment. As shown in Figure 5, the training data DB 111 includes information such as "training data ID", "dataset name", "description", and "data quantity", and stores images to be used as training data for each training data ID. The "training data ID" column of the training data DB 111 indicates identification information for the training data group. The "dataset name", "description", and "data quantity" columns of the training data DB 111 each indicate basic information associated with the training data ID.

[0054] The “Dataset Name” in Training Data DB111 indicates the name of the identification information associated with the training data ID. The “Description” in Training Data DB111 is information added for the user's reference. The “Data Quantity” in Training Data DB111 indicates the quantity of image data included in the associated training data ID.

[0055] In this example, the training data ID "T01" is associated with the dataset named "For slope collapse detection", described as "Images of slope collapse occurrences", and containing "200 images". The training data ID "T02" is associated with the dataset named "For false positive prevention", described as "Normal images (ground truth images)", and containing "50 images".

[0056] Figure 5 also shows information about two image data files (filenames "20230701090001_100_120.png" and "20230702150002_600_800.jpg") stored in the training data DB111 and associated with the training data ID "T01". The filenames of the image data stored as training data may include information indicating the location where the event occurred, such as "100_120" and "600_800" mentioned above. The learning unit 101 can create an image language model by training using at least one dataset stored in the training data DB111.

[0057] (2: Configuration of the pre-trained model DB112) Figure 6 is a data table showing an example of the configuration of a trained model DB 112 stored in the data storage unit 106 of the analysis device 10 according to the embodiment. As shown in Figure 6, the trained model DB 112 stores information such as "model ID", "model name", "training category", "creation date", "training data ID", and "description". The "model ID" column of the trained model DB 112 indicates identification information of the trained model. The "model name", "training category", "creation date", "training data ID", and "description" columns of the trained model DB 112 indicate basic information associated with the model ID.

[0058] The "Model Name" in the Trained Model DB112 indicates the name of the trained model. The "Training Category" in the Trained Model DB112 indicates whether it is initial training or additional training. Initial training corresponds to a trained model created without using a base trained model. Additional training corresponds to a trained model created by additionally training a base trained model. The "Creation Date" in the Trained Model DB112 indicates the creation date of the associated trained model. The "Teacher Data ID" in the Trained Model DB112 indicates the teacher data ID used in initial training or additional training. The "Description" in the Trained Model DB112 is additional information for the user's reference.

[0059] In this example, the model ID "M01" is associated with the model name "Base Model", learning category "Initial Learning", creation date "2024 / 7 / 26", training data ID "T01", and description "For Base Use". The model ID "M02" is associated with the model name "20240727 Updated Model", learning category "Additional Learning", creation date "2024 / 7 / 27", training data ID "T02", and description "Additional Learning for False Detection Prevention".

[0060] (3: Configuration of Camera Information DB113) Figure 7 is a data table showing an example of the configuration of the camera information DB 113 stored in the data storage unit 106 of the analysis device 10 according to the embodiment. As shown in Figure 7, the camera information DB 113 stores information such as "camera ID", "installation location name", "installation location", and "detection target". The "camera ID" column of the camera information DB 113 shows the identification information of each camera 31 included in the camera set 30. The "installation location name", "installation location", and "detection target" columns of the camera information DB 113 show basic information associated with the camera ID.

[0061] The “Installation Location Name” in Camera Information DB113 indicates the name of the installation location of camera 31. The “Installation Location” in Camera Information DB113 indicates the specific installation location information of camera 31. The installation location of camera 31 is indicated, for example, by a kilometer post (KP). However, it is not limited to this, and the installation location of camera 31 may be indicated by GPS (Global Positioning System) coordinates, etc. The “Detection Target” in Camera Information DB113 indicates keyword information for at least one assumed event that is to be detected.

[0062] In this example, the camera ID "C01" is associated with the installation location name "XXX_0KP", the installation location "0KP", and the detection target "(1) Slope collapse". The camera ID "C02" is associated with the installation location information "XXX_0.1KP", the installation location "0.1KP", and the detection target "(1) Slope collapse (2) Water accumulation due to muddy water". Thus, the installation location name may be a name that combines the installation location and KP.

[0063] (4: Prompt evaluation DB114 configuration) Figure 8 is a data table showing an example of the configuration of the prompt evaluation DB 114 stored in the data storage unit 106 of the analysis device 10 according to the embodiment. As shown in Figure 8, the prompt evaluation DB 114 stores information such as “evaluation data ID”, “type”, “dataset name”, “format”, “quantity”, “keyword”, and “description”. The “evaluation data ID” column of the prompt evaluation DB 114 indicates identification information of the prompt evaluation data. The “type”, “dataset name”, “format”, “quantity”, “keyword”, and “description” columns of the prompt evaluation DB 114 indicate basic information associated with the evaluation data ID.

[0064] The “Type” field in the Prompt Evaluation DB114 indicates the data type of the evaluation data, for example, whether it is an image or a video. The “Dataset Name” field in the Prompt Evaluation DB114 indicates the name of the dataset of evaluation data associated with the Prompt ID. The “Format” field in the Prompt Evaluation DB114 indicates the file extension of the data format of the associated evaluation data ID. The “Quantity” field in the Prompt Evaluation DB114 indicates the quantity of image data or the duration of the video included in the associated evaluation data ID. The “Keywords” field in the Prompt Evaluation DB114 indicates the keywords of the detected event associated with the evaluation data ID. The “Description” field in the Prompt Evaluation DB114 is additional information for the user's reference.

[0065] In this example, the evaluation data ID "E01" is associated with the following data: type "image", dataset name "slope collapse images", data format "PNG", quantity "50 images", and keyword "slope collapse". The evaluation data ID "E02" is associated with the following data: type "video", dataset name "flooded video", data format "MOV", quantity "1:20:07", and keyword "flooded due to muddy water".

[0066] Figure 8 also shows information about two image data files (filenames "202307010900001_100_120.png" and "202307021500020_600_800.png") stored in the prompt evaluation DB 114 and associated with evaluation data ID "E01". The filenames of the image data stored as evaluation data may include information indicating the location where the event occurred, such as "100_120" and "600_800" mentioned above. The prompt generation unit 104 can perform accuracy evaluation of prompts and prompt combinations by using at least one dataset stored in the prompt evaluation DB 114.

[0067] (5: Prompt DB115 configuration) Figure 9 is a data table showing an example of the configuration of the prompt DB 115 stored in the data storage unit 106 of the analysis device 10 according to the embodiment. As shown in Figure 9, the prompt DB 115 stores information such as “prompt ID”, “prompt name”, “creation date”, “number of prompts”, “accuracy”, “purpose”, and “usage date”. The “prompt ID” column of the prompt DB 115 indicates identification information for a prompt set containing at least one prompt. The “prompt name”, “creation date”, “number of prompts”, “accuracy”, “purpose”, and “usage date” of the prompt DB 115 indicate basic information associated with the prompt ID.

[0068] The “Prompt Name” in Prompt DB115 indicates the name of the prompt set associated with the prompt ID. The “Creation Date” in Prompt DB115 indicates the creation date of the associated prompt set. The “Number of Prompts” in Prompt DB115 indicates the number of prompts included in the prompt set. The “Accuracy” in Prompt DB115 indicates the result of the accuracy evaluation by the prompt generation unit 104. The “Purpose” in Prompt DB115 indicates the purpose of the prompt set. The “Usage Date” in Prompt DB115 indicates the usage date of the prompt set.

[0069] In this example, the prompt associated with prompt ID "P01" contains the prompt name "Collapse Detection Set", creation date "2024 / 7 / 26", number of prompts "5", accuracy "90%", and purpose "Slope Collapse Detection". The prompt associated with prompt ID "P02" contains the prompt name "Water Aquifer Detection Set", creation date "2024 / 7 / 28", number of prompts "3", accuracy "85%", and purpose "Water Aquifer Detection".

[0070] Figure 9 also shows information on two prompts (prompt content "Is a slope collapse occurring?" and "Is the slope collapsing?") stored in the prompt DB115 and associated with prompt ID "P01". In the accuracy evaluation by the prompt generation unit 104, an accuracy evaluation is performed for each of these prompts. In this example, the accuracy of "Is a slope collapse occurring?" is 95%, and the accuracy of "Is the slope collapsing?" is 70%. The accuracy of each prompt ID in the prompt DB115 can be calculated according to the accuracy of each associated prompt. Specifically, since the evaluation data includes images and information on the events and objects to be detected, it can be determined whether the prompt's response to the evaluation data is correct or not. For this reason, the correct response rate of each prompt for multiple evaluation data including the events to be detected can be used as the accuracy score.

[0071] (6: Configuration of Detection Results DB116) Figure 10 is a data table showing an example of the configuration of the detection result DB 116 stored in the data storage unit 106 of the analysis device 10 according to the embodiment. As shown in Figure 10, the detection result DB 116 includes information such as "detection ID", "applicable model ID", "camera ID", "prompt ID", "detection time", "detection prompt", "score", and "detection data", and can store an image corresponding to the detection result for each detection ID. The image corresponding to the detection result may be the original image or video at the time of detection, or it may be an image or video with the detection result superimposed on the original image or video. The "detection ID" column of the detection result DB 116 indicates identification information of the detection result. The "applicable model ID", "camera ID", "prompt ID", "detection time", "detection prompt", and "score" columns of the detection result DB 116 indicate basic information associated with the detection ID.

[0072] The “Applied Model ID” in the Detection Results DB116 indicates the identification information of the trained model used in the detection process. The “Camera ID” in the Detection Results DB116 indicates the identification information of the monitored camera 31. The “Prompt ID” in the Detection Results DB116 indicates the identification information of the prompt set used in the detection process. The “Detection Time” in the Detection Results DB116 indicates the time when the target was detected. The “Detection Prompt” in the Detection Results DB116 indicates the prompt used to detect the target from the image. The “Score” in the Detection Results DB116 indicates the score of the detection result. The “Detection Data” in the Detection Results DB116 indicates the file name of the detection result data.

[0073] In this example, the detection ID "D01" is associated with the applied model ID "M02", camera ID "C01", prompt ID "P01", detection time "2024 / 7 / 26 09:15:20", detection prompt "Has a slope collapse occurred?", score "0.9", and detection data "20240729091520_001_001.png". The detection ID "D02" is associated with the applied model ID "M01", camera ID "C02", prompt ID "P02", detection time "2024 / 7 / 27 14:35:50", detection prompt "Has the slope collapsed?", score "0.8", and detection data "20240802143550_002_001.png".

[0074] (7: Configuration of Alert Issuance History DB117) Figure 11 is a data table showing an example of the configuration of the alert history DB 117 stored in the data storage unit 106 of the analysis device 10 according to the embodiment. As shown in Figure 11, the alert history DB 117 includes information such as “Alert ID”, “Alert Date and Time”, “Detection ID”, “Recipient”, and “Alert Content”. The “Alert ID” column of the alert history DB 117 indicates identification information of the alert result. The “Alert Date and Time”, “Detection ID”, “Recipient”, and “Alert Content” columns of the alert history DB 117 indicate basic information associated with the alert ID.

[0075] The “Issuance Date and Time” in the Alert History DB117 indicates the date and time the alert was issued. The “Detection ID” in the Alert History DB117 indicates the detection ID associated with the alert ID. The “Recipient” in the Alert History DB117 indicates the recipient of the alert instruction. The “Issuance Details” in the Alert History DB117 indicates a description of the event that triggered the alert for that detection ID.

[0076] In this example, the following information is stored under the notification ID "A01": notification date and time "2024 / 7 / 26 09:15:20", detection ID "D01", recipient "Control Center", and notification content "The slope at 0KP has collapsed". The following information is stored under the notification ID "A02": notification date and time "2024 / 7 / 28 14:35:55", detection ID "D02", recipient "Control Center", and notification content "The slope at 0.1KP has collapsed".

[0077] <2> operation Next, the operation of the analysis system 1 according to the embodiment will be described. The operation of the analysis device 10 described below can be achieved by the processor 11 executing a program stored in the memory device 12.

[0078] <2-1> Overview of Question Answering AI Figure 12 is a schematic diagram showing an overview of the question-answering AI by the analysis device 10 according to this embodiment. As shown in Figure 12, a predetermined trained model is set in the detection unit 105. The prompt set used in this example includes four prompts Q1 to Q4. Each prompt Q used in the question-answering AI is written in question format.

[0079] When image data and a set of prompts are input to the detection unit 105, the detection unit 105 analyzes the image data using a trained model to recognize events and objects contained in the image data. Based on the analysis results, the detection unit 105 then creates answers for each of the four prompts Q1 to Q4.

[0080] In this example, the detection unit 105 outputs answers A1 to A4, which are associated with the four prompts Q1 to Q4, as detection results. Answers A1 to A4 include, for example, binary information (YES or NO) as answers to the associated prompts. In addition to answers to questions, answers to prompts may also include the position of a rectangle indicating the answer in the image, a description of the image, a score map indicating a region in the image, or an anomaly score map. The score map corresponds to an image colored pixel by pixel based on the detection result score (detection score). The anomaly score map corresponds to a portion (pixel, etc.) in the score map where the detection score exceeds a predetermined threshold. The anomaly score map may be displayed overlaid on the normal score map.

[0081] Thus, question-answering AI uses a general format of images and questions. Therefore, even when the events or objects to be detected are changed or added, the question-answering AI can flexibly respond simply by preparing questions that match the events or objects to be detected. The image language model used in question-answering AI can learn sets of images, questions, and answers, and can provide appropriate answers depending on the question.

[0082] <2-2> Method for generating an image language model Figure 13 is a flowchart illustrating an example of how to create an image language model using the analysis device 10 according to the embodiment. Below, an example of how to create an image language model using the analysis device 10 according to the embodiment will be described with reference to Figure 13.

[0083] After the user has prepared the training data for creating the image language model, the series of processes shown in Figure 13 begins (start).

[0084] First, the training data prepared by the user is pre-registered in the training data DB 111 of the data storage unit 106 (step S11).

[0085] Next, it is checked whether or not it is initial training (step S12). Whether or not it is initial training may be checked by whether or not a base pre-trained model is selected when creating the image language model.

[0086] If it is confirmed in step S12 that it is initial training (step S12: YES), training data is selected and a trained model is created (step S13). The selection of training data is performed, for example, by the user operating the terminal device 20 to access the analysis device 10. Then, the created trained model is saved in the trained model DB 112 (step S14). When the process in step S14 is completed, the series of processes in Figure 13 is finished (end).

[0087] If it is confirmed in step S12 that it is not initial training (step S12: NO), a base trained model is selected (step S15). The selection of the base trained model is performed, for example, by the user operating the terminal device 20 to access the analysis device 10. Then, training data is selected and a trained model is created by additional training on the base trained model (step S16). Then, the trained model created by additional training is saved in the trained model DB 112 (step S17). The trained model created by additional training may overwrite the base trained model or may be saved as a new model. When the process in step S17 is completed, the series of processes in Figure 13 is finished (end).

[0088] Figures 14 and 15 are examples of images used as training data, showing road RD and slope SL adjacent to road RD. Figure 14 corresponds to an example of a ground truth image. Figure 15 corresponds to an example of an image containing the object to be detected. In the ground truth image shown in Figure 14, slope SL has not collapsed. Thus, ground truth images are prepared that do not contain the event or object to be detected. By using ground truth images for training, false positives by the trained model can be suppressed. On the other hand, in the image shown in Figure 15, a part of slope SL has collapsed, and a landslide LS, which is the object to be detected by the image language model in this example, has occurred. Thus, images containing the object to be detected are prepared as training data. Keywords such as "slope collapse" are associated with these training data images.

[0089] <2-3> Analysis Processing Figure 16 is a flowchart showing an example of the analysis process performed by the analysis device 10 according to the embodiment. An example of the analysis process performed by the analysis device 10 according to the embodiment will be described below with reference to Figure 16.

[0090] When the user instructs the analysis device 10 to perform the analysis process, the series of processes shown in Figure 16 begins (start).

[0091] First, the model setting function of the setting unit 102 selects a trained model to be used in the detection process from the trained model DB 112 (step S21). In step S21, the trained model to be used can also be specified by the user by operating the terminal device 20 to access the analysis device 10.

[0092] Next, the keyword to be detected is entered into the detection event setting function of the setting unit 102 (step S22). Keyword input is performed by the user operating the terminal device 20 to access the analysis device 10. In the process of step S22, multiple keywords may be entered into the detection event setting function, text data may be entered, or image data may be entered. If text data is entered, at least one keyword is extracted from the text data. If image data is entered, at least one keyword is extracted from the image.

[0093] Next, a setup process is performed (step S23). The setup process involves selecting and configuring the camera 31 to be analyzed. Then, a prompt generation process is performed (step S24). The prompt generation process generates a prompt set containing at least one prompt based on the keyword entered in step S22. After that, a detection process is performed (step S25). The detection process is performed using the selected trained model, the set prompt set, and the input image, and is a process corresponding to the question-answering AI. Note that in the flowchart shown in Figure 16, the order of steps S23 and S24 may be reversed.

[0094] (1: Setup process) Figure 17 is a flowchart showing an example of the setting process by the analysis device 10 according to the embodiment. An example of the setting process by the analysis device 10 according to the embodiment will be described below with reference to Figure 17.

[0095] When the setting process in step S23 shown in Figure 16 starts (start), it is first checked whether or not it is a manual setting (step S231).

[0096] If the process in step S231 confirms that the setting is manual (step S231: YES), the user selects the camera 31 to be analyzed (monitored) from the camera information DB 113 (step S232). In the process in step S232, multiple cameras 31 may be selected, or a camera 31 may be selected based on its camera ID or installation location. Then, the camera setting function of the setting unit 102 is configured to acquire real-time video from the selected camera 31 (step S233). Specifically, the setting unit 102 changes the settings of the input unit 103 so that it transmits the real-time video acquired from the selected camera 31 to the detection unit 105. After that, the series of processes shown in Figure 17 are completed (end).

[0097] If the process in step S231 confirms that the setting is not manual, i.e., automatic (step S231: NO), the camera setting function of the setting unit 102 compares the entered keyword with the expected event in the camera information DB 113 and extracts the camera 31 in which the event is likely to occur (step S234). Multiple cameras 31 may be extracted in the process of step S234. Then, the camera setting function of the setting unit 102 sets the system to acquire real-time video from the extracted camera 31 (step S235). Specifically, the setting unit 102 automatically changes the settings of the input unit 103 so that it transmits the real-time video acquired from the camera 31 extracted in the process of step S234 to the detection unit 105. After that, the series of processes shown in Figure 17 are completed (end).

[0098] (2: Prompt generation process) Figure 18 is a schematic diagram showing an overview of the prompt generation process by the analysis device 10 according to the embodiment. As shown in Figure 18, in the prompt generation process, at least one keyword is input to the prompt generation unit 104. The prompt generation unit 104 then generates multiple prompts and performs accuracy evaluation of the generated prompts, and outputs a prompt set containing at least one prompt. As the prompt set output by the prompt generation unit 104, for example, the prompt set with the highest accuracy as a result of the accuracy evaluation is selected. In this example, the prompt generation unit 104 performs prompt generation and accuracy evaluation based on the keyword input, and as a result, outputs a prompt set containing four prompts Q1 to Q4.

[0099] Figure 19 is a flowchart showing an example of prompt generation processing by the analysis device 10 according to the embodiment. An example of prompt generation processing by the analysis device 10 according to the embodiment will be described below with reference to Figure 19.

[0100] When the setup process in step S24 shown in Figure 16 begins (start), the prompt generation unit 104 first generates multiple prompts based on the keywords entered in the process of step S22 (step S241). Specifically, the automatic prompt generation function of the prompt generation unit 104 automatically generates a large number of prompts using a generation AI or the like based on the keywords of the detected event.

[0101] Next, the prompt generation unit 104 evaluates the accuracy of the multiple generated prompts using the evaluation data from the prompt evaluation DB 114 (step S242). At this time, the prompt selection function of the prompt generation unit 104 calculates the accuracy of the prompts and the combination of prompts using the evaluation data from the prompt evaluation DB 114.

[0102] Next, the prompt generation unit 104 extracts the most accurate prompts and prompt combinations from the multiple prompts generated (step S243). Then, the prompt generation unit 104 automatically sets the extracted prompts and prompt combinations in the setting unit 102 (step S244). After that, the prompt generation unit 104 saves the information of the set prompts in the prompt DB 115 (step S245). When the process in step S245 is completed, the series of processes shown in Figure 19 is completed (end).

[0103] (3: Detection process) Figure 20 is a flowchart showing an example of the detection process by the analysis device 10 according to the embodiment. An example of the detection process by the analysis device 10 according to the embodiment will be described below with reference to Figure 20.

[0104] When the detection process in step S25 shown in Figure 16 begins (start), the frame division function of the detection unit 105 first divides the camera video (real-time video) or video file acquired from the input unit 103 into frame units (step S251). Note that if image data is input from the input unit 103, or if the image data stored in the data storage unit 106 is the target of analysis, the processing in step S251 may be omitted.

[0105] Next, the frame division function of the detection unit 105 transmits one of the divided frames and at least one set prompt (prompt set) to the AI ​​detection function of the detection unit 105 (step S252).

[0106] Next, the AI ​​detection function of the detection unit 105 detects events and objects within the frame based on the selected trained model and at least one set prompt (prompt set) (step S253).

[0107] The AI ​​detection function of the detection unit 105 then checks whether or not the frame contains the event and object to be detected (step S254). If it is confirmed in step S254 that the frame contains the event and object to be detected (step S254: YES), the AI ​​detection function of the detection unit 105 saves the detection result to the detection result DB 116 (step S255) and proceeds to step S256. If it is confirmed in step S254 that the frame does not contain the event and object to be detected (step S254: NO), the AI ​​detection function of the detection unit 105 proceeds to step S256.

[0108] In step S256, the AI ​​detection function of the detection unit 105 checks whether the processing of all divided frames has been completed. These "all divided frames" correspond to the image data divided into frames in step S251. If it is confirmed in step S256 that the processing of all divided frames has not been completed (step S256: NO), the detection unit 105 selects the next frame (step S257) and proceeds to step S252. That is, the detection unit 105 performs the question-answering AI processing on the next frame image. If it is confirmed in step S256 that the processing of all divided frames has been completed (step S256: YES), the detection unit 105 terminates the series of processes shown in Figure 20. Note that if the processing of step S251 is omitted, the processing of steps S256 and S257 may also be omitted.

[0109] <2-4> How to use the detection results Figure 21 is a flowchart illustrating an example of how to utilize the detection results from the analysis device 10 according to this embodiment. One possible method of utilizing the detection results from the analysis device 10 is to use the alarm device 40 to notify the user of the danger when the score of the detected event and object exceeds a predetermined value (for example, when a dangerous condition is detected). Below, an example of how to utilize the detection results from the analysis device 10 according to this embodiment will be described with reference to Figure 21.

[0110] For example, when the detection result DB116 is updated, the series of processes shown in Figure 21 are executed (start). First, the detection unit 105 or the data storage unit 106 sends the updated detection result from the detection result DB116 to the notification unit 107 (step S31). Then, the detection result determination function of the notification unit 107 compares the score of the updated detection result with a predetermined threshold (step S32). This predetermined threshold can be changed as appropriate depending on how the score is set.

[0111] Next, the detection result determination function of the notification unit 107 determines whether or not to issue an alert based on the result of the processing in step S32 (step S33). If an alert is not issued (step S33: NO), the notification unit 107 terminates the series of processes shown in Figure 21 (termination). If an alert is issued (step S33: YES), the notification function of the notification unit 107 instructs the alarm device 40 to issue an alert (step S34) and saves the content of the alert in the alert history DB 117 (step S35). If the analysis system 1 is configured to have multiple alarm devices 40, the notification unit 107 instructs the alarm to be issued to the alarm destination set for each camera 31, for example. Once the processing in step S35 is completed, the notification unit 107 terminates the series of processes shown in Figure 21 (termination).

[0112] The setting unit 102 may be configured to allow setting of expected responses to prompts. In this case, the notification unit 107 can determine whether or not a target has been detected based on the expected response to the prompt.

[0113] <3> Effects of the Embodiment The analysis device 10 in the analysis system 1 according to this embodiment includes a question-answering AI that receives an image and a prompt as input and can obtain an answer to the prompt related to the image. The question-answering AI of this embodiment can also answer even when multiple prompts are set. For this reason, in the analysis device 10, prompts for capturing various events and objects to be detected in advance are set in the monitoring operation, and the prompts can be changed according to the situation.

[0114] As a result, the analysis device 10 according to the embodiment can detect events and objects in response to changes in each environment, even when the same image language model is applied to analysis processing for multiple cameras 31. Consequently, the analysis device 10 according to the embodiment eliminates the need to adjust the AI ​​model for each of the numerous cameras 31, contributing to the advancement and efficiency of monitoring operations in various fields. Therefore, the analysis device 10 according to the embodiment can be realized at low cost while having high detection accuracy.

[0115] Furthermore, the analysis device 10 according to the embodiment can create a prompt set that includes at least one prompt based on a keyword. By automatically generating prompts in this way, the detection accuracy of events and objects can be improved. In addition, since events and objects related to the keyword are automatically generated, various events and objects that can be expected from the keyword being detected can be additionally detected. As a result, the analysis device 10 according to the embodiment can further improve the safety of the monitored object.

[0116] Furthermore, the analysis device 10 can also obtain the following effects (1) to (8). (1) The analysis device 10 acquires real-time video from the camera 31 and performs analysis processing, thereby enabling real-time AI monitoring of surveillance camera footage. (2) The analysis device 10 can obtain an answer simply by prompting a question, and learning only requires training the combination of questions and images. Therefore, the analysis device 10 can easily improve its accuracy. (3) The analysis device 10 can not only monitor in real time, but also retrospectively review recorded video footage to confirm events. (4) The analysis device 10 can automatically set the camera 31 and prompt set by inputting only the keyword of the event to be detected. Therefore, the analysis device 10 can reduce the effort required for monitoring. (5) The analysis device 10 can broadly monitor various events and objects by generating a set of prompts using the prompt generation unit 104. (6) The analysis device 10 can determine whether the result is abnormal or normal based on the difference from the expected response. Therefore, the analysis device 10 can reduce the effort required for monitoring. (7) The analysis device 10 does not require manual work such as listing candidate prompts or evaluating and selecting their accuracy. The analysis device 10 automatically lists candidate prompts using the generation AI and performs accuracy evaluation using evaluation data registered in the prompt evaluation data DB 114. As a result, the analysis device 10 can easily extract effective prompts. (8) The analysis device 10 can avoid notifying the same event multiple times by referring to past information recorded in the detection result DB 116 and the alert issuance history DB 117.

[0117] <4> Variations, etc. In this embodiment, the use of the analysis system 1 on roads was illustrated as an example, but it is not limited to this. The analysis system 1 can also be applied to safety monitoring, such as the wearing of helmets and protective boots, at production sites. By applying the analysis system 1 to safety monitoring at production sites, it is possible to achieve both improved safety at the site and reduced workload for supervisors. Furthermore, the question-answering AI function of the analysis device 10 can also be used to search for specific scenes from broadcast content or surveillance camera footage.

[0118] In this embodiment, an example was given in which the analysis device 10 can create an image language model to be used as a question-answering AI, but it is not limited to this. The analysis device 10 may also acquire a trained model from an external source. In this case, the learning unit 101 may be omitted from the analysis device 10.

[0119] The analysis device 10 may utilize an external generative AI that functions as a question-answering AI using an image language model in the detection process. In this case, the analysis device 10 is configured to input, for example, the image to be detected and a set of prompts (questions) to the external generative AI, and to obtain answers to the set of prompts (questions) for the input image from the external generative AI. Specifically, in the process of step S252 of the flowchart shown in Figure 20, the destination of the transmission of one of the divided frames and at least one set prompt is changed to the external generative AI. Then, the process of step S253 is executed by the external generative AI, and the analysis device 10 receives answers to the prompts from the external generative AI. Other operations are the same as in the above embodiment. When the analysis device 10 performs detection processing using an external generative AI, the learning unit 101 and the detection unit 105 may be omitted from the analysis device 10.

[0120] In the embodiments, the flowchart used to describe the operation is merely an example. The operation described using the flowchart may have its order of processes changed, additional processes added, some processes omitted, or some processes executed in parallel, to the extent possible. Each of the processes described in the above embodiments may be implemented by dedicated hardware. The processes described in the above embodiments may be a mixture of processes executed by software and processes executed by hardware, or may consist of only one or the other.

[0121] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0122] 1…Analysis System 10…Analysis device 11… Processor 12…Memory device 13…Storage device 14...Display device 15…Input Interface 16…Communication module 20…Terminal device 30...Camera set 31... Camera 40… Alarm device 101...Learning Department 102...Settings section 103...Input section 104... Prompt generation unit 105...Detection unit 106...Data storage unit 107…Notification department 111...Teacher data database 112... Pre-trained model database 113...Camera Information Database 114…Prompt Evaluation DB 115... Prompt DB 116...Detection Result DB 117... Alert Issuance History Database

Claims

1. An input unit capable of acquiring at least one image, each associated with identification information, A setting unit sets a prompt set that includes at least one prompt for each piece of identification information, and sets a pre-trained image language model common to the identification information. An analysis device comprising: a detection unit that inputs a first image associated with a first identification information from among the at least one of the aforementioned images, and a first prompt set set for the first identification information, to the image language model, and outputs a response of the first prompt set for the first image.

2. The system further comprises a prompt generation unit that generates the prompt set based on at least one keyword relating to the event or object to be detected. The analysis apparatus according to claim 1.

3. The prompt generation unit generates a plurality of prompts based on the at least one keyword, performs an accuracy evaluation of each of the plurality of prompts and an accuracy evaluation of the combination of prompts in the plurality of prompts, and sets the at least one prompt extracted based on the evaluation results as the prompt set. The analysis apparatus according to claim 2.

4. The input unit is further configured to acquire images and information from each of a plurality of cameras, each associated with different identification information. The setting unit compares at least one keyword relating to the event or object to be detected with the information, and based on the comparison result, sets at least one camera from the plurality of cameras to be used for detection. The analysis apparatus according to claim 1.

5. The system further includes a notification unit configured to determine whether or not a target has been detected based on the aforementioned response, and to output a notification based on the determination result. The analysis apparatus according to claim 1.

6. The setting unit can set the expected response to the prompt, The notification unit determines whether or not the target has been detected based on the expected response. The analysis apparatus according to claim 5.

7. The aforementioned image language model is a question-answering AI. The analysis apparatus according to claim 1.

8. The aforementioned input unit is configured to acquire video files, The detection unit divides the video file into frames and inputs the images obtained by adding identification information to the divided frames into the image language model. The analysis apparatus according to claim 1.

9. The input unit is configured to acquire image files or video files. The detection unit inputs the image file or the video file into the image language model. The analysis apparatus according to claim 1.

10. The system further comprises a data storage unit configured to store the aforementioned image, the prompt set, the image language model, and the response. The analysis apparatus according to claim 1.

11. The system further includes a notification unit configured to determine whether or not a target has been detected based on the aforementioned response, and to output a notification based on the determination result. The data storage unit stores past judgment results, The notification unit determines whether or not to output a notification based on the determination result and the past determination results. The analysis apparatus according to claim 10.

12. On the computer, Obtain at least one image, each associated with identification information, A prompt set is set, which includes at least one prompt for each of the aforementioned identification pieces of information. Setting a common pre-trained image language model for the aforementioned identification information, The image language model is input to a first image associated with a first identification information from among the at least one of the aforementioned images, and a first prompt set set for the first identification information, and the response of the first prompt set to the first image is output. A program that executes something.

Images