System and method for registering targets for detection.

Abstract

This invention provides a detection target registration system that can further improve the accuracy of a detection system that uses generation AI or the like to determine whether or not an image contains a target for detection. [Solution] The detection target registration system is a detection target registration system that registers instruction information to a detection system that uses a first generation AI, etc., which generates and outputs a detection result when an image and instruction information for generating a detection result indicating whether or not a detection target is included in the image are input, and comprises: a reception unit that receives verification text indicating a detection target from a user; a first acquisition unit that inputs information based on the verification text to a second generation AI, etc., which generates and outputs an image based on the input text and acquires the output image as a verification image; and a registration unit that registers instruction information reflecting the received verification text to the detection system based on the comparison result between the verification text and an explanatory text that explains the verification image.

Description

Technical Field

[0001] The present invention relates to a detection target registration system and a detection target registration method.

Background Art

[0002] In recent years, detection systems that use generative AI or AI agents to determine whether a detection target is included in an image have been developed.

[0003] Patent Document 1 discloses an event determination device that determines the presence or absence of an event in moving image data. This event determination device specifies, on an input screen, the moving image for which determination is to be performed, the event to be detected, and the text moving image classification model to be used. The event determination device creates keywords related to the detection event, calculates the existence probability of the keywords in the moving image to obtain time-series probability data, and analyzes this time-series probability data using a time-series analysis model that has been trained with a moving image reproducing the detection event and the keywords.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0005] In detection systems that use generative AI or AI agents to determine whether a detection target is included in an image, there is a demand for further improving determination accuracy.

[0006] An object of the present invention is to provide a detection target registration system and a detection target registration method capable of further improving the determination accuracy in a detection system that uses generative AI or AI agents to determine whether a detection target is included in an image.

Means for Solving the Problems

[0007] To solve the above problems, the present invention provides a detection target registration system for registering instruction information to a detection system that uses a first generating AI or AI agent that generates and outputs a detection result when an image and instruction information for generating a detection result indicating whether or not a detection target is included in the image are input, the system comprising: a reception unit that receives verification text indicating a detection target from a user; a first acquisition unit that inputs information based on the verification text to a second generating AI or AI agent that generates and outputs an image based on the input text and acquires the output image as a verification image; a second acquisition unit that inputs the verification image to a third generating AI or AI agent that generates and outputs text explaining the input image and acquires the output text as an explanation text; a comparison unit that compares the verification text and the explanation text; and a registration unit that registers instruction information reflecting the received verification text to the detection system based on the comparison result of the verification text and the explanation text.

[0008] In this detection target registration system, it is preferable that the image encoder of the first generating AI or AI agent and the text encoder of the second generating AI or AI agent are trained so that the features generated by the image encoder and the features generated by the text encoder are correlated.

[0009] In this detection target registration system, it is preferable that the image encoder of the first generating AI or AI agent and the text encoder of the second generating AI or AI agent are trained using the same training data.

[0010] In this detection target registration system, it is preferable for the detection system to input an input image of the monitoring area to a first generating AI or AI agent, and to determine the presence or absence of a detection target in the monitoring area based on the output detection result.

[0011] In this detection target registration system, it is preferable for the first acquisition unit to input the verification text to a fourth generating AI or AI agent that outputs an image generation prompt for generating an image based on the input text, and then input the outputted image generation prompt to a second generating AI or AI agent to acquire the verification image.

[0012] In this detection target registration system, the comparison unit preferably inputs the verification text and the explanatory text to a fifth generating AI or AI agent that outputs a comparison result of comparing the two input texts, and then compares the verification text and the explanatory text based on the output comparison result.

[0013] In this detection target registration system, it is preferable that the first acquisition unit acquires multiple verification images from a single verification text, the second acquisition unit acquires explanatory text for each of the multiple verification images, and the comparison unit compares the verification text with common elements or feature elements of the explanatory text for each of the multiple verification images.

[0014] In this detection target registration system, it is preferable that the first acquisition unit acquires multiple verification images from a single verification text, the second acquisition unit acquires explanatory text for each of the multiple verification images, and the registration unit notifies the user of a summary of the explanatory text for each of the multiple verification images.

[0015] To solve these problems, the present invention provides a detection target registration method for registering instruction information to a detection system that uses a first generating AI or AI agent that generates and outputs a detection result when an image and instruction information for generating a detection result indicating whether or not a detection target is included in the image are input, the method comprising: receiving verification text indicating a detection target from a user, inputting information based on the verification text to a second generating AI or AI agent that generates and outputs an image based on the input text, acquiring the output image as a verification image, inputting the verification image to a third generating AI or AI agent that generates and outputs text explaining the input image, acquiring the output text as an explanation text, comparing the verification text and the explanation text, and registering instruction information reflecting the received verification text in the detection system based on the comparison result of the verification text and the explanation text. [Effects of the Invention]

[0016] The detection target registration system and detection target registration method according to the present invention make it possible to further improve the determination accuracy in a detection system that uses a generating AI or AI agent to determine whether or not an image contains a detection target. [Brief explanation of the drawing]

[0017] [Figure 1] This diagram shows the overall system configuration of monitoring system 1. [Figure 2] This flowchart shows an example of how the detection process works. [Figure 3] This is a schematic diagram illustrating the data flow in monitoring system 1. [Figure 4] This flowchart shows an example of how the registration process works. [Figure 5] (A) shows an example of a prompt generation prompt, (B) shows an example of a response text, and (C) shows an example of an image generation prompt. [Figure 6] (A) shows an example of an explanatory prompt, and (B) to (E) show examples of explanatory text. [Figure 7] (A) to (D) show an example of explanatory text. [Figure 8] (A) shows an example of a comparison prompt, and (B) shows an example of a comparison result. [Figure 9] (A) shows an example of a first comparison prompt, (B) shows an example of a first comparison result, (C) shows an example of a second comparison prompt, and (D) shows an example of a second comparison result. [Figure 10] (A) shows an example of an element prompt, (B) shows an example of element text, (C) shows an example of a correction prompt, and (D) shows an example of corrected text. [Mode for Carrying Out the Invention]

[0018] Hereinafter, a monitoring system according to an embodiment will be described with reference to the drawings.

[0019] Figure 1 is a diagram showing the overall system configuration of a monitoring system 1. As shown in Figure 1, the monitoring system 1 includes a detection system 2 and a detection target registration system 3. The detection system 2 includes an imaging device T, a detection device 10 and the like, and the detection target registration system 3 includes a registration device 20 and the like. The monitoring system 1 further includes a server device 30 and the like. The number of each of the imaging device T, the detection device 10, the registration device 20 and the server device 30 is not limited to one, and may be plural. The imaging device T, the detection device 10, the registration device 20 and the server device 30 are connected to be communicable with each other via a network N. The network N is a wide area communication network such as an intranet or the Internet. The imaging device T is installed at a monitoring target such as a house, a store, an office, a commercial facility, or a factory, and images one or a plurality of monitoring areas included in the monitoring target. The detection device 10 is installed at a monitoring console or the like of a monitoring center located inside or outside the monitoring area, and aggregates and manages images captured by the imaging device T. The registration device 20 is installed at a monitoring console or the like of a monitoring center, and registers information in the detection system 2. The server device 30 is installed in a monitoring center or similar location. The server device 30 may also be a cloud server on a wide-area communication network such as the Internet.

[0020] The imaging device T comprises an imaging unit and a communication unit. The imaging unit includes, for example, a photoelectric conversion element such as a CCD element or a C-MOS element, an imaging optical system that forms an image on the photoelectric conversion element, and an A / D converter that amplifies the electrical signal output from the photoelectric conversion element and performs analog-to-digital (A / D) conversion. The imaging unit is positioned in the monitoring area included in the object to be monitored and sequentially generates input images of the monitoring area at a predetermined frame period. The communication unit has a communication interface circuit conforming to wired / wireless communication standards such as Ethernet® and IEEE802.11, and communicates with the detection device 10 via the network N to transmit the input images.

[0021] The detection device 10 detects the presence or absence of a detection target in the monitoring area based on the input image generated by the imaging device T. The detection target is an event (abnormal event) such as the intrusion of a suspicious person, a fire, or a person collapsing. The detection target may also be an object such as a suspicious person or a suspicious object. The detection device 10 has a first input unit 11, a first output unit 12, a first communication unit 13, a first storage unit 14, and a first control unit 15.

[0022] The first input unit 11 has an interface circuit that receives signals from an operating device such as a keyboard, mouse, or touch panel, and accepts operations from the user, and outputs a signal corresponding to the accepted operation to the first control unit 15.

[0023] The first output unit 12 has a display device such as a liquid crystal display or an organic EL display and an interface circuit that outputs images to the display device, and displays various information such as images and text according to instructions from the first control unit 15. The first output unit 12 also has an audio output device such as a speaker and an interface circuit that outputs audio to the audio output device, and outputs audio according to instructions from the first control unit 15.

[0024] The first communication unit 13 has a communication interface circuit conforming to wired / wireless communication standards such as Ethernet (registered trademark) and IEEE 802.11, and communicates with the imaging device T, registration device 20, and server device 30 via the network N to send and receive various information.

[0025] The first storage unit 14 includes semiconductor memory, magnetic storage media, and / or optical storage media. The first storage unit 14 stores the code and various data of a computer program executed by the first control unit 15 to control the operation of the detection device 10. The computer program is installed in the first storage unit 14 by known methods via a computer-readable storage medium such as a CD-ROM or DVD-ROM, or via a communication line. The computer program may also be distributed from a server or the like and installed in the first storage unit 14.

[0026] The first control unit 15 has a processor such as a CPU or multiprocessor and its peripheral circuits, and the processor controls the operation of the detection device 10 by executing a computer program stored in the first storage unit 14. A DSP, LSI, ASIC, FPGA, etc. may be used as the first control unit 15. The first control unit 15 has a detection unit 151, etc., as a functional module of the program that runs on the processor. The first control unit 15 periodically receives input images from the imaging device T and detects the presence or absence of a detection target in the monitoring area based on the received input images. The detection device 10 may have an imaging unit similar to the imaging unit of the imaging device T, and the first control unit 15 may periodically receive input images from the imaging unit.

[0027] The registration device 20 registers information about the object to be detected with the detection device 10. The registration device 20 includes a second input unit 21, a second output unit 22, a second communication unit 23, a second storage unit 24, and a second control unit 25.

[0028] The second input unit 21 has an interface circuit that receives signals from an operating device such as a keyboard, mouse, or touch panel, and accepts operations from the user, and outputs a signal corresponding to the accepted operation to the second control unit 25.

[0029] The second output unit 22 has a display device such as a liquid crystal display or an organic EL display and an interface circuit that outputs images to the display device, and displays various information such as images and text according to instructions from the second control unit 25. The second output unit 22 also has an audio output device such as a speaker and an interface circuit that outputs audio to the audio output device, and outputs audio according to instructions from the second control unit 25.

[0030] The second communication unit 23 has a communication interface circuit conforming to wired / wireless communication standards such as Ethernet (registered trademark) and IEEE 802.11, and communicates with the detection device 10 and the server device 30 via the network N to send and receive various information.

[0031] The second storage unit 24 includes semiconductor memory, magnetic storage media, and / or optical storage media. The second storage unit 24 stores the code and various data of the computer program executed by the second control unit 25 to control the operation of the registration device 20. The computer program is installed in the second storage unit 24 by known methods via a computer-readable storage medium such as a CD-ROM or DVD-ROM, or via a communication line. The computer program may also be distributed from a server or the like and installed in the second storage unit 24.

[0032] The second control unit 25 includes a processor such as a CPU or multiprocessor and its peripheral circuits, and the processor controls the operation of the registration device 20 by executing a computer program stored in the second storage unit 24. A DSP, LSI, ASIC, FPGA, etc. may be used as the second control unit 25. The second control unit 25 includes a reception unit 251, a first acquisition unit 252, a second acquisition unit 253, a comparison unit 254, and a registration unit 255 as functional modules of the program that runs on the processor.

[0033] The server device 30 stores one or more types of generating AI or AI agents, and generates images or text using the generating AI or AI agents in accordance with requests from the detection device 10 or registration device 20, and transmits them to the detection device 10 or registration device 20. The server device 30 includes a third input unit 31, a third output unit 32, a third communication unit 33, a third storage unit 34, and a third control unit 35.

[0034] The third input unit 31 has an interface circuit that receives signals from an operating device such as a keyboard, mouse, or touch panel, and accepts operations from the user, and outputs a signal corresponding to the accepted operation to the third control unit 35.

[0035] The third output unit 32 has a display device such as a liquid crystal display or an organic EL display and an interface circuit for outputting images to the display device, and displays various information such as images and text according to instructions from the third control unit 35. The third output unit 32 also has an audio output device such as a speaker and an interface circuit for outputting audio to the audio output device, and outputs audio according to instructions from the third control unit 35.

[0036] The third communication unit 33 has a communication interface circuit conforming to wired / wireless communication standards such as Ethernet (registered trademark) and IEEE 802.11, and communicates with the detection device 10 and the registration device 20 via the network N to send and receive various information.

[0037] The third storage unit 34 includes semiconductor memory, magnetic storage media, and / or optical storage media. The third storage unit 34 stores the code and various data of the computer program executed by the third control unit 35 to control the operation of the server device 30. The computer program is installed in the third storage unit 34 by known methods via a computer-readable storage medium such as a CD-ROM or DVD-ROM, or via a communication line. The computer program may also be distributed from a server or the like and installed in the third storage unit 34.

[0038] The third memory unit 34 stores the first model 341, the second model 342, the third model 343, the fourth model 344, the fifth model 345, the sixth model 346, and the seventh model 347, etc. Each model is called a so-called generative AI (Artificial Intelligence) or learning model. Each model is pre-trained to generate and output information according to the input information. The first model 341 is an example of the first generative AI, the third model 343 is an example of the second generative AI, and the fourth model 344 is an example of the third generative AI. Furthermore, the second model 342 is an example of the fourth generative AI, the fifth model 345 is an example of the fifth generative AI, and the sixth model 346 is an example of the sixth generative AI.

[0039] The first model 341 and the fourth model 344 are Vision Language Models (VLMs) that generate and output information corresponding to the input image and natural language (text) when an image and natural language (text) are input. The first model 341 and the fourth model 344 generate and output text that describes the context (event, situation, content, object, etc.) depicted in the input image. The first model 341 and the fourth model 344 may be the same model or different models. That is, the first model 341 and the fourth model 344 may be configured to include a common VLM that generates and outputs text that describes the input image. Examples of the first model 341 and the fourth model 344 include LLaVA (Large Language and Vision Assistant), GPT (Generative Pre-trained Transformer)-4v, GPT-4o, etc. LLaVA is a VLM (Vertical Language Model) that combines Llama2, a Large Language Model (LLM), with CLIP (Contrastive Language-Image Pre-Training), which includes an image encoder and a text encoder. LLaVA takes an image and a question about that image in natural language and outputs an answer that matches the image and the question. LLaVA converts the image into a feature vector using CLIP's image encoder, converts that feature vector into a format that can be input to Llama2 (projection), and converts the natural language into a feature vector using a tokenizer (embedding). LLaVA combines the feature vector converted from the image and the feature vector converted from the natural language and inputs them into Llama2, outputting an answer that matches the image and the question.

[0040] Models 342 (2nd), 345 (5th), 346 (6th), and 347 (7th) are Large-Scale Language Models (LLMs) that generate and output information corresponding to the natural language input. Models 342 (2nd), 345 (5th), 346 (6th), and 347 (7th) generate and output answers according to the input instructions (prompts). Models 342 (2nd), 345 (5th), 346 (6th), and 347 (7th) may be the same model or different models. Examples of Models 342 (2nd), 345 (5th), 346 (6th), and 347 (7th) include Llama2, Vicuna, Mistral, and BERT (Bidirectional Encoder Representations from Transformers).

[0041] The third model 343 is an image generation AI that generates and outputs images corresponding to natural language input. The third model 343 generates and outputs images that match the input instructions, i.e., images that represent the object and / or state specified by the input instructions. The third model 343 is, for example, Stable Diffusion. Stable Diffusion is an image generation AI that combines CLIP, which includes an image encoder and a text encoder, with a Diffusion Model, which is an image generation AI model. Stable Diffusion converts the input natural language into a feature vector using CLIP's text encoder, inputs that feature vector into the Diffusion Model, and outputs an image that matches that natural language.

[0042] The same image encoder and text encoder included in CLIP are used as the image encoder included in the VLM of the first model 341, and as the text encoder included in the image generation AI of the third model 343. As mentioned above, CLIP includes an image encoder that converts images into feature vectors and a text encoder that converts text into feature vectors. CLIP is pre-trained using training data that includes many combinations of images and text describing those images, so that the feature vectors of the images and the feature vectors of the text in each combination are close to each other.

[0043] CLIP calculates a score indicating the similarity between the image feature vector and the text feature vector for each combination of image and text feature vectors. In other words, CLIP calculates scores such that combinations where the text better describes the image receive higher scores, and combinations where the text does not better describe the image receive lower scores. Thus, the image encoder and text encoder included in the same CLIP are trained to be particularly similar to each other, so that the feature vectors (features) generated by the image encoder and the feature vectors (features) generated by the text encoder correlate. Therefore, the image encoder and text encoder can assign common meaning to corresponding images and texts, and can have a common understanding of them.

[0044] Furthermore, the image encoder and text encoder included in the same CLIP are trained using the same training data. That is, the image encoder is trained on images corresponding to text that the text encoder has already trained on, and not on images corresponding to text that the text encoder has not yet trained on. Therefore, the image encoder can understand images corresponding to text that the text encoder understands, and not on images corresponding to text that the text encoder does not understand. Similarly, the text encoder is trained on text corresponding to images that the image encoder has already trained on, and not on text corresponding to images that the image encoder has not yet trained on. Therefore, the text encoder can understand text corresponding to images that the image encoder understands, and not on text corresponding to images that the image encoder does not understand. Consequently, the level of visual knowledge of the image encoder and the level of vocabulary of the text encoder are almost identical.

[0045] In other words, within the same CLIP, the concepts that an image encoder can recognize from an image—that is, the concepts that can translate an image into language—are at the same level as the concepts that a text encoder can recognize from text—that is, the concepts that can convert text into images.

[0046] The VLM in the first model 341 is pre-trained to convert the feature vectors transformed from an image using CLIP's image encoder into feature vectors that can be handled by Llama2, and then generate text corresponding to those feature vectors. Similarly, the Diffusion Model included in the third model 343 is pre-trained to generate images corresponding to the feature vectors transformed from text using CLIP's text encoder. Therefore, the first model 341 and the third model 343 share a common understanding and possess the same level of knowledge through CLIP, and the concepts that the first model 341 can verbalize and the concepts that the third model 343 can visualize are at the same level.

[0047] Furthermore, the image encoder in the first model 341 and the text encoder in the third model 343 may be image encoders and text encoders from any model other than CLIP that includes both image encoders and text encoders. Also, the image encoder in the first model 341 and the text encoder in the third model 343 may be pre-trained at different times, as long as they are pre-trained using the same training data. Furthermore, the image encoder in the first model 341 and the text encoder in the third model 343 may be image encoders and text encoders included in different models, as long as they are pre-trained using the same training data. In addition, the image encoder in the first model 341 and the text encoder in the third model 343 do not need to be trained so that the features generated by each encoder are correlated. Furthermore, the image encoder in the first model 341 and the text encoder in the third model 343 do not need to be trained using the same training data. The number of existing image and text combinations is finite, and the training data will be depleted as training is repeated. Therefore, even between image encoders and text encoders included in different CLIPs, the amount of knowledge each of the image encoder and text encoders will gradually converge.

[0048] The third control unit 35 includes a processor such as a CPU or multiprocessor and its peripheral circuits. The processor controls the operation of the server device 30 by executing a computer program stored in the third storage unit 34. A DSP, LSI, ASIC, FPGA, or the like may be used as the third control unit 35.

[0049] Figure 2 is a flowchart illustrating an example of the operation of the detection process by the detection device 10. This flowchart is executed primarily by the first control unit 15 in cooperation with each element of the detection device 10, based on a program pre-stored in the first storage unit 14.

[0050] First, the detection unit 151 acquires an input image (step S101). The detection unit 151 acquires the input image by sending an input image request signal to the imaging device T via the first communication unit 13 and receiving the input image from the imaging device T via the first communication unit 13. The imaging device T may also spontaneously send an input image to the detection device 10 each time it generates one. Alternatively, the detection unit 151 may acquire an input image from the imaging unit of the detection device 10. The input image may be a still image or a video.

[0051] Next, the detection unit 151 uses the first model 341 to obtain a judgment text (step S102). The judgment text is text indicating whether or not the input image contains a detection target. The judgment text is an example of a detection result. Note that the detection result may be in the form of information other than text, such as an image or binary format, and this information may be obtained from the first model 341 which generates such information.

[0052] Figure 3 is a schematic diagram illustrating the data flow in the monitoring system 1. As shown in Figure 3, the detection unit 151 inputs the input image and judgment prompt to the first model 341, which is a VLM, and obtains the judgment text output from the first model 341. The judgment prompt is an example of instruction information and is text containing a command to the first model 341 to generate judgment text from the input image. The judgment prompt includes the detection target and an instruction to determine whether or not the input image contains that detection target. For example, the judgment prompt may say, "Determine whether the input image contains 'falling beside the bed,' and output in the format ○ if it contains it, and × if it does not."

[0053] The detection unit 151 sends a judgment request signal to the server device 30 via the first communication unit 13 to input the input image and judgment prompt to the first model 341 and cause the first model 341 to generate judgment text. The judgment request signal includes the input image and the judgment prompt. When the third control unit 35 of the server device 30 receives the judgment request signal from the detection device 10 via the third communication unit 33, it inputs the input image and judgment prompt included in the received judgment request signal to the first model 341 and causes the first model 341 to generate judgment text. The first model 341 generates and outputs judgment text when it receives an image and a judgment prompt to generate judgment text indicating whether or not the image contains a detection target. The third control unit 35 sends the generated judgment text to the detection device 10 via the third communication unit 33. The detection unit 151 acquires the judgment text by receiving it from the server device 30 via the first communication unit 13. In this manner, the detection unit 151 inputs the input image and judgment prompt to the first model 341, causing the first model 341 to generate judgment text.

[0054] Next, the detection unit 151 determines whether or not a detection target exists (is occurring) in the monitoring area (step S103). If the acquired determination text is text that indicates that the input image contains a detection target, the detection unit 151 determines that a detection target exists (is occurring) in the monitoring area. For example, if the determination prompt is "Determine whether the input image contains 'falling beside the bed', and output in the format ○ if it does, and × if it does not," and a determination text indicating a positive content such as "○" is output, the detection unit 151 determines that a detection target exists (is occurring) in the monitoring area. On the other hand, if the acquired determination text is text that indicates that the input image does not contain a detection target, the detection unit 151 determines that there is no detection target (is not occurring) in the monitoring area. For example, if the judgment prompt is "Determine whether the input image contains 'falling beside the bed,' and output in the format ○ if it contains it, and × if it does not," and a judgment text indicating a negative, such as "×," is output, the detection unit 151 determines that there is no detection target (no occurrence) in the monitoring area. In this way, the detection unit 151 uses the first model 341 to detect the presence or absence of a detection target in the monitoring area based on the judgment text output from the first model 341. This allows the detection device 10 to determine with high accuracy whether or not a detection target exists in the monitoring area. If the detection unit 151 determines that there is no detection target in the monitoring area, it returns to step S101 without performing any special processing.

[0055] On the other hand, if the detection unit 151 determines that a detection target exists in the monitoring area, it notifies the user that a detection target exists (is occurring) in the monitoring area by displaying it on the first output unit 12 or by outputting it as sound (step S104), and returns the process to step S101.

[0056] Figure 4 is a flowchart illustrating an example of the operation of the registration process by the registration device 20. This flowchart is executed primarily by the second control unit 25 in cooperation with each element of the registration device 20, based on a program pre-stored in the second storage unit 24.

[0057] First, the reception unit 251 receives the verification text entered by the user using the first input unit 11 (step S201). The verification text is used to indicate the detection target and to create a judgment prompt. The user can set as the verification text a term or sentence whose validity they want to verify as to whether or not it can be used as a detection target specified in the judgment prompt used by the detection system 2. For example, the verification text is the same string as the judgment prompt. The verification text may be only a part of the judgment prompt, for example, only the detection target. For example, if the judgment prompt is "Determine whether the input image contains 'falling beside the bed', and output in the format ○ if it is contained, and × if it is not contained," the verification text may be "falling beside the bed," etc.

[0058] Next, the first acquisition unit 252 uses the second model 342 to acquire an image generation prompt (step S202). The image generation prompt is an example of information based on the verification text and is text containing instructions to cause the third model 343 to generate a verification image. The image generation prompt is a prompt to generate an image based on the input text. The verification image is an image based on the verification text, generated using the third model 343. First, the first acquisition unit 252 generates a prompt generation prompt. The prompt generation prompt is text containing instructions to cause the second model 342 to generate an image generation prompt.

[0059] Figure 5(A) shows an example of a prompt generation prompt. As shown in Figure 5(A), the prompt generation prompt includes a validation text and an instruction to generate an image generation prompt that causes the image generation model to generate an image containing the detection target indicated in the validation text (an image representing the detection target). The prompt generation prompt may also include an instruction to extract targets related to the detection target contained in the validation text and to include the extracted targets in the image generation prompt. Furthermore, the prompt generation prompt may include an instruction to cause the image generation prompt to generate multiple images. This allows the first acquisition unit 252 to acquire multiple validation images from a single validation text.

[0060] As shown in Figure 3, the first acquisition unit 252 inputs a prompt generation prompt to the second model 342, which is an LLM, and acquires the response text output from the second model 342.

[0061] The first acquisition unit 252 sends a prompt generation request signal to the server device 30 via the second communication unit 23, instructing the second model 342 to generate a response text by inputting a prompt generation prompt to the second model 342. The prompt generation request signal includes a prompt generation prompt. When the third control unit 35 of the server device 30 receives the prompt generation request signal from the registration device 20 via the third communication unit 33, it inputs the prompt generation prompt included in the received prompt generation request signal to the second model 342, causing the second model 342 to generate a response text. The second model 342 generates and outputs a response text when a prompt generation prompt is input. The third control unit 35 transmits the generated response text to the registration device 20 via the third communication unit 33. The first acquisition unit 252 acquires the response text by receiving it from the server device 30 via the second communication unit 23. In this way, the first acquisition unit 252 inputs a prompt generation prompt to the second model 342 and causes the second model 342 to generate a response text.

[0062] Figure 5(B) shows an example of response text. As shown in Figure 5(B), the response text output from the second model 342 includes an image generation prompt.

[0063] Figure 5(C) shows an example of an image generation prompt. The image generation prompt shown in Figure 5(C) is the image generation prompt included in the response text shown in Figure 5(B). As shown in Figure 5(C), the image generation prompt indicates the detection target included in the verification text in more detail and / or specifically. The first acquisition unit 252 extracts the image generation prompt from the acquired response text.

[0064] Next, the first acquisition unit 252 acquires a verification image using the third model 343 (step S203). As shown in Figure 3, the first acquisition unit 252 inputs an image generation prompt to the third model 343, which is an image generation model, and acquires the image output from the third model 343 as a verification image. The first acquisition unit 252 acquires one or more verification images from a single verification text. By generating verification images using an image generation prompt, the registration device 20 can acquire verification images that appropriately correspond to the verification text, regardless of the information input by the user.

[0065] The first acquisition unit 252 sends an image generation request signal to the server device 30 via the second communication unit 23, instructing the third model 343 to generate an image by inputting an image generation prompt to the third model 343. The image generation request signal includes an image generation prompt. When the third control unit 35 of the server device 30 receives the image generation request signal from the registration device 20 via the third communication unit 33, it inputs the image generation prompt included in the received image generation request signal to the third model 343, causing the third model 343 to generate an image. If text is input, the third model 343 generates and outputs an image based on the input text. The third control unit 35 sends the generated image to the registration device 20 via the third communication unit 33. The first acquisition unit 252 receives the image from the server device 30 via the second communication unit 23 and acquires it as a verification image. In this way, the first acquisition unit 252 inputs an image generation prompt to the third model 343, causing the third model 343 to generate an image.

[0066] Next, the second acquisition unit 253 acquires explanatory text using the fourth model 344 (step S204). The explanatory text is text that describes the verification image, and describes the objects or events depicted in the verification image. First, the second acquisition unit 253 generates an explanatory prompt. The explanatory prompt is text that includes instructions for the fourth model 344 to generate the explanatory text.

[0067] Figure 6(A) shows an example of a descriptive prompt. As shown in Figure 6(A), the descriptive prompt includes instructions to generate descriptive text that describes the objects or events in the image (verification image) input along with the descriptive prompt. The descriptive prompt may also include instructions to generate descriptive text that shows all objects or events in the image.

[0068] As shown in Figure 3, the second acquisition unit 253 inputs the verification image and explanation prompt to the fourth model 344, which is a VLM, and acquires the explanation text output from the fourth model 344. The second acquisition unit 253 inputs all the verification images acquired in step S203 to the fourth model 344 and acquires the corresponding explanation text for each verification image. That is, the second acquisition unit 253 acquires the explanation text for each verification image.

[0069] The second acquisition unit 253 sends an explanation request signal to the server device 30 via the second communication unit 23 to input the verification image and explanation prompt to the fourth model 344 and cause the fourth model 344 to generate explanation text. The explanation request signal includes the verification image and explanation prompt. When the third control unit 35 of the server device 30 receives the explanation request signal from the registration device 20 via the third communication unit 33, it inputs the verification image and explanation prompt included in the received explanation request signal to the fourth model 344 and causes the fourth model 344 to generate explanation text. When the fourth model 344 receives the verification image and explanation prompt, it generates and outputs explanation text that explains the input verification image. The third control unit 35 sends the generated explanation text to the registration device 20 via the third communication unit 33. The second acquisition unit 253 acquires the explanation text by receiving it from the server device 30 via the second communication unit 23. In this manner, the second acquisition unit 253 inputs the verification image and the explanation prompt to the fourth model 344 and causes the fourth model 344 to generate the explanation text, that is, it acquires the text output from the fourth model 344 as the explanation text.

[0070] Figures 6(B)-(E) and 7(A)-(D) show examples of descriptive text. Figures 6(B)-(E) and 7(A)-(D) are the descriptive texts corresponding to each of the eight verification images generated from a single verification text. As shown in Figures 6(B)-(E) and 7(A)-(D), each descriptive text indicates the object or event depicted in the corresponding verification image.

[0071] Next, the comparison unit 254 obtains a comparison result by comparing the verification text and the explanatory text using the fifth model 345 (step S205). The comparison result is a text that indicates the determination result of whether the verification text is similar to the explanatory text (similarity) and / or whether it is included in the explanatory text (matching). First, the comparison unit 254 generates a comparison prompt. The comparison prompt is a text that contains instructions for the fifth model 345 to generate the comparison result. The comparison prompt includes the verification text, each explanatory text, and instructions to determine whether the verification text is similar to the explanatory text and / or whether the detection target indicated in the verification text is included in the explanatory text.

[0072] As shown in Figure 3, the comparison unit 254 inputs a comparison prompt to the fifth model 345, which is an LLM, and obtains the comparison result output from the fifth model 345.

[0073] The comparison unit 254 sends a comparison request signal to the server device 30 via the second communication unit 23, instructing the fifth model 345 to input a comparison prompt and generate a comparison result. The comparison request signal includes a comparison prompt. When the third control unit 35 of the server device 30 receives a comparison request signal from the registration device 20 via the third communication unit 33, it inputs the comparison prompt included in the received comparison request signal to the fifth model 345, causing the fifth model 345 to generate a comparison result. When a comparison prompt including verification text and explanatory text is input to the fifth model 345, it generates and outputs a comparison result by comparing the two input texts (verification text and explanatory text). The third control unit 35 transmits the generated comparison result to the registration device 20 via the third communication unit 33. The comparison unit 254 obtains the comparison result by receiving it from the server device 30 via the second communication unit 23. In this way, the comparison unit 254 inputs a comparison prompt to the fifth model 345 and causes the fifth model 345 to generate a comparison result.

[0074] Figure 8(A) shows an example of a comparison prompt. The "****" portion in Figure 8(A) contains the descriptive texts obtained in step S204. As shown in Figure 8(A), the comparison prompt includes the verification text obtained in step S201, the descriptive texts obtained in step S204, and an instruction to determine whether the detection target indicated in the verification text is included in the description indicated in the descriptive text. Figure 8(B) shows an example of the comparison results. As shown in Figure 8(B), the comparison results include the determination of whether the detection target shown in the verification text is included in the explanation shown in the explanation text, and the basis for that determination.

[0075] The comparison unit 254 may generate comparison prompts in multiple stages and obtain comparison results from the fifth model 345. For example, the comparison unit 254 generates a first comparison prompt for each explanatory text, which includes an instruction to determine whether the verification text corresponds to each explanatory text. For example, whether the verification text corresponds to an explanatory text is determined by whether the detection target indicated in the verification text can be detected from the verification image corresponding to the explanatory text. The comparison unit 254 inputs the generated first comparison prompt to the fifth model 345 and obtains the first comparison text output from the fifth model 345. Next, the comparison unit 254 generates a second comparison prompt, which includes an instruction to determine whether the verification text is similar to the explanatory text, based on the number of explanatory texts determined to correspond to the verification text and the number of explanatory texts determined not to correspond to the verification text in the first comparison result. The comparison unit 254 inputs the generated second comparison prompt to the fifth model 345 and obtains the second comparison text output from the fifth model 345 as the comparison result.

[0076] Figure 9(A) shows an example of the first comparison prompt. The "****" portion in Figure 9(A) contains the descriptive texts obtained in step S204. As shown in Figure 9(A), the first comparison prompt includes the verification text obtained in step S201, the descriptive texts obtained in step S204, and an instruction to determine whether the detection target indicated in the verification text is detectable from the verification image corresponding to each descriptive text. Figure 9(B) shows an example of the first comparison results. As shown in Figure 9(B), the first comparison results show, for each explanatory text, the determination result of whether or not the detection target indicated in the verification text can be detected from the verification image corresponding to each explanatory text, and the basis for that determination. Figure 9(C) shows an example of a second comparison prompt. As shown in Figure 9(C), the second comparison prompt includes instructions to determine whether the validation text is similar to the explanatory text based on the results of the first comparison. Figure 9(D) shows an example of the second comparison results. As shown in Figure 9(C), the second comparison results show the determination of whether the verification text is similar to the explanatory text, and the basis for that determination.

[0077] The comparison unit 254 may also compare the verification text with common or characteristic elements of the explanatory text for each verification image. Common elements are explanatory elements that represent a common concept (meaning) in the explanatory text for each verification image. The comparison unit 254 may also compare the verification text with the common elements. Feature elements are explanatory elements (summaries) that concisely represent the main topic in the explanatory text for each verification image. The comparison unit 254 may also compare the verification text with the feature elements. In this case, the comparison unit 254 generates a comparison prompt to compare the verification text with the common or characteristic elements of the explanatory text for each verification image. For example, the comparison unit 254 generates a first comparison prompt that includes the verification text, each explanatory text, and instructions to extract the common or characteristic elements of each explanatory text, and generates a second comparison prompt that includes instructions to determine whether the verification text is similar to the extracted common or characteristic elements. The comparison unit 254 may also generate a first comparison prompt that includes an instruction to extract common or characteristic elements of the description texts that correspond to the verification text (i.e., the detection target shown in the verification text can be detected from the corresponding verification image). In this case as well, the registration device 20 can compare the verification text and each description text with high accuracy.

[0078] Next, the comparison unit 254 compares the verification text and the explanatory text based on the comparison result output from the fifth model 345 and determines whether the verification text is similar to the explanatory text or whether it is included in the explanatory text (step S206). If the comparison result indicates that the verification text is similar to the explanatory text, the comparison unit 254 determines that the verification text is similar to the explanatory text, and if the comparison result indicates that the verification text is not similar to the explanatory text, the comparison unit 254 determines that the verification text is included in the explanatory text, and if the comparison result indicates that the verification text is not included in the explanatory text, the comparison unit 254 determines that the verification text is included in the explanatory text. By using the fifth model 345 to compare the verification text and the explanatory text, the registration device 20 can determine the similarity or agreement between the verification text and the explanatory text with higher accuracy.

[0079] If the verification text is similar to or included in the explanatory text, the registration unit 255 registers a judgment prompt reflecting the verification text received in step S201 with the detection system 2 (step S207), and ends the series of steps. The registration unit 255 sends a request signal to the detection device 10 via the second communication unit 23, requesting that it generate (update) a judgment prompt that includes the verification text and is based on that verification text. When the detection unit 151 of the detection device 10 receives the request signal from the registration device 20 via the first communication unit 13, it stores and registers the verification text included in the request signal in the first storage unit 14. The detection unit 151 also generates and stores (updates) a judgment prompt based on the received verification text. For example, the detection unit 151 uses the verification text as is as the judgment prompt. The detection unit 151 may also generate and store a judgment prompt by performing modification processing on the verification text, such as adding predetermined standard phrases such as cautionary notes, changing some particles, or converting between polite and informal language. Thereafter, the detection unit 151 uses the newly generated judgment prompt to execute the detection process shown in Figure 2.

[0080] The registration unit 255 may generate a judgment prompt based on the verification text and send a request signal to the detection device 10 via the second communication unit 23, which includes the generated judgment prompt and requests that the judgment prompt be registered.

[0081] Thus, the registration unit 255 registers a judgment prompt reflecting the verification text with the detection system 2 based on the comparison result of the verification text and the explanatory text. As described above, the first model 341 used in the detection process and the third model 343 used in the registration process share a common understanding and possess the same level of knowledge through CLIP. Therefore, if the explanatory text that describes the verification image generated from the verification text using the third model 343 appropriately represents the detection target indicated in the verification text, then the verification text is appropriately understood by the third model 343 and is highly likely to be appropriately understood by the first model 341 as well. Consequently, the first model 341 is highly likely to appropriately understand the judgment prompt generated based on the verification text and is highly likely to correctly determine whether the input image represents the detection target indicated in the judgment prompt. Therefore, the registration device 20 can improve the detection accuracy in the detection system 2 by registering the verification text with the detection system 2 and updating the judgment prompt according to the comparison result of the verification text and the explanatory text.

[0082] On the other hand, if the verification text is not similar to the descriptive text or is not included in the descriptive text, the registration unit 255 uses the sixth model 346 to obtain element text (step S208). Element text is text that shows elements such as objects or events related to the detection target extracted from each descriptive text. For each descriptive text, the element text shows elements related to the detection target extracted from each descriptive text. The element text may show only elements extracted from descriptive texts that are determined to correspond to the verification text. The element text may also show elements extracted from descriptive texts that are determined not to correspond to the verification text. Furthermore, the element text may show elements extracted collectively from multiple descriptive texts. In that case, the element text may show common elements or characteristic elements from multiple descriptive texts. First, the registration unit 255 generates an element prompt. The element prompt is text that includes instructions for the sixth model 346 to generate element text. The element prompt includes the verification text, each descriptive text, the determination result of whether the verification text corresponds to each descriptive text, and instructions to extract elements related to the detection target from each descriptive text.

[0083] As shown in Figure 3, the registration unit 255 inputs an element prompt to the sixth model 346, which is an LLM, and obtains the element text output from the sixth model 346.

[0084] The registration unit 255 sends an element request signal to the server device 30 via the second communication unit 23 to input an element prompt to the sixth model 346 and cause the sixth model 346 to generate element text. The element request signal includes an element prompt. When the third control unit 35 of the server device 30 receives an element request signal from the registration device 20 via the third communication unit 33, it inputs the element prompt included in the received element request signal to the sixth model 346 and causes the sixth model 346 to generate element text. The sixth model 346 generates and outputs element text when an element prompt is input. The third control unit 35 sends the generated element text to the registration device 20 via the third communication unit 33. The registration unit 255 acquires the element text by receiving it from the server device 30 via the second communication unit 23. In this way, the registration unit 255 inputs an element prompt to the sixth model 346 and causes the sixth model 346 to generate element text.

[0085] The registration unit 255 may also input an element prompt to the fifth model 345, which has output the comparison results, and obtain the element text output from the fifth model 345. In this case, since the fifth model 345 has already recognized the verification text, the explanatory text, and the determination result of whether the verification text corresponds to each explanatory text, the verification text, each explanatory text, and / or their determination results may be omitted in the element prompt.

[0086] Figure 10(A) shows an example of an element prompt. Figure 10(A) shows an element prompt for input to the fifth model 345, which outputs the comparison results. As shown in Figure 10(A), the element prompt includes instructions to extract elements related to the detection target from the descriptive text. Figure 10(B) shows an example of element text. As shown in Figure 10(B), the element text shows elements that were determined to be related to the detection target in the explanatory text. In particular, this element text shows only elements that were determined to be related to the detection target in the explanatory text that corresponds to the verification text.

[0087] Next, the registration unit 255 determines whether the verification text is modifiable (step S209). The registration unit 255 calculates the ratio of the number of verification images in which the explanatory text is determined not to correspond to the verification text to the total number of acquired verification images (hereinafter sometimes referred to as the failure ratio). If the failure ratio is greater than or equal to a threshold, the registration unit 255 determines that the verification text is not modifiable, and if the failure ratio is less than the threshold, it determines that the verification text is modifiable. The threshold is set to the ratio in which it can be considered that appropriate verification images, i.e., verification images in which the detection target is represented, have been acquired.

[0088] If the verification text cannot be modified, the registration unit 255 notifies the user that the verification text is inappropriate by displaying it on the second output unit 22 (step S210), and terminates the series of steps. This allows the user to recognize that the verification text was inappropriate and to modify it. The registration unit 255 may also notify the user that the verification text is inappropriate by displaying each element shown in the acquired element text as a summary of the explanatory text for each verification image on the second output unit 22, along with the fact that the verification text is inappropriate. This allows the user to recognize the elements in the verification text that are effective for detecting the target, and / or the elements that caused the verification text to be determined to be inappropriate, and to modify the verification text appropriately.

[0089] On the other hand, if the verification text is editable, the registration unit 255 generates and outputs editable verification text using the seventh model 347 (step S211), and terminates the series of steps. Edited verification text is text that has been modified from the verification text. The registration unit 255 obtains the editable text using the seventh model 347 and extracts the editable verification text from the editable text. Editable text is text that contains the editable verification text and is generated based on elements related to the verification text, as shown in the element text. First, the registration unit 255 generates an edit prompt. The edit prompt is text that contains instructions for the seventh model 347 to generate editable text. The edit prompt includes the verification text, element text, and instructions to edit the verification text using the element text. The correction prompt may specify that the validation text should be modified using elements extracted from the explanatory text that is determined to correspond to the validation text. The correction prompt may also specify that elements extracted from the explanatory text that corresponds to the validation text that overlap with elements extracted from the explanatory text that does not correspond to the validation text should not be used. Furthermore, the correction prompt may specify that elements extracted from the explanatory text that does not correspond to the validation text may be used negatively.

[0090] As shown in Figure 3, the first acquisition unit 252 inputs a correction prompt to the seventh model 347, which is an LLM, and acquires the corrected text output from the seventh model 347.

[0091] The registration unit 255 sends a correction request signal to the server device 30 via the second communication unit 23 to input a correction prompt to the seventh model 347 and cause the seventh model 347 to generate the corrected text. The correction request signal includes a correction prompt. When the third control unit 35 of the server device 30 receives a correction request signal from the registration device 20 via the third communication unit 33, it inputs the correction prompt included in the received correction request signal to the seventh model 347 and causes the seventh model 347 to generate the corrected text. The seventh model 347 generates and outputs the corrected text when a correction prompt is input. The third control unit 35 sends the generated corrected text to the registration device 20 via the third communication unit 33. The registration unit 255 obtains the corrected text by receiving it from the server device 30 via the second communication unit 23. In this way, the registration unit 255 inputs a correction prompt to the seventh model 347 and causes the seventh model 347 to generate the corrected text.

[0092] The registration unit 255 may also input a correction prompt to the fifth model 345, which outputs the comparison result, or the sixth model 346, which outputs the element text, and obtain the corrected text output from the fifth model 345 or the sixth model 346. In this case, since the fifth model 345 or the sixth model 346 has already recognized the verification text and the element text, the verification text and / or element text may be omitted in the correction prompt.

[0093] Figure 10(C) shows an example of a correction prompt. Figure 10(C) shows a correction prompt to be input to the fifth model 345, which has output comparison results and element text. As shown in Figure 11(C), the correction prompt includes instructions to correct the validation text using the element text. Figure 10(D) shows an example of corrected text. As shown in Figure 10(D), the corrected text includes corrected validation text that has been modified relative to the validation text. The corrected validation text may have the form of a judgment prompt.

[0094] The registration unit 255 generates the corrected verification text by extracting it from the corrected text. The registration unit 255 outputs the generated corrected verification text by displaying it on the second output unit 22 to notify the user. This allows the user to appropriately correct the verification text based on the corrected verification text. The registration unit 255 may also automatically register the corrected verification text with the detection system 2 in the same manner as the process in step S207. That is, the registration unit 255 may register instruction information that reflects the corrected verification text with the detection system 2. The corrected verification text is an example of instruction information that reflects the verification text. The registration device 20 can automatically register the corrected verification text with the detection system 2 and update the judgment prompt without requiring any action from the user, thereby improving the detection accuracy in the detection system 2.

[0095] As described above, the correction prompt includes descriptive text corresponding to the verification image and a determination result indicating whether the verification text corresponds to each descriptive text. That is, the registration unit 255 generates the corrected verification text based on the determination result for the verification image and the descriptive text corresponding to that verification image. As a result, the registration device 20 can generate corrected verification text that appropriately modifies the verification text.

[0096] Furthermore, as described above, the correction prompt specifies that the verification text should be modified using elements extracted from the explanatory text that is determined to correspond to the verification text. That is, the registration unit 255 generates the modified verification text based on elements related to the verification text contained in the explanatory text for the verification image that is determined to contain the detection target. As a result, the registration device 20 can generate modified verification text that more appropriately modifies the verification text.

[0097] Furthermore, as described above, the correction prompt specifies that elements extracted from the explanatory text determined to correspond to the verification text that overlap with elements extracted from the explanatory text determined not to correspond to the verification text should not be used. The correction prompt also specifies that elements extracted from the explanatory text determined not to correspond to the verification text may be used negatively. In other words, the registration unit 255 generates the revised verification text so that elements included in the explanatory text for the verification image determined not to contain the detection target are not included in the verification image generated based on the revised verification text. As a result, the registration device 20 can generate revised verification text that more appropriately modifies the verification text.

[0098] Note that the processing in step S202 may be omitted, and instead of the image generation prompt, the validation text itself may be input to the third model 343. In that case, the validation text itself is an example of information based on the validation text. Furthermore, in step S205, the comparison unit 254 may compare the verification text and the explanatory text without using the fifth model 345. For example, the comparison unit 254 vectorizes the verification text and the explanatory text using known methods such as BoW (Bag of Words) or Doc2Vec. The comparison unit 254 calculates the similarity between the verification text and the explanatory text using the cosine similarity of each vector, and determines whether the verification text is similar to the explanatory text based on whether the calculated similarity is above a threshold. Furthermore, the processing in step S208 may be executed after the processing in step S209. Also, the processing in step S209 may be omitted, and either the processing in step S210 or the processing in step S211 may be executed permanently. Furthermore, the processing from step S209 onward may be omitted. Furthermore, the explanatory prompt input to the fourth model 344, which is a VLM, may include instructions that cause the fourth model 344 to generate element text and / or modified text. In that case, the processing in steps S208, S208 and / or S211 may be executed together.

[0099] As explained above, the detection system 2 uses VLM to determine whether or not an image contains a target for detection. The detection target registration system 3 receives verification text from the user indicating the target for detection and obtains a verification image generated from the verification text using image generation AI. The detection target registration system 3 registers the verification text with the detection system 2 according to the comparison result between the verification text and the descriptive text describing the verification image, and the detection system 2 uses the registered verification text to generate a judgment prompt to be input into VLM. As a result, the detection system 2 can generate a judgment prompt that can appropriately detect the target for detection, and the detection target registration system 3 can further improve the judgment accuracy of the detection system 2, which uses generation AI to determine whether or not an image contains a target for detection.

[0100] Because VLM is a black box, it is difficult to determine whether VLM can detect the events that the user wants to detect. The detection target registration system 3 can appropriately determine the validity of the prompts to be input to VLM by using images generated by an image generation AI that has a text encoder trained to correlate with the VLM's image encoder.

[0101] Although preferred embodiments have been described above, the embodiments are not limited to the examples described above. For example, the detection target registration system 3 may have an AI agent generate each piece of information instead of a generating AI. In that case, the first model 341, the second model 342, the third model 343, the fourth model 344, the fifth model 345, the sixth model 346 and / or the seventh model 347 are AI agents that use the corresponding generating AI. The first model 341 is an example of the first AI agent, the third model 343 is an example of the second AI agent, and the fourth model 344 is an example of the third AI agent. The second model 342 is an example of the fourth AI agent, the fifth model 345 is an example of the fifth AI agent, and the sixth model 346 is an example of the sixth AI agent. Each model is pre-trained to generate and output information according to the input information.

[0102] The first model 341 and / or the fourth model 344 are one or more autonomous AI agents using VLM that generate and output information corresponding to the input image and natural language when an image and natural language are input. The first model 341 and the fourth model 344 may be the same model or different models. The first model 341 generates and outputs judgment text when an image and a judgment prompt for generating judgment text indicating whether or not the image contains a detection target are input. The fourth model 344 generates and outputs descriptive text describing the input image when an image and a description prompt are input. The second model 342, the fifth model 345, the sixth model 346, and / or the seventh model 347 are one or more autonomous AI agents that use LLM to generate and output information corresponding to the natural language input. The second model 342, the fifth model 345, the sixth model 346, and the seventh model 347 may be the same model or different models. The second model 342 outputs a response text when a prompt generation prompt is input. The fifth model 345 outputs a comparison result of comparing two input texts when two texts are input. The sixth model 346 outputs element text when an element prompt is input. The seventh model 347 outputs corrected text when a correction prompt is input. The third model 343 is one or more autonomous AI agents that use an image generation AI to generate and output images corresponding to natural language input. The third model 343 also generates and outputs images based on text input when text is received. An autonomous AI agent, given a goal, will have a generative AI generate tasks to achieve that goal, collect information necessary for the generative AI to execute those tasks, and repeat the process of having the generative AI execute the tasks until it outputs information that will achieve the goal. The AI ​​agent may also autonomously acquire environmental information such as temperature and lighting of the monitored area, and output information that will achieve the goal while taking the acquired environmental information into consideration.

[0103] In each process in which the models shown in Figures 2 and 4 are used, the prompts may not be generated, and an AI agent generated to achieve the goal corresponding to each prompt may be used.

[0104] As a result, the detection system 2 can generate a judgment prompt that can appropriately detect the target, and the detection target registration system 3 can further improve the judgment accuracy of the detection system 2, which uses an AI agent to determine whether or not the image contains the target.

[0105] Furthermore, the first model 341 may be stored in the first storage unit 14 of the detection device 10 instead of the third storage unit 34 of the server device 30. In that case, when the first control unit 15 of the detection device 10 uses the first model 341, it does not send each request signal to the server device 30, but instead causes the first model 341 stored in the first storage unit 14 to generate the information. Similarly, the second model 342, third model 343, fourth model 344, fifth model 345, sixth model 346, and / or seventh model 347 may be stored in the second storage unit 24 of the registration device 20 instead of the third storage unit 34 of the server device 30. In that case, when the second control unit 25 of the registration device 20 uses each model, it does not send each request signal to the server device 30, but instead causes the models stored in the second storage unit 24 to generate the information.

[0106] Furthermore, the image encoder of the fourth model 344 and the text encoder of the third model 343 may be trained so that the features generated by each encoder are correlated. Also, the image encoder of the fourth model 344 and the text encoder of the third model 343 may be trained using the same training data. As mentioned above, the first model 341 and the fourth model 344 may be the same model, but the fourth model 344 is used infrequently because it is only used when verifying the verification text. Therefore, it is preferable for the fourth model 344 to use a training model with high image recognition performance, even if its processing speed is slower than that of the first model 341.

[0107] Furthermore, the detection system 2 and the detection target registration system 3 may be used in any system other than the monitoring system 1, such as an image matching system or an image authentication system. [Explanation of Symbols]

[0108] 2 Detection system, 3 Detection target registration system, 10 Detection device, 20 Registration device, 251 Reception unit, 252 First acquisition unit, 253 Second acquisition unit, 254 Comparison unit, 255 Registration unit, 341 First model, 342 Second model, 343 Third model, 344 Fourth model, 345 Fifth model

Claims

1. A detection target registration system that registers the instruction information to a first generation AI or AI agent that generates and outputs a detection result when an image and instruction information for generating a detection result indicating whether or not the image contains a detection target are input, A reception unit that receives verification text from the user indicating the target of detection, A first acquisition unit inputs information based on the verification text to a second generating AI or AI agent that generates and outputs an image based on the input text, and acquires the output image as a verification image. The verification image is input to a third generating AI or AI agent that generates and outputs text describing the input image, and a second acquisition unit acquires the output text as explanatory text. A comparison unit that compares the verification text and the explanatory text, A registration unit registers the instruction information reflecting the received verification text into the detection system based on the comparison result of the verification text and the explanatory text. A detection target registration system characterized by having the following features.

2. The detection target registration system according to claim 1, wherein the image encoder of the first generating AI or AI agent and the text encoder of the second generating AI or AI agent are trained so that the feature quantities generated by the image encoder and the feature quantities generated by the text encoder are correlated.

3. The detection target registration system according to claim 1 or 2, wherein the image encoder of the first generating AI or AI agent and the text encoder of the second generating AI or AI agent are trained using the same training data.

4. The detection system inputs an input image of a monitoring area to the first generating AI or AI agent, and determines the presence or absence of the detection target in the monitoring area based on the output detection result, according to claim 1 or 2.

5. The detection target registration system according to claim 1 or 2, wherein the first acquisition unit inputs the verification text to a fourth generation AI or AI agent that outputs an image generation prompt for generating an image based on the input text, and the output image generation prompt is input to the second generation AI or AI agent to acquire the verification image.

6. The detection target registration system according to claim 1 or 2, wherein the comparison unit inputs the verification text and the explanatory text to a fifth generating AI or AI agent that outputs a comparison result of comparing two input texts, and compares the verification text and the explanatory text based on the output comparison result.

7. The first acquisition unit acquires a plurality of verification images from a single verification text, The second acquisition unit acquires the explanatory text for each of the plurality of verification images, The detection target registration system according to claim 1 or 2, wherein the comparison unit compares the verification text with common elements or feature elements of the descriptive text for each of the plurality of verification images.

8. The first acquisition unit acquires a plurality of verification images from a single verification text, The second acquisition unit acquires the explanatory text for each of the plurality of verification images, The detection target registration system according to claim 1 or 2, wherein the registration unit notifies the user of a summary of the descriptive text for each of the plurality of verification images.

9. The detection target registration system according to claim 1 or 2, wherein the first generating AI or AI agent and the third generating AI or AI agent are configured to include a common VLM that generates and outputs text describing an input image.

10. The first acquisition unit acquires a plurality of verification images from a single verification text, The second acquisition unit acquires the explanatory text for each of the plurality of verification images, The detection target registration system according to claim 1 or 2, wherein the registration unit generates a modified verification text by modifying the verification text based on the explanatory text for each of the plurality of verification images, and registers the instruction information reflecting the modified verification text in the detection system.

11. The detection target registration system according to claim 10, wherein the registration unit inputs the descriptive text for each of the plurality of verification images to a sixth generating AI or AI agent that outputs element text indicating elements related to the detection target of the input text, and generates the modified verification text based on the output element text.

12. A detection target registration method for registering instruction information to a detection system that uses a first generating AI or AI agent that generates and outputs a detection result when an image and instruction information for generating a detection result indicating whether or not the image contains a detection target, wherein the instruction information is registered to the detection target registration method, The system receives verification text from the user indicating the target of detection. The information based on the verification text is input to a second generating AI or AI agent that generates and outputs an image based on the input text, and the output image is acquired as the verification image. The verification image is input to a third generating AI or AI agent that generates and outputs text describing the input image, and the output text is acquired as the explanatory text. The verification text is compared with the explanatory text, Based on the comparison result between the verification text and the explanatory text, the instruction information reflecting the received verification text is registered in the detection system. A method for registering targets for detection, characterized by the above.

Images