Object detection device, object detection method, and object detection program

Abstract

To reduce missed detections and false detections when detecting an object using an object detection model. [Solution] A detection control unit (21) inputs an input image to an object detection model (31) that detects objects with specified attributes from image data, and causes the model to detect objects from the input image. The object detection model (31) outputs a detection result including the detection attribute and the certainty of the detection for each detected object. If the detection result includes multiple detection attributes with certainty levels equal to or higher than a lower limit, a determination control unit (22) inputs the input image and the multiple detection attributes with certainty levels equal to or higher than a lower limit to a determination model (32), generates a description of the input image, and determines from the description whether an object with any of the multiple detection attributes is included in the input image.

Description

[Technical Field]

[0001] The present disclosure relates to a technology for detecting an object with a specified attribute from image data. [Background technology]

[0002] Patent Document 1 describes detecting an object included in image data using an object detection model trained using deep learning. In particular, Patent Document 1 describes detecting an object using a first method that requires less computational effort than a method using deep learning, and then determining whether the detected object has a specified attribute using a second method using deep learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6782825 Summary of the Invention [Problem to be solved by the invention]

[0004] When detecting objects from image data, it is desirable to reduce the occurrence of false positives, where an object that should not be detected is detected, and false negatives, where an object that should be detected is not detected. In the method described in Patent Document 1, by raising the threshold value that serves as the standard for determining that an object is a specified object in the second method, false positives can be reduced to a certain extent, but false negatives increase accordingly. Conversely, by lowering the threshold value, false negatives can be reduced to a certain extent, but false positives increase accordingly. An object of the present disclosure is to reduce missed detections and false detections when detecting an object using an object detection model. [Means for solving the problem]

[0005] An object detection device according to the present disclosure includes: a determination control unit that, when an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, inputs the input image to a determination model that is a trained model different from the object detection model, generates an explanatory text for the input image, and determines from the explanatory text whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image; Equipped with The object detection model outputs a detection result including a detection attribute and a certainty of detection for each detected object; When the detection result includes multiple detection attributes whose certainty is equal to or greater than a lower limit, the judgment control unit inputs the input image and the multiple detection attributes whose certainty is equal to or greater than a lower limit to the judgment model, and determines from the description whether an object having any of the multiple detection attributes is included in the input image. [Effects of the Invention]

[0006] In the present disclosure, it is possible to reduce false positives by setting a low threshold value used when detecting an object using an object detection model. Furthermore, even if the threshold value is set low, false positives can be reduced because double-checking is performed using the object detection model and other models. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a configuration diagram of an object detection device 10 according to a first embodiment. [Figure 2] 3 is a flowchart of processing by the object detection device 10 according to the first embodiment. [Figure 3] FIG. 3 is an explanatory diagram of input to a determination model 32 according to the first embodiment. [Figure 4] FIG. 3 is an explanatory diagram of an output from a determination model 32 according to the first embodiment. [Figure 5] 10 is a flowchart of processing by the object detection device 10 according to the second embodiment. [Figure 6] FIG. 10 is an explanatory diagram of input to a determination model 32 according to the second embodiment. [Figure 7] FIG. 10 is an explanatory diagram of an output from a determination model 32 according to the second embodiment. [Figure 8] 10 is a flowchart of processing by the object detection device 10 according to the third embodiment. [Figure 9] FIG. 11 is an explanatory diagram of input to an explanation generation model according to the third embodiment. [Figure 10] FIG. 11 is an explanatory diagram of an output from an explanation generation model according to the third embodiment. [Figure 11] FIG. 11 is an explanatory diagram of input to an attribute detection model according to the third embodiment. [Figure 12] FIG. 11 is an explanatory diagram of an output from an attribute detection model according to the third embodiment. [Figure 13] 10 is a flowchart of processing by the object detection device 10 according to the fourth embodiment. [Figure 14] FIG. 11 is an explanatory diagram of input to an attribute detection model according to the fourth embodiment. [Figure 15] FIG. 11 is an explanatory diagram of an output from an attribute detection model according to the fourth embodiment. [Figure 16] FIG. 10 is a configuration diagram of an object detection device 10 according to a fifth embodiment. [Figure 17] 10 is a flowchart of processing by the object detection device 10 according to the fifth embodiment. [Figure 18] FIG. 13 is an explanatory diagram of input to a condition determination model 33 according to the fifth embodiment. [Figure 19] FIG. 13 is an explanatory diagram of an output from a condition determination model 33 according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Embodiment 1 ***Configuration Description*** The configuration of an object detection device 10 according to the first embodiment will be described with reference to FIG. The object detection device 10 is a computer. The object detection device 10 includes hardware such as a processor 11, a memory 12, a storage 13, and a communication interface 14. The processor 11 is connected to other hardware via signal lines and controls the other hardware.

[0009] The processor 11 is an IC that performs processing. IC stands for Integrated Circuit. Specific examples of the processor 11 include a CPU, a DSP, and a GPU. CPU stands for Central Processing Unit. DSP stands for Digital Signal Processor. GPU stands for Graphics Processing Unit.

[0010] The memory 12 is a storage device that temporarily stores data. Specific examples of the memory 12 include SRAM and DRAM. SRAM stands for Static Random Access Memory. DRAM stands for Dynamic Random Access Memory.

[0011] The storage 13 is a storage device that stores data. A specific example of the storage 13 is an SSD. SSD is an abbreviation for Solid State Drive. The storage 13 may also be a portable recording medium such as an SD (registered trademark) memory card, CompactFlash (registered trademark), NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. SD is an abbreviation for Secure Digital. DVD is an abbreviation for Digital Versatile Disk.

[0012] The communication interface 14 is an interface for communicating with external devices. Specific examples of the communication interface 14 include Ethernet (registered trademark), USB, and HDMI (registered trademark) ports. USB stands for Universal Serial Bus. HDMI stands for High-Definition Multimedia Interface.

[0013] The object detection device 10 includes, as functional components, a detection control unit 21, a determination control unit 22, and a notification unit 23. The functions of the functional components of the object detection device 10 are realized by software. The storage 13 stores a program that realizes the function of each functional component of the object detection device 10. The program is loaded into the memory 12 by the processor 11 and executed by the processor 11. In this way, the function of each functional component of the object detection device 10 is realized.

[0014] The storage 13 stores an object detection model 31, which is a trained model. The object detection model 31 is a model that detects an object with a specified attribute from image data. The object detection model 31 is what is known as AI. AI stands for artificial intelligence. The object detection model 31 is a model constructed using deep learning.

[0015] Furthermore, the object detection device 10 is connected via the communication interface 14 to a determination model 32, which is a trained model. The determination model 32 is a so-called generative AI. Specific examples of the determination model 32 include algorithms such as BERT and GPT. BERT stands for Bidirectional Encoder Representations from Transformers. GPT stands for Generative Pretrained Transformer. The learning model 112 may be configured by combining a plurality of algorithms including these algorithms.

[0016] Here, the object detection model 31 is stored in the storage 13, and the determination model 32 is connected via the communication interface 14. In other words, the object detection model 31 is located inside the object detection device 10, and the determination model 32 is located outside the object detection device 10. However, both the object detection model 31 and the determination model 32 may be located inside the object detection device 10, or both the object detection model 31 and the determination model 32 may be located outside the object detection device 10.

[0017] 1 shows only one processor 11. However, there may be a plurality of processors 11, and the plurality of processors 11 may cooperate to execute programs that realize the respective functions.

[0018] ***Explanation of Operation*** The operation of the object detection device 10 according to the first embodiment will be described with reference to FIGS. The operation procedure of object detection device 10 according to embodiment 1 corresponds to the object detection method according to embodiment 1. Furthermore, a program that realizes the operation of object detection device 10 according to embodiment 1 corresponds to the object detection program according to embodiment 1.

[0019] The processing of the object detection device 10 according to the first embodiment will be described with reference to FIG. (Step S11: Image input processing) The detection control unit 21 sets the frame images that make up the video data as input images.

[0020] (Step S12: Detection control process) The detection control unit 21 inputs an input image to the object detection model 31, causing it to detect an object from the input image. At this time, the detection control unit 21 inputs one or more attributes to be detected together with the input image to the object detection model 31, causing it to detect an object having any of the one or more attributes. The detection control unit 21 acquires the detection result output from the object detection model 31. The detection result includes, for the detected object, a detection frame indicating the detected position, attributes, and a certainty indicating the certainty of the detection.

[0021] (Step S13: Final determination process) The detection control unit 21 determines whether the certainty factor included in the detection result acquired in step S12 is equal to or greater than a confirmation threshold. If the confidence level is equal to or greater than the confirmation threshold, the detection control unit 21 determines that an object has been detected by the object detection model 31, and proceeds to step S14. On the other hand, if the confidence level is less than the confirmation threshold, the detection control unit 21 ends processing of the input image.

[0022] (Step S14: Determination control process) The determination control unit 22 inputs an input image to the determination model 32, and causes it to determine whether or not the input image contains an object with a detection attribute that is an attribute of the object detected by the object detection model 31. At this time, the determination control unit 22 instructs the determination model 32 to determine whether or not the input image contains an object with a detection attribute, and to create an explanation for the input image. For example, as shown in FIG. 3, the determination control unit 22 inputs an input image and attribute information indicating the detection attribute as a prompt to the determination model 32. At this time, the determination control unit 22 determines whether or not an object with the detection attribute is included in the input image, and inputs a prompt to the determination model 32 instructing it to create an explanatory text for the input image. In FIG. 3, an instruction is given to output in JSON Schema format. Outputting in JSON Schema format makes it easier to use the output data in subsequent processing. Here, ${attribute information} represents the attribute information specified in the input information ("white cane" in the above example). In other words, "${attribute information}" is read as "white cane". Other ${***} are similarly replaced with the information specified as *** in the input information. It is desirable that the attribute information be such that the target attribute can be clearly identified, such as "a white cane for the visually impaired," rather than simply specifying "a white cane." If "a white cane" is simply specified, there is a possibility that a white cane that is not for the visually impaired may also be determined to correspond to the detection attribute. Therefore, when the attribute information is simply designated as "white cane," the determination control unit 22 may operate to replace "white cane" with "white cane for visually impaired people" using preset information. Other examples of preset information include replacing "wheelchair" designated as attribute information with "wheelchair without attendant," or replacing "safety vest" designated as attribute information with "safety vest worn by civil engineering workers." In this way, the determination control unit 22 may replace attribute information designated using preset information with detailed attribute information.

[0023] The determination control unit 22 acquires the determination result and the explanatory text output from the determination model 32. The determination result indicates whether or not an object with the detection attribute is included in the input image. When the input image and attribute information shown in Figure 3 are entered as a prompt, the determination result and explanatory text are output in the format shown in Figure 4. In Figure 4, the determination result is true, which indicates that the object with the detection attribute is included in the input image, and the explanatory text is "A woman is walking with a man using a white cane..."

[0024] (Step S15: Detection and Determination Processing) The determination control unit 22 determines whether or not the determination result acquired in step S14 indicates that an object with the detection attribute is included in the input image. If the determination result indicates that the object with the detection attribute is included in the input image, the determination control unit 22 proceeds to step S16. On the other hand, if the determination result does not indicate that the object with the detection attribute is included in the input image, the determination control unit 22 ends the processing for the input image.

[0025] (Step S16: Notification process) The notification unit 23 notifies the user of the object detection device 10 that an object with the detection attribute has been detected and the explanatory text obtained in step 14. For example, the notification unit 23 displays the information that an object with the detection attribute has been detected and the explanatory text on a display device used by the user of the object detection device 10.

[0026] ***Effects of the First Embodiment*** As described above, when an object is detected from an input image using the object detection model 31, the object detection device 10 according to embodiment 1 uses the judgment model 32 to determine whether an object with the detected attributes is present in the input image. This makes it possible to reduce missed detections and false positives. In other words, it is possible to reduce missed detections by setting a low confirmation threshold used when detecting an object with the object detection model 31. Furthermore, even if the confirmation threshold is set low, double-checking is performed using the object detection model 31 and the judgment model 32, so it is possible to keep false positives to a minimum.

[0027] Furthermore, when notifying that an object with a detection attribute has been detected, the object detection device 10 according to the first embodiment also notifies an explanatory text about the input image. By referring to the explanatory text, the user can easily understand the situation, thereby improving usability.

[0028] The determination model 32 is a so-called generative AI. Because processing by the generative AI takes time, it is difficult to perform processing in real time to determine whether or not an object is included in each of the frame images that make up a video. The object detection device 10 according to the first embodiment inputs an input image to the determination model 32 only when an object is detected by the object detection model 31. This makes it easier to realize detection processing in real time using the determination model 32, which is a generative AI.

[0029] Furthermore, the object detection device 10 according to the first embodiment can also be used for intrusion detection by devising a method for setting attribute information to be input to the determination model 32. For example, suppose you want to detect a person standing on the yellow track side of a station platform. In this case, by setting the attribute information to "a person standing on the yellow track side" instead of simply setting it to "a person," the determination model 32 can detect only people standing on the yellow track side. Normally, when performing intrusion detection, it is necessary to set the area on the yellow track side as the detection target area for each camera. However, the object detection device 10 according to the first embodiment can easily perform intrusion detection without such setting.

[0030] Embodiment 2 The second embodiment differs from the first embodiment in that when the confidence level output from the object detection model 31 is equal to or greater than the confirmation threshold but is not so high, the determination model 32 is made to determine which attribute of an object is included in the input image. In the second embodiment, this difference will be explained, and explanation of the same points will be omitted. In the second embodiment, it is assumed that there are multiple attributes to be detected.

[0031] ***Explanation of Operation*** The processing of the object detection device 10 according to the second embodiment will be described with reference to FIG. The processing from step S21 to step S23 is the same as the processing from step S11 to step S13 in FIG.

[0032] (Step S24: Selection determination process) The detection control unit 21 determines whether the certainty factor included in the detection result acquired in step S22 is equal to or greater than a selection threshold. If the certainty factor is equal to or greater than the selection threshold, the detection control unit 21 proceeds to step S25. In step S25, the processes of steps S14 to S16 in Fig. 2 are executed. On the other hand, if the certainty factor is less than the determination threshold, the detection control unit 21 proceeds to step S26.

[0033] (Step S26: Determination control process) The determination control unit 22 inputs an input image to the determination model 32 and determines whether or not an object with any of the multiple attributes to be detected is included in the input image. At this time, the determination control unit 22 instructs the determination model 32 to determine whether or not an object with any of the multiple attributes is included in the input image, and to create an explanation for the input image. 6, the determination control unit 22 inputs an input image and attribute information indicating multiple attributes of the detection target to the determination model 32. At this time, the determination control unit 22 determines whether or not an object with any of the multiple attributes is included in the input image, and inputs a prompt to the determination model 32 to instruct it to create an explanation for the input image.

[0034] The determination control unit 22 acquires the determination result and the explanatory text output from the determination model 32. The determination result indicates the attribute of the object included in the input image. When the input image, attribute information, and prompt shown in Fig. 6 are input, the determination result and explanatory text are output in the format shown in Fig. 7. In Fig. 7, the determination result indicates that a white cane is included in the input image, and the explanatory text is "A woman is walking with a man while using a white cane..."

[0035] (Step S27: Detection and Determination Processing) The determination control unit 22 determines whether or not the determination result acquired in step S26 indicates that an object with any one of the multiple attributes is included in the input image. If the determination result indicates that an object with any of the multiple attributes is included in the input image, the determination control unit 22 proceeds to step S28. On the other hand, if the determination result indicates that an object with none of the multiple attributes is included in the input image, the determination control unit 22 ends the processing for the input image.

[0036] (Step S28: Notification process) The notification unit 23 notifies that an object having the attribute indicated by the determination result obtained in step S26 has been detected, and notifies the explanatory text obtained in step S26.

[0037] ***Effects of the Second Embodiment*** As described above, the object detection device 10 according to the second embodiment has the determination model 32 determine the attribute of an object contained in the input image when the confidence level is equal to or higher than the confirmation threshold but is not so high. This makes it possible to appropriately identify the attribute of the object using the determination model 32 when there is a high possibility that the object to be detected is contained. Here, the determination model 32 is a so-called generative AI. Although the processing by the generative AI takes time, it is possible to identify the attributes of an object with relatively high accuracy. The reason why the processing by the generation AI is so accurate is thought to be because it is possible to make judgments taking into account surrounding information when identifying the attributes of the detected object. For example, if the attribute information is a white cane, even if the person is holding a white cane, it will not be determined that a white cane has been detected if the person is dressed for hiking.

[0038] ***Other Configurations*** <Variation 1> In the second embodiment, an input image is input to the determination model 32, and it is determined whether or not an object having any of a plurality of attributes to be detected is included in the input image. Here, a case where the confidence level is equal to or greater than the confirmation threshold but is not very high may occur when the detection results are split between two of the multiple attributes targeted for detection. In other words, it may be difficult to distinguish between the two attributes, so the confidence level may not be very high. Taking such cases into consideration, the determination control unit 22 may input an input image to the determination model 32 and determine whether or not an object having any of the attributes of a plurality of attributes for which the certainty factor is equal to or greater than a lower limit is included in the input image. In other words, instead of determining whether or not an attribute of all attributes to be detected is included, it may be determined whether or not an attribute of attributes for which the certainty factor according to the object detection model 31 was relatively high is included. Alternatively, if there are multiple attributes for which the difference in certainty is less than a certain value, regardless of the level of certainty, the multiple attributes may be input and a determination may be made as to whether any of the attributes is included.

[0039] <Variation 2> If the attribute of the object indicated by the determination result obtained in the processing of step S26 is the same as the attribute indicated by the detection result obtained in step S22 during the past reference period or past reference time, the detection control unit 21 may lower the determination threshold. By lowering the determination threshold, the number of cases in which the process proceeds from step S24 to step S25 increases and the number of cases in which the process proceeds to step S26 decreases, which may reduce the occurrence of incorrect determination by the generation AI in step S26 throughout the process. Therefore, if there is a high possibility that the obtained results will be the same, it is desirable to make step S25 easier to execute.

[0040] Embodiment 3 The third embodiment differs from the first embodiment in that a description of the input image is generated and it is determined from the description whether an object with the detection attribute is included in the input image. In the third embodiment, this difference will be explained, and explanations of the same points will be omitted.

[0041] ***Explanation of Operation*** Referring to FIG. 8, the processing of the object detection device 10 according to the third embodiment will be described. The processes from step S31 to step S33 are the same as the processes from step S11 to step S13 in Fig. 2. The processes from step S36 and step S37 are the same as the processes from step S15 and step S16 in Fig. 2.

[0042] Here, the description will be given assuming that the judgment model 32 includes two separate generation AIs: an explanation generation model and an attribute detection model. The explanation generation model is a model specialized in image processing that takes image data as input and generates explanatory text for the image data. The attribute detection model is a model specialized in language processing that detects specified attributes from text. However, the judgment model 32 may be a standalone generative AI. In this case, the explanation generation model, attribute detection model, and judgment model 32 in the following description should be read as interchangeable.

[0043] (Step S34: Explanation generation process) The determination control unit 22 inputs an input image to the explanation generation model, causing it to create an explanation for the input image. 9, the determination control unit 22 inputs, together with the input image, a prompt to instruct the explanation generation model to create an explanation for the input image. At this time, the length of the explanation may be specified to be longer so that it can be expressed in more detail than the explanation for notification created in the first embodiment.

[0044] The determination control unit 22 acquires the explanation output from the explanation generation model. When the input image and prompt shown in Fig. 9 are input, the explanation shown in Fig. 10 is output.

[0045] (Step S35: Object determination process) The determination control unit 22 inputs the description acquired in step S34 and the detection attribute to the attribute detection model, and determines from the description whether or not an object with the detection attribute is included in the input image. 11, the determination control unit 22 inputs a description and attribute information indicating a detection attribute to the attribute detection model. At this time, the determination control unit 22 determines whether or not an object with the detection attribute is included in the description, and inputs a prompt to the attribute detection model to instruct it to summarize the description.

[0046] The determination control unit 22 acquires the determination result output from the attribute detection model. The determination result indicates whether or not the object with the detected attribute is included in the description. When the description, attribute information, and prompt shown in FIG. 11 are input, the determination result and summarized description are output in the format shown in FIG. 12.

[0047] ***Effects of the Third Embodiment*** As described above, the object detection device 10 according to the third embodiment generates a description of the input image and determines from the description whether an object with a detection attribute is included in the input image. By making a determination after creating a description rather than directly from the input image, the interpretability of the determination result is improved.

[0048] Furthermore, the object detection device 10 according to the third embodiment uses an explanation generation model specialized for image processing and an attribute detection model specialized for language processing, thereby enabling faster processing speed and higher processing accuracy.

[0049] ***Other Configurations*** <Variation 3> In the third embodiment, a description is input to the attribute detection model, and the description determines whether or not an object with the detection attribute is included in the description. However, the determination control unit 22 may determine whether or not an object with the detection attribute is included in the input image from the description by a simple word search. In other words, the determination control unit 22 may determine whether or not a word indicating the detection attribute is present in the description, and then determine whether or not an object with the detection attribute is included in the input image.

[0050] Embodiment 4 The fourth embodiment differs from the third embodiment in that when the confidence level output from the object detection model 31 is equal to or greater than the confirmation threshold but is not so high, the determination model 32 is made to determine which attribute of an object is included in the input image. In the fourth embodiment, this difference will be explained, and explanation of the same points will be omitted. In the fourth embodiment, it is assumed that there are multiple attributes to be detected.

[0051] ***Explanation of Operation*** The processing of the object detection device 10 according to the fourth embodiment will be described with reference to FIG. The processing from step S41 to step S43 is the same as the processing from step S31 to step S33 in FIG.

[0052] (Step S44: Selection determination process) Similar to step S24 in FIG. 5, the detection control unit 21 determines whether the certainty factor included in the detection result acquired in step S42 is equal to or greater than the selection threshold. If the certainty factor is equal to or greater than the selection threshold, the detection control unit 21 proceeds to step S45. In step S45, the processes of steps S34 to S37 in Fig. 8 are executed. On the other hand, if the certainty factor is less than the determination threshold, the detection control unit 21 proceeds to step S46.

[0053] (Step S46: Explanation generation process) 8, the determination control unit 22 inputs an input image to the explanation generation model to create an explanation for the input image. Then, the determination control unit 22 acquires the explanation output by the explanation generation model.

[0054] (Step S47: Object determination process) The determination control unit 22 inputs the description acquired in step S34 to the attribute detection model, and determines from the description whether or not an object with any of the multiple attributes of the detection target is included in the input image. For example, as shown in Fig. 14, a description and attribute information indicating multiple attributes of the detection target are input to the attribute detection model. At this time, the determination control unit 22 determines whether or not an object with the detection attribute is included in the input image, and inputs a prompt to the attribute detection model to instruct it to summarize the description.

[0055] The determination control unit 22 acquires the determination result output from the determination model 32. The determination result indicates the attributes of the object included in the input image. When the description, attribute information, and prompt shown in FIG. 14 are input, the determination result and summarized description are output in the format shown in FIG. 15. In FIG. 15, the determination result indicates that a wheelchair is included in the input image.

[0056] (Step S48: Detection and Determination Processing) As in step S27 of FIG. 5, the determination control unit 22 determines whether or not the determination result acquired in step S47 indicates that an object with any one of a plurality of attributes is included in the input image. If the determination result indicates that an object with any of the multiple attributes is included in the input image, the determination control unit 22 proceeds to step S49. On the other hand, if the determination result indicates that an object with none of the multiple attributes is included in the input image, the determination control unit 22 ends the processing for the input image.

[0057] (Step S49: Notification process) The notification unit 23 notifies the user that an object having the attribute indicated by the determination result obtained in step S47 has been detected, and notifies the user of the summarized description obtained in step S47.

[0058] ***Effects of the Fourth Embodiment*** As described above, the object detection device 10 according to the fourth embodiment has the determination model 32 determine the attribute of an object contained in the input image when the confidence level is equal to or higher than the confirmation threshold but is not so high. This makes it possible to appropriately identify the attribute of the object using the determination model 32 when there is a high possibility that the object to be detected is contained.

[0059] Embodiment 5. The fifth embodiment differs from the first to fourth embodiments in that when an object is detected, it is determined whether or not to issue a notification. In the fifth embodiment, this difference will be explained, and explanation of the same points will be omitted. In the fifth embodiment, a case will be described in which a function is added to the first embodiment. However, it is also possible to add a function to the second to fourth embodiments.

[0060] ***Configuration Description*** The configuration of an object detection device 10 according to the fifth embodiment will be described with reference to FIG. The object detection device 10 is connected to a condition determination model 33 via a communication interface 14. The condition determination model 33 is a so-called generative AI, similar to the determination model 32. The condition determination model 33 is a model that determines conditions from sentences.

[0061] ***Explanation of Operation*** The processing of the object detection device 10 according to the fifth embodiment will be described with reference to FIG. The processing from step S51 to step S55 is the same as the processing from step S11 to step S15 in FIG.

[0062] (Step S56: Condition determination process) The notification unit 23 inputs the explanatory text acquired in step S54 to the condition determination model 33, and causes the condition determination model 33 to determine the notification level from the explanatory text. For example, as shown in Fig. 18, a prompt is input to the condition determination model 33 along with an explanatory text, a level classification setting, and a notification setting. Fig. 18 is based on the premise that an object with an attribute such as a wheelchair is to be identified, and two level classification settings are set: an assistant is present (no action required) and an assistant is not present (action required). The notification setting is set to false (no notification) when an assistant is present (action required), and to true (notification) when an assistant is not present (action required).

[0063] The notification unit 23 acquires the determination result output from the condition determination model 33. The determination result indicates the result of level classification and the result of whether or not notification is possible. When the information shown in Fig. 18 is input, the result of level classification and the result of whether or not notification is possible are displayed in the format shown in Fig. 19.

[0064] (Step S57: Notification determination process) The notification unit 23 determines whether the result of the determination as to whether or not to notify, acquired in step S56, indicates that notification should be made. If the result of the notification possibility determination indicates that notification should be made, the notification unit 23 proceeds to the process at step S58. On the other hand, if the result of the notification possibility determination indicates that notification should not be made, the notification unit 23 ends the process for the input image, returns the process to step S51, and proceeds to the process for the next frame image.

[0065] (Step S58: Notification process) The notification unit 23 notifies that an object with the detection attribute has been detected and the explanatory text obtained in step 54 .

[0066] ***Effects of the Fifth Embodiment*** As described above, when an object is detected, the object detection device 10 according to the fifth embodiment determines whether or not to issue a further notification. This reduces unnecessary notifications and reduces the burden on the user of responding to notifications.

[0067] ***Other Configurations*** <Variation 4> In the fifth embodiment, the condition determination process of step S56 and the notification determination process of step S57 are executed after the process of the determination control unit 22. The condition determination process of step S56 and the notification determination process of step S57 may be executed without the process of the determination control unit 22. In other words, if the certainty factor is equal to or greater than the confirmation threshold in step S53, the processes of steps S54 and S55 may be skipped and the process may proceed to step S56. In this case, double-checking using the judgment model 32 is no longer performed. However, checking is performed using the condition judgment model 33 instead of the judgment model 32. Therefore, although the roles are slightly different, a certain degree of double-checking effect can be obtained.

[0068] <Variation 5> In the above-described embodiments, each functional component is realized by software. However, as a fifth modification, each functional component may be realized by hardware. The following describes the differences between this fifth modification and the above-described embodiments.

[0069] When each functional component is realized by hardware, the object detection device 10 includes an electronic circuit instead of the processor 11, the memory 12, and the storage 13. The electronic circuit is a dedicated circuit that realizes the functions of each functional component, the memory 12, and the storage 13.

[0070] Possible electronic circuits include single circuits, composite circuits, programmed processors, parallel programmed processors, logic ICs, GAs, ASICs, and FPGAs. GA stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. Each functional component may be realized by one electronic circuit, or each functional component may be realized by distributing it among a plurality of electronic circuits.

[0071] <Variation 6> As a sixth modification, some of the functional components may be realized by hardware, and other functional components may be realized by software.

[0072] The processor 11, memory 12, storage 13, and electronic circuitry are collectively referred to as a processing circuit. In other words, the functions of the functional components are realized by the processing circuit.

[0073] Furthermore, the term "unit" in the above description may be read as a "circuit," "step," "procedure," "process," or "processing circuit."

[0074] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) When an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, the input image is input to a judgment model that is a trained model different from the object detection model, and a judgment control unit judges whether an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image. An object detection device comprising: (Appendix 2) The determination control unit inputs the input image to the determination model, generates a description of the input image, and determines from the description whether an object having a detection attribute that is an attribute of an object detected by the object detection model is included in the input image. 10. The object detection device of claim 1. (Appendix 3) The determination control unit inputs the input image and the detection attribute to the determination model, generates a description of the input image, and determines from the description whether an object of the detection attribute is included in the input image. 3. An object detection device as defined in claim 2. (Appendix 4) the object detection model outputs a detection result including a confidence level that the object having the specified attribute has been detected; When the certainty included in the detection result is equal to or greater than a confirmation threshold and equal to or greater than a selection threshold higher than the confirmation threshold, the determination control unit inputs the input image and the detection attribute to the determination model to generate a description of the input image and determine from the description whether or not an object of the detection attribute is included in the input image; when the certainty is equal to or greater than the confirmation threshold and less than the selection threshold, the determination control unit inputs the input image and a plurality of attributes to be detected to the determination model to generate a description of the input image and determine from the description whether or not an object of any of the plurality of attributes is included in the input image. 4. The object detection device of claim 3. (Appendix 5) The determination control unit inputs the description and the detection attribute to an attribute detection model that detects an attribute specified from a sentence, and determines from the description whether an object with the detection attribute is included in the input image. 3. An object detection device as defined in claim 2. (Appendix 6) the object detection model outputs a detection result including a confidence level that the object having the specified attribute has been detected; When the certainty included in the detection result is equal to or greater than a confirmation threshold and equal to or greater than a selection threshold higher than the confirmation threshold, the determination control unit inputs the explanatory text and the detection attribute to the attribute detection model, and determines from the explanatory text whether or not an object having the detection attribute is included in the input image; when the certainty is equal to or greater than the confirmation threshold and equal to or less than the selection threshold, the determination control unit inputs the explanatory text and a plurality of attributes to be detected to the attribute detection model, and determines from the explanatory text whether or not an object having any of the plurality of attributes is included in the input image. 6. The object detection device of claim 5. (Appendix 7) The object detection device further comprises: a notification unit that notifies, together with the explanatory text, that an object having the detection attribute has been detected when the determination control unit determines that the object having the detection attribute is included in the input image. 6. The object detection device according to any one of Supplementary Notes 2 to 5, comprising: (Appendix 8) The object detection device further comprises: a notification unit that notifies, together with the explanatory text, that an object having one of the plurality of attributes has been detected when the determination control unit determines that the object having one of the plurality of attributes is included in the input image. 7. The object detection device according to claim 4 or 6, comprising: (Appendix 9) The object detection device further comprises: a notification unit that inputs the explanatory text into a condition determination model that determines conditions from text, determines a notification level from the explanatory text, and notifies in a manner corresponding to the determined notification level; 9. The object detection device according to any one of Supplementary Notes 2 to 8, comprising: (Appendix 10) An object detection method in which, when an object is detected from an input image using an object detection model that detects objects with specified attributes from image data, a computer inputs the input image into a judgment model, which is a trained model different from the object detection model, and determines whether the input image contains an object with detection attributes that are attributes of the object detected by the object detection model. (Appendix 11) a determination control process in which, when an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, the input image is input to a determination model that is a trained model different from the object detection model, and the determination process determines whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image; An object detection program that causes a computer to function as an object detection device that performs the above.

[0075] The embodiments and modifications of the present disclosure have been described above. Some of these embodiments and modifications may be combined and implemented. Also, one or more of them may be implemented partially. Note that the present disclosure is not limited to the above embodiments and modifications, and various modifications are possible as needed. [Explanation of symbols]

[0076] 10 object detection device, 11 processor, 12 memory, 13 storage, 14 communication interface, 21 detection control unit, 22 judgment control unit, 23 notification unit, 31 object detection model, 32 judgment model, 33 condition judgment model.

Claims

1. a determination control unit that, when an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, inputs the input image to a determination model that is a trained model different from the object detection model, generates an explanatory text for the input image, and determines from the explanatory text whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image; Equipped with The object detection model outputs a detection result including a detection attribute and a certainty of detection for each detected object; When the detection result includes multiple detection attributes whose certainty is equal to or greater than a lower limit, the judgment control unit inputs the input image and the multiple detection attributes whose certainty is equal to or greater than a lower limit to the judgment model, and determines from the description whether an object having any of the multiple detection attributes is included in the input image.

2. a determination control unit that, when an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, inputs the input image to a determination model that is a trained model different from the object detection model, generates an explanatory text for the input image, and determines from the explanatory text whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image; Equipped with The object detection model outputs a detection result including a detection attribute and a certainty of detection for each detected object; When the detection result includes multiple attributes whose difference in confidence is smaller than a certain value, the judgment control unit inputs the input image and the multiple detection attributes whose difference in confidence is smaller than a certain value to the judgment model, and determines from the description whether an object with any of the multiple detection attributes is included in the input image.

3. The object detection device further comprises: a notification unit that notifies, together with the explanatory text, that an object having the detection attribute has been detected when the determination control unit determines that the object having the detection attribute is included in the input image. The object detection device according to claim 1 or 2, comprising:

4. The object detection device further comprises: a notification unit that inputs the explanatory text into a condition determination model that determines conditions from text, determines a notification level from the explanatory text, and notifies in a manner corresponding to the determined notification level; The object detection device according to claim 1 or 2, comprising:

5. When an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, the computer inputs the input image to a judgment model that is a trained model different from the object detection model, generates an explanatory text for the input image, and determines from the explanatory text whether an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image; The object detection model outputs a detection result including a detection attribute and a certainty of detection for each detected object; An object detection method in which, when the detection result includes multiple detection attributes whose certainty is equal to or greater than a lower limit, a computer inputs the input image and the multiple detection attributes whose certainty is equal to or greater than a lower limit to the judgment model, and determines from the description whether an object having any of the multiple detection attributes is included in the input image.

6. When an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, the computer inputs the input image to a judgment model that is a trained model different from the object detection model, generates an explanatory text for the input image, and determines from the explanatory text whether an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image; The object detection model outputs a detection result including a detection attribute and a certainty of detection for each detected object; An object detection method in which, when the detection result includes multiple attributes for which the difference in confidence is smaller than a certain value, a computer inputs the input image and multiple detection attributes for which the difference in confidence is smaller than a certain value into the judgment model, and determines from the description whether an object with any of the multiple detection attributes is included in the input image.

7. A determination control process in which, when an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, the input image is input to a determination model that is a trained model different from the object detection model, an explanatory text is generated for the input image, and whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image is determined from the explanatory text. The computer functions as an object detection device that performs the following: The object detection model outputs a detection result including a detection attribute and a certainty of detection for each detected object; In the judgment control process, if the detection result includes multiple detection attributes whose certainty is equal to or greater than a lower limit, the input image and the multiple detection attributes whose certainty is equal to or greater than a lower limit are input to the judgment model, and the object detection program determines from the description whether an object with any of the multiple detection attributes is included in the input image.

8. A determination control process in which, when an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, the input image is input to a determination model that is a trained model different from the object detection model, an explanatory text is generated for the input image, and whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image is determined from the explanatory text. The computer functions as an object detection device that performs the following: The object detection model outputs a detection result including a detection attribute and a certainty of detection for each detected object; In the judgment control process, if the detection result includes multiple attributes whose difference in confidence is smaller than a certain value, the input image and the multiple detection attributes whose difference in confidence is smaller than a certain value are input to the judgment model, and the object detection program determines from the description whether an object with any of the multiple detection attributes is included in the input image.

Images