OBJECT DETECTION DEVICE, OBJECT DETECTION METHOD, AND OBJECT DETECTION PROGRAM

Abstract

To reduce missed detections and false detections when detecting an object using an object detection model. [Solution] A detection control unit 21 inputs an input image to an object detection model 31 that detects an object with a specified attribute from image data, and detects the object from the input image. When an object is detected by the object detection model 31, a judgment control unit 22 inputs the input image to a judgment model 32, which is a trained model different from the object detection model 31, and judges whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model 31 is included in the input image.

Description

[Technical field]

[0001] The present disclosure relates to a technique for detecting an object with a specified attribute from image data. [Background technology]

[0002] Patent Document 1 describes a method of detecting an object included in image data using an object detection model trained using deep learning. In particular, Patent Document 1 describes a method of detecting an object using a first method that requires less computational effort than a method using deep learning, and then determining whether or not the detected object has a specified attribute using a second method using deep learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6782825 Summary of the Invention [Problem to be solved by the invention]

[0004] When detecting an object from image data, it is desirable to reduce the occurrence of false positives, where an object that should not be detected is detected, and false negatives, where an object that should be detected is not detected. In the method described in Patent Document 1, false positives can be reduced to a certain extent by increasing the threshold value that is the standard for determining that an object is a specified object in the second method, but false positives increase accordingly. Conversely, lowering the threshold value can reduce false negatives to a certain extent, but false positives increase accordingly. An object of the present disclosure is to reduce missed detections and false positives when detecting an object using an object detection model. [Means for solving the problem]

[0005] The object detection device according to the present disclosure comprises: When an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, the input image is input to a judgment model that is a trained model different from the object detection model, and a judgment control unit judges whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image. Equipped with. Effect of the Invention

[0006] In the present disclosure, it is possible to reduce missed detections by setting a low threshold value used when detecting an object with an object detection model. Even if the threshold value is set low, it is possible to reduce false positives by performing a double check using the object detection model and other models. [Brief description of the drawings]

[0007] [Figure 1] 1 is a configuration diagram of an object detection device 10 according to a first embodiment. [Diagram 2] 3 is a flowchart of processing by the object detection device 10 according to the first embodiment. [Diagram 3] FIG. 4 is an explanatory diagram of an input to a judgment model 32 according to the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram of an output from a determination model 32 according to the first embodiment. [Diagram 5] 11 is a flowchart of processing by the object detection device 10 according to the second embodiment. [Figure 6] FIG. 11 is an explanatory diagram of an input to a judgment model 32 according to the second embodiment. [Figure 7] FIG. 11 is an explanatory diagram of an output from a judgment model 32 according to the second embodiment. [Figure 8] 11 is a flowchart of processing by the object detection device 10 according to the third embodiment. [Figure 9] FIG. 13 is an explanatory diagram of an input to an explanation generation model in the third embodiment. [Figure 10] FIG. 13 is an explanatory diagram of output from an explanation generation model according to the third embodiment. [Figure 11]FIG. 11 is an explanatory diagram of an input to an attribute detection model according to the third embodiment. [Figure 12] FIG. 11 is an explanatory diagram of an output from an attribute detection model according to the third embodiment. [Figure 13] 11 is a flowchart of processing by the object detection device 10 according to the fourth embodiment. [Figure 14] FIG. 13 is an explanatory diagram of an input to an attribute detection model according to the fourth embodiment. [Figure 15] FIG. 13 is an explanatory diagram of an output from an attribute detection model according to the fourth embodiment. [Figure 16] FIG. 13 is a configuration diagram of an object detection device 10 according to a fifth embodiment. [Figure 17] 13 is a flowchart of processing by the object detection device 10 according to the fifth embodiment. [Figure 18] FIG. 13 is an explanatory diagram of an input to a condition determination model 33 according to the fifth embodiment. [Figure 19] FIG. 13 is an explanatory diagram of an output from a condition determination model 33 according to the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Embodiment 1 ***Configuration Description*** The configuration of an object detection device 10 according to the first embodiment will be described with reference to FIG. The object detection device 10 is a computer. The object detection device 10 includes the following hardware components: a processor 11, a memory 12, a storage 13, and a communication interface 14. The processor 11 is connected to other hardware components via signal lines and controls the other hardware components.

[0009] The processor 11 is an IC that performs processing. IC is an abbreviation for Integrated Circuit. Specific examples of the processor 11 include a CPU, a DSP, and a GPU. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0010] The memory 12 is a storage device that temporarily stores data. Specific examples of the memory 12 include SRAM and DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory.

[0011] The storage 13 is a storage device that stores data. A specific example of the storage 13 is an SSD. SSD is an abbreviation for Solid State Drive. The storage 13 may also be a portable recording medium such as an SD (registered trademark) memory card, CompactFlash (registered trademark), NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. SD is an abbreviation for Secure Digital. DVD is an abbreviation for Digital Versatile Disk.

[0012] The communication interface 14 is an interface for communicating with an external device. Specific examples of the communication interface 14 include Ethernet (registered trademark), USB, and HDMI (registered trademark) ports. USB is an abbreviation for Universal Serial Bus. HDMI is an abbreviation for High-Definition Multimedia Interface.

[0013] The object detection device 10 includes, as functional components, a detection control unit 21, a determination control unit 22, and a notification unit 23. The functions of the functional components of the object detection device 10 are realized by software. The storage 13 stores a program that realizes the function of each functional component of the object detection device 10. The program is loaded into the memory 12 by the processor 11 and executed by the processor 11. In this way, the function of each functional component of the object detection device 10 is realized.

[0014] The storage 13 stores an object detection model 31, which is a trained model. The object detection model 31 is a model that detects an object with a specified attribute from image data. The object detection model 31 is a so-called AI. AI stands for artificial intelligence. The object detection model 31 is a model constructed using deep learning.

[0015] In addition, the object detection device 10 is connected via the communication interface 14 to a determination model 32, which is a trained model. The judgment model 32 is a so-called generative AI. As a specific example, the judgment model 32 may be configured using algorithms such as BERT and GPT. BERT is an abbreviation for Bidirectional Encoder Representations from Transformers. GPT is an abbreviation for Generative Pretrained Transformer. The learning model 112 may be configured by combining a plurality of algorithms including these algorithms.

[0016] Here, the object detection model 31 is stored in the storage 13, and the judgment model 32 is connected via the communication interface 14. In other words, the object detection model 31 is inside the object detection device 10, and the judgment model 32 is outside the object detection device 10. However, both the object detection model 31 and the judgment model 32 may be inside the object detection device 10, or both the object detection model 31 and the judgment model 32 may be outside the object detection device 10.

[0017] 1 shows only one processor 11. However, there may be a plurality of processors 11, and the plurality of processors 11 may cooperate to execute programs that realize the respective functions.

[0018] ***Explanation of Operation*** The operation of the object detection device 10 according to the first embodiment will be described with reference to FIGS. The operation procedure of the object detection device 10 according to the embodiment 1 corresponds to the object detection method according to the embodiment 1. Moreover, the program for realizing the operation of the object detection device 10 according to the embodiment 1 corresponds to the object detection program according to the embodiment 1.

[0019] The process of the object detection device 10 according to the first embodiment will be described with reference to FIG. (Step S11: Image input processing) The detection control unit 21 sets a frame image constituting the video data as an input image.

[0020] (Step S12: Detection control process) The detection control unit 21 inputs an input image to the object detection model 31, and causes the object detection model 31 to detect an object from the input image. At this time, the detection control unit 21 inputs one or more attributes to be detected together with the input image to the object detection model 31, and causes the object detection model 31 to detect an object having any of the one or more attributes. The detection control unit 21 acquires the detection result output from the object detection model 31. The detection result includes, for the detected object, a detection frame indicating the detected position, attributes, and a certainty indicating the certainty of the detection.

[0021] (Step S13: Final decision process) The detection control unit 21 determines whether or not the certainty factor included in the detection result acquired in step S12 is equal to or greater than a determination threshold. If the confidence level is equal to or greater than the definite threshold, the detection control unit 21 determines that an object has been detected by the object detection model 31, and proceeds to step S14. On the other hand, if the confidence level is less than the definite threshold, the detection control unit 21 ends the processing for the input image.

[0022] (Step S14: Judgment control process) The determination control unit 22 inputs an input image to the determination model 32, and causes it to determine whether or not an object with a detection attribute, which is an attribute of an object detected by the object detection model 31, is included in the input image. At this time, the determination control unit 22 instructs the determination model 32 to determine whether or not an object with a detection attribute is included in the input image, and to create an explanatory text for the input image. For example, as shown in FIG. 3, the judgment control unit 22 inputs an input image and attribute information indicating a detection attribute as a prompt to the judgment model 32. At this time, the judgment control unit 22 judges whether or not an object of the detection attribute is included in the input image, and inputs a prompt to the judgment model 32 instructing to create an explanatory text for the input image. In FIG. 3, an instruction is given to output in the JSON Schema format. By outputting in the JSON Schema format, the output data can be easily used in subsequent processing. Here, ${attribute information} represents the attribute information specified in the input information ("white cane" in the above example). In other words, "${attribute information}" is read as "white cane". Other ${***} are similarly replaced with information specified as *** in the input information. It is preferable that the attribute information be specified so that the target attribute can be clearly identified, such as "a white cane for the visually impaired," rather than simply specifying "a white cane." If "a white cane" is simply specified, a white cane that is not for the visually impaired may also be determined to correspond to the detection attribute. Therefore, when the attribute information is simply designated as "white cane", the determination control unit 22 may operate to replace "white cane" with "white cane for visually impaired people" using preset information. Other examples of preset information include replacing "wheelchair" designated as attribute information with "wheelchair without attendant", and replacing "safety vest" designated as attribute information with "safety vest worn by construction workers". In this way, the determination control unit 22 may replace attribute information designated using preset information with detailed attribute information.

[0023] The determination control unit 22 acquires the determination result and the explanatory text output from the determination model 32. The determination result indicates whether or not an object with the detection attribute is included in the input image. When the input image and attribute information shown in Fig. 3 are input as a prompt, the determination result and explanatory text are output in the format shown in Fig. 4. In Fig. 4, the determination result is true, which indicates that the object with the detection attribute is included in the input image, and the explanatory text is "A woman is walking with a man while using a white cane...".

[0024] (Step S15: Detection and determination process) The determination control unit 22 determines whether or not the determination result acquired in step S14 indicates that an object with a detection attribute is included in the input image. If the determination result indicates that the object with the detection attribute is included in the input image, the determination control unit 22 advances the process to step S16. On the other hand, if the determination result does not indicate that the object with the detection attribute is included in the input image, the determination control unit 22 ends the process for the input image.

[0025] (Step S16: Notification process) The notification unit 23 notifies the user of the object detection device 10 that an object with the detection attribute has been detected and the explanatory text obtained in step 14. For example, the notification unit 23 displays, on a display device used by the user of the object detection device 10, that an object with the detection attribute has been detected and also displays the explanatory text.

[0026] ***Advantages of the First Embodiment*** As described above, when an object is detected from an input image by the object detection model 31, the object detection device 10 of embodiment 1 uses the judgment model 32 to determine whether or not an object with the detected attributes is present in the input image. This makes it possible to reduce missed detections and false positives. In other words, it is possible to reduce missed detections by setting a low confirmation threshold used when detecting an object with the object detection model 31. Even if the confirmation threshold is set low, double checking is performed using the object detection model 31 and the judgment model 32, so it is possible to reduce false positives.

[0027] Furthermore, when notifying that an object with a detection attribute has been detected, the object detection device 10 according to the first embodiment also notifies an explanatory text of the input image. By referring to the explanatory text, the user can easily understand the situation, improving usability.

[0028] The judgment model 32 is a so-called generative AI. Since processing by the generative AI takes time, it is difficult to perform processing in real time to judge whether or not an object is included in all frame images constituting a video. The object detection device 10 according to the first embodiment inputs an input image to the judgment model 32 only when an object is detected by the object detection model 31. This makes it easier to realize detection processing in real time using the judgment model 32, which is a generative AI.

[0029] Furthermore, the object detection device 10 according to the first embodiment can also be used for intrusion detection by devising a method for setting attribute information to be input to the determination model 32. For example, suppose that it is desired to detect a person standing on the yellow track side of a station platform. In this case, by setting the attribute information to "a person standing on the yellow track side" instead of simply setting it to "a person", it becomes possible for the determination model 32 to detect only the person standing on the yellow track side. When performing intrusion detection, it is usually necessary to set the detection target area to be the area on the yellow track side for each camera. However, the object detection device 10 according to the first embodiment can easily perform intrusion detection without such setting.

[0030] Embodiment 2 The second embodiment differs from the first embodiment in that, when the confidence level output from the object detection model 31 is equal to or higher than the determination threshold but is not so high, the determination model 32 is made to determine which object has which attribute is included in the input image. In the second embodiment, this difference will be described, and the same points will not be described. In the second embodiment, it is assumed that there are multiple attributes to be detected.

[0031] ***Explanation of Operation*** The process of the object detection device 10 according to the second embodiment will be described with reference to FIG. The processes from step S21 to step S23 are the same as the processes from step S11 to step S13 in FIG.

[0032] (Step S24: Selection determination process) The detection control unit 21 determines whether or not the confidence level included in the detection result acquired in step S22 is equal to or greater than a selection threshold value. If the confidence level is equal to or greater than the selection threshold, the detection control unit 21 advances the process to step S25. In step S25, the processes of steps S14 to S16 in Fig. 2 are executed. On the other hand, if the confidence level is less than the determination threshold, the detection control unit 21 advances the process to step S26.

[0033] (Step S26: Judgment control process) The determination control unit 22 inputs an input image to the determination model 32 and determines whether or not an object having any of a plurality of attributes to be detected is included in the input image. At this time, the determination control unit 22 instructs the determination model 32 to determine whether or not an object having any of a plurality of attributes is included in the input image, and to create an explanatory text for the input image. 6, for example, the determination control unit 22 inputs an input image and attribute information indicating a plurality of attributes of a detection target to the determination model 32. At this time, the determination control unit 22 determines whether or not an object having any of the plurality of attributes is included in the input image, and inputs a prompt to the determination model 32 to instruct the determination model 32 to create an explanatory text for the input image.

[0034] The determination control unit 22 acquires the determination result and the explanatory text output from the determination model 32. The determination result indicates the attributes of the object included in the input image. When the input image, attribute information, and prompt shown in Fig. 6 are input, the determination result and explanatory text are output in the format shown in Fig. 7. In Fig. 7, the determination result indicates that a white cane is included in the input image, and the explanatory text is "A woman is walking with a man while using a white cane...".

[0035] (Step S27: Detection and determination process) The determination control unit 22 determines whether or not the determination result acquired in step S26 indicates that an object having any one of the multiple attributes is included in the input image. If the determination result indicates that an object having any one of the multiple attributes is included in the input image, the determination control unit 22 advances the process to step S28. On the other hand, if the determination result indicates that an object having none of the multiple attributes is included in the input image, the determination control unit 22 ends the process for the input image.

[0036] (Step S28: Notification process) The notification unit 23 notifies the user that an object having the attribute indicated by the determination result obtained in step S26 has been detected, and notifies the user of the explanatory text obtained in step S26.

[0037] ***Effects of the second embodiment*** As described above, the object detection device 10 according to the second embodiment causes the determination model 32 to determine which object's attribute is included in the input image when the confidence level is equal to or higher than the determination threshold but is not so high. This makes it possible to appropriately identify the object's attribute using the determination model 32 when the object to be detected is likely to be included. Here, the judgment model 32 is a so-called generative AI. Although the processing by the generative AI takes time, it is possible to identify the attributes of an object with relatively high accuracy. The reason why the processing by the generation AI is so accurate is thought to be because it is possible to make judgments taking into account surrounding information when identifying the attributes of the detected object. For example, if the attribute information is a white cane, even if the person is holding a white cane, if the person is dressed for hiking, it will not be determined that a white cane has been detected.

[0038] ***Other configurations*** <Variation 1> In the second embodiment, an input image is input to the determination model 32, and it is determined whether or not an object having any of a plurality of attributes to be detected is included in the input image. Here, a case where the certainty is equal to or greater than the confirmation threshold but is not very high may occur when the detection result is split between two of the multiple attributes targeted for detection, etc. In other words, it may be difficult to distinguish between the two attributes, so the certainty may not be very high. Taking such a case into consideration, the determination control unit 22 may input an input image to the determination model 32 and determine whether or not an object having any of the attributes of a plurality of attributes whose certainty is equal to or greater than a lower limit is included in the input image. In other words, instead of determining whether or not any of all attributes to be detected are included, it may be determined whether or not any of the attributes whose certainty was relatively high according to the object detection model 31 are included. Alternatively, if there are multiple attributes whose difference in certainty is smaller than a certain value regardless of the level of certainty, the multiple attributes may be input and a determination may be made as to whether any of the attributes are included.

[0039] <Variation 2> If the attribute of the object indicated by the determination result acquired in the processing of step S26 is the same as the attribute indicated by the detection result acquired in step S22 in the past reference period or past reference time, the detection control unit 21 may lower the determination threshold. By lowering the determination threshold, the number of cases proceeding from step S24 to step S25 increases and the number of cases proceeding to step S26 decreases, which may reduce the occurrence of incorrect determination by the generation AI in step S26 as a whole process. Therefore, if there is a high possibility that the obtained results will be the same, it is preferable to make step S25 easier to execute.

[0040] Embodiment 3 The third embodiment differs from the first embodiment in that an explanatory text for an input image is generated, and it is determined from the explanatory text whether an object with a detection attribute is included in the input image. In the third embodiment, this difference will be explained, and explanations of the same points will be omitted.

[0041] ***Explanation of Operation*** The process of the object detection device 10 according to the third embodiment will be described with reference to FIG. The processes in steps S31 to S33 are the same as those in steps S11 to S13 in Fig. 2. The processes in steps S36 and S37 are the same as those in steps S15 and S16 in Fig. 2.

[0042] Here, the judgment model 32 will be described as including two separate generation AIs, an explanation generation model and an attribute detection model. The explanation generation model is a model specialized in image processing that takes image data as input and generates an explanation for the image data. The attribute detection model is a model specialized in language processing that detects specified attributes from a sentence. However, the judgment model 32 may be a standalone generative AI. In this case, the explanation generation model, the attribute detection model, and the judgment model 32 in the following description may be read as such.

[0043] (Step S34: Explanation generation process) The determination control unit 22 inputs an input image to the explanation generation model, and causes the model to create an explanation for the input image. For example, as shown in Fig. 9, the determination control unit 22 inputs a prompt to generate an explanatory text for the input image together with the input image to the explanatory text generation model. At this time, the length of the explanatory text may be specified to be longer so that the explanatory text can be expressed in more detail than the explanatory text for notification generated in the first embodiment.

[0044] The determination control unit 22 acquires the explanatory text output from the explanation generation model. When the input image and the prompt shown in Fig. 9 are input, the explanatory text as shown in Fig. 10 is output.

[0045] (Step S35: Object determination process) The determination control unit 22 inputs the explanatory text acquired in step S34 and the detection attribute to the attribute detection model, and causes the model to determine, from the explanatory text, whether or not an object with the detection attribute is included in the input image. For example, as shown in Fig. 11, the determination control unit 22 inputs a description and attribute information indicating a detection attribute to the attribute detection model. At this time, the determination control unit 22 determines whether or not an object of the detection attribute is included in the description, and inputs a prompt to the attribute detection model to instruct the description to be summarized.

[0046] The determination control unit 22 obtains the determination result output from the attribute detection model. The determination result indicates whether or not an object with the detected attribute is included in the description. When the description, attribute information, and prompt shown in FIG. 11 are input, the determination result and the summarized description are output in the format shown in FIG. 12.

[0047] ***Effects of the Third Embodiment*** As described above, the object detection device 10 according to the third embodiment generates a description of the input image and judges from the description whether or not an object with a detection attribute is included in the input image. By making a judgment after creating a description rather than directly from the input image, the interpretability of the judgment result is improved.

[0048] Moreover, the object detection device 10 according to the third embodiment uses an explanation generation model specialized for image processing and an attribute detection model specialized for language processing, which makes it possible to increase the processing speed and the processing accuracy.

[0049] ***Other configurations*** <Variation 3> In the third embodiment, a description is input to the attribute detection model, and whether or not an object with the detection attribute is included in the description is determined from the description. However, the determination control unit 22 may determine whether or not an object with the detection attribute is included in the input image from the description by a simple text search. In other words, the determination control unit 22 may determine whether or not a word indicating the detection attribute is present in the description, and determine whether or not an object with the detection attribute is included in the input image.

[0050] Embodiment 4 The fourth embodiment differs from the third embodiment in that, when the confidence level output from the object detection model 31 is equal to or higher than the determination threshold but is not so high, the determination model 32 is made to determine which object has which attribute is included in the input image. In the fourth embodiment, this difference will be described, and the same points will not be described. In the fourth embodiment, it is assumed that there are multiple attributes to be detected.

[0051] ***Explanation of Operation*** The process of the object detection device 10 according to the fourth embodiment will be described with reference to FIG. The processing from step S41 to step S43 is the same as the processing from step S31 to step S33 in FIG.

[0052] (Step S44: Selection determination process) Similar to step S24 in FIG. 5, the detection control unit 21 determines whether or not the certainty factor included in the detection result acquired in step S42 is equal to or greater than the selection threshold value. If the confidence level is equal to or greater than the selection threshold, the detection control unit 21 advances the process to step S45. In step S45, the processes of steps S34 to S37 in Fig. 8 are executed. On the other hand, if the confidence level is less than the determination threshold, the detection control unit 21 advances the process to step S46.

[0053] (Step S46: Explanation generation process) 8, the determination control unit 22 inputs the input image to the explanation generation model to create an explanation for the input image. Then, the determination control unit 22 acquires the explanation output by the explanation generation model.

[0054] (Step S47: Object determination process) The determination control unit 22 inputs the explanatory text acquired in step S34 to the attribute detection model, and causes the model to determine from the explanatory text whether or not an object having any of the multiple attributes of the detection target is included in the input image. For example, as shown in Fig. 14, a description and attribute information indicating a plurality of attributes of a detection target are input to the attribute detection model. At this time, the determination control unit 22 determines whether or not an object with the detection attribute is included in the input image, and inputs a prompt to the attribute detection model to instruct it to summarize the description.

[0055] The determination control unit 22 acquires the determination result output from the determination model 32. The determination result indicates the attributes of the object included in the input image. When the explanatory text, attribute information, and prompt shown in FIG. 14 are input, the determination result and a summarized explanatory text are output in a format as shown in FIG. 15. In FIG. 15, the determination result indicates that a wheelchair is included in the input image.

[0056] (Step S48: Detection and determination process) Similar to step S27 in FIG. 5, the determination control unit 22 determines whether or not the determination result acquired in step S47 indicates that an object having any one of a plurality of attributes is included in the input image. If the determination result indicates that an object having any one of the multiple attributes is included in the input image, the determination control unit 22 advances the process to step S49. On the other hand, if the determination result indicates that an object having none of the multiple attributes is included in the input image, the determination control unit 22 ends the process for the input image.

[0057] (Step S49: Notification process) The notification unit 23 notifies the user that an object having the attributes indicated by the determination result obtained in step S47 has been detected, and notifies the user of the summarized description obtained in step S47.

[0058] ***Effects of the fourth embodiment*** As described above, the object detection device 10 according to the fourth embodiment causes the determination model 32 to determine which object's attribute is included in the input image when the confidence level is equal to or higher than the determination threshold but is not so high. This makes it possible to appropriately identify the object's attribute using the determination model 32 when the object to be detected is likely to be included.

[0059] Embodiment 5. The fifth embodiment differs from the first to fourth embodiments in that, when an object is detected, it is determined whether or not to issue a notification. In the fifth embodiment, this difference will be described, and a description of the same points will be omitted. In the fifth embodiment, a case will be described in which functions are added to the first embodiment. However, it is also possible to add functions to the second to fourth embodiments.

[0060] ***Configuration Description*** The configuration of an object detection device 10 according to the fifth embodiment will be described with reference to FIG. The object detection device 10 is connected to a condition determination model 33 via the communication interface 14. The condition determination model 33 is a so-called generative AI, similar to the determination model 32. The condition determination model 33 is a model that determines conditions from sentences.

[0061] ***Explanation of Operation*** The process of the object detection device 10 according to the fifth embodiment will be described with reference to FIG. The processing from step S51 to step S55 is the same as the processing from step S11 to step S15 in FIG.

[0062] (Step S56: Condition determination process) The notification unit 23 inputs the explanatory text acquired in step S54 to the condition determination model 33, and causes the condition determination model 33 to determine the notification level from the explanatory text. For example, as shown in Fig. 18, a prompt is input to the condition determination model 33 together with an explanatory text, a level setting, and a notification setting. In Fig. 18, it is assumed that an object with an attribute such as a wheelchair is to be identified, and two level settings are set: when an assistant is present (no action required) and when an assistant is not present (action required). The notification setting is set to false (no notification) when an assistant is present (action required), and to true (notification) when an assistant is not present (action required).

[0063] The notification unit 23 acquires the judgment result output from the condition judgment model 33. The judgment result indicates the result of the level classification and the result of whether or not the notification is possible. When the information shown in Fig. 18 is input, the result of the level classification and the result of whether or not the notification is possible are displayed in the format shown in Fig. 19.

[0064] (Step S57: Notification determination process) The notification unit 23 determines whether or not the result of the determination as to whether or not to notify, acquired in step S56, indicates that notification should be made. If the result of the notification possibility indicates that notification should be performed, the notification unit 23 advances the process to step S58. On the other hand, if the result of the notification possibility indicates that notification should not be performed, the notification unit 23 ends the process for the input image, returns the process to step S51, and moves to the process for the next frame image.

[0065] (Step S58: Notification process) The notification unit 23 notifies the user that an object with the detection attribute has been detected, and the explanatory text obtained in step 54 .

[0066] ***Effects of the Fifth Embodiment*** As described above, when an object is detected, the object detection device 10 according to the fifth embodiment determines whether or not to issue a further notification. This reduces unnecessary notifications and eases the burden on the user of responding to notifications.

[0067] ***Other configurations*** <Modification 4> In the fifth embodiment, the condition determination process in step S56 and the notification determination process in step S57 are executed after the process of the determination control unit 22. The condition determination process in step S56 and the notification determination process in step S57 may be executed without the process of the determination control unit 22. In other words, if the certainty factor in step S53 is equal to or greater than the final threshold, the processes in steps S54 and S55 may be skipped and the process may proceed to step S56. In this case, double checking using the judgment model 32 is no longer performed. However, checking is performed using the condition judgment model 33 instead of the judgment model 32. Therefore, although the roles are slightly different, a certain degree of double checking effect can be obtained.

[0068] <Variation 5> In the above-described embodiment, each functional component is realized by software. However, as a modification 5, each functional component may be realized by hardware. The following describes the differences between the above-described embodiment and the modification 5.

[0069] When each functional component is realized by hardware, the object detection device 10 includes an electronic circuit instead of the processor 11, the memory 12, and the storage 13. The electronic circuit is a dedicated circuit for realizing the functions of each functional component, the memory 12, and the storage 13.

[0070] The electronic circuits include single circuits, composite circuits, programmed processors, parallel programmed processors, logic ICs, GAs, ASICs, and FPGAs. GA stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. Each functional component may be realized by one electronic circuit, or each functional component may be realized by distributing it among a plurality of electronic circuits.

[0071] <Variation 6> As a sixth modification, some of the functional components may be realized by hardware, and other functional components may be realized by software.

[0072] The processor 11, the memory 12, the storage 13, and the electronic circuitry are collectively referred to as a processing circuit. In other words, the functions of the functional components are realized by the processing circuit.

[0073] Furthermore, the term "part" in the above description may be read as a "circuit," "step," "procedure," "processing," or "processing circuit."

[0074] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) When an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, the input image is input to a judgment model that is a trained model different from the object detection model, and a judgment control unit judges whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image. An object detection device comprising: (Appendix 2) The determination control unit inputs the input image to the determination model, generates an explanatory text for the input image, and determines from the explanatory text whether or not an object having a detection attribute that is an attribute of an object detected by the object detection model is included in the input image. 2. An object detection device as described in claim 1. (Appendix 3) The determination control unit inputs the input image and the detection attribute to the determination model, generates an explanatory text for the input image, and determines whether or not an object having the detection attribute is included in the input image from the explanatory text. 3. An object detection device as described in claim 2. (Appendix 4) The object detection model outputs a detection result including a confidence level that an object having the specified attribute has been detected; When the certainty included in the detection result is equal to or greater than a confirmation threshold and the certainty is equal to or greater than a selection threshold higher than the confirmation threshold, the determination control unit inputs the input image and the detection attribute to the determination model to generate an explanatory text for the input image and determine from the explanatory text whether or not an object of the detection attribute is included in the input image; when the certainty is equal to or greater than the confirmation threshold and less than the selection threshold, the determination control unit inputs the input image and a plurality of attributes to be detected to the determination model to generate an explanatory text for the input image and determine from the explanatory text whether or not an object of any of the plurality of attributes is included in the input image. 4. An object detection device as described in claim 3. (Appendix 5) The determination control unit inputs the explanatory text and the detection attribute to an attribute detection model that detects a specified attribute from a sentence, and determines whether or not an object having the detection attribute is included in the input image from the explanatory text. 3. An object detection device as described in claim 2. (Appendix 6) The object detection model outputs a detection result including a confidence level that an object having the specified attribute has been detected; The determination control unit, when the certainty included in the detection result is equal to or greater than a determination threshold and the certainty is equal to or greater than a selection threshold higher than the determination threshold, inputs the explanatory text and the detection attribute to the attribute detection model, and determines from the explanatory text whether or not an object having the detection attribute is included in the input image, and when the certainty is equal to or greater than the determination threshold and less than the selection threshold, inputs the explanatory text and a plurality of attributes to be detected to the attribute detection model, and determines from the explanatory text whether or not an object having any of the plurality of attributes is included in the input image. 6. An object detection device as described in claim 5. (Appendix 7) The object detection device further comprises: a notification unit that notifies, when the determination control unit determines that the object having the detection attribute is included in the input image, that the object having the detection attribute has been detected together with the explanatory text. 6. An object detection device according to any one of claims 2 to 5, comprising: (Appendix 8) The object detection device further comprises: a notification unit that notifies, when the determination control unit determines that an object having any one of the plurality of attributes is included in the input image, that an object having any one of the plurality of attributes has been detected together with the explanatory text. 7. An object detection device according to claim 4 or 6, comprising: (Appendix 9) The object detection device further comprises: A notification unit that inputs the explanatory text into a condition determination model that determines a condition from a text, determines a notification level from the explanatory text, and performs a notification in a manner corresponding to the determined notification level. 9. An object detection device according to any one of claims 2 to 8, comprising: (Appendix 10) An object detection method in which, when an object is detected from an input image by an object detection model that detects objects with specified attributes from image data, a computer inputs the input image into a judgment model, which is a trained model different from the object detection model, and determines whether the input image contains an object with detection attributes that are attributes of the object detected by the object detection model. (Appendix 11) a judgment control process for inputting the input image to a judgment model, which is a trained model different from the object detection model, when an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, and judging whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image; An object detection program that causes a computer to function as an object detection device that performs the above-mentioned steps.

[0075] The above describes the embodiments and modifications of the present disclosure. Some of these embodiments and modifications may be combined and implemented. Also, one or some of them may be partially implemented. Note that the present disclosure is not limited to the above embodiments and modifications, and various modifications are possible as necessary. [Explanation of symbols]

[0076] 10 object detection device, 11 processor, 12 memory, 13 storage, 14 communication interface, 21 detection control unit, 22 judgment control unit, 23 notification unit, 31 object detection model, 32 judgment model, 33 condition judgment model.

Claims

1. A determination control unit that, when an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, inputs the input image to a determination model that is a trained model different from the object detection model, generates an explanatory text for the input image, and determines from the explanatory text whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image. An object detection device comprising:

2. The determination control unit inputs the input image and the detection attribute to the determination model, generates an explanatory text for the input image, and determines whether or not an object having the detection attribute is included in the input image from the explanatory text. The object detection device according to claim 1 .

3. The object detection model outputs a detection result including a confidence level that an object having the specified attribute has been detected; When the certainty included in the detection result is equal to or greater than a confirmation threshold and the certainty is equal to or greater than a selection threshold higher than the confirmation threshold, the determination control unit inputs the input image and the detection attribute to the determination model to generate an explanatory text for the input image and determine from the explanatory text whether or not an object of the detection attribute is included in the input image; when the certainty is equal to or greater than the confirmation threshold and less than the selection threshold, the determination control unit inputs the input image and a plurality of attributes to be detected to the determination model to generate an explanatory text for the input image and determine from the explanatory text whether or not an object of any of the plurality of attributes is included in the input image. The object detection device according to claim 2 .

4. The determination control unit inputs the explanatory text and the detection attribute to an attribute detection model that detects a specified attribute from a sentence, and determines whether or not an object having the detection attribute is included in the input image from the explanatory text. The object detection device according to claim 1 .

5. The object detection model outputs a detection result including a confidence level that an object having the specified attribute has been detected; The determination control unit, when the certainty included in the detection result is equal to or greater than a determination threshold and the certainty is equal to or greater than a selection threshold higher than the determination threshold, inputs the explanatory text and the detection attribute to the attribute detection model, and determines from the explanatory text whether or not an object having the detection attribute is included in the input image, and when the certainty is equal to or greater than the determination threshold and less than the selection threshold, inputs the explanatory text and a plurality of attributes to be detected to the attribute detection model, and determines from the explanatory text whether or not an object having any of the plurality of attributes is included in the input image. The object detection device according to claim 4.

6. The object detection device further comprises: a notification unit that notifies, when the determination control unit determines that the object having the detection attribute is included in the input image, that the object having the detection attribute has been detected together with the explanatory text. The object detection device according to claim 1 .

7. The object detection device further comprises: a notification unit that notifies, when the determination control unit determines that an object having any one of the plurality of attributes is included in the input image, that an object having any one of the plurality of attributes has been detected together with the explanatory text. The object detection device according to claim 3 .

8. The object detection device further comprises: A notification unit that inputs the explanatory text into a condition determination model that determines a condition from a text, determines a notification level from the explanatory text, and performs a notification in a manner corresponding to the determined notification level. The object detection device according to claim 1 .

9. An object detection method in which, when an object is detected from an input image by an object detection model that detects objects with specified attributes from image data, a computer inputs the input image into a judgment model, which is a trained model different from the object detection model, generates an explanatory text for the input image, and determines from the explanatory text whether an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image.

10. A judgment control process for inputting an input image to a judgment model, which is a trained model different from the object detection model, when an object is detected from an input image by an object detection model that detects an object with a specified attribute from image data, generating an explanatory text for the input image, and judging from the explanatory text whether or not an object with a detection attribute that is an attribute of the object detected by the object detection model is included in the input image. An object detection program that causes a computer to function as an object detection device that performs the above-mentioned steps.

Images