Information processing system, information processing device, method, and program

Abstract

This information processing system comprises: an acquisition unit that acquires information indicating an estimated scene output by an image recognition model that recognizes an input image, the estimated scene being represented in the image, and object information output by the image recognition model and indicating at least one of an estimated human or an estimated non-human object included in the image; a determination unit that determines the degree of matching between at least one of a human or a non-human object indicated by explanatory information associated with the acquired information indicating the estimated scene and at least one of the human or the non-human object indicated by the acquired object information; and a correction unit that, on the basis of the determined degree of matching, corrects at least one of an index value indicating reliability of the estimation by the image recognition model or a first threshold of the estimated scene used for determination of the index value.

Description

Information Processing System, Information Processing Apparatus, Method, and Program

[0001] The present disclosure relates to an information processing system, an information processing apparatus, a method, and a program for recognizing an image.

[0002] Various techniques for recognizing an image have been conventionally proposed. As an example of such a technique, an arithmetic device disclosed in Patent Document 1 uses sensor data from a sensor group that detects a recognition target and a first classifier that classifies the recognition target to calculate the recognition result of the recognition target and the reliability of the recognition result, and classifies the sensor data into either a related target with which the recognition result is associated or an unrelated target with which the recognition result is not associated based on the calculated reliability of the recognition result.

[0003] International Publication No. 2020 / 183776

[0004] However, the arithmetic device disclosed in Patent Document 1 does not detect a scene represented in an image. Therefore, there has been a problem that an object recognition result cannot be appropriately output based on the reliability of estimation by an image recognition model.

[0005] One object of the present disclosure is to provide, in view of the above problem, an information processing system, an information processing apparatus, a method, and a program capable of appropriately outputting an object recognition result based on the reliability of estimation by an image recognition model.

[0006] The information processing system according to the present disclosure comprises: an acquisition unit that acquires information indicating an estimated scene represented in an image output by an image recognition model that recognizes an input image, and object information indicating at least one of an estimated person and an object included in the image output by the image recognition model; a determination unit that determines a degree of matching between at least one of a person and an object indicated by description information associated with the acquired information indicating the estimated scene and at least one of a person and an object indicated by the acquired object information; and a correction unit that corrects at least one of an index value indicating reliability of estimation by the image recognition model and a first threshold for the estimated scene used for determination of the index value based on the determined degree of matching.

[0007] The information processing device relating to this disclosure includes: an acquisition unit that acquires information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image; object information indicating at least one of an estimated person and object included in the image, output by the image recognition model; a determination unit that determines the degree of agreement between at least one of the person and object indicated by explanatory information associated with the acquired estimated scene information and at least one of the person and object indicated by the acquired object information; and a correction unit that corrects at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold value for the estimated scene used to determine the index value, based on the determined degree of agreement.

[0008] The method relating to this disclosure involves a computer acquiring information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image, and object information indicating at least one of an estimated person and object contained in the image, output by the image recognition model; determining the degree of agreement between at least one of the person and object indicated by the explanatory information associated with the acquired estimated scene information and at least one of the person and object indicated by the acquired object information; and correcting at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold for the estimated scene used to determine the index value, based on the determined degree of agreement.

[0009] The program relating to this disclosure causes a computer to obtain information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image, and object information indicating at least one of an estimated person and object contained in the image, output by the image recognition model; to determine the degree of agreement between at least one of the person and object indicated by the explanatory information associated with the obtained estimated scene information and at least one of the person and object indicated by the obtained object information; and to correct at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold value for the estimated scene used to determine the index value, based on the determined degree of agreement.

[0010] This disclosure provides an information processing system, information processing device, method, and program that can appropriately output object recognition results based on the reliability of estimations by an image recognition model.

[0011] This figure shows an example of the hardware configuration of the information processing device related to this disclosure. This figure shows the functional configuration of the information processing device related to this disclosure. This figure shows an example of a threshold table. This figure shows an example of a log information table. This figure shows the input and output information of the image recognition model related to this disclosure. This flowchart shows an example of a process performed by the information processing device related to this disclosure. This flowchart shows another example of a process performed by the information processing device related to this disclosure. This flowchart shows an example of a threshold update process. This figure shows the main components of the information processing system related to this disclosure. This flowchart shows the main processes performed by the information processing device related to this disclosure.

[0012] Hereinafter, an exemplary embodiment will be described with reference to the drawings. Figure 1 is a diagram showing an example of the hardware configuration of the information processing device 10 according to this disclosure. Specific examples of the information processing device 10 include various information processing devices such as servers and PCs (Personal Computers). The information processing device 10 corresponds to an information processing system.

[0013] The information processing device 10 comprises a processor 11, a communication interface (I / F) 12, a memory 13, a storage device 14, and a bus 15. These devices exchange data via the bus 15.

[0014] The processor 11 is a device that performs overall control of the information processing device 10. Specific examples of the processor 11 include CPUs (Central Processing Units), GPUs (Graphics Processing Units), MPUs (Micro Processing Units), and other processors. The processor 11 loads instructions contained in the program stored in the storage device 14 into the memory 13 and executes them. In other embodiments, at least a portion of the functions performed by the processor 11 may also be performed by integrated circuits such as FPGAs (Field-Programmable Gate Arrays) and ASICs (Application Specific Integrated Circuits). These devices are equivalent to computers. Furthermore, the information processing device 10 can be equipped with multiple processors 11.

[0015] The communication interface 12 is an interface for data communication between the information processing device 10 and other devices. The storage device 14 is a storage device that stores various data such as programs executed by the processor 11 and data processed by the processor 11.

[0016] Figure 2 shows the functional configuration of the information processing device 10 according to this disclosure. The information processing device 10 includes an acquisition unit 110, an image recognition unit 111, an image recognition model 112, a recognition result output unit 113, a score determination unit 114, a registration unit 115, an object determination unit 116, a correction unit 117, and an acquired information determination unit 118. These functions can be implemented by a program.

[0017] The acquisition unit 110 acquires the image to be detected. The acquisition unit 110 also acquires a first threshold for each scene ID (Identification), which is identification information of the scene (situation) represented by the image, from the threshold table stored in the storage device 14. Figure 3 is a diagram showing an example of a threshold table. The threshold table registers the first threshold for each scene. More specifically, the threshold table registers the first threshold for each scene ID and descriptive information representing the scene indicated by that scene ID, associated with each scene ID. The descriptive information includes objects such as people and objects. In this embodiment, the initial value of the first threshold for each scene ID is set according to the score, which is an index value indicating the reliability of the estimation by the image recognition model 112. For example, if the image recognition model 112 calculates a score in the range of 0 or more and 1 or less, the initial value of the first threshold can be 1 or a value close to 1. The higher the score, the higher the reliability of the estimation by the image recognition model 112. Therefore, by setting the initial value of the first threshold to a high value, false detection of scenes and objects can be suppressed.

[0018] Furthermore, the acquisition unit 110 acquires log information from the log information table in the storage device 14. Figure 4 shows an example of the log information table. The log information table registers the scene ID, object information, and score output by the image recognition model 112. The scene ID is the identification information of the scene estimated by the image recognition model 112. The object information is the identification information of the object estimated by the image recognition model 112. The object includes at least one of a person and an object. In addition, time information such as the image generation time or image recognition time is registered in the log information table in association with the scene ID. Furthermore, the descriptive information registered in the threshold table is registered in the log information table in association with the scene ID.

[0019] The image recognition unit 111 inputs an image to the image recognition model 112 and outputs a score, scene ID, and object information.

[0020] As shown in Figure 5, the image recognition model 112 is a machine learning model that, upon input of an image, outputs the scene ID of the estimated scene represented in the image, object information indicating the estimated objects contained in the image, and a score. The score is a numerical value in the range of 0 or greater and 1 or less. Specific examples of the image recognition model 112 include large-scale language models such as Vision Transformer. The image recognition model 112 can be trained using machine learning methods such as deep learning.

[0021] The score determination unit 114 determines whether the score output by the image recognition model 112 is equal to or greater than a first threshold associated with the scene ID output by the image recognition model 112.

[0022] Furthermore, the score determination unit 114 may determine whether the score output by the image recognition model 112 is greater than or equal to a second threshold (minimum threshold) which is smaller than a first threshold associated with the scene ID output by the image recognition model 112. The second threshold can be any value smaller than all of the first thresholds. The second threshold is set to suppress false detections by the image recognition model 112. The second threshold may be common to all scenes, or it may be set individually for each scene.

[0023] The recognition result output unit 113 outputs the recognition result of the image recognition model 112 corresponding to an index value greater than or equal to the first threshold to an output device such as a display device. Specifically, the recognition result output unit 113 outputs the recognition result based on the scene ID and object information output by the image recognition model 112, and the score output by the image recognition model 112 to the output device.

[0024] The registration unit 115 registers the score, scene ID, and object information output by the image recognition model 112 in the log information table. The registration unit 115 also registers the threshold values ​​corrected by the correction unit 117 in the threshold table.

[0025] The object determination unit 116 determines the degree of agreement between the object indicated by the object information output by the image recognition model 112 and the object indicated by the descriptive information associated with the scene ID output by the image recognition model 112. Specifically, the object determination unit 116 can determine whether the degree of agreement between these objects is above a predetermined threshold. If the degree of agreement between these objects is above the predetermined threshold, the object determination unit 116 determines that these objects are a match. On the other hand, if the degree of agreement between these objects is below the predetermined threshold, the object determination unit 116 determines that these objects are not a match.

[0026] The correction unit 117 corrects the first threshold associated with the scene ID based on the degree of agreement between the object indicated by the object information output by the image recognition model 112 and the object indicated by the descriptive information associated with the scene ID output by the image recognition model 112.

[0027] Specifically, the correction unit 117 reduces the score associated with objects that are determined not to match. For example, in the example shown in Figure 4, the object "child" indicated by the descriptive information associated with the scene ID "scene_1" output by the image recognition model 112, which is identified by the time information "20241115134142020", does not match the object "woman" indicated by the object information output by the image recognition model 112. In this case, the correction unit 117 reduces the score "0.61" associated with this descriptive information and object information. For example, the correction unit 117 can subtract an arbitrary value, such as 0.1, from the score. Then, the correction unit 117 can correct the first threshold by multiplying the subtracted score by the first threshold associated with the scene ID. This improves the undetection of scenes and objects caused by setting a high initial value for the first threshold.

[0028] In this embodiment, it is preferable that the correction unit 117 corrects the first threshold associated with the scene ID not only using the subtracted score, but also using other scores associated with the same scene ID. In this case, the correction unit 117 can correct the first threshold by taking a weighted average of each score associated with the same scene ID and multiplying the resulting value by the first threshold associated with the scene ID.

[0029] Alternatively, the correction unit 117 may directly correct the first threshold without correcting the score. In this case, the correction unit 117 can correct the first threshold by reducing it, for example by subtracting an arbitrary value from the first threshold.

[0030] The information acquisition determination unit 118 determines whether or not there is other descriptive information and object information to be acquired for each scene ID registered in the log information table.

[0031] Figure 6 is a flowchart showing an example of a process performed by the information processing device 10. In step S1, the acquisition unit 110 acquires the image to be detected and a first threshold value for each scene ID. In step S2, the image recognition unit 111 inputs the image to the image recognition model 112 and outputs a score, scene ID, and object information.

[0032] In step S3, the score determination unit 114 determines whether the score output in step S2 is equal to or greater than a first threshold associated with the scene ID output in step S2. If it is determined that the score is equal to or greater than the first threshold (YES), the process branches to step S4. In step S4, the recognition result output unit 113 outputs the recognition result based on the scene ID and object information output in step S2, along with the score output in step S2, to the output device.

[0033] On the other hand, if it is determined that the score output in step S2 is less than the first threshold (NO), the process branches to step S5. In step S5, the score determination unit 114 determines whether the score output in step S2 is equal to or greater than the second threshold. If it is determined that the score output in step S2 is less than the second threshold (NO), the process shown in Figure 6 ends. As a result, if the score is less than the second threshold, which is the minimum threshold, the score, scene ID, and object information are not registered in the log information table. Therefore, since the first threshold is not corrected using the score, it is possible to suppress false detection of scenes and objects that may occur due to an excessive reduction in the first threshold.

[0034] On the other hand, if it is determined that the score output in step S2 is equal to or greater than the second threshold (YES), the process branches to step S6.

[0035] In step S6, the registration unit 115 registers the score, scene ID, and object information output in step S2 into the log information table, and the process shown in Figure 6 is completed. At this time, the registration unit 115 also registers the descriptive information associated with the scene ID in the threshold table into the log information table.

[0036] Figure 7 is a flowchart showing another example of the process performed by the information processing device 10. In step S11, the acquisition unit 110 acquires log information from the storage device 14. In step S12, the information processing device 10 performs a threshold update process for each scene ID, and the process shown in Figure 7 is completed.

[0037] Figure 8 is a flowchart showing an example of threshold update processing. In step S21, the acquisition unit 110 acquires descriptive information and object information associated with a certain scene ID from the log information table. In step S22, the object determination unit 116 determines whether the object indicated by the acquired object information matches the object indicated by the acquired descriptive information. If it is determined that these objects do not match (NO), the process branches to step S23.

[0038] In step S23, the correction unit 117 obtains a first threshold associated with a certain scene ID from the threshold table and corrects the first threshold. In step S24, the registration unit 115 registers the corrected first threshold in association with a certain scene ID in the threshold table, and the process shown in Figure 24 is completed.

[0039] In step S25, the information acquisition determination unit 118 refers to the log information table and determines whether there is any other descriptive information and object information to be acquired for a given scene ID. If it is determined that there is any other descriptive information and object information to be acquired (YES), the process returns to step S21. On the other hand, if it is determined that there is no other descriptive information and object information to be acquired (NO), the process shown in Figure 8 ends.

[0040] As the processes described in Figures 6 to 8 are repeatedly executed, the first threshold for each scene, which is initially set to a high value, gradually decreases. This suppresses the detection of undetected scenes and objects, and optimizes the first threshold for each scene.

[0041] Figure 9 shows the main components of the information processing system 10 according to this disclosure. The information processing device 10 includes an acquisition unit 110, an object determination unit 116, and a correction unit 117. The acquisition unit 110 acquires information indicating an estimated scene represented in an image, output by an image recognition model 112 that recognizes an input image, and object information indicating at least one of an estimated person and object included in the image, output by the image recognition model 112. The object determination unit 116 determines the degree of agreement between at least one of the person and object indicated by the explanatory information associated with the acquired estimated scene information and at least one of the person and object indicated by the acquired object information. The correction unit 117 corrects at least one of an index value indicating the reliability of the estimation by the image recognition model 112 and a first threshold value for each scene used to determine the index value, based on the determined degree of agreement.

[0042] FIG. 10 is a flowchart illustrating main processing executed by an information processing apparatus 10 according to the present disclosure. The information processing apparatus 10 acquires information indicating an estimated scene represented in an image, which is output by an image recognition model 112 that recognizes an input image, and object information indicating at least one of an estimated person and an object included in the image, which is output by the image recognition model 112 (step S30). Next, the information processing apparatus 10 determines a degree of matching between at least one of a person and an object indicated by explanation information associated with the acquired information indicating the estimated scene, and at least one of a person and an object indicated by the acquired object information (step S31). Then, the information processing apparatus 10 corrects at least one of an index value indicating reliability of estimation by the image recognition model 112, and a first threshold value for the estimated scene used for determination of the index value, based on the determined degree of matching (step S32).

[0043] By adopting this configuration, the threshold value for each scene used for determining an index value indicating reliability of estimation by the image recognition model 112 can be individually corrected. As a result, an object recognition result obtained by the image recognition model 112 can be appropriately output.

[0044] Further, in the above-described embodiment, a recognition result output unit 113 outputs, to an output device, a recognition result of the image recognition model 112 corresponding to an index value equal to or greater than the first threshold. In other words, the recognition result output unit 113 does not output, to the output device, a recognition result of the image recognition model 112 corresponding to an index value less than the first threshold. An object determination unit 116 determines whether at least one of a person and an object indicated by explanation information of an estimated scene corresponding to an index value less than the first threshold and equal to or greater than a second threshold smaller than a minimum value of the first threshold for each scene matches at least one of a person and an object indicated by object information corresponding to an index value less than the first threshold and equal to or greater than the second threshold. When it is determined that at least one of these persons and objects does not match, a correction unit 117 corrects the first threshold of the estimated scene.

[0045] By adopting this configuration, the first threshold used to determine the output of the recognition result of the image recognition model 112 is corrected using recognition results corresponding to index values ​​that are below the first threshold but above the second threshold, which are not output to the output device. Scenes corresponding to index values ​​that are below the first threshold but above the second threshold may include scenes in which the image recognition model 112 has not been sufficiently trained, such as rare scenes. Therefore, by correcting the first threshold using recognition results corresponding to index values ​​that are below the first threshold but above the second threshold, the first threshold can be corrected to take into account scenes in which training has not been sufficient.

[0046] In the embodiment described above, a single information processing device 10 implements the functional means shown in Figure 2, but in other embodiments, the functional means shown in Figure 2 may be implemented by distributing them among multiple information processing devices.

[0047] In the examples described above, the program includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disk (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically, or otherwise propagating signals.

[0048] The present disclosure has been described above with reference to the embodiments, but the present disclosure is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0049] The drawings are merely illustrative for the purpose of describing one or more embodiments. Each drawing is not necessarily associated with only one specific embodiment, but may be associated with one or more other embodiments. As can be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to produce embodiments that are not explicitly illustrated or described. Not all of the features or steps shown in any one drawing to describe the exemplary embodiments are necessarily essential, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0050] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) An information processing system comprising: an acquisition unit that acquires information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image; object information indicating at least one of an estimated person and object included in the image, output by the image recognition model; a determination unit that determines the degree of agreement between at least one of the person and object indicated by explanatory information associated with the acquired estimated scene information and at least one of the person and object indicated by the acquired object information; and a correction unit that corrects at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold of the estimated scene used to determine the index value, based on the determined degree of agreement. (Note 2) The information processing system according to Note 1, further comprising a recognition result output unit that outputs the recognition result of the image recognition model corresponding to an index value greater than or equal to the first threshold. (Note 3) The information processing system according to Note 1 or 2, wherein the determination unit determines whether at least one of the people and objects indicated by the descriptive information of the estimated scene, which corresponds to an index value less than the first threshold and greater than or equal to a second threshold that is smaller than the minimum value of the first threshold for each scene, matches at least one of the people and objects indicated by the object information, which corresponds to an index value less than the first threshold and greater than or equal to the second threshold. (Note 4) The information processing system according to any one of Notes 1 to 3, wherein the correction unit determines that at least one of the people and objects indicated by the descriptive information does not match at least one of the people and objects indicated by the object information, and performs a correction to reduce the index value representing the confidence level of the estimation. (Note 5) The information processing system according to any one of Notes 1 to 3, wherein the correction unit corrects the first threshold by multiplying the first threshold of the estimated scene by a value obtained by taking a weighted average of each index value corresponding to each of the multiple pieces of information indicating the same estimated scene output by the image recognition model.(Note 6) An information processing device comprising: an acquisition unit that acquires information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image; object information indicating at least one of an estimated person and object included in the image, output by the image recognition model; a determination unit that determines the degree of agreement between at least one of a person and object indicated by explanatory information associated with the acquired estimated scene information and at least one of a person and object indicated by the acquired object information; and a correction unit that corrects at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold value of the estimated scene used to determine the index value, based on the determined degree of agreement. (Note 7) The information processing device according to Note 6, further comprising a recognition result output unit that outputs the recognition result of the image recognition model corresponding to an index value greater than or equal to the first threshold value. (Note 8) The information processing device according to Note 6 or 7, wherein the determination unit determines whether at least one of the people and objects indicated by the descriptive information of the estimated scene, which corresponds to an index value less than the first threshold and greater than or equal to a second threshold that is smaller than the minimum value of the first threshold for each scene, matches at least one of the people and objects indicated by the object information, which corresponds to an index value less than the first threshold and greater than or equal to the second threshold. (Note 9) The information processing device according to any one of Notes 6 to 8, wherein the correction unit performs a correction to reduce the index value representing the confidence level of the estimation when it is determined that at least one of the people and objects indicated by the descriptive information does not match at least one of the people and objects indicated by the object information. (Note 10) The information processing device according to any one of Notes 6 to 8, wherein the correction unit corrects the first threshold by multiplying the first threshold of the estimated scene by a value obtained by taking a weighted average of each index value corresponding to each of the multiple pieces of information indicating the same estimated scene output by the image recognition model.(Note 11) A method comprising: a computer obtaining information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image, and object information indicating at least one of an estimated person and object included in the image, output by the image recognition model; determining the degree of agreement between at least one of the person and object indicated by explanatory information associated with the obtained estimated scene information and at least one of the person and object indicated by the obtained object information; and correcting at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold for the estimated scene used to determine the index value, based on the determined degree of agreement. (Note 12) The method according to Note 11, wherein the computer outputs the recognition result of the image recognition model corresponding to an index value greater than or equal to the first threshold. (Note 13) The method according to Note 11 or 12, wherein the computer determines whether at least one of the people and objects indicated by the descriptive information of the estimated scene, which corresponds to an index value less than the first threshold and greater than or equal to a second threshold that is smaller than the minimum value of the first threshold for each scene, matches at least one of the people and objects indicated by the object information, which corresponds to an index value less than the first threshold and greater than or equal to the second threshold. (Note 14) The method according to any one of Notes 11 to 13, wherein if the computer determines that at least one of the people and objects indicated by the descriptive information does not match at least one of the people and objects indicated by the object information, it performs a correction to reduce the index value representing the confidence level of the estimation. (Note 15) The method according to any one of Notes 11 to 13, wherein the computer corrects the first threshold by multiplying the first threshold of the estimated scene by a value obtained by taking a weighted average of each index value corresponding to each of the multiple pieces of information indicating the same estimated scene output by the image recognition model.(Note 16) A program that causes a computer to acquire information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image, and object information indicating at least one of an estimated person and object included in the image, output by the image recognition model; to determine the degree of agreement between at least one of the person and object indicated by the explanatory information associated with the acquired estimated scene information and at least one of the person and object indicated by the acquired object information; and to correct at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold for the estimated scene used to determine the index value, based on the determined degree of agreement. (Note 17) The program described in Note 16, which causes the computer to output the recognition result of the image recognition model corresponding to an index value greater than or equal to the first threshold. (Note 18) A program according to Note 16 or 17 that causes the computer to determine whether at least one of the people and objects indicated by the descriptive information of the estimated scene, which corresponds to an index value less than the first threshold and greater than or equal to a second threshold that is smaller than the minimum value of the first threshold for each scene, matches at least one of the people and objects indicated by the object information, which corresponds to an index value less than the first threshold and greater than or equal to the second threshold. (Note 19) A program according to any one of Notes 16 to 18 that causes the computer to perform a correction by reducing the index value representing the confidence level of the estimation, if it is determined that at least one of the people and objects indicated by the descriptive information does not match at least one of the people and objects indicated by the object information. (Note 20) A program according to any one of Notes 16 to 18 that causes the computer to correct the first threshold by multiplying the first threshold of the estimated scene by a value obtained by taking a weighted average of each index value corresponding to each of the multiple pieces of information indicating the same estimated scene output by the image recognition model.

[0051] Some or all of the elements described in any appendix may apply to various hardware, software, recording means, systems, and methods for recording software.

[0052] 10: Information processing device, information processing system 11: Processor 12: Communication interface 13: Memory 14: Storage device 15: Bus 110: Acquisition unit 111: Image recognition unit 112: Image recognition model 113: Recognition result output unit 114: Score determination unit 115: Registration unit 116: Object determination unit 117: Correction unit 118: Acquired information determination unit

Claims

1. An information processing system comprising: an acquisition unit that acquires information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image; object information indicating at least one of an estimated person and object included in the image, output by the image recognition model; a determination unit that determines the degree of agreement between at least one of the person and object indicated by explanatory information associated with the acquired estimated scene information and at least one of the person and object indicated by the acquired object information; and a correction unit that corrects at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold value for the estimated scene used to determine the index value, based on the determined degree of agreement.

2. The information processing system according to claim 1, further comprising a recognition result output unit that outputs the recognition result of the image recognition model corresponding to an index value greater than or equal to the first threshold.

3. The information processing system according to claim 1 or 2, wherein the determination unit determines whether at least one of the people and objects indicated by the descriptive information of the estimated scene, which corresponds to an index value less than the first threshold and an index value greater than or equal to a second threshold that is smaller than the minimum value of the first threshold for each scene, matches at least one of the people and objects indicated by the object information, which corresponds to an index value less than the first threshold and greater than or equal to the second threshold.

4. The information processing system according to any one of claims 1 to 3, wherein the correction unit determines that at least one of the person and object indicated by the descriptive information does not match at least one of the person and object indicated by the object information, and performs a correction to reduce the index value representing the reliability of the estimation.

5. The information processing system according to any one of claims 1 to 3, wherein the correction unit corrects the first threshold by multiplying the first threshold of the estimated scene by a value obtained by taking a weighted average of each index value corresponding to each of the multiple pieces of information indicating the same estimated scene output by the image recognition model.

6. An information processing device comprising: an acquisition unit that acquires information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image; object information indicating at least one of an estimated person and object included in the image, output by the image recognition model; a determination unit that determines the degree of agreement between at least one of the person and object indicated by explanatory information associated with the acquired estimated scene information and at least one of the person and object indicated by the acquired object information; and a correction unit that corrects at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold value of the estimated scene used to determine the index value, based on the determined degree of agreement.

7. The information processing apparatus according to claim 6, comprising a recognition result output unit that outputs the recognition result of the image recognition model corresponding to an index value greater than or equal to the first threshold.

8. The information processing apparatus according to claim 6 or 7, wherein the determination unit determines whether at least one of the people and objects indicated by the descriptive information of the estimated scene, which corresponds to an index value less than the first threshold and an index value greater than or equal to a second threshold that is smaller than the minimum value of the first threshold for each scene, matches at least one of the people and objects indicated by the object information, which corresponds to an index value less than the first threshold and greater than or equal to the second threshold.

9. The information processing apparatus according to any one of claims 6 to 8, wherein the correction unit determines that at least one of the person and object indicated by the descriptive information does not match at least one of the person and object indicated by the object information, and performs a correction to reduce the index value representing the confidence level of the estimation.

10. The information processing apparatus according to any one of claims 6 to 8, wherein the correction unit corrects the first threshold by multiplying the first threshold of the estimated scene by a value obtained by taking a weighted average of each index value corresponding to each of the multiple pieces of information indicating the same estimated scene output by the image recognition model.

11. A method comprising: a computer acquiring information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image; object information indicating at least one of an estimated person and object included in the image, output by the image recognition model; determining the degree of agreement between at least one of the person and object indicated by explanatory information associated with the acquired estimated scene information and at least one of the person and object indicated by the acquired object information; and correcting at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold value for the estimated scene used to determine the index value, based on the determined degree of agreement.

12. The method according to claim 11, wherein the computer outputs the recognition result of the image recognition model corresponding to an index value greater than or equal to the first threshold.

13. The method according to claim 11 or 12, wherein the computer determines whether at least one of the people and objects indicated by the descriptive information of the estimated scene, which corresponds to an index value less than the first threshold and greater than or equal to a second threshold that is less than the minimum value of the first threshold for each scene, matches at least one of the people and objects indicated by the object information, which corresponds to an index value less than the first threshold and greater than or equal to the second threshold.

14. The method according to any one of claims 11 to 13, wherein the computer determines that at least one of the person and object indicated by the descriptive information does not match at least one of the person and object indicated by the object information, and performs a correction to reduce the index value representing the confidence level of the estimation.

15. The method according to any one of claims 11 to 13, wherein the computer corrects the first threshold by multiplying the first threshold of the estimated scene by a value obtained by taking a weighted average of each index value corresponding to each of the multiple pieces of information representing the same estimated scene output by the image recognition model.

16. A program that causes a computer to acquire information indicating an estimated scene represented in an image, output by an image recognition model that recognizes an input image, and object information indicating at least one of an estimated person and object included in the image, output by the image recognition model; to determine the degree of agreement between at least one of the person and object indicated by the explanatory information associated with the acquired estimated scene information and at least one of the person and object indicated by the acquired object information; and to correct at least one of an index value indicating the reliability of the estimation by the image recognition model and a first threshold value for the estimated scene used to determine the index value, based on the determined degree of agreement.

17. The program according to claim 16, which causes the computer to output the recognition result of the image recognition model corresponding to an index value greater than or equal to the first threshold.

18. The program according to claim 16 or 17, which causes the computer to determine whether at least one of the people and objects indicated by the descriptive information of the estimated scene, which corresponds to an index value less than the first threshold and greater than or equal to a second threshold that is smaller than the minimum value of the first threshold for each scene, matches at least one of the people and objects indicated by the object information, which corresponds to an index value less than the first threshold and greater than or equal to the second threshold.

19. The program according to any one of claims 16 to 18, which causes the computer to perform a correction by reducing the index value representing the confidence level of the estimation if it determines that at least one of the person and object indicated by the descriptive information does not match at least one of the person and object indicated by the object information.

20. The program according to any one of claims 16 to 18, which causes the computer to correct the first threshold by multiplying the first threshold of the estimated scene by a value obtained by taking a weighted average of each index value corresponding to each of the multiple pieces of information representing the same estimated scene output by the image recognition model.

Images