Image processing device, image processing method, and recording medium

The image processing apparatus and method improve object detection accuracy by using a reference object to evaluate the validity of other objects' information, addressing erroneous detections in object detection engines.

WO2026239924A1PCT designated stage Publication Date: 2026-11-19NEC CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2026/016727
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-13
Filing Date
2026-04-27
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Existing object detection engines face issues with erroneous detection due to flexible learning, which can reduce their effectiveness in real-world scenarios, necessitating a method to accurately judge errors in object detection results.

Method used

An image processing apparatus and method that includes an information acquisition unit to gather type, size, and accuracy information for objects, a reference determination unit to identify a suitable reference object, and an evaluation unit to assess the validity of other objects' information using the reference object's size and accuracy, thereby improving the accuracy of object detection.

Benefits of technology

The solution enhances the accuracy of object detection by reducing erroneous detections and improving the efficiency of image-based information processing tasks such as object tracking.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2026016727_19112026_PF_FP_ABST
    Figure JP2026016727_19112026_PF_FP_ABST
Patent Text Reader

Abstract

This image processing device comprises an information acquisition unit, a reference determination unit, and an evaluation unit. The information acquisition unit acquires, for each of a plurality of objects included in an image, first information indicating a type estimation result, second information indicating a size in the image, and third information indicating accuracy of the estimation result. The reference determination unit determines a reference object that serves as a reference related to the size in the image, on the basis of the third information acquired for each of the plurality of objects. The evaluation unit evaluates the second information of each object other than the reference object by using the second information of the reference object, and thereby determines the validity of the first information acquired for each of the other objects.
Need to check novelty before this filing date? Find Prior Art

Description

Image processing apparatus, image processing method, and recording medium

[0001] The present invention relates to an image processing apparatus, an image processing method, and a recording medium.

[0002] Engines that detect objects from images (object detection engines) are used in a variety of scenarios. Patent Document 1 listed below discloses an example of technology related to such an object detection engine. For example, Patent Document 1 listed below discloses a technology that suppresses "erroneous detection of an object of an abnormally impossible size" by performing the following processes (1) and (2) on each object detected from an image. (1) Acquire restriction information on the size of the image region corresponding to the position of the object in the image and the type of the object, and information on the possibility of the existence of the object. (2) Based on the acquired information and the size of the detected object on the image, it is determined whether there is an erroneous detection of the object.

[0003] Japanese Unexamined Patent Publication No. 2023-131274

[0004] In order to suppress erroneous detection by the object detection engine, flexible learning adapted to data of various environments (zoom ratio, distance to the object) may sometimes be adopted. However, the result of such flexible learning may conversely become a factor that induces erroneous detection in actual business usage scenarios, and may reduce the usefulness of the object detection engine in actual operation. To address such a problem, a technology that accurately judges errors in object detection results is desired.

[0005] An image processing apparatus according to one aspect of the present disclosure includes: an information acquisition unit configured to acquire, for each of a plurality of objects included in an image, first information indicating an estimation result of a type, second information indicating a size in the image, and third information indicating a certainty of the estimation result; a reference determination unit configured to determine a reference object that serves as a reference for the size in the image, based on the third information acquired for each of the plurality of objects; and an evaluation unit configured to evaluate the second information of each of the other objects other than the reference object by using the second information of the reference object, so as to judge the validity of the first information acquired for each of the other objects.

[0006] An image processing method in another aspect of the present disclosure includes a computer acquiring, for each of a plurality of objects contained in an image, first information indicating a type estimation result, second information indicating the size in the image, and third information indicating the accuracy of the estimation result; determining a reference object that serves as a reference for the size in the image based on the third information acquired for each of the plurality of objects; and determining the validity of the first information acquired for each of the other objects by evaluating the second information of each of the other objects other than the reference object using the second information of the reference object.

[0007] A recording medium in another further aspect of this disclosure records a program. The program, when executed by a computer, causes the computer to perform operations including: obtaining, for each of a plurality of objects contained in an image, first information indicating the type estimation result, second information indicating the size in the image, and third information indicating the accuracy of the estimation result; determining a reference object that serves as a reference for the size in the image based on the third information obtained for each of the plurality of objects; and determining the validity of the first information obtained for each of the other objects by evaluating the second information of each of the other objects other than the reference object using the second information of the reference object.

[0008] According to one example of this disclosure, a technique is provided for accurately determining errors in object detection results.

[0009] This figure shows a first example of the functional configuration of the image processing apparatus in this disclosure. This figure shows an example of the hardware configuration of a computer that implements the image processing apparatus in this disclosure. This flowchart shows a first example of processing performed by the image processing apparatus in this disclosure. This figure shows an example of object information obtained by analyzing an image. This is an example of information indicating the real-world size (actual size) determined according to the type of object. This figure illustrates a second functional configuration of the image processing apparatus in this disclosure. This flowchart shows a second example of processing performed by the image processing apparatus in this disclosure. This figure shows an example of the analysis results of multiple images generated by repeatedly photographing a specific object while changing the shooting conditions.

[0010] The embodiments relating to this disclosure will be described below with reference to the drawings. In this disclosure, the drawings are associated with one or more embodiments. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted as appropriate. Unless otherwise specified, in each block diagram, each block represents a functional unit configuration, not a hardware unit configuration. When elements such as blocks in the diagrams are linked by arrows, the direction of the arrows is merely to make the flow of information easier to understand. In this case, unless otherwise specified, the direction of the arrows does not limit the direction of communication (one-way communication / two-way communication).

[0011] Furthermore, in the following description, "acquisition" includes at least one of the following: the device retrieving data or information stored in another device or storage medium (active acquisition), and the device inputting data or information output from another device into its own device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a reply, and accessing and reading data from another device or storage medium. Examples of passive acquisition include receiving information that is delivered (or transmitted, push notification, etc.). Moreover, "acquisition" may also mean selecting and acquiring data or information from among the received data or information, or selecting and receiving data or information that has been delivered.

[0012] • First Embodiment <Functional Configuration> Figure 1 is a diagram showing a first example of the functional configuration of an image processing apparatus in this disclosure. The image processing apparatus 10 illustrated in this figure comprises an information acquisition unit 110, a reference determination unit 120, and an evaluation unit 130.

[0013] The information acquisition unit 110 acquires information about objects detected from an image (hereinafter also referred to as "object information"). The object information includes at least: first information indicating the estimated type of object detected from the image; second information indicating the size of the object in the image; and third information indicating the accuracy of the estimated type of object. If multiple objects are present in the image, the first, second, and third information described above are generated for each of the multiple objects. The information acquisition unit 110 may be configured to acquire the object information described above from another image analysis device (not shown), or it may be configured to analyze the image to be processed and generate the object information described above in response to the input image.

[0014] The reference determination unit 120 determines information that will serve as a standard for judging the validity of the image analysis results (object information generated for each object). As an example, the reference determination unit 120 determines an object to be used as a size criterion within the image (hereinafter also referred to as the "reference object"). For example, if multiple objects are detected in the image, the reference determination unit 120 identifies an object that is "suitable as a standard" based on third information (information indicating the accuracy of the estimation result of the type of object) obtained for each of the multiple objects. Then, the reference determination unit 120 determines the object identified from among the multiple objects based on the third information as the reference object.

[0015] The evaluation unit 130 uses the information determined by the reference determination unit 120 to evaluate whether the image analysis results (detection results for each object) are valid. For example, the evaluation unit 130 uses the second information (size in the image) of the object determined as the reference object to evaluate the second information (size in the image) of other objects other than the reference object. Then, based on the evaluation results of the second information (size in the image) of each of the other objects, the evaluation unit 130 determines the validity of the first information (estimation result of the type of object) of each of the other objects.

[0016] <Example Hardware Configuration> Figure 2 shows an example of the hardware configuration of a computer that implements the image processing apparatus described in this disclosure.

[0017] The computer 1000 illustrated in this figure includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0018] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0019] Processor 1020 includes components such as the Central Processing Unit (CPU) and the Graphics Processing Unit (GPU).

[0020] Memory 1030 is a main memory device implemented as Random Access Memory (RAM), etc.

[0021] The storage device 1040 is an auxiliary storage device implemented as a Hard Disk Drive (HDD), Solid State Drive (SSD), memory card, or Read Only Memory (ROM). The storage device 1040 stores at least a program module that implements the functions of the image processing device 10 described above (information acquisition unit 110, reference determination unit 120, and evaluation unit 130).

[0022] The processor 1020 realizes the functions corresponding to program modules by loading the program modules read from the storage device 1040 onto the memory 1030 and executing them. For example, the processor 1020 realizes the functions of the information acquisition unit 110 as described in this disclosure by reading the program module corresponding to the information acquisition unit 110 onto the memory 1030 and executing it. Similarly, the processor 1020 realizes the functions of the reference determination unit 120 as described in this disclosure by reading the program module corresponding to the reference determination unit 120 onto the memory 1030 and executing it. Similarly, the processor 1020 realizes the functions of the evaluation unit 130 as described in this disclosure by reading the program module corresponding to the evaluation unit 130 onto the memory 1030 and executing it. This operation of the processor 1020 is common to all embodiments included in this disclosure.

[0023] The program module described above may be recorded on a recording medium other than the storage device 1040. The recording medium for recording the program module includes any medium usable by a non-temporary, tangible computer 1000. The recording medium for recording the program module may have program code embedded in it that can be read by the computer 1000 (processor 1020).

[0024] The input / output interface 1050 is an interface for connecting the computer 1000 (image processing device 10) to various devices. The term "various devices" here is not particularly limited, but includes, for example, input devices such as operation buttons, keyboards, touch panels, mice and microphones, and output devices such as displays and speakers.

[0025] The network interface 1060 is an interface for connecting the computer 1000 (image processing device 10) to various networks. These networks include, for example, a Local Area Network (LAN) and a Wide Area Network (WAN). The network interface 1060 supports various wireless and wired communication standards. For example, the computer 1000 (image processing device 10) communicates via the network interface 1060 with other devices (not shown) that store the image to be processed or the analysis results of the image. This allows the computer 1000 (image processing device 10) to acquire the image to be processed or the analysis results of the image.

[0026] Furthermore, the computer 1000 (image processing device 10) may be connected to the aforementioned "various devices" via the network interface 1060.

[0027] <Example of operation of the image processing device> Below, an example of the operation of the image processing device in this disclosure will be explained with reference to a diagram. Figure 3 is a flowchart of a first example of processing performed by the image processing device in this disclosure.

[0028] First, the information acquisition unit 110 acquires information about objects in the image to be processed (object information) (step S102). For each object detected in the image to be processed, the object information includes first information indicating the estimated type of the object, second information indicating the size of the object within the target image, and third information indicating the accuracy of the estimated result.

[0029] The reference determination unit 120 determines a reference object based on third information (information indicating the accuracy of the type estimation result) for each object (step S104). For example, the reference determination unit 120 identifies the object with the highest accuracy of the type estimation result indicated by the third information as an object that is "suitable as a reference". The reference determination unit 120 then determines the object identified based on the accuracy of the type estimation result indicated by the third information as the reference object.

[0030] The evaluation unit 130 uses the second information of the reference object (information indicating its size in the image) to evaluate the size of other objects in the image other than the reference object (step S106). As will be described in detail later, for example, the evaluation unit 130 uses the second information of the reference object to evaluate the size of other objects in the image in light of the allowable size range set for each type of object. If the size of a certain other object in the image is within the allowable range set for that other object, the evaluation unit 130 generates an evaluation result indicating that it is "appropriate". On the other hand, if the size of a certain other object in the image is outside the allowable range set for that other object, the evaluation unit 130 generates an evaluation result indicating that it is "inappropriate".

[0031] The evaluation unit 130 then determines whether an evaluation result of "valid" has been generated for each of the other objects other than the reference object (step S108). Here, suppose an evaluation result indicating "valid" has been generated as a result of evaluating the size of a certain other object in the image. In this case (step S108: YES), the evaluation unit 130 confirms the estimation result (first information) of the type of the corresponding other object as the correct result (step S110). On the other hand, suppose an evaluation result indicating "inappropriate" has been generated as a result of evaluating the size of a certain other object in the image. In this case (step S108: NO), the evaluation unit 130 rejects the estimation result (first information) of the type of the corresponding other object as the incorrect result (step S112).

[0032] In an example of an image processing apparatus described in this disclosure, if the type of each of several objects depicted in an image is estimated by analyzing the image, the validity of the estimated type of each object is determined. For example, as shown in the above example of operation, an index related to the size of an object (a size tolerance) is set for each type of object based on the size of a reference object in the image, which is determined using information indicating the "accuracy of the estimated type of each object". Then, using the index set for each type of object, it is determined whether or not the estimated type of each object is valid.

[0033] Here, the "accuracy" of the type estimation result can also be considered an indicator for determining whether the corresponding estimation result is true or false. For example, suppose the accuracy of the estimation result for the first object is higher than the accuracy of the estimation result for the second object. This means that the type estimation result for the first object is more reliable than the type estimation result for the second object. In other words, based on the reliability of the type estimation results for the detected objects, the "suitability as a criterion" for the first object is considered higher than the "suitability as a criterion" for the second object.

[0034] From the above perspective, according to an example of an image processing device in this disclosure, when the validity of the estimation result of the type of each object (image analysis result of each object) is judged using the size of each object in the image, it is expected that the accuracy of the judgment can be improved. In other words, according to this disclosure, an image processing device is provided that solves the problem of "accurately judging errors in object detection results". Furthermore, according to this disclosure, an image processing method executed by the image processing device (computer), a program for realizing each function of the image processing device (computer), and a recording medium for recording the program are also provided.

[0035] The following describes a detailed example of the image processing apparatus used in this disclosure.

[0036] <Flow for determining the reference object> An example of the flow for determining the reference object by the reference determination unit 120 will be explained below. As described above, the reference determination unit 120 identifies an object with high "suitability as a reference" based on the third information (information indicating the accuracy of the type estimation result) acquired for each of the multiple objects. The reference determination unit 120 then determines the object with high "suitability as a reference" as the reference object.

[0037] For example, suppose that an image is analyzed and object information is obtained as shown in Figure 4. Here, Figure 4 is a diagram showing an example of object information obtained by analyzing an image. In this example, the accuracy of the estimation result for the object estimated as "Class C" (the fourth object from the top in the table shown in this figure) is "0.96". This accuracy of "0.96" is the highest among the accuracy of the four objects detected from the image. In other words, the object estimated as "Class C" is considered to have the highest "suitability as a criterion". Therefore, the criterion determination unit 120 determines the object with the highest accuracy of the estimation result, i.e., the object estimated as "Class C", as the criterion object.

[0038] Furthermore, the reference determination unit 120 may be configured to determine a reference object by narrowing down the objects to those whose estimation result accuracy exceeds a certain threshold (for example, 80% of the maximum possible accuracy). With such a configuration, for example, operation in situations where the accuracy of the type estimation result (i.e., "suitability as a reference") is low for all objects can be avoided. This makes it less likely for the accuracy to decrease when judging the validity of the type estimation result for each object using the information of the reference object.

[0039] Furthermore, the criterion determination unit 120 may be configured to output a predetermined notification if it detects that the accuracy of the estimation results for multiple objects (i.e., "suitability as a criterion") is low for all of them. For example, the criterion determination unit 120 may display a message on a display connected via the input / output interface 1050 or the like indicating that it was not possible to determine a criterion object (an object whose estimation accuracy exceeds a certain level). Alternatively, the criterion determination unit 120 may display a message on a display connected via the input / output interface 1050 or the like prompting the user to input a criterion object.

[0040] <Example of Evaluation Method> As an example, the evaluation unit 130 generates an evaluation index based on the size of the reference object in the image, indicated by the second information of the reference object, and the real-world size of the reference object, which is determined according to the type of reference object. For example, the evaluation unit 130 calculates a coefficient for converting the size in the image to the real-world size based on the size of the reference object in the image and the real-world size, which is determined according to the type of reference object. The evaluation unit 130 then uses the evaluation index described above to determine the validity of the estimation results for each type of other object. For example, the evaluation unit 130 calculates the real-world size based on the size of the other object in the image (hereinafter also referred to as "real-world converted size") by multiplying the size of the other object in the image, indicated by the second information of the other object other than the reference object, by the coefficient described above. The evaluation unit 130 then evaluates the real-world converted size calculated for each object against the real-world size (hereinafter also referred to as "actual size"), which is determined according to the estimation result of the type of object (the type indicated by the first information of each object). The evaluation unit 130 determines the validity of the type of estimation result (first information obtained for each of the other objects) for each of the other objects, based on the evaluation results of the real-world equivalent size of each of the other objects other than the reference object.

[0041] For example, in the above explanation based on the example in Figure 4, the size of the object determined as the reference object (the object estimated to be "Class C") in the image is "15 [pixels]". Here, suppose that information indicating the real-world size (actual size) determined according to the type of object is defined as shown in Figure 5. Figure 5 is an example of information indicating the real-world size (actual size) determined according to the type of object. Information like that exemplified in Figure 5 is generated, for example, based on the actual measurement results of the actual size of each object and is pre-stored in an arbitrary storage area such as the storage device 1040. The evaluation unit 130 refers to the information as shown in Figure 5 and determines the actual size of the front of the reference object (the object estimated to be "Class C") to be "60 [cm]". 2 It can be identified as ].

[0042] In addition, the evaluation unit 130 can calculate, based on a numerical value indicating the size on the image of the object determined as the reference object (the object estimated as "class C") and a numerical value indicating the actual size of the same object, for example, a coefficient (a magnification of one numerical value when the other numerical value is used as a reference) for converting a numerical value indicating a size in the image into a numerical value indicating a size in the real world. In the example of FIG. 4, the numerical value indicating the size of the reference object (the object estimated as "class C") in the image is "15". Also, in the example of FIG. 5, the numerical value indicating the actual size of the reference object (the object estimated as "class C") is "60". The evaluation unit 130 can calculate a numerical value of "4" as the coefficient (magnification) for converting the numerical value indicating the size in the image into the numerical value indicating the size in the real world, based on the two numerical values.

[0043] Then, the evaluation unit 130 uses the above-mentioned coefficient (magnification) of "4" to convert the numerical value (the numerical value indicated by the second information) indicating the size in the image of another object other than the reference object into a numerical value indicating the size in the real world. For example, the numerical value related to the size in the image of the first object estimated as "class A" is "10". The evaluation unit 130 uses the above coefficient to convert the numerical value "10" related to the size in this image into the numerical value "40" related to the size in the real world. Similarly, the evaluation unit 130 converts the numerical value "5" related to the size in the image of the second object estimated as "class A" into the numerical value "20" related to the size in the real world. Further, the evaluation unit 130 similarly converts the numerical value "15" related to the size in the image of the third object estimated as "class B" into the numerical value "60" related to the size in the real world.

[0044] Then, the evaluation unit 130 acquires information related to the size in the real world (actual size) determined according to the type of each object other than the reference object, based on the estimation result of the type of each object. For example, the evaluation unit 130 refers to the information in FIG. 5 to acquire a numerical value of "40" as the information indicating the actual size of the "class A" object. Similarly, the evaluation unit 130 refers to the information in FIG. 5 to acquire a numerical value of "50" as the information indicating the actual size of the object of type "class B".

[0045] Then, the evaluation unit 130 compares the real-world size of another object converted based on the information of the reference object (hereinafter also referred to as "real-world converted size") with the full-scale size corresponding to the type of each object. For example, the evaluation unit 130 determines whether or not the numerical value of the real-world converted size calculated for each object detected from the image falls within an allowable range based on the numerical value of the full-scale size corresponding to each type.

[0046] Regarding the "allowable range", the evaluation unit 130 can set a range having an arbitrary width based on, for example, the numerical value of the full-scale size corresponding to the type of each object. As an example, the evaluation unit 130 sets a range of ±20% of the numerical value of the full-scale size corresponding to the type of each object as the "allowable range".

[0047] In this case, for the first object estimated as "Class A", the evaluation unit 130 sets a numerical range of "more than 32 and less than 48" as the allowable range based on the numerical value "40" relating to the full-scale size of the type "Class A". The numerical value relating to the size in the image of the first object estimated as "Class A" and the numerical value "40" calculated by the above coefficient are within the allowable range set in this manner. Therefore, the evaluation unit 130 determines that the type estimation result for the first object (that is, the estimation result of "Class A") is valid. In this case, the evaluation unit 130 generates an evaluation result indicating "valid" as the evaluation result for the first object.

[0048] The evaluation unit 130 performs the same processing for the second object estimated as "Class A" and the third object estimated as "Class B". For the second object estimated as "Class A", the numerical value "20" calculated from the size in the image and the coefficient mentioned above falls outside the acceptable range set for the "Class A" type. Therefore, the evaluation unit 130 determines that the estimation result for the type of the second object (i.e., the estimation result of "Class A") is incorrect. In this case, the evaluation unit 130 generates an evaluation result indicating that the second object is "incorrect". Furthermore, for the third object estimated as "Class B", the evaluation unit 130 sets the numerical range "greater than 40 and less than 60" as the acceptable range based on the numerical value "50" for the actual size of an object of the "Class B" type. The numerical value "60" calculated from the size in the image and the coefficient mentioned above for the third object estimated as "Class B" falls outside the acceptable range set in this way. Therefore, the evaluation unit 130 also determines that the estimated type result for the third object (i.e., the estimated result of "Class B") is "unfair". In this case, the evaluation unit 130 generates an evaluation result indicating that the third object is "unfair".

[0049] As described above, according to the example of the image processing device in this disclosure, the validity of the estimation results for each object in an image can be determined using the size of a reference object. This reduces the possibility of misdetecting the type of object. As a result, it is expected that the efficiency of various image-based information processing tasks, such as object tracking, can be improved.

[0050] - Depending on the orientation (shooting angle) of the imaging device that generates the image of the second embodiment, the shape of objects in the image may be distorted. For example, when an object is photographed from directly in front (directly above or horizontally), the shape of the object in the image is not distorted. On the other hand, when an object is photographed at an angle, for example from diagonally above or diagonally below, the shape of the object will be distorted depending on its position in the image (coordinate value in the image coordinate system). That is, in an image taken at an angle to an object, the size of the object in the image may change depending on its position in the image. If an evaluation using the size of a reference object in the image and its actual size is performed on such an image, the accuracy of judging the validity of the estimation results for the type of each object may decrease. The image processing device in this disclosure may further have functions to resolve such problems.

[0051] <Example of Functional Configuration> Figure 6 is a diagram illustrating a second functional configuration of the image processing apparatus in this disclosure. The image processing apparatus 10 illustrated in this figure further includes an angle information acquisition unit 112 in addition to the configuration illustrated in Figure 1. In the example shown in this figure, the information acquisition unit 110 includes the angle information acquisition unit 112.

[0052] The angle information acquisition unit 112 acquires information indicating the shooting angle of the image to be processed (hereinafter also referred to as "angle information"). By referring to the metadata of the image to be processed, the angle information acquisition unit 112 can acquire information to identify the shooting angle relative to objects in the image, such as the orientation and posture of the imaging device when the image was taken.

[0053] The reference determination unit 120 determines whether the shooting angle of the image indicated by the angle information satisfies predetermined conditions. If the shooting angle of the image indicated by the angle information satisfies predetermined conditions, the reference determination unit 120 acquires reference information instead of determining a reference object. In this disclosure, “reference information” is information that indicates characteristics related to the change in the size of an object according to its position in the image (coordinate value in the image coordinate system).

[0054] When the above-mentioned reference information is obtained, the evaluation unit 130 evaluates the combination of the position of each of the multiple objects in the image and the second information indicating the size of each object in the image based on the reference information. Then, based on the evaluation results for each of the multiple objects, the evaluation unit 130 determines the validity of the first information indicating the type of estimation result obtained for each of the multiple objects.

[0055] <Example of operation of the image processing device> Below, an example of the operation of the image processing device in this disclosure will be explained with reference to a diagram. Figure 7 is a flowchart of a second example of processing performed by the image processing device in this disclosure.

[0056] Similar to the process illustrated in the flowchart of Figure 3, the information acquisition unit 110 acquires information about objects in the image to be processed (object information) (step S102). The object information includes, for each object detected in the image to be processed, first information indicating the estimated type of the object, second information indicating the size of the object in the image, and third information indicating the accuracy of the estimation result.

[0057] The angle information acquisition unit 112 also acquires angle information indicating the shooting angle of the image to be processed (step S202). For example, the angle information acquisition unit 112 can refer to metadata associated with the image to be processed and acquire information that can identify the shooting angle of the image to be processed.

[0058] The reference determination unit 120 determines whether the shooting angle of the image to be processed, indicated by the angle information, satisfies predetermined conditions (step S204). Here, the “predetermined conditions” are conditions for determining whether the image is prone to distortion (change in shape) of objects depending on their position in the image, that is, whether the orientation of the imaging device relative to the object is tilted above a certain level. For example, the reference determination unit 120 uses a range of angles defined in advance as the above conditions (for example, a numerical range of 15 to 75 degrees) to determine whether the shooting angle based on the angle information is within the range of said conditions. In addition, factors such as shooting distance, zoom magnification, or the type of optical system such as a lens can also affect the likelihood of distortion (change in shape) of objects depending on their position in the image. The predetermined conditions may also include factors other than these shooting angles. In such cases, the reference determination unit 120 can obtain corresponding information (shooting distance, zoom magnification, type and characteristics of the lens used for shooting, etc.) from the metadata associated with the image to be processed and determine whether the predetermined conditions are met.

[0059] If the shooting angle of the image to be processed, as indicated by the angle information, does not meet the predetermined conditions (step S204: NO), the reference determination unit 120 determines a reference object, for example, as described above. For example, if the shooting angle indicated by the angle information is outside the range of angles defined as the above-mentioned conditions (e.g., a numerical range of 15 to 75 degrees), the reference determination unit 120 determines a reference object based on the third information of each object (information indicating the accuracy of the estimation result) (step S104). The subsequent processing flow is the same as the flow shown in the flowchart of Figure 3.

[0060] On the other hand, if the shooting angle of the image to be processed, as indicated by the angle information, satisfies a predetermined condition (step S204: YES), the reference determination unit 120 acquires reference information instead of determining a reference object (step S206). For example, if the shooting angle indicated by the angle information is within the range of angles defined as the above-mentioned condition (e.g., a numerical range of 15 to 75 degrees), the reference determination unit 120 acquires reference information that indicates characteristics related to the change in the shape (size in the image) of an object according to its position in the image.

[0061] The reference information here can be generated, for example, based on the results of analyzing multiple images produced by repeatedly photographing a specific object while changing the shooting conditions. Here, "shooting conditions" can include various elements such as shooting angle, shooting distance, zoom magnification, and the position of the specific object in the image coordinate system.

[0062] For example, suppose that the results shown in Figure 8 are obtained as a result of analyzing multiple images generated by repeatedly photographing a specific object while changing the shooting conditions. Figure 8 is a diagram showing an example of the results of analyzing multiple images generated by repeatedly photographing a specific object while changing the shooting conditions. The information exemplified in Figure 8 is generated by analyzing multiple images using the following procedure, for example.

[0063] (1) Recognize an image region corresponding to a specific object using an arbitrary object recognition algorithm or based on user-specified input to the image. (2) Calculate the size of the specific object in the image (height and width) based on the recognized image region. (3) Obtain the position of the recognized image region (coordinate value in the image coordinate system). (4) Store the calculated size of the specific object in the image and its position in the image (for example, its position in the Y-axis direction of the image coordinate system) in association with each other.

[0064] As shown in Figure 8, reference information is generated that indicates characteristics related to the change in the shape (size) of an object according to its position in the image. For example, a function corresponding to the graph drawn in this figure may be generated as reference information. This function accepts the position of an object in the image (coordinate value in the image coordinate system) as an input parameter and returns the height and width of the object in the image (size of the object in the image) as output. Such a function (reference information) is generated for each type of object, for example. The reference determination unit 120 can identify reference information corresponding to each of the multiple objects based on the multiple first pieces of information (information indicating the estimation result of the type of object) detected from the image.

[0065] In the example shown in Figure 8, for the sake of explanation, elements related to the imaging device, such as the shooting angle, shooting distance, and zoom magnification, are not particularly limited. Strictly speaking, the above-described function (reference information) can be generated after considering various elements related to the imaging device, such as the shooting angle, shooting distance, and zoom magnification. Therefore, the "reference information" in this disclosure can also be understood as information that shows the above-described characteristics using various elements related to the imaging device (for example, at least one of the shooting angle, shooting distance, and zoom magnification).

[0066] The functional component that performs the above-described processing to generate reference information may be provided in the image processing device 10, or it may be provided in another device (not shown).

[0067] Returning to Figure 7, the evaluation unit 130 evaluates the size of each object in the image using the acquired reference information instead of determining a reference object (step S208). For example, the evaluation unit 130 identifies the position (position in the image coordinate system) of each object detected in the image to be processed. Then, for each object detected in the image to be processed, the evaluation unit 130 identifies the size of the object according to its position in the image (hereinafter also referred to as the "reference size") based on the reference information corresponding to the type of object. The evaluation unit 130 evaluates whether the difference between the size of each object detected in the image to be processed and the reference size identified using the reference information corresponding to the type of object is within an acceptable range. If the difference between the size of an object in the image and the reference size identified using the reference information corresponding to the type of object is within an acceptable range, the evaluation unit 130 generates an evaluation result indicating that it is "appropriate". On the other hand, if the difference between the size of an object in the image and the reference size identified using the reference information corresponding to the type of object is outside an acceptable range, the evaluation unit 130 generates an evaluation result indicating that it is "inappropriate".

[0068] The evaluation unit 130 then determines whether an evaluation result of "valid" has been generated for each object detected from the image (step S210). Here, suppose an evaluation result indicating "valid" has been generated as a result of evaluating the size of an object in the image using reference information corresponding to the type of object. In this case (step S210: YES), the evaluation unit 130 confirms the estimation result (first information) of the type of object in question as the correct result (step S212). On the other hand, suppose an evaluation result indicating "incorrect" has been generated as a result of evaluating the size of an object in the image using reference information corresponding to the type of object. In this case (step S210: NO), the evaluation unit 130 rejects the estimation result (first information) of the type of object in question as the incorrect result (step S214).

[0069] In one example of an image processing device described in this disclosure, when determining the validity of the estimation results for the type of each object detected from an image, the device switches between performing a process to determine a reference object based on the shooting angle of the image being processed, or performing a process to acquire reference information. In other words, the logic for determining the validity of the estimation results for the type of object switches depending on whether or not the image being processed contains objects whose size (shape) may change depending on their position within the image. With this configuration, when processing images containing objects that are deformed (at least a part of their appearance is distorted) depending on their position within the image, it is possible to avoid a decrease in the accuracy of the judgment regarding the validity of the estimation results for the type of each object.

[0070] <Modification> In the above example, reference information is generated for each type of object, but generalized reference information that is not limited to the type of object may also be generated. For example, reference information showing the rate of change in size according to the position in the image can be generated using the results shown in Figure 8 and the actual size of the specific object used to obtain those results. By correcting the real-world equivalent size mentioned above using such reference information, the accuracy of judgments regarding the validity of the estimation results for each type of object is less likely to decrease when evaluating using the size of the reference object in the image and its actual size.

[0071] While the present disclosure has been described above with reference to exemplary embodiments, the disclosure is not limited to the embodiments described above. For example, each embodiment can be combined with other embodiments as appropriate. Furthermore, various modifications to the structure and details of the present disclosure are possible, as can be understood by those skilled in the art within the scope of the present disclosure.

[0072] Furthermore, while the flowcharts used in the above description show multiple steps (processes) in sequence, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impair the content.

[0073] Some or all of the above embodiments may also be described as follows, but are not limited to the following: 1. An image processing apparatus comprising: an information acquisition means for acquiring, for each of a plurality of objects contained in an image, first information indicating the type estimation result, second information indicating the size in the image, and third information indicating the accuracy of the estimation result; a reference determination means for determining a reference object that serves as a reference for the size in the image based on the third information acquired for each of the plurality of objects; and an evaluation means for determining the validity of the first information acquired for each of the other objects by evaluating the second information of each of the other objects other than the reference object using the second information of the reference object. 2. 1. The image processing apparatus according to the present invention. The evaluation means calculates a coefficient for converting the size in the image to the size in the real world using the size of the reference object in the image indicated by the second information of the reference object and the size of the reference object in the real world determined according to the type of reference object, and determines the validity of the first information acquired for each of the other objects by comparing the size calculated using the second information of each of the other objects and the coefficient with the size of the other objects in the real world determined according to the type of each of the other objects. 3. The image processing apparatus according to the present invention. The information acquisition means further acquires angle information indicating the shooting angle of the image, and the reference determination means, if the shooting angle of the image indicated by the angle information satisfies predetermined conditions, acquires reference information indicating characteristics regarding the change in size according to the position in the image instead of determining the reference object, and the evaluation means determines the validity of the first information acquired for each of the plurality of objects by evaluating the combination of the position of each of the plurality of objects in the image and the second information based on the reference information. 4. The image processing apparatus according to 3. wherein the aforementioned reference information is generated for each type of object, and the reference determination means identifies the respective reference information for each of the plurality of objects based on the respective first information for each of the plurality of objects.5. The image processing apparatus according to 3. or 4., wherein the reference determination means determines the reference object when the shooting angle indicated by the angle information is outside a predetermined range, and acquires the reference information when the shooting angle indicated by the angle information is within the predetermined range. 6. The image processing apparatus according to any one of 3. to 5., wherein the reference information is information that indicates the characteristics using at least one of the angle, distance, and zoom magnification at the time of image capture. 7. An image processing method comprising: a computer acquiring, for each of a plurality of objects included in an image, first information indicating the type estimation result, second information indicating the size in the image, and third information indicating the accuracy of the estimation result; determining a reference object that serves as a reference for the size in the image based on the third information acquired for each of the plurality of objects; and determining the validity of the first information acquired for each of the other objects by evaluating the second information of each of the other objects other than the reference object using the second information of the reference object. 8. A program to cause a computer to function as: an information acquisition means for acquiring first information indicating the type estimation result, second information indicating the size in the image, and third information indicating the accuracy of the estimation result for each of a plurality of objects contained in an image; a reference determination means for determining a reference object that serves as a reference for the size in the image based on the third information acquired for each of the plurality of objects; and an evaluation means for determining the validity of the first information acquired for each of the other objects by evaluating the second information of each of the other objects other than the reference object using the second information of the reference object.

[0074] Furthermore, some or all of the configurations described in Appendices 2 to 6, which are dependent on Appendice 1 (image processing device) as described above, may also be dependent on Appendice 7 (image processing method) and Appendice 8 (program) in the same dependent relationship as in Appendices 2 to 6. Moreover, not limited to Appendice 1 (image processing device), Appendice 7 (image processing method), and Appendice 8 (program), some or all of the configurations described as appendices may also be dependent on various hardware, software, various recording means for recording software, or systems, without departing from the embodiments described above.

[0075] This application claims priority based on Japanese Patent Application No. 2025-080384, filed on 13 May 2025, and incorporates all of its disclosures herein.

[0076] 10 Image processing device 1000 Computer 1010 Bus 1020 Processor 1030 Memory 1040 Storage device 1050 Input / output interface 1060 Network interface 110 Information acquisition unit 112 Angle information acquisition unit 120 Reference determination unit 130 Evaluation unit

Claims

1. An image processing apparatus comprising: an information acquisition means for acquiring, for each of a plurality of objects contained in an image, first information indicating the type estimation result, second information indicating the size in the image, and third information indicating the accuracy of the estimation result; a reference determination means for determining a reference object that serves as a reference for the size in the image based on the third information acquired for each of the plurality of objects; and an evaluation means for determining the validity of the first information acquired for each of the other objects by evaluating the second information of each of the other objects other than the reference object using the second information of the reference object.

2. The image processing apparatus according to claim 1, wherein the evaluation means calculates a coefficient for converting the size in the image to the size in the real world using the size of the reference object in the image shown by the second information of the reference object and the size of the reference object in the real world determined according to the type of reference object, and determines the validity of the first information obtained for each of the other objects by comparing the size calculated using the second information of each of the other objects and the coefficient with the size of each of the other objects in the real world determined according to the type of each of the other objects.

3. The image processing apparatus according to claim 1 or 2, wherein the information acquisition means further acquires angle information indicating the shooting angle of the image, the reference determination means, if the shooting angle of the image indicated by the angle information satisfies predetermined conditions, acquires reference information indicating characteristics relating to the change in size according to the position in the image, instead of determining the reference object, and the evaluation means determines the validity of the first information acquired for each of the plurality of objects by evaluating the combination of the position of each of the plurality of objects in the image and the second information based on the reference information.

4. The image processing apparatus according to claim 3, wherein the reference information is generated for each type of object, and the reference determination means identifies the reference information for each of the plurality of objects based on the first information for each of the plurality of objects.

5. The image processing apparatus according to claim 3 or 4, wherein the reference determination means determines the reference object when the shooting angle indicated by the angle information is outside a predetermined range, and acquires the reference information when the shooting angle indicated by the angle information is within the predetermined range.

6. The image processing apparatus according to any one of claims 3 to 5, wherein the reference information is information that indicates the characteristics using at least one of the angle, distance, and zoom magnification at the time of image capture.

7. An image processing method comprising: a computer acquiring, for each of a plurality of objects contained in an image, first information indicating the type estimation result, second information indicating the size in the image, and third information indicating the accuracy of the estimation result; determining a reference object that serves as a reference for the size in the image based on the third information acquired for each of the plurality of objects; and determining the validity of the first information acquired for each of the other objects by evaluating the second information of each of the other objects other than the reference object using the second information of the reference object.

8. The image processing method according to claim 7, further comprising: the computer calculating a coefficient for converting the size in the image to the size in the real world using the size of the reference object in the image indicated by the second information of the reference object and the size of the reference object in the real world determined according to the type of reference object; and determining the validity of the first information obtained for each of the other objects by comparing the size calculated using the second information of each of the other objects and the coefficient with the size of each of the other objects in the real world determined according to the type of each of the other objects.

9. The image processing method according to claim 7 or 8, further comprising: the computer acquiring angle information indicating the shooting angle of the image; if the shooting angle of the image indicated by the angle information satisfies predetermined conditions, instead of determining the reference object, acquiring reference information indicating characteristics relating to the change in size according to the position in the image; and determining the validity of the first information acquired for each of the plurality of objects by evaluating the combination of the position of each of the plurality of objects in the image and the second information based on the reference information.

10. The image processing method according to claim 9, wherein the reference information is generated for each type of object, and the computer further comprises identifying the reference information for each of the plurality of objects based on the first information for each of the plurality of objects.

11. The image processing method according to claim 9 or 10, further comprising the computer determining the reference object when the shooting angle indicated by the angle information is outside a predetermined range, and acquiring the reference information when the shooting angle indicated by the angle information is within the predetermined range.

12. The image processing method according to any one of claims 9 to 11, wherein the reference information is information that indicates the characteristics using at least one of the angle, distance, and zoom magnification at the time of image capture.

13. A recording medium that records a program, which, when executed by a computer, causes the computer to acquire, for each of a plurality of objects contained in an image, first information indicating the type estimation result, second information indicating the size in the image, and third information indicating the accuracy of the estimation result; determine a reference object that serves as a reference for the size in the image based on the third information acquired for each of the plurality of objects; and determine the validity of the first information acquired for each of the other objects by evaluating the second information of each of the other objects other than the reference object using the second information of the reference object.

14. The recording medium according to claim 13, further comprising: calculating a coefficient for converting the size in the image to the size in the real world using the size of the reference object in the image shown by the second information of the reference object and the size of the reference object in the real world determined according to the type of reference object; and determining the validity of the first information obtained for each of the other objects by comparing the size calculated using the second information of each of the other objects and the coefficient with the size of each of the other objects in the real world determined according to the type of each of the other objects.

15. The recording medium according to claim 13 or 14, further comprising: acquiring angle information indicating the shooting angle of the image; if the shooting angle of the image indicated by the angle information satisfies predetermined conditions, instead of determining the reference object, acquiring reference information indicating characteristics relating to the change in size according to the position in the image; and determining the validity of the first information acquired for each of the plurality of objects by evaluating the combination of the position of each of the plurality of objects in the image and the second information based on the reference information.

16. The recording medium according to claim 15, wherein the reference information is generated for each type of object, and the operation further comprises identifying the reference information for each of the plurality of objects based on the first information for each of the plurality of objects.

17. The recording medium according to claim 15 or 16, further comprising determining the reference object when the shooting angle indicated by the angle information is outside a predetermined range, and acquiring the reference information when the shooting angle indicated by the angle information is within the predetermined range.

18. The recording medium according to any one of claims 15 to 17, wherein the reference information is information that indicates the characteristics using at least one of the angle, distance, and zoom magnification at the time of image capture.