Image recognition device, image recognition system, image recognition program, and image recognition method

Abstract

When multiple overlapping regions are mistakenly detected for the same hand, a method is needed to leave one region for each hand and determine that the other regions detecting the same hand are mistakenly detected as overlapping regions. [Solution] The image recognition device has an area detection means for detecting an object area containing a target object from an image, an acquisition means for acquiring an evaluation value indicating the likelihood that the object area is an area containing the target object, a determination means for determining a distribution area within the object area in which detection target points on the target object are distributed, and a determination means for determining that an object area with a lower evaluation value out of the first object area and the second object area is an erroneous detection when a first distribution area of ​​a first object area and a second distribution area of ​​a second object area in a first image satisfy predetermined conditions based on the first distribution area and an area where the first distribution area and the second distribution area overlap.

Description

[Technical Field]

[0001] The present invention relates to an image recognition device that detects an area of ​​an object or the like from an image. [Background technology]

[0002] As an image recognition method for detecting a region of a target object or the like from an image, a method of performing detection in stages to improve detection accuracy is known. In staged detection, the region of the entire object is first detected from the image, and then processing is performed to further detect a part of the object within the region of the entire object.

[0003] However, if the detection result when detecting the entire object area from the image is incorrect, the detection results at subsequent stages will also be incorrect. Therefore, a technology is needed to determine whether the detected area was detected correctly (positive detection) or whether the detected area was detected incorrectly (false detection).

[0004] In Patent Document 1, the target object to be detected is the hand of a wearer wearing a head-mounted display (hereinafter referred to as HMD).The technology disclosed uses at least one of information regarding the movement of the device worn on the hand, the distance between the HMD and the hand, the direction of hand movement, the direction of hand extension, and the presence or absence of the device, and determines that a hand other than the wearer's hand has been detected as the target object as a false detection. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-32409 Summary of the Invention [Problem to be solved by the invention]

[0006] When detecting an area containing hands from a captured image, multiple overlapping areas may be detected. In this case, the captured image may actually contain multiple overlapping hands, and the areas for each of the multiple hands may be correctly detected. However, the captured image may actually contain only one hand, and multiple overlapping areas may be mistakenly detected for the same hand. In such cases where multiple overlapping areas for the same hand are mistakenly detected, a method is needed to leave one area for one hand and determine that the other areas detecting the same hand are mistakenly detected as overlapping areas. If multiple overlapping areas for the same hand are mistakenly detected, the detection results in subsequent stages will also be incorrect.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an image recognition device that can determine whether each detected object region is a correct detection or an incorrect detection, even when multiple object regions are detected for the same object to be detected. [Means for solving the problem]

[0008] The image recognition device of the present invention is characterized by comprising: region detection means for detecting an object region including a target object from an image; acquisition means for acquiring an evaluation value indicating the likelihood that the object region is a region including the target object; determination means for determining a distribution region within the object region in which detection target points of the target object are distributed; and determination means for determining that an object region with a lower evaluation value out of the first object region and the second object region is an erroneous detection when a predetermined condition is satisfied in a first distribution region determined by the determination means from a first object region detected by the region detection means from a first image and a second distribution region determined by the determination means from a second object region detected by the region detection means from the first image based on the first distribution region and an area where the first distribution region and the second distribution region overlap. [Effects of the Invention]

[0009] According to the present invention, even when a plurality of object regions are detected for the same detection target object, it is possible to determine whether each detected object region is a correct detection or an erroneous detection. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an internal configuration of an image recognition device according to a first embodiment of the present invention. [Figure 2] 1 is a diagram schematically illustrating an example of image data processed by an image recognition device according to a first embodiment of the present invention. [Figure 3] FIG. 2 is a flowchart illustrating the operation of the image recognition device according to the first embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Each embodiment will be described below with reference to the drawings. Identical or equivalent components, members, and processes shown in each drawing will be assigned the same reference numerals, and redundant descriptions will be omitted where appropriate. Also, some components, members, and processes will be omitted in each drawing. The following embodiments do not limit the present invention, and not all combinations of features described in the present embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). In the following embodiments, identical components will be described with the same reference numerals.

[0012] (First embodiment) <Internal configuration of image recognition device> 1 is a block diagram illustrating a head-mounted display (HMD) 100, which is an image recognition device according to this embodiment. In this embodiment, an HMD is assumed as an example of a device constituting the image recognition device, but the present invention is not limited to these embodiments. For example, the HMD may be a smartphone or tablet terminal equipped with a camera, or may be other devices such as a personal computer (PC) or digital camera.

[0013] The HMD 100 is a head-mounted display device (electronic device) that can be worn on the head of a user. The HMD 100 includes a camera for capturing an image of the area in front of the user and a display for displaying the image to the user. The display of the HMD 100 displays a composite image that combines an image captured by the HMD 100 of the area in front of the user with content such as CG (computer graphics) in a format that corresponds to the posture of the HMD 100. This allows the user to experience virtual reality or mixed reality with their eyes.

[0014] The internal configuration of the HMD 100 will be described with reference to Fig. 1. In the HMD 100, a control unit 101, a ROM 102, a RAM 103, and an imaging unit 104 are connected to a system bus 105. The control unit 101, the ROM 102, the RAM 103, the imaging unit 104, and the system bus 105 are hardware resources that make up the HMD 100. Note that each component of the HMD 100 may be an image recognition system configured with individual hardware.

[0015] The control unit 101 controls each unit of the HMD 100 in accordance with input signals and programs described below. The control unit 101 has at least one CPU that executes programs stored in a ROM 102 and at least one other circuit. Note that instead of the control unit 101 controlling the entire device, the entire device may be controlled by multiple pieces of hardware sharing the processing.

[0016] The ROM 102 is an electrically erasable and recordable non-volatile memory, and stores programs and the like to be executed by the control unit 101. When the HMD 100 is powered on, the control unit 101 reads the programs from the ROM 102 and starts controlling the HMD 100. The ROM 102 is, for example, a flash memory.

[0017] The RAM 103 is used as a work area by the programs executed by the control unit 101. The RAM 103 is made up of, for example, a volatile memory (DRAM) using a semiconductor element.

[0018] Although the control unit 101, ROM 102, and RAM 103 have been described as separate hardware resources, these functions may be integrated into a single LSI.

[0019] The imaging unit 104 is composed of a stereo camera, and captures color or monochrome images of a scene using the two cameras mounted on the left and right sides, and outputs the video signals to the system bus 105. This video signal undergoes various image processing by the control unit 101, and is stored in the RAM 103 as left and right image data. The imaging unit 104 is composed of, for example, an optical system that controls an optical lens unit and aperture, zoom, focus, etc., and an imaging element that converts the light (image) introduced through the optical lens unit into an electrical video signal. The imaging element generally used is a CMOS imaging element (CMOS image sensor) that uses a CMOS, or a CCD imaging element (CCD image sensor) that uses a CCD.

[0020] <Flow for determining whether each detected object region is a correct or incorrect detection> A flowchart illustrating the operation of the HMD 100 according to the first embodiment will be described below with reference to FIGS. 2(a), 2(b), 2(c), 2(d), 2(e), 2(f), and 3. The process illustrated in this flowchart is implemented by the control unit 101 of the HMD 100 controlling each unit of the HMD 100 in accordance with input signals and programs. That is, this process is implemented by the control unit 101 expanding a program stored in the ROM 102 into the RAM 103 and executing it. The process illustrated in the flowchart of FIG. 3 is executed when the user launches an application on the HMD 100 and is executed each time the imaging unit 104 acquires an image (each time an image is captured). The application is, for example, an application that the user selects on the home screen (home space) after launching the HMD 100, and includes an app that allows the user to interact with a virtual object using hand gestures. The timing at which this flowchart is executed is not limited to when the user launches an application on the HMD 100. For example, it may be the timing when the user starts up the HMD 100 or the timing when a virtual object is displayed in the mixed reality space.

[0021] 2(a), 2(b), 2(c), 2(d), 2(e), and 2(f) are diagrams schematically illustrating an example of image data processed by the image recognition device of the first embodiment. In FIG. 2(a), an image 200 is an example of image data generated by the control unit 101 from a video signal acquired by the imaging unit 104, and the image 200 includes a hand 201 of the HMD wearer. Also, FIGS. 2(b), 2(c), 2(d), 2(e), and 2(f) will be described later.

[0022] In step S301, the control unit 101 acquires the video signal output from the imaging unit 104 to the system bus 105, generates image data, and proceeds to step S302. For example, the control unit 101 stores the image 200 in FIG.

[0023] In step S302, the control unit 101 executes a process for detecting a human hand, which is a target object to be detected, from the image data generated in step S301, and then proceeds to step S303. The process for detecting a human hand is a region detection process for detecting a region of a predetermined shape (hereinafter referred to as an object region) that is estimated to include a hand. For example, a rectangle is an example of the predetermined shape. Furthermore, the region of such a predetermined shape is detected as the smallest predetermined shape that is estimated to include a hand. Note that, depending on the image, the rectangular region estimated to include a hand may not be the smallest region. In the process for detecting a human hand, depending on the appearance of the subject or the like included in the image data, only one object region, multiple object regions, or no object region at all may be detected. Note that the process for detecting a human hand may be realized, for example, using a trained deep learning model or a rule-based algorithm. Furthermore, in this embodiment, the predetermined shape of the object region is a rectangle, but the predetermined shape is not necessarily limited to a rectangle. It may be, for example, a polygon, a circle, an ellipse, or the like, as long as all object regions have the same shape.

[0024] Figure 2(b) is an example of the result of performing the processing of step S302 on an image 200 that includes the hand 201 of the HMD wearer in this embodiment, in which the detected object region 211 and object region 221 are superimposed on the image 200.

[0025] In step S303, the control unit 101 acquires an evaluation value indicating the likelihood that the object region detected in step S302 contains a human hand. If multiple object regions are detected, the control unit 101 acquires an evaluation value for each object region. The evaluation value is, for example, a value output by the deep learning model along with the object region. The control unit 101 may use the evaluation value output from the deep learning model in step S303 as the evaluation value for the object region detected in step S302 in the processing of step S304 and subsequent steps. Alternatively, the control unit 101 may use an evaluation value corrected based on at least one of the vertical width, horizontal width, and area of ​​the object region. For example, if the width of the object region is 1 / 5 of the width of the image 200, the control unit 101 determines that the likelihood that a human hand is contained is low and determines a new evaluation value to be a lower value. Alternatively, the control unit 101 may determine the evaluation value by comparing the vertical width and / or area in addition to the horizontal width. Furthermore, the evaluation value may be determined based on the result of judging any one of the width, height, or area of ​​the object region based on a predetermined threshold value without comparing it with image data. In this embodiment, it is assumed that control unit 101 obtains an evaluation value of 80 for object region 211 and an evaluation value of 40 for object region 221. The evaluation value is determined in the range from 0 to 100, with 0 being the minimum value and 100 being the maximum value. The range of the evaluation value is not limited to the range from 0 to 100, and may be determined relatively without having a range, for example.

[0026] In step S304, the control unit 101 executes a position detection process to detect the positions of detection target points for all object regions detected in step S302, and then proceeds to step S305. For example, the control unit 101 executes a position detection process to detect the positions of the joint points (joint points) of the hand, including the fingertips and wrist of each finger, and then proceeds to step S305. This process of detecting the joint point positions may be realized, for example, using a trained deep learning model or a rule-based algorithm. Here, the joint points acquired for each finger include, for example, four points from the fingertip to the base of the finger (DIP joint, PIP joint, MP joint, and IP joint) and one point on the wrist. At least 21 joint points are acquired as joint points for each finger.

[0027] FIG. 2(c) is a diagram showing an example of the results of executing the processing of step S304 on image 200 in this embodiment, in which object regions to be processed and the positions of each joint point detected for each object region are superimposed on image 200. In FIG. 2(c), black circles represent the positions of each joint point detected for object region 211, and black dot 212 among the black circles represents the position of the uppermost joint point among the joint points detected in object region 211. Furthermore, black dot 213 among the black circles represents the position of the rightmost joint point among the joint points detected in object region 211. Black dot 214 among the black circles represents the positions of the leftmost and lowermost joint points (hereinafter referred to as the lower left end) among the joint points detected in object region 211. In FIG. 2(c), white circles represent the positions of each joint point detected for object region 221, and white dot 222 among the white circles represents the position of the uppermost joint point among the joint points detected in object region 221. Furthermore, a white point 223 among the white circles indicates the position of the rightmost joint point among the joint points detected within the object region 221. A white point 224 among the white circles indicates the position of the leftmost and bottommost (hereinafter referred to as bottom left) joint point among the joint points detected within the object region 221.

[0028] Next, a description will be given of the loop process for determining an erroneously detected object region, which is performed from step S305 to step S313. Note that here, if the detected region is correctly detected, it is determined to be a correct detection, and if the detected region is incorrectly detected, it is determined to be an erroneous detection.

[0029] In step S305, control unit 101 starts loop processing (hereinafter referred to as loop A) for applying the processing from steps S306 to S312 to the object region detected in step S302. Furthermore, control unit 101 selects one object region per loop and executes loop A. Note that if no object region is detected in step S302, loop A is not applied and the process proceeds to step S314.

[0030] In step S306, control unit 101 starts a loop process (hereinafter referred to as loop B) for applying the processes from steps S307 to S311 to the object region detected in step S302. Furthermore, control unit 101 selects one object region for each loop and executes loop B.

[0031] In step S307, control unit 101 determines whether the object region selected in step S305 and the object region selected in step S306 are the same object region. If control unit 101 determines that the object region selected in step S305 and the object region selected in step S306 are different object regions, the process proceeds to step S308. On the other hand, if control unit 101 determines that the object region selected in step S305 and the object region selected in step S306 are the same object region, the process proceeds to step S312.

[0032] For example, if the object region 211 is selected in step S305 and the object region 221 is selected in step S306, the control unit 101 proceeds to step S308. Also, if the object region 211 is selected in steps S305 and S306, or if the object region 221 is selected in steps S305 and S306, the control unit 101 proceeds to step S312.

[0033] In step S308, the control unit 101 determines the smallest region of a predetermined shape (hereinafter, "distribution region") that contains the joint point positions acquired in step S304 for the object region selected in step S305. Here, a rectangular region is determined as the predetermined shape. That is, a rectangle is determined as the width from the rightmost point to the leftmost point and the height from the topmost point to the bottommost point of the joint points acquired in step S304 for the object region selected in step S305. Note that, although the control unit 101 determines a rectangle as the distribution region here, this distribution region is not necessarily limited to a rectangle and may be, for example, a polygon, a circle, an ellipse, or the like.

[0034] For example, when the control unit 101 selects the object region 211 in step S305, it determines a rectangle whose width extends from the black point 212 at the top to the black point 214 at the bottom left corner, and whose height extends from the black point 213 at the right corner to the black point 214 at the bottom left corner, from among the joint point positions acquired in step S303. That is, the black point 212 is on the top side of the rectangle, the black point 213 is on the right side of the rectangle, and the black point 214 is at the intersection of the left and bottom sides of the rectangle (the bottom left vertex of the rectangle).

[0035] Here, with reference to FIG. 2(d), an object region 211 extracted from the image 200 will be described. As a result of executing the processing of step S304, each joint point is arranged in the object region 211. Furthermore, as a result of executing the processing of step S308, a distribution region 215 is arranged (superimposed) on the object region 211 as a rectangular frame that contains each joint point. Here, in this embodiment, the upper left of the image 200 is the origin, the vertical downward direction is the v coordinate axis, and the horizontal right direction is the u coordinate axis. The v coordinate 216 of the upper end of the distribution region 215 is defined as vt1, the v coordinate 217 of the lower end as vb1, the u coordinate 218 of the left end as ul1, and the u coordinate 219 of the right end as ur1. Furthermore, in FIG. 2(d), vt1 = 85, vb1 = 170, ul1 = 20, and ur1 = 75.

[0036] In step S309, the control unit 101 performs the same process as in step S308 on the object region selected in step S306 to determine a distribution region.

[0037] For example, when the control unit 101 selects the object region 221 in step S306, it determines a rectangle from among the joint point positions acquired in step S303, with a width extending from the white point 222 at the top to the white point 224 at the bottom left, and a height extending from the white point 223 at the right to the white point 224 at the bottom left. That is, the white point 222 is on the top side of the rectangle, the white point 223 is on the right side of the rectangle, and the white point 224 is at the intersection of the left and bottom sides of the rectangle (the bottom left vertex of the rectangle).

[0038] 2(e), an object region 221 extracted from the image 200 will be described. As a result of executing the processing of step S304, each joint point is arranged in the object region 221. Furthermore, as a result of executing the processing of step S309, a distribution region 225 is arranged (superimposed) on the object region 221 as a rectangular frame that contains each joint point. Here, the v coordinate 226 of the upper end of the distribution region 225 is set to vt2, the v coordinate 227 of the lower end to vb2, the u coordinate 228 of the left end to ul2, and the u coordinate 229 of the right end to ur2. In this embodiment, vt2=90, vb2=180, ul2=10, and ur2=80.

[0039] In step S310, the control unit 101 determines whether the ratio of the area of ​​the overlapping region between the distribution region determined in step S308 and the distribution region determined in step S309, with the distribution region determined in step S308 as the reference, is greater than a predetermined threshold. Note that, hereinafter, the area where the distribution region determined in step S308 and the distribution region determined in step S309 overlap is referred to as an overlapping region, and the ratio of the area of ​​the overlapping region with the distribution region determined in step S308 as the reference is referred to as ratio R. If the control unit 101 determines that ratio R is greater than the predetermined threshold, the process proceeds to step S311. If the control unit 101 does not determine that ratio R is greater than the predetermined threshold, the process proceeds to step S312.

[0040] In this embodiment, it is assumed that 0.5 is set as the predetermined threshold value. That is, when 0.5 is set as the predetermined threshold value, if the area of ​​the overlapping region is larger than half the area of ​​distribution region 215 determined in step S308, ratio R is larger than 0.5. Note that if it is not determined that ratio R is larger than the predetermined threshold value, that is, if ratio R is equal to or smaller than the predetermined threshold value, both selected object regions are not determined to be erroneous detections, and the process proceeds to step S312. For example, when 0.5 is set as the predetermined threshold value, if the area of ​​the overlapping region is smaller than half the area of ​​distribution region 215 determined in step S308, ratio R is smaller than 0.5. Therefore, neither object region 211 nor object region 221 is determined to be erroneous detections, and the process proceeds to step S312.

[0041] 2(f) is a diagram showing the result of control unit 101 executing the processes in steps S308 and S309 in this embodiment, in which object region 211 and distribution region 215, and object region 221 and distribution region 225 are superimposed in the coordinate space of image 200. The overlapping region 230 indicated by diagonal lines is the region where distribution region 215 and distribution region 225 overlap.

[0042] For example, if object region 211 is selected in step S305 and object region 221 is selected in step S306, the v coordinates of the upper and lower endpoints of distribution region 215 in object region 211 are vt1 = 85 and vb1 = 170, respectively. Furthermore, the u coordinates of the left and right endpoints of distribution region 215 in object region 211 are ul1 = 20 and ur1 = 75. Therefore, the area of ​​distribution region 215 is (vb1 - vt1) × (ur1 - ul1) = (170 - 85) × (75 - 20) = 4675. Furthermore, the v coordinates of the upper and lower endpoints of distribution region 225 in object region 221 are vt2 = 90 and vb2 = 180, respectively, and the u coordinates of the left and right endpoints of distribution region 225 in object region 221 are ul2 = 10 and ur2 = 80. Then, the area of ​​overlapping region 230 is (vb1-vt2)×(ur1-ul1)=(170-90)×(75-20)=4400. Therefore, area of ​​overlapping region 230 / area of ​​distribution region 215=4400 / 4675=approximately 0.94>0.5 (=predetermined threshold). Therefore, in this case, control unit 101 determines that ratio R is greater than the predetermined threshold, and proceeds to the processing of step S311.

[0043] In the above-described step S310, instead of determining whether the ratio R is greater than a predetermined threshold, it may be determined whether the ratio R is equal to or greater than a predetermined threshold. Furthermore, the numerator and denominator used when calculating the ratio may be interchanged, and a condition for determining whether the ratio R is smaller than a predetermined threshold (whether it is less than the predetermined threshold) or equal to or less than the predetermined threshold may be used. Furthermore, instead of determining based on the area ratio as described above, the determination may be made based on the ratio of the vertical width or horizontal width of the overlapping region to the distribution region.

[0044] In step S311, control unit 101 compares the evaluation values ​​of the object regions selected in step S305 and step S306, determines the object region with the lower evaluation value to be a false detection, and proceeds to step S312. In this embodiment, for example, as described above, control unit 101 determines that object region 211 has an evaluation value of 80 and object region 221 has an evaluation value of 40. Therefore, control unit 101 determines that object region 221 is a false detection because 80>40 (= evaluation value of object region 211>evaluation value of object region 221). Also, in this embodiment, control unit 101 determines that an object region is a false detection and proceeds to the next process, but it may also determine that the other object region that was not determined to be a false detection is a correct detection. For example, if control unit 101 determines that the object region selected in step S306 is a false detection by the process of step S311, it may determine that the object region selected in step S305 is a correct detection.

[0045] Furthermore, if the evaluation values ​​are the same, one of the object regions may be arbitrarily determined to be an erroneous detection. Alternatively, in step S311, if the evaluation values ​​are the same, a rule-based determination may be added using information on the vertical width, horizontal width, or area of ​​each of the object region and distribution region in order to determine that one of the object regions is an erroneous detection. For example, the difference in area between object region 211 and distribution region 215 selected in step S305 may be compared with the difference in area between object region 221 and distribution region 225 selected in step S306, and the larger of the two may be determined to be an erroneous detection.

[0046] In step S312, the control unit 101 determines whether to end loop B. That is, if all object regions have been selected in step S306, loop B is ended and the process proceeds to step S313;

[0047] For example, when image 200 is generated in step S301, object region 211 is first selected in step S305, and then object region 211 is selected in step S306. In step S307, the object region selected in loop A and the object region selected in loop B are the same, so the process proceeds to step S312 without performing the processes from step S308 to step S311. Next, in step S312, it is determined that object region 221 has not yet been selected, so the process returns to step S306 and continues loop B. When object region 211 and object region 221 are selected in step S306, all object regions detected in image 200 have been selected, so the process proceeds to step S313.

[0048] In step S313, the control unit 101 determines whether to end loop A. If all object regions have been selected in step S305, the control unit 101 determines to end loop A, and proceeds to step S314. If all object regions have not been selected in step S305, the control unit 101 determines not to end loop A, returns to step S305, and continues loop A.

[0049] For example, when image 200 is generated in step S301, object region 211 is first selected in step S305, and then object region 211 is selected in step S306. When loop B ends as described above and the process proceeds to step S313, not all object regions have been selected in loop A, so the process returns to step S305. Next, in step S305, object region 221 is selected, and the process of loop A is performed again. Here, when loop B ends again and the process proceeds to step S313, it is determined that all object regions have been selected in loop A, and the process proceeds to step S314.

[0050] In step S314, the control unit 101 determines that, of all object regions detected in step S302, all object regions other than those determined to be erroneous detections at least once in step S311 are correct detections, and proceeds to step S315. That is, regions that have never been determined to be erroneous detections in step S311 are determined to be correct detections.

[0051] In step S315, the control unit 101 determines whether to terminate the processing of the flowchart in Fig. 3. For example, if a user inputs an end instruction via an operation device (not shown) or if image data has not been acquired, the control unit 101 determines to terminate the processing of Fig. 3. If the application is to be terminated, the control unit 101 proceeds to termination. On the other hand, if the control unit 101 determines not to terminate the processing of Fig. 3, the control unit 101 proceeds to step S301.

[0052] As described above, in this embodiment, the control unit 101 detects an object region including a human hand, which is a detection target object, from image data and obtains an evaluation value indicating the likelihood that the region includes a human hand. Furthermore, the control unit 101 detects each joint point position, including the fingertips and wrist of each finger, within the object region and determines the smallest rectangle that contains the joint point positions as a distribution region. If the ratio of the area of ​​the overlapping area of ​​the distribution regions for two of the detected object regions to the area of ​​a single reference distribution region is equal to or greater than a predetermined threshold, the control unit 101 determines that the object region with a low evaluation value is an erroneous detection. This makes it possible to determine whether each detected object region is a correct or erroneous detection when multiple overlapping object regions are detected around the detection target object.

[0053] (Other embodiments) The present invention can also be realized by executing the following process: software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various storage media, and the computer (or control unit, MPU, etc.) of the system or device reads and executes the program code. In this case, the program and the storage medium storing the program constitute the present invention.

[0054] While the present invention has been described in detail above based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0055] Note that each functional unit in each of the above embodiments (variations) may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of multiple functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Furthermore, each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP. For example, an apparatus may have a processor and a memory (storage medium) in which a control program is stored. Then, the functions of at least some of the functional units of the apparatus may be realized by the processor reading and executing the control program from the memory.

[0056] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0057] In addition, in each of the examples described above, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs) and dedicated processors (e.g., GPUs, ASICs, FPGAs, and programmable logic devices, etc.).

[0058] The disclosure of this embodiment includes the following configuration, method, and program.

[0059] [Configuration 1] an area detection means for detecting an object area including a target object from an image; an acquisition means for acquiring an evaluation value indicating a likelihood that the object region is a region including a target object; a determining means for determining a distribution region in which detection target points on the target object are distributed from within the object region; and a determination means for determining that an object region having a low evaluation value out of the first object region and the second object region is an erroneous detection when a predetermined condition based on a first distribution region determined by the determination means from a first object region detected from a first image by the region detection means and a second distribution region determined by the determination means from a second object region detected from the first image by the region detection means is satisfied, based on the first distribution region and an overlapping region of the first distribution region and the second distribution region. An image recognition device characterized by:

[0060] [Configuration 2] The area detection means detects the smallest area of ​​a predetermined shape that includes the target object as the object area. 2. The image recognition device according to configuration 1,

[0061] [Configuration 3] The determining means determines the minimum area of ​​a predetermined shape that includes the detection target points as the distribution area. 3. The image recognition device according to configuration 1 or 2.

[0062] [Configuration 4] The predetermined shape is a rectangle. 4. The image recognition device according to configuration 2 or 3.

[0063] [Configuration 5] The acquisition means determines the evaluation value according to at least one of a vertical width, a horizontal width, and an area of ​​the object region that includes the target object. 5. The image recognition device according to any one of configurations 1 to 4.

[0064] [Configuration 6] The area detection means detects an area including a human hand as the object area including the target object. 6. The image recognition device according to any one of configurations 1 to 5.

[0065] [Configuration 7] The object detection device further includes a position detection unit for detecting the position of the detection target point on the target object within the object region detected by the region detection unit. 7. The image recognition device according to any one of configurations 1 to 6,

[0066] [Configuration 8] The position detection means detects the positions of the joints of the fingers and wrist of a person's hand as the positions of the detection target points. 8. The image recognition device according to configuration 7,

[0067] [Configuration 9] The determining means determines whether a ratio of an area of ​​the overlapping region of the first distribution region and the second distribution region, based on an area of ​​the first distribution region, is greater than a predetermined threshold value as the predetermined condition. 9. The image recognition device according to any one of configurations 1 to 8.

[0068] [Configuration 10] The determining means does not determine that both the first object region and the second object region are erroneously detected unless a ratio of an area of ​​the first distribution region and an area of ​​the second distribution region overlapping each other based on an area of ​​the first distribution region is greater than a predetermined threshold value. 10. The image recognition device according to any one of configurations 1 to 9.

[0069] [Configuration 11] When three or more object regions are detected by the region detection means, the determination means determines whether all of the three or more object regions are correct detections or erroneous detections, and determines only the object regions that have never been determined to be erroneous detections in all determinations as correct detections. 11. The image recognition device according to any one of configurations 1 to 10.

[0070] [Configuration 12] The determination means determines that an object region having a higher evaluation value out of the first object region and the second object region is correctly detected when a predetermined condition based on the first distribution region and an overlapping region of the first distribution region and the second distribution region is satisfied. 12. The image recognition device according to any one of configurations 1 to 11.

[0071] [method] a region detection step of detecting an object region including a target object from the image; an acquisition step of acquiring an evaluation value indicating a likelihood that the object region is a region including a target object; a determining step of determining a distribution region in which detection target points on the target object are distributed from within the object region; a determining step of determining that an object region having a low evaluation value out of the first object region and the second object region is an erroneous detection when a first distribution region determined in the determining step from a first object region detected from a first image in the region detecting step and a second distribution region determined in the determining step from a second object region detected from the first image in the region detecting step satisfy a predetermined condition based on the first distribution region and an overlapping region of the first distribution region and the second distribution region. An image recognition method comprising:

[0072] [program] A program for causing a computer to function as each of the means of the image recognition device according to any one of configurations 1 to 12.

[0073] [system] an area detection device that detects an object area including a target object from an image; an acquisition device that acquires an evaluation value indicating a likelihood that the object region is a region including a target object; a determination device that determines a distribution region in which detection target points on the target object are distributed from within the object region; a determination device that determines an object region having a low evaluation value as an erroneous detection, out of the first object region and the second object region, when a first distribution region determined by the determination device from a first object region detected by the region detection device from a first image and a second distribution region determined by the determination device from a second object region detected by the region detection device from the first image satisfy a predetermined condition based on the first distribution region and an area where the first distribution region and the second distribution region overlap. An image recognition system characterized by:

Claims

1. an area detection means for detecting an object area including a target object from an image; an acquisition means for acquiring an evaluation value indicating a likelihood that the object region is a region including a target object; a determining means for determining a distribution region in which detection target points on the target object are distributed from within the object region; and a determination means for determining that an object region having a low evaluation value out of the first object region and the second object region is an erroneous detection when a first distribution region determined by the determination means from a first object region detected from a first image by the region detection means and a second distribution region determined by the determination means from a second object region detected from the first image by the region detection means satisfy a predetermined condition based on the first distribution region and an area where the first distribution region and the second distribution region overlap. An image recognition device characterized by:

2. The area detection means detects the smallest area of ​​a predetermined shape that includes the target object as the object area.

2. The image recognition device according to claim 1.

3. The determining means determines the minimum area of ​​a predetermined shape that includes the detection target points as the distribution area.

2. The image recognition device according to claim 1.

4. The predetermined shape is a rectangle.

4. The image recognition device according to claim 2 or 3.

5. The acquisition means determines the evaluation value according to at least one of a vertical width, a horizontal width, and an area of ​​the object region that includes the target object.

2. The image recognition device according to claim 1.

6. The area detection means detects an area including a human hand as the object area including the target object.

2. The image recognition device according to claim 1.

7. The object detection device further includes a position detection unit for detecting the position of the detection target point on the target object within the object region detected by the region detection unit.

2. The image recognition device according to claim 1.

8. The position detection means detects the positions of the joints of the fingers and wrist of a person's hand as the positions of the detection target points.

8. The image recognition device according to claim 7.

9. The determining means determines whether a ratio of an area of ​​the overlapping region of the first distribution region and the second distribution region, based on an area of ​​the first distribution region, is greater than a predetermined threshold value as the predetermined condition.

2. The image recognition device according to claim 1.

10. The determining means does not determine that both the first object region and the second object region are erroneously detected unless a ratio of an area of ​​the first distribution region and an area of ​​the second distribution region overlapping each other based on an area of ​​the first distribution region is greater than a predetermined threshold value.

2. The image recognition device according to claim 1.

11. When three or more object regions are detected by the region detection means, the determination means determines whether all of the three or more object regions are correct detections or erroneous detections, and determines only the object regions that have never been determined to be erroneous detections in all determinations as correct detections.

2. The image recognition device according to claim 1.

12. The determination means determines that an object region having a higher evaluation value out of the first object region and the second object region is correctly detected when a predetermined condition based on the first distribution region and an overlapping region of the first distribution region and the second distribution region is satisfied.

2. The image recognition device according to claim 1.

13. a region detection step of detecting an object region including a target object from the image; an acquisition step of acquiring an evaluation value indicating a likelihood that the object region is a region including a target object; a determining step of determining a distribution region in which detection target points on the target object are distributed from within the object region; a determining step of determining that an object region having a low evaluation value out of the first object region and the second object region is an erroneous detection when a first distribution region determined in the determining step from a first object region detected from a first image in the region detecting step and a second distribution region determined in the determining step from a second object region detected from the first image in the region detecting step satisfy a predetermined condition based on the first distribution region and an overlapping region of the first distribution region and the second distribution region. An image recognition method comprising:

14. A program for causing a computer to function as each of the means of the image recognition device according to claim 1.

15. an area detection device that detects an object area including a target object from an image; an acquisition device that acquires an evaluation value indicating a likelihood that the object region is a region including a target object; a determination device that determines a distribution region in which detection target points on the target object are distributed from within the object region; a determination device that determines an object region having a low evaluation value as an erroneous detection, out of the first object region and the second object region, when a first distribution region determined by the determination device from a first object region detected by the region detection device from a first image and a second distribution region determined by the determination device from a second object region detected by the region detection device from the first image satisfy a predetermined condition based on the first distribution region and an area where the first distribution region and the second distribution region overlap. An image recognition system characterized by:

Images