Image recognition device, image recognition system, image recognition program, and image recognition method

Abstract

If multiple areas detected as containing a target object overlap, there is a risk that an area may be mistakenly determined to be correctly detected, even though it is actually an area that has been incorrectly detected. [Solution] The image recognition device has a position detection means for detecting the position of a detection target point in a target object for each object region when multiple object regions are detected from one image, and a judgment means for judging whether or not each object region is a false detection when multiple object regions are detected from one image, and the judgment means judges that the first object region is a false detection when, in a first object region and a second object region detected from one image by the region detection means, a specific detection target point detected from the second object region by the position detection means is present within the first object region.

Description

[Technical Field]

[0001] The present invention relates to an image recognition device that detects an area of ​​an object or the like from an image. [Background technology]

[0002] As an image recognition method for detecting a region of a target object or the like from an image, a method of performing detection in stages to improve detection accuracy is known. In staged detection, the region of the entire object is first detected from the image, and then processing is performed to further detect a part of the object within the region of the entire object.

[0003] However, if the detection result when detecting the entire object area from the image is incorrect, the detection results at subsequent stages will also be incorrect. Therefore, a technology is needed to determine whether the detected area was detected correctly (positive detection) or whether the detected area was detected incorrectly (false detection).

[0004] In Patent Document 1, the target object to be detected is the hand of a wearer wearing a head-mounted display (hereinafter referred to as HMD).The technology disclosed uses at least one of information regarding the movement of the device worn on the hand, the distance between the HMD and the hand, the direction of hand movement, the direction of hand extension, and the presence or absence of the device, and determines that a hand other than the wearer's hand has been detected as the target object as a false detection. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-32409 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology disclosed in Patent Document 1 does not consider a method for reducing the possibility of an erroneous determination of whether a detection is correct or incorrect even when multiple overlapping regions among regions detected as including a target object. In other words, when multiple overlapping regions among regions detected as including a target object, there is a risk that a region may be erroneously determined as correctly detected, even when it is actually an incorrectly detected region. Therefore, an object of the present invention is to provide an image recognition device that reduces the possibility of an erroneous determination of whether a detection is correct or incorrect even when multiple overlapping regions among regions detected as including a target object. [Means for solving the problem]

[0007] In order to achieve the above object, the image recognition device of the present invention comprises: an area detection means for detecting an object area containing a target object from an image; a position detection means for detecting, for each object area, the position of a detection target point in the target object when the area detection means detects multiple object areas from a single image; and a judgment means for judging, for each object area when the area detection means detects multiple object areas from a single image, whether the object area is a false detection, wherein, in a first object area and a second object area detected from a single image by the area detection means, when a specific detection target point detected from the second object area by the position detection means is present within the first object area, the judgment means judges the first object area to be a false detection. [Effects of the Invention]

[0008] According to the present invention, when a plurality of overlapping object regions are detected around a target object, it is possible to determine whether each detected object region is a correct detection or an erroneous detection. [Brief explanation of the drawings]

[0009] [Figure 1]1 is a diagram illustrating an internal configuration of an image recognition device according to a first embodiment of the present invention. [Figure 2] 1 is a diagram schematically illustrating an example of image data processed by an image recognition device according to a first embodiment of the present invention. [Figure 3] FIG. 2 is a flowchart illustrating the operation of the image recognition device according to the first embodiment of the present invention. [Figure 4] FIG. 10 is a diagram schematically illustrating an example of image data processed by an image recognition device according to a second embodiment of the present invention. [Figure 5] FIG. 10 is a flowchart illustrating the operation of an image recognition device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Each embodiment will be described below with reference to the drawings. Identical or equivalent components, members, and processes shown in each drawing will be assigned the same reference numerals, and redundant descriptions will be omitted where appropriate. Also, some components, members, and processes will be omitted in each drawing. The following embodiments do not limit the present invention, and not all combinations of features described in the present embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). In the following embodiments, identical components will be described with the same reference numerals.

[0011] (First embodiment) <Internal configuration of image recognition device> 1 is a block diagram illustrating a head-mounted display (HMD) 100, which is an image recognition device according to this embodiment. In this embodiment, an HMD is assumed as an example of a device constituting the image recognition device, but the present invention is not limited to these embodiments. For example, the HMD may be a smartphone or tablet terminal equipped with a camera, or may be other devices such as a personal computer (PC) or digital camera.

[0012] The HMD 100 is a head-mounted display device (electronic device) that can be worn on the head of a user. The HMD 100 includes a camera for capturing an image of the area in front of the user and a display for displaying the image to the user. The display of the HMD 100 displays a composite image that combines an image captured by the HMD 100 of the area in front of the user with content such as CG (computer graphics) in a format that corresponds to the posture of the HMD 100. This allows the user to experience virtual reality or mixed reality with their eyes.

[0013] The internal configuration of the HMD 100 will be described with reference to Fig. 1. In the HMD 100, a control unit 101, a ROM 102, a RAM 103, and an imaging unit 104 are connected to a system bus 105. The control unit 101, the ROM 102, the RAM 103, the imaging unit 104, and the system bus 105 are hardware resources that make up the HMD 100. Note that each component of the HMD 100 may be an image recognition system configured with individual hardware.

[0014] The control unit 101 controls each unit of the HMD 100 in accordance with input signals and programs described below. The control unit 101 has at least one CPU that executes programs stored in a ROM 102 and at least one other circuit. Note that instead of the control unit 101 controlling the entire device, the entire device may be controlled by multiple pieces of hardware sharing the processing.

[0015] The ROM 102 is an electrically erasable and recordable non-volatile memory, and stores programs and the like to be executed by the control unit 101. When the HMD 100 is powered on, the control unit 101 reads the programs from the ROM 102 and starts controlling the HMD 100. The ROM 102 is, for example, a flash memory.

[0016] The RAM 103 is used as a work area by the programs executed by the control unit 101. The RAM 103 is made up of, for example, a volatile memory (DRAM) using a semiconductor element.

[0017] Although the control unit 101, ROM 102, and RAM 103 have been described as separate hardware resources, these functions may be integrated into a single LSI.

[0018] The imaging unit 104 is composed of a stereo camera, and captures color or monochrome images of a scene using the two cameras mounted on the left and right sides, and outputs the video signals to the system bus 105. This video signal undergoes various image processing by the control unit 101, and is stored in the RAM 103 as left and right image data. The imaging unit 104 is composed of, for example, an optical system that controls an optical lens unit and aperture, zoom, focus, etc., and an imaging element that converts the light (image) introduced through the optical lens unit into an electrical video signal. The imaging element generally used is a CMOS imaging element (CMOS image sensor) that uses a CMOS, or a CCD imaging element (CCD image sensor) that uses a CCD.

[0019] <Flow for determining whether each detected object region is a correct or incorrect detection> A flowchart illustrating the operation of the HMD 100 according to the first embodiment will be described below with reference to FIGS. 2(a), 2(b), 2(c), and 3. The process illustrated in this flowchart is implemented by the control unit 101 of the HMD 100 controlling each unit of the HMD 100 in accordance with an input signal and a program. That is, this process is implemented by the control unit 101 expanding a program stored in the ROM 102 into the RAM 103 and executing it. The process illustrated in the flowchart of FIG. 3 is executed when the user launches an application on the HMD 100 and is executed each time the imaging unit 104 acquires an image (each time an image is captured). The application is, for example, an application selected by the user on the home screen (home space) after launching the HMD 100, and includes an app that allows the user to interact with a virtual object using hand gestures. The timing of execution of this flowchart is not limited to the timing when the user launches an application on the HMD 100. For example, the timing may be when the user launches the HMD 100 or when a virtual object is displayed in the mixed reality space.

[0020] 2(a), (b), and (c) are diagrams schematically illustrating an example of image data processed by the image recognition device of this embodiment. In FIG. 2(a), an image 200 is an example of image data generated by the control unit 101 from a video signal acquired by the imaging unit 104, and the image 200 includes a hand 201 of the HMD wearer. Also, FIGS. 2(b) and 2(c) will be described later.

[0021] In step S301, the control unit 101 acquires the video signal output from the imaging unit 104 to the system bus 105, generates image data, and proceeds to step S302. For example, the control unit 101 stores the image 200 in FIG.

[0022] In step S302, the control unit 101 executes a process for detecting a human hand, which is a target object to be detected, from the image data generated in step S301, and then proceeds to step S303. The process for detecting a human hand is a region detection process for detecting a region of a predetermined shape (hereinafter referred to as an object region) that is estimated to include a hand. For example, a rectangle is an example of the predetermined shape. Furthermore, as for such a region of a predetermined shape, a region of the smallest predetermined shape that is estimated to include a hand is detected. Note that, depending on the image, the rectangular region that is estimated to include a hand may not be the smallest region. In the process for detecting a human hand, depending on the appearance of the subject or the like included in the image data, only one object region, multiple object regions, or no object region at all may be detected. Note that the process for detecting a human hand may be realized, for example, using a trained deep learning model or a rule-based algorithm. Furthermore, in this embodiment, the predetermined shape of the object region is a rectangle, but the predetermined shape is not necessarily limited to a rectangle. It may be, for example, a polygon, a circle, an ellipse, or the like, as long as all object regions have the same shape.

[0023] 2(b) is a diagram showing an example of the results of executing the processing of step S302 on image 200 in this embodiment, in which detected object region 211 and object region 221 are superimposed on image 200. In this embodiment, the upper left corner of image 200 is the origin, the vertical downward direction is the v coordinate axis, and the horizontal rightward direction is the u coordinate axis. The v coordinate 213 of the upper end of object region 211 is set to vt1, the v coordinate of the lower end is vb1, and the v coordinate 223 of the upper end of object region 221 is set to vt2, where vt1 = 100, vb1 = 200, and vt2 = 150.

[0024] In step S303, the control unit 101 executes a position detection process for detecting the positions of detection target points for all object regions detected in step S302, and then proceeds to step S304. For example, the control unit 101 executes a position detection process for detecting the positions of the joint points (joint points) of the hand, including the fingertips and wrist of each finger, and then proceeds to step S304. This process for detecting the joint point positions may be realized, for example, using a trained deep learning model or a rule-based algorithm. Here, the joint points acquired for each finger include, for example, four points from the fingertip to the base of the finger (DIP joint, PIP joint, MP joint, and IP joint) and one point on the wrist. At least 21 joint points are acquired as joint points for each finger.

[0025] 2(c) is an example of the results of executing the processing of step S303 on image 200 in the first embodiment, in which the object region to be processed and the detected joint point positions are combined with image 200. Black circles represent the joint point positions detected for object region 211, and among the black circles, 212 represents the wrist joint point position. White circles represent the joint point positions detected for object region 221, and among the white circles, 222 represents the wrist joint point position.

[0026] The processing from step S304 to step S311 is a loop processing for determining an object region that is a false detection. Note that here, if the detected region is correctly detected, it is determined to be a true detection, and if the detected region is incorrectly detected, it is determined to be a false detection.

[0027] In step S304, the control unit 101 starts a loop process (hereinafter referred to as loop A) for applying the processes from steps S305 to S310 to the object region detected in step S302. Note that if no object region is detected in step S302, loop A is not applied and the process proceeds to step S312.

[0028] In step S305, the control unit 101 starts a loop process (hereinafter referred to as loop B) for applying the processes from steps S306 to S309 to the object region detected in step S302.

[0029] In step S306, control unit 101 determines whether the object region selected in step S304 and the object region selected in step S305 are the same object region. If control unit 101 determines that the object region selected in step S304 and the object region selected in step S305 are different object regions, the process proceeds to step S307. On the other hand, if control unit 101 determines that the object region selected in step S304 and the object region selected in step S305 are the same object region, the process proceeds to step S310.

[0030] For example, if the object region 211 is selected in step S304 and the object region 221 is selected in step S305, the control unit 101 proceeds to step S307. Also, if the object region 211 is selected in steps S304 and S305, or if the object region 221 is selected in steps S304 and S305, the control unit 101 proceeds to step S310.

[0031] In step S307, the control unit 101 determines whether the wrist joint point position detected in S303 is present within the object region selected in step S305, relative to the object region selected in step S304. If the wrist joint point position detected in step S303 is not present within the object region selected in step S304, the control unit 101 proceeds to step S310. If the wrist joint point position detected in step S303 is present within the object region selected in step S304, the control unit 101 proceeds to step S308. Note that in this embodiment, the wrist joint point position is the target of determination, but the target of determination is not necessarily limited to the wrist joint point position. Furthermore, the target of determination is not necessarily limited to one point, and may be multiple points. For example, the target may be a joint point position including a fingertip, or multiple joint point positions including the wrist joint point position and the fingertip.

[0032] In this embodiment, with reference to FIG. 2(c), it is assumed that object region 211 is selected in step S304, and object region 221 is selected in step S305. In this case, wrist joint point position 212 detected in S303 for object region 211 is present within object region 221, so control unit 101 proceeds to step S308. Also with reference to FIG. 2(c), it is assumed that object region 221 is selected in step S304, and object region 211 is selected in step S305. In this case, wrist joint point position 222 detected in S303 for object region 221 is not present within object region 211, so control unit 101 proceeds to step S310.

[0033] In step S308, control unit 101 determines whether the object region selected in step S305 is located below a predetermined position (threshold) relative to the object region selected in step S304. If control unit 101 determines that the object region selected in step S305 is located below the predetermined position relative to the object region selected in step S304, control unit 101 proceeds to step S309. If control unit 101 determines that the object region selected in step S305 is not located below the predetermined position relative to the object region selected in step S304, control unit 101 proceeds to step S310. In this embodiment, the predetermined position (threshold) is set to a v coordinate that is ¼ of the vertical width of the object region selected in step S304 below the v coordinate of the top end of the object region selected in S304. Then, the coordinate position of the v coordinate of the predetermined position is compared with the v coordinate of the top end of the object region selected in step S305.

[0034] For example, if object region 211 is selected in step S304 and object region 221 is selected in step S305, the v coordinate of the upper end of object region 211 is vt1 = 100, as described above. Furthermore, the v coordinate of the lower end of object region 211 is vb1 = 200, and the v coordinate of the upper end of object region 221 is vt2 = 150. The vertical width of object region 211 selected in S304 is vb1 - vt1 = 100 pixels, and the v coordinate thv representing a predetermined position 1 / 4 below the vertical width of object region 211 is thv = vt1 + (vb1 - vt1) / 4 = 100 + 100 / 4 = 125. Because vt2 is located below thv, control unit 101 determines that object region 221 is located below the predetermined position with respect to object region 211, and proceeds to step S309.

[0035] The predetermined position (threshold) used in step S308 may be set based on the vertical width or horizontal width of the object region selected in step S304. Also, the predetermined position (threshold) used in step S308 may be set based on the vertical width or horizontal width of the object region selected in step S305. Also, the predetermined position (threshold) used in step S308 may be a position that is a predetermined position below the object region selected in step S304.

[0036] Note that the processing of step S308 may be omitted. That is, in step S307, if the wrist joint point position detected in step S303 is present within the object region selected in step S304, the control unit 101 may proceed to step S309. Here, as described above, if the wrist joint point position detected in step S303 is not present within the object region selected in step S304, the control unit 101 proceeds to step S310.

[0037] In step S309, control unit 101 determines that the object region selected in step S305 is an erroneous detection, and proceeds to step S310. That is, control unit 101 determines that the object region selected in step S305 is a region that does not accurately include a hand.

[0038] For example, if object region 221 is selected in step S305, control unit 101 determines that object region 221 is an erroneous detection, and proceeds to step S310. The predetermined position used in the determination in step S308 may be, for example, the position of any of the detected object regions or a position based on the length of one side of any of the detected object regions.

[0039] In step S310, the control unit 101 determines whether to end loop B. That is, if all object regions have been selected in step S305, loop B is ended and the process proceeds to step S311;

[0040] For example, if the control unit 101 first selects the object region 211 in step S305 and proceeds to step S310, it determines in step S310 that the object region 221 has not yet been selected, and the process returns to step S305 to continue loop B. If the object region 211 and the object region 221 are selected in step S305, it is determined that all object regions have been selected, and the process proceeds to step S311.

[0041] In step S311, the control unit 101 determines whether to end loop A. That is, if all object regions have been selected in step S304, loop A ends and the process proceeds to step S312;

[0042] For example, if the control unit 101 first selects the object region 211 in step S304 and proceeds to step S311, it determines in step S311 that the object region 221 has not yet been selected, and the process returns to S304 to continue loop A. If the object region 211 and the object region 221 are selected in step S304, it is determined that all object regions have been selected, and the process proceeds to step S312.

[0043] In step S312, the control unit 101 determines that, of all object regions detected in step S302, the object regions other than those determined to be erroneous detections in S309 are correctly detected, and the process proceeds to step S313.

[0044] In step S313, the control unit 101 determines whether to terminate the processing of the flowchart in Fig. 3. For example, if the user inputs an end instruction via an operation device (not shown) or if the control unit 101 is unable to acquire image data, the control unit 101 determines to terminate the processing of Fig. 3. On the other hand, if it is determined not to terminate in step S313, the control unit 101 proceeds to step S301.

[0045] As described above, in this embodiment, the control unit 101 detects an object region including a human hand, which is a target object, from image data, and detects each joint point position, including the fingertips and wrist, of each finger within that object region. Then, if the wrist joint point position among the joint point positions detected in one object region is present within another object region and the other object region is present below a predetermined position relative to the first object region, the control unit 101 determines that the other object region is an erroneous detection. This makes it possible to determine whether each detected object region is a correct detection or an erroneous detection when multiple overlapping object regions are detected around the target object.

[0046] (Second embodiment) The configuration of the image recognition device according to this embodiment is the same as that of the first embodiment, and therefore a description thereof will be omitted.

[0047] <Flow for determining whether each detected object region is a correct or incorrect detection> A flowchart illustrating the operation of the HMD 100 according to the first embodiment will be described below with reference to FIGS. 4 and 5. The process illustrated in this flowchart is implemented by the control unit 101 of the HMD 100 controlling each unit of the HMD 100 in accordance with an input signal and a program. That is, this process is implemented by the control unit 101 expanding a program stored in the ROM 102 into the RAM 103 and executing it. The process illustrated in the flowchart of FIG. 5 is executed when the user launches an application on the HMD 100 and is executed each time the imaging unit 202 acquires an image (each time an image is captured). The application is, for example, an application selected by the user on the home screen (home space) after launching the HMD 100, and includes an app that allows the user to interact with a virtual object using hand gestures. The timing of execution of this flowchart is not limited to the timing when the user launches an application on the HMD 100. For example, the timing may be when the user launches the HMD 100 or when a virtual object is displayed in the mixed reality space.

[0048] The processing in steps S301 and S302 is the same as in the flow of FIG. 3, and therefore a description thereof will be omitted.

[0049] In step S503, the control unit 101 executes a process for detecting the positions of each joint point, including the fingertip and wrist, and the wrist sub-point for all object regions detected in step S302, and then proceeds to step S304. Here, the wrist sub-point is defined as any point that exists between the position of the wrist joint point and the elbow joint and within the object region to be processed. The process for detecting the positions of each joint point and the wrist sub-point may be realized, for example, using a trained deep learning model or a rule-based algorithm.

[0050] FIG. 4 is a diagram showing an example of the results of executing the processing of step S503 on image 200 in this embodiment, in which the object region to be processed and the positions of each detected joint point and wrist-bottom point are superimposed on image 200. The object region 211, the wrist joint point 212 within the object region 211, the object region 221, and the wrist joint point 222 within the object region 221 are the same as those in FIG. 2(c), and therefore their description will be omitted. Also, in FIG. 4, as in FIG. 2(c), black circles represent the joint points detected for the object region 211, and white circles represent the joint points detected for the object region 221. Point 413 among the black circles represents the position of the wrist-bottom point. Point 423 among the white circles represents the position of the wrist-bottom point. Angle 414 is the angle of the arm, represented by the angle between a line connecting the wrist joint point position 212 and the wrist-bottom point position 413 and a line drawn perpendicularly from the wrist-bottom point position 413. Angle 424 is the angle of the arm represented by the angle formed by a line connecting wrist joint point position 222 and wrist bottom point position 423, and a line dropped vertically from wrist bottom point position 423. The coordinates of wrist joint point position 212 are (u12, v12), the coordinates of wrist bottom point position 413 are (u13, v13), the coordinates of wrist joint point position 222 are (u22, v22), and the coordinates of wrist bottom point position 423 are (u23, v23). Furthermore, let u12 = 100, u13 = 80, u22 = 50, u23 = 30, v12 = 160, v13 = 180, v22 = 210, and v23 = 230.

[0051] The processes in steps S304, S305, and S306 are the same as those in the flow of FIG. 3, and therefore will not be described here.

[0052] In step S507, the control unit 101 determines whether the positions of the wrist joint point and wrist bottom point detected in S503 for the object region selected in step S304 are present within the object region selected in step S305. If the control unit 101 determines that the positions of the wrist joint point and wrist bottom point detected in S503 for the object region selected in step S304 are not present within the object region selected in step S305, the control unit 101 proceeds to step S310. If the control unit 101 determines that the positions of the wrist joint point and wrist bottom point detected in S503 for the object region selected in step S304 are present within the object region selected in step S305, the control unit 101 proceeds to step S508.

[0053] In this embodiment, for example, if object region 211 is selected in step S304 and object region 221 is selected in step S305, wrist joint point position 212 and wrist bottom point position 413 detected in S503 for object region 211 are located within object region 221. Therefore, control unit 101 proceeds to step S508. Also, if object region 221 is selected in step S304 and object region 211 is selected in step S305, wrist joint point position 222 and wrist bottom point position 423 detected in S503 for object region 221 are not located within object region 211. Therefore, control unit 101 proceeds to step S310.

[0054] In step S508, the control unit 101 determines the arm angle based on the positional relationship between the wrist joint point and the wrist sub-point detected in S503 for the object region selected in step S304. The control unit 101 also determines the arm angle based on the positional relationship between the wrist joint point and the wrist sub-point detected in S503 for the object region selected in step S305. The control unit 101 then compares the arm angle within the object region selected in step S304 with the arm angle within the object region selected in step S305, and determines whether the difference between the two angles is within a predetermined range (whether the absolute value of the difference between the two angles is smaller than a threshold). In this embodiment, the predetermined range is defined as th_angle1 or greater and th_angle2 or less, where th_angle1=-30° and th_angle2=30°.

[0055] For example, suppose that object region 211 is selected in step S304, and object region 221 is selected in step S305. As described above, the coordinates of the position of the wrist joint point in object region 211 are coordinates (u12, v12) = (100, 160), and the coordinates of the position of the wrist-under point in object region 211 are coordinates (u13, v13) = (80, 180). Furthermore, the coordinates of the position of the wrist joint point in object region 221 are coordinates (u22, v22) = (50, 180), and the coordinates of the position of the wrist-under point in object region 221 are coordinates (u23, v23) = (30, 210). Furthermore, arm angle 414 is arctan((u13-u12) / (v13-v12)) = -45.0°, and arm angle 424 is arctan((u23-u22) / (v23-v22)) = -41.8°, so the difference between the angles of the two arms is 3.2°. Since this is greater than or equal to th_angle1 and less than th_angle2, control unit 101 determines that the difference between the angles of the two arms is within a predetermined range, and proceeds to step S309. Note that, although the angle formed by the line connecting the wrist joint point and the wrist bottom point relative to the vertical direction of the image is used as the arm angle in the above-mentioned determination, the angle used in the above-mentioned determination is not limited to this. For example, the angle formed by the line connecting the wrist joint point and the wrist bottom point relative to the horizontal direction of the image may also be used as the arm angle in the above-mentioned determination.

[0056] The processes in steps S309, S310, S311, and S312 are the same as those in the flow of FIG. 3, and therefore will not be described here.

[0057] In step S513, if the control unit 101 determines that all of the object regions detected in step S302 are false detections, it proceeds to step S514, and if there is even one object region that is determined to be a correct detection, it proceeds to step S313.

[0058] In step S514, control unit 101 identifies the object region that exists at the top of all the object regions detected in step S302, and determines that the object region identified as existing at the top is a correct detection.

[0059] The process in step S313 is the same as that in the flow of FIG. 3, and therefore the description thereof will be omitted.

[0060] As described above, in this embodiment, the control unit 101 detects an object region including a human hand, which is a target object, from image data, and detects the positions of each joint point and wrist point, including the fingertips and wrist, of each finger within the object region. Next, if the positions of the wrist joint point and wrist point detected in one object region exist within another object region, the arm angles calculated based on the positional relationship of the wrist joint point and wrist point in each object region are compared. If the difference between the two angles is within a predetermined range, the other object region is determined to be an erroneous detection. However, if all detected object regions are determined to be erroneous detections, the object region located at the top of all object regions is determined to be a correct detection. This makes it possible to determine whether each detected object region is a correct or erroneous detection when multiple overlapping object regions are detected around the target object.

[0061] (Other embodiments) The present invention can also be realized by executing the following process: software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various storage media, and the computer (or control unit, MPU, etc.) of the system or device reads and executes the program code. In this case, the program and the storage medium storing the program constitute the present invention.

[0062] Although the present invention has been described in detail above based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0063] Note that each functional unit in each of the above embodiments (variations) may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of multiple functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Furthermore, each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP. For example, an apparatus may have a processor and a memory (storage medium) in which a control program is stored. Then, the functions of at least some of the functional units of the apparatus may be realized by the processor reading and executing the control program from the memory.

[0064] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0065] In addition, in each of the examples described above, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs) and dedicated processors (e.g., GPUs, ASICs, FPGAs, and programmable logic devices, etc.).

[0066] The disclosure of this embodiment includes the following configuration, method, and program.

[0067] [Configuration 1] an area detection means for detecting an object area including a target object from an image; a position detection means for detecting a position of a detection target point on the target object for each of the object regions when the region detection means detects a plurality of the object regions from one image; and a determination means for determining whether or not each of the object regions is an erroneous detection when the region detection means detects a plurality of the object regions from one image, The determination means determines that a first object region is a false detection when, in a first object region and a second object region detected from one image by the region detection means, a specific detection target point among the detection target points detected from the second object region by the position detection means is present in the first object region. An image recognition device characterized by:

[0068] [Configuration 2] The area detection means detects the smallest area of ​​a predetermined shape that includes the target object as the object area. 2. The image recognition device according to configuration 1,

[0069] [Configuration 3] The predetermined shape is a rectangle. 3. The image recognition device according to configuration 2.

[0070] [Configuration 4] The determination means determines that the first object region is a false detection when, in a first object region and a second object region detected from one image by the region detection means, a specific detection target point among the detection target points detected from the second object region by the position detection means is present in the first object region and the upper end of the first object region is located lower than the upper end of the second object region by a threshold value. 4. The image recognition device according to any one of configurations 1 to 3.

[0071] [Configuration 5] the target object is a human hand, The detection target points are joint points that are points that estimate the positions of at least one of the joints and fingertips of the hand. 5. The image recognition device according to any one of configurations 1 to 4.

[0072] [Configuration 6] The specific detection target points do not include the estimated points of the DIP joints, PIP joints, MP joints, and IP joints of each finger. 6. The image recognition device according to configuration 5.

[0073] [Configuration 7] The specific detection target point is a point including the wrist joint point. 7. The image recognition device according to configuration 5 or 6,

[0074] [Configuration 8] The position detection means further detects a first point existing between the wrist and the elbow joint as the detection target point. 8. The image recognition device according to any one of configurations 5 to 7, wherein:

[0075] [Configuration 9] The specific detection target points include the wrist joint point and the first point. 9. The image recognition device according to configuration 8,

[0076] [Configuration 10] The method further includes acquiring means for acquiring an arm orientation in the image based on the position of the wrist joint point detected by the position detection means and the position of the first point, The determination means determines whether or not the first object region is a false detection when, in a first object region and a second object region detected from one image by the region detection means, a specific detection target point among the detection target points detected in the second object region by the position detection means is present in the first object region and an angle difference between a first arm orientation in the first object region and a second arm orientation in the second object region acquired by the acquisition means is smaller than a threshold value. 10. The image recognition device according to configuration 9,

[0077] [Configuration 11] The determination means determines that the first object region is a false detection when the specific detection target point of the second object region exists within the first object region, the specific detection target point of the first object region exists within the second object region, and the upper end of the second object region is located above the upper end of the first object region. 11. The image recognition device according to any one of configurations 1 to 10.

[0078] [Configuration 12] The determination means determines that the first object region is a false detection when the region detection means detects three or more object regions including the first object region from one image, and at least one of the specific detection target points of each of the object regions other than the first object region exists within the first object region. 12. The image recognition device according to any one of configurations 1 to 11.

[0079] [method] a region detection step of detecting an object region including a target object from the image; a position detection step of detecting a position of a detection target point on the target object for each of the object regions when the region detection step detects a plurality of the object regions from one image; and a determination step of determining whether or not each of the object regions is an erroneous detection when a plurality of the object regions are detected from one image by the region detection step, In the determination step, in a first object region and a second object region detected from one image by the region detection step, if a specific detection target point among the detection target points detected from the second object region by the position detection step exists in the first object region, the first object region is determined to be a false detection. An image recognition method comprising:

[0080] [program] A program for causing a computer to function as each of the means of the image recognition device according to any one of configurations 1 to 12.

[0081] [system] an area detection device that detects an object area including a target object from an image; a position detection device that detects a position of a detection target point on the target object for each of the object regions when the region detection device detects a plurality of the object regions from one image; a determination device that, when a plurality of object regions are detected from one image by the region detection device, determines whether or not each of the object regions is an erroneous detection, The determination device determines that a first object region is a false detection when, in a first object region and a second object region detected from one image by the region detection device, a specific detection target point among the detection target points detected from the second object region by the position detection device is present in the first object region. An image recognition system characterized by:

Claims

1. an area detection means for detecting an object area including a target object from an image; a position detection means for detecting a position of a detection target point on the target object for each of the object regions when the region detection means detects a plurality of the object regions from one image; and a determination means for determining whether or not each of the object regions is an erroneous detection when the region detection means detects a plurality of the object regions from one image, The determination means determines that a first object region is a false detection when, in a first object region and a second object region detected from one image by the region detection means, a specific detection target point among the detection target points detected from the second object region by the position detection means is present in the first object region. An image recognition device characterized by:

2. The area detection means detects the smallest area of ​​a predetermined shape that includes the target object as the object area.

2. The image recognition device according to claim 1.

3. The predetermined shape is a rectangle.

3. The image recognition device according to claim 2.

4. The determination means determines that the first object region is a false detection when, in a first object region and a second object region detected from one image by the region detection means, a specific detection target point among the detection target points detected from the second object region by the position detection means is present in the first object region and the upper end of the first object region is located lower than the upper end of the second object region by a threshold value.

2. The image recognition device according to claim 1.

5. the target object is a human hand, The detection target points are joint points that are points that estimate the positions of at least one of the joints and fingertips of the hand.

2. The image recognition device according to claim 1.

6. The specific detection target points do not include estimated points of the DIP joints, PIP joints, MP joints, and IP joints of each finger.

6. The image recognition device according to claim 5.

7. The specific detection target point is a point including the wrist joint point.

6. The image recognition device according to claim 5.

8. The position detection means further detects a first point located between the wrist and the elbow joint as the detection target point.

6. The image recognition device according to claim 5.

9. The specific detection target points include the wrist joint point and the first point.

9. The image recognition device according to claim 8.

10. The present invention further includes an acquisition means for acquiring an arm orientation in the image based on the position of the wrist joint point detected by the position detection means and the position of the first point, The determination means determines whether or not the first object region is a false detection when, in a first object region and a second object region detected from one image by the region detection means, a specific detection target point among the detection target points detected in the second object region by the position detection means is present in the first object region and an angular difference between an orientation of a first arm in the first object region and an orientation of a second arm in the second object region, both acquired by the acquisition means, is smaller than a threshold value.

10. The image recognition device according to claim 9.

11. The determination means determines that the first object region is a false detection when the specific detection target point of the second object region exists within the first object region, the specific detection target point of the first object region exists within the second object region, and the upper end of the second object region is located above the upper end of the first object region.

2. The image recognition device according to claim 1.

12. The determination means determines that the first object region is a false detection when the region detection means detects three or more object regions including the first object region from one image, and at least one of the specific detection target points of each of the object regions other than the first object region exists within the first object region.

2. The image recognition device according to claim 1.

13. a region detection step of detecting an object region including a target object from the image; a position detection step of detecting a position of a detection target point on the target object for each of the object regions when the object regions are detected from one image by the region detection step; a determination step of determining whether or not the object region is an erroneous detection for each of the object regions when the region detection step detects a plurality of the object regions from one image, In the determination step, in a first object region and a second object region detected from one image by the region detection step, if a specific detection target point among the detection target points detected from the second object region by the position detection step exists in the first object region, the first object region is determined to be a false detection. An image recognition method comprising:

14. A program for causing a computer to function as each of the means of the image recognition device according to claim 1.

15. an area detection device that detects an object area including a target object from an image; a position detection device that detects a position of a detection target point on the target object for each of the object regions when the region detection device detects a plurality of the object regions from one image; a determination device that, when a plurality of object regions are detected from one image by the region detection device, determines whether or not the object region is an erroneous detection for each of the object regions; The determination device determines that a first object region is a false detection when, in a first object region and a second object region detected from one image by the region detection device, a specific detection target point among the detection target points detected from the second object region by the position detection device is present in the first object region. An image recognition system characterized by:

Images