Object recognition apparatus, object recognition method, and program

Abstract

To provide an object recognition apparatus, an object recognition method, and a program capable of recognizing a position and an orientation of an object with high versatility.SOLUTION: An object recognition apparatus for recognizing a position and an attitude of an object, comprises: a data acquisition unit 111 for acquiring a plurality of pieces of local data from a direction intersecting a central axis of the object; a first estimation unit 112 for estimating a voting vector corresponding to each of the plurality of pieces of local data based on the plurality of pieces of local data acquired by the data acquisition unit 111, by referring to an estimation model 121 that defines a relation between the local data of the object and the voting vector extending toward the central axis of the object; and a second estimation unit 113 for estimating the central axis of the object based on the voting vector corresponding to each of the plurality of pieces of local data estimated by the first estimation unit 112. The second estimation unit 113 estimates the position of the object in at least one direction intersecting the central axis and the attitude of the object around the axis extending in the direction.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an object recognition device, an object recognition method, and a program.

Background Art

[0002] Conventionally, various techniques for recognizing the posture of an object having a three-dimensional shape have been proposed. For example, in the device described in Patent Document 1, the position and posture of a six-degree-of-freedom object are calculated by fitting a model to the edges on a two-dimensional image, and then, based on the position of the feature that determines the posture in a specific axis direction in the model coordinate system, a technique for calculating the posture of the object around the specific axis is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technique described in Patent Document 1, since the shape of the object to be recognized is limited to a rotationally symmetric shape, there is still room for improvement in enhancing its versatility.

[0005] The present invention has been made in view of such circumstances, and an object thereof is to provide an object recognition device, an object recognition method, and a program that can recognize the position and posture of an object with high versatility.

Means for Solving the Problems

[0006] To solve the above problems, an object recognition apparatus according to an aspect of the present invention is an object recognition apparatus that recognizes the position and orientation of an object, including: a data acquisition unit that acquires a plurality of local data from a direction intersecting the central axis of the object; and a first estimation unit that estimates a voting vector corresponding to each of the plurality of local data by referring to an estimation model that defines the relationship between the local data of the object and the voting vector extending toward the central axis of the object based on the plurality of local data acquired by the data acquisition unit; and a second estimation unit that estimates the central axis of the object based on the voting vectors corresponding to each of the plurality of local data estimated by the first estimation unit, wherein the second estimation unit estimates the position of the object in at least one direction intersecting the central axis and the orientation of the object around the axis extending in the direction.

[0007] An object recognition method according to an aspect of the present invention is an object recognition method that recognizes the position and orientation of an object, including: a data acquisition step of acquiring a plurality of local data from a direction intersecting the central axis of the object; a first estimation step of estimating a voting vector corresponding to each of the plurality of local data by referring to an estimation model that defines the relationship between the local data of the object and the voting vector extending toward the central axis of the object based on the plurality of local data acquired in the data acquisition step; and a second estimation step of estimating the central axis of the object based on the voting vectors corresponding to each of the plurality of local data estimated in the first estimation step, wherein the second estimation step includes estimating the position of the object in at least one direction intersecting the central axis and the orientation of the object around the axis extending in the direction.

[0008] A program according to an aspect of the present invention causes a computer to execute data acquisition processing for acquiring a plurality of local data from a direction intersecting the central axis of an object, and based on the plurality of local data acquired by the data acquisition processing, referring to an estimation model that defines the relationship between the local data of the object and a voting vector extending toward the central axis of the object, a first estimation process for estimating a voting vector corresponding to each of the plurality of local data, and a second estimation process for estimating the central axis of the object based on the voting vectors corresponding to each of the plurality of local data estimated by the first estimation process, and the second estimation process includes estimating the position of the object in at least one direction intersecting the central axis and the posture of the object around the axis extending in the direction.

Advantages of the Invention

[0009] According to the present invention, the position and posture of an object can be recognized with high versatility.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8A

Figure 8B

Best Mode for Carrying Out the Invention

[0011] Hereinafter, an embodiment of the object recognition device will be described with reference to the drawings.

[0012] As shown in FIG. 1, the object recognition device 100 is a device that recognizes the position and orientation of an object, and is realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Further, some or all of these components may be realized by hardware (including a circuitry such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit)), or may be realized by the cooperation of software and hardware. The program may be stored in a storage unit 120 such as an HDD or a flash memory of the object recognition device 100, or may be stored in a removable storage unit 120 such as a DVD or a CD-ROM, and may be installed in the HDD or the flash memory of the object recognition device 100 when the storage unit 120 is mounted on a drive device.

[0013] The control unit 110 of the object recognition device 100 includes, for example, a data acquisition unit 111, a first estimation unit 112, a second estimation unit 113, and a third estimation unit 114.

[0014] The data acquisition unit 111 acquires a plurality of local data from a direction intersecting the central axis of the object. The data acquisition unit 111, for example, acquires the data of the object and extracts a plurality of local data from the acquired data. The data of the object may be, for example, an image of the object captured by the camera 10. The camera 10 may be a two-dimensional camera or a three-dimensional camera. For example, based on the image of the object captured by the camera 10, as local data, a distance image of the object including object point cloud data regarding the distance for each point cloud of the object may be acquired. Alternatively, the data of the object may be, for example, object point cloud data including distance information of the object acquired by a distance sensor such as LiDAR, and based on the object point cloud data, as local data, a distance image of the object including information regarding the distance for each point cloud of the object may be acquired.

[0015] The first estimation unit 112 refers to an estimation model 121 that defines the relationship between the object data and the voting vector based on the plurality of local data acquired by the data acquisition unit 111, and estimates a voting vector corresponding to each of the plurality of local data. The voting vector is a vector starting from a predetermined position of the object and ending at the central axis of the object. The estimation model 121 is a model that outputs a voting vector corresponding to the local data when the local data of the object is input. The estimation model 121 is a model learned by machine learning using learning data in which a plurality of local data of the object and the voting vectors in the plurality of local data of the object are associated. The estimation model 121 is associated with CAD data 122 indicating the three-dimensional shape of the object. In the storage unit 120, CAD data 122 (for example, "CAD data A" to "CAD data N") indicating the three-dimensional shape of the object is stored for each type of the object (for example, N types), and for each of these CAD data 122, an estimation model 121 (for example, "estimation model A" to "estimation model N") is associated. Then, when the data acquisition unit 111 acquires the data of the object, the first estimation unit 112 specifies the CAD data 122 corresponding to the type of the object, reads out the estimation model 121 corresponding to the specified CAD data 122 from the storage unit 120, and uses it for the estimation of the voting vector.

[0016] The second estimation unit 113 estimates the position of the object in at least one direction intersecting the central axis of the object and the posture of the object around the axis extending in the direction based on the voting vectors corresponding to each of the plurality of local data estimated by the first estimation unit 112. Thus, the second estimation unit 113 estimates the central axis of the object. For example, the second estimation unit 113 estimates the position of the object in two directions intersecting the estimated central axis (the positions in the Y-axis direction and the Z-axis direction in the example shown in FIG. 3) and the posture of the object around the axis extending in each direction (the yaw angle and the pitch angle in the example shown in FIG. 3). For example, the second estimation unit 113 estimates the central axis of the object by performing statistical processing on the end point group data which is the data of the end points of the voting vectors corresponding to the plurality of local data. As the statistical processing, for example, RANSAC (Random Sample Consensus), which is an example of a method of learning the parameters of a mathematical model by excluding the influence of outliers from the data including outliers, can be mentioned. For example, when the object is long and slender, the second estimation unit 113 estimates the axis extending in the longitudinal direction of the object as the central axis of the object. For example, the second estimation unit 113 may estimate the central axis of the object based on the voting vectors whose estimation accuracy of the voting vectors estimated by the first estimation unit 112 satisfies a predetermined condition. For example, the second estimation unit 113 may discard the voting vectors whose estimation accuracy of the voting vectors estimated by the first estimation unit 112 is less than a predetermined value, and estimate the central axis of the object using the voting vectors whose estimation accuracy is equal to or higher than the predetermined value.

[0017] The third estimation unit 114 estimates the position of the object in the direction along the central axis of the object (the position in the X-axis direction in the example shown in FIG. 3) and the posture of the object around the axis of the central axis based on the central axis of the object estimated by the second estimation unit 113. For example, the third estimation unit 114 estimates the position of the object in the direction along the central axis of the object and the posture of the object around the axis of the central axis using, for example, template matching. For example, when the object is flat, the third estimation unit 114 estimates the axis extending in the direction along the plane of the object by the second estimation unit 113 as the central axis of the object, and based on the estimated central axis of the object, estimates the position on the plane of the object as the position of the object in the direction along the central axis of the object, and estimates the roll angle on the plane of the object as the posture of the object around the axis of the central axis.

[0018] FIG. 2 is a diagram for explaining an outline of processing when the object recognition device 100 according to the present embodiment recognizes the position and posture of an object.

[0019] As shown in FIG. 2, the data acquisition unit 111 acquires object point cloud data and generates a distance image obtained by converting the acquired object point cloud data into image data including distance information of the object.

[0020] Next, the first estimation unit 112 inputs a plurality of local images extracted as local data from the distance image to an estimation model 121 associated with CAD data 122 corresponding to the type of the object, and estimates a voting vector corresponding to each of the plurality of local images.

[0021] Next, the second estimation unit 113 estimates the central axis of the object based on the plurality of estimated voting vectors. In this case, when the estimated central axis of the object is the X-axis, the second estimation unit 113 estimates the position of the object in the axial direction of the Y-axis that intersects the X-axis, and estimates the rotation angle (Yaw) of the object around the axis of the Y-axis as the posture of the object. Further, the second estimation unit 113 estimates the position of the object in the axial direction of the Z-axis that intersects the X-axis, and estimates the rotation angle (Pitch) of the object around the axis of the Z-axis as the posture of the object (see FIG. 3). That is, the second estimation unit 113 determines the 4-degree-of-freedom data (Y, Z, Yaw, Pitch) of the object through the estimation of the central axis of the object.

[0022] Next, the third estimation unit 114 estimates the position of the object in the axial direction of the X-axis using template matching, and estimates the rotation angle (Roll) of the object around the axis of the X-axis as the posture of the object. That is, the third estimation unit 114 determines the remaining 2-degree-of-freedom data (X, Roll) of the object through template matching.

[0023] FIG. 4 is a diagram for explaining an outline of processing when the object recognition device 100 according to the present embodiment estimates the central axis of an object using the estimation model 121.

[0024] As shown in FIG. 4, first, as pre-learning, the object recognition device 100 refers to the CAD data 122 stored in the storage unit 120 and learns a function f indicating the correlation between the local image X of the object and the voting vector y of the object. Specifically, the object recognition device 100 collates each of the local images X of the plurality of objects with the corresponding CAD data 122, specifies the position corresponding to the local image X of the object in each CAD data 122, uses the specified position as a starting point, and uses the central axis of the object included in each CAD data 122 as an end point to calculate the voting vector y of the object corresponding to each local image X for each local image X of the object. Then, the object recognition device 100 learns a function f indicating the correlation between the local image X of the object and the voting vector y of the object for each local image X of the object based on the calculated voting vector y of the object.

[0025] Next, as an operation, the object recognition device 100 extracts a plurality of local images of the object from the image data of the object captured by the camera 10. Further, the object recognition device 100 inputs the extracted local images of the object into each corresponding function f obtained by pre-learning, and estimates the voting vector y of the object for each local image of the object. Then, the object recognition device 100 estimates the central axis of the object by performing statistical processing on the end point cloud data of the voting vector y of the object estimated for each local image of the object.

[0026] FIG. 5 is a diagram showing an example of the outline of the RANSAC process.

[0027] As shown in FIG. 5, the second estimation unit 113 randomly selects a combination of two points from the end point cloud data of the voting vector, and draws a straight line connecting the two points. Then, the second estimation unit 113 counts the number of data within a range of a predetermined distance from the straight line connecting the two points as a score, and estimates the straight line with the best score (score = "8" in the example shown in the figure) as the central axis of the object.

[0028] FIG. 6 is a diagram showing an example of the outline of the pattern matching process.

[0029] As shown in FIG. 6, when performing template matching, the third estimation unit 114 first reads out the CAD data 122 corresponding to the object to be recognized from the storage unit 120. Next, the third estimation unit 114 collates the four-degree-of-freedom data (Y, Z, Yaw, Pitch) of the object based on the central axis of the object estimated in advance by the second estimation unit 113 with the CAD data 122 read out from the storage unit 120 to determine the remaining two-degree-of-freedom data (X, Roll) of the object. Then, the third estimation unit 114 recognizes the position and orientation of the object based on the determined six-degree-of-freedom data of the object.

[0030] Next, the object recognition process executed by the object recognition device 100 of the present embodiment will be described with reference to the flowchart shown in FIG. 7. Hereinafter, an example of object recognition processing using the object recognition device 100 will be described. Note that the flowchart shown in FIG. 7 is executed, for example, at a predetermined cycle.

[0031] As shown in FIG. 7, first, the data acquisition unit 111 acquires a plurality of local data about the object (step S11). The content of the plurality of local data is as already described.

[0032] Next, the first estimation unit 112 performs a first estimation step based on the plurality of local data (step S12). Specifically, the first estimation unit 112 inputs the local data extracted in the previous step S11 into the estimation model 121 and estimates a voting vector corresponding to the local data.

[0033] Next, the second estimation unit 113 performs a second estimation step based on the voting vector estimated in the previous step S13 (step S14). As an example, the second estimation unit 113 may estimate four degrees of freedom of the object. Here, the estimation of the central axis of the object includes, for example, estimating the central axis of the object based on the voting vectors that satisfy a predetermined condition among the voting vectors estimated in the previous step 13.

[0034] Next, the third estimation unit 114 performs a third estimation step (step S15). Specifically, the third estimation step 114 estimates the remaining two degrees of freedom of the object using template matching.

[0035] In this way, through S11 to S15, the position and orientation of the object can be recognized.

[0036] Next, the operation of the object recognition device 100 according to the present embodiment will be described, particularly focusing on the operation when estimating the central axis of the object.

[0037] As shown in FIG. 8A, the positions of the object point cloud data included in the image of the object captured by the camera 10 generally have large position errors. Even if an attempt is made to estimate the position of the object by matching using the CAD data 122 based on these point cloud data, sufficient estimation accuracy may not be obtained. Also, when there is an overlap between multiple objects or a shielding object, etc., it becomes difficult to obtain the object point cloud data included in the image of the object. In this regard as well, it becomes a factor for reducing the recognition accuracy of the position and orientation of the object.

[0038] Regarding this point, as shown in FIG. 8B, in the present embodiment, a plurality of local images are extracted from the image of the object captured by the camera 10, and based on the plurality of extracted local images, the central axis of the object is estimated. In this way, the object point cloud data that is assumed to be located on the central axis of the object is estimated using the plurality of local images, and the central axis of the object is estimated by performing statistical processing on the estimated object point cloud data. Therefore, even if there is a slight error in the position of the object point cloud data included in the image of the object captured by the camera 10, the position and orientation of the object can be accurately recognized. Also, based on the plurality of local images extracted from the image of the object, in order to estimate the central axis of the object, even if there is an overlap between multiple objects or a shielding object, etc., the robustness in recognizing the position and orientation of the object is improved. Further, for example, objects of various shapes such as flat plate shape, curved plate shape, quadrangular prism shape, cylindrical shape, shaft shape, etc. can be targeted, and the position and orientation of the object can be recognized with high versatility.

[0039] Note that the above embodiment can also be implemented in the following forms. In the above embodiment, the third estimation unit 114 may calculate the length in the major axis direction of the central axis estimated based on the plurality of local images by the second estimation unit 113, and use the central axis of the object estimated by the second estimation unit 113 for recognizing the position and orientation of the object on the condition that the calculated length is equal to or greater than a predetermined value.

[0040] In the above embodiment, the second estimation unit 113 may classify the images of the object captured by the camera 10 into a plurality of image groups, and estimate the central axis of the object based on each of the classified image groups. In this case, when the axial direction of the central axis of the object estimated based on the first image group and the axial direction of the central axis of the object estimated based on the second image group match, the second estimation unit 113 may limit the search range of the central axis of the object. Thereby, the third estimation unit 114 can reduce the processing load when recognizing the position and orientation of the object based on the central axis of the object estimated by the second estimation unit 113.

[0041] In the above embodiment, the object recognition device 100 may be configured such that the estimation result of the central axis of the object by the second estimation unit 113 can be changed by receiving an operation from the user.

[0042] [Appendix] The technical idea that can be grasped from the above embodiment is described below.

[0043] <1> An object recognition device for recognizing the position and orientation of an object, a data acquisition unit that acquires a plurality of local data from a direction intersecting the central axis of the object, a first estimation unit that refers to an estimation model defining the relationship between the local data of the object and the voting vector extending toward the central axis of the object based on the plurality of local data acquired by the data acquisition unit, and estimates a voting vector corresponding to each of the plurality of local data, a second estimation unit that estimates the central axis of the object based on the voting vectors corresponding to each of the plurality of local data estimated by the first estimation unit, and the second estimation unit estimates the position of the object in at least one direction intersecting the central axis and the orientation of the object around the axis extending in the direction, an object recognition device.

[0044] <2> The second estimation unit estimates the position of the object in two directions intersecting the estimated central axis and the orientation of the object around the axis extending in each direction. The object recognition device described in <1>.

[0045] <3> Further comprising a storage unit storing an estimation model, The estimation model is a model learned by machine learning using learning data in which local data of an object is associated with a voting vector in the local data of the object. The object recognition device described in <1>.

[0046] <4> Further comprising a third estimation unit that estimates the position of the object in the direction along the central axis estimated by the second estimation unit and the posture of the object around the axis extending in the direction. The object recognition device according to any one of <1> to <3>.

[0047] <5> The third estimation unit estimates the position of the object in the direction along the central axis estimated by the second estimation unit and the posture of the object around the axis extending in the direction using template matching. The object recognition device described in <4>.

[0048] <6> When the object is elongated, the second estimation unit estimates the axis extending in the longitudinal direction of the object as the central axis. The object recognition device according to any one of <1> to <5>.

[0049] <7> The local data is a distance image of the object including information regarding the distance for each point group of the object. The object recognition device according to any one of <1> to <6>.

[0050] <8> The second estimation unit estimates the central axis based on the voting vectors whose estimation accuracy satisfies a predetermined condition among the voting vectors estimated by the first estimation unit. The object recognition device according to any one of <1> to <7>.

[0051] <9> An object recognition method for recognizing the position and orientation of an object, a data acquisition step of acquiring a plurality of local data from a direction intersecting the central axis of the object, referring to an estimation model that defines the relationship between the local data of the object and the voting vector extending toward the central axis of the object based on the plurality of local data acquired in the data acquisition step, and estimating a voting vector corresponding to each of the plurality of local data in a first estimation step, a second estimation step of estimating the central axis of the object based on the voting vectors corresponding to each of the plurality of local data estimated in the first estimation step, including the second estimation step includes estimating the position of the object in at least one direction intersecting the central axis and the orientation of the object around the axis extending in the direction, object recognition method.

[0052] <10> causing a computer to perform a data acquisition process of acquiring a plurality of local data from a direction intersecting the central axis of the object, referring to an estimation model that defines the relationship between the local data of the object and the voting vector extending toward the central axis of the object based on the plurality of local data acquired in the data acquisition process, and estimating a voting vector corresponding to each of the plurality of local data in a first estimation process, a second estimation process of estimating the central axis of the object based on the voting vectors corresponding to each of the plurality of local data estimated in the first estimation process, and causing it to execute, the second estimation process includes estimating the position of the object in at least one direction intersecting the central axis and the orientation of the object around the axis extending in the direction, program.

[0053] Note that the embodiments described above are for facilitating the understanding of the present invention and are not for limiting the interpretation of the present invention. The present invention can be modified / improved without departing from its gist, and equivalents thereof are also included in the present invention. That is, what those skilled in the art appropriately modify in each embodiment is also included in the scope of the present invention as long as it has the features of the present invention. For example, each element included in each embodiment and its arrangement, material, conditions, shape, size, etc. are not limited to those illustrated and can be appropriately changed. In addition, each embodiment is an example, and it goes without saying that partial substitution or combination of the configurations shown in different embodiments is possible, and these are also included in the scope of the present invention as long as they include the features of the present invention.

Explanation of Reference Numerals

[0054] 10... Camera, 100... Object recognition device, 110... Control unit, 111... Data acquisition unit, 112... First estimation unit, 113... Second estimation unit, 114... Third estimation unit, 120... Storage unit, 121... Estimation model, 122... CAD data.

Claims

1. An object recognition device for recognizing the position and orientation of an object, comprising: a data acquisition unit that acquires a plurality of local data from a direction intersecting the central axis of the object; a first estimation unit that refers to an estimation model defining the relationship between the local data of the object and a voting vector extending toward the central axis of the object, and estimates the voting vector corresponding to each of the plurality of local data; a second estimation unit that estimates the central axis of the object based on the voting vectors corresponding to each of the plurality of local data estimated by the first estimation unit; wherein the second estimation unit estimates the position of the object in at least one direction intersecting the central axis and the orientation of the object around an axis extending in the direction; an object recognition device.

2. The object recognition device according to claim 1, wherein the second estimation unit estimates the position of the object in two directions intersecting the estimated central axis and the orientation of the object around an axis extending in each direction. The object recognition device according to claim 1.

3. further comprising a storage unit that stores the estimation model, wherein the estimation model is a model learned by machine learning using learning data associating local data of an object with the voting vector in the local data of the object; The object recognition device according to claim 1.

4. The object recognition device according to claim 1, further comprising a third estimation unit that estimates the position of the object in the direction along the central axis estimated by the second estimation unit and the orientation of the object around an axis extending in the direction. The object recognition device according to claim 1.

5. The object recognition device according to claim 4, wherein the third estimation unit uses template matching to estimate the position of the object in the direction along the central axis estimated by the second estimation unit and the orientation of the object around an axis extending in the direction. The object recognition device according to claim 4.

6. The object recognition device according to claim 1, wherein when the object is elongated, the second estimation unit estimates the axis extending in the longitudinal direction of the object as the central axis. The object recognition device according to claim 1.

7. The local data is a distance image of the object including information regarding the distance for each point cloud of the object. The object recognition device according to claim 1.

8. The object recognition device according to claim 1, wherein the second estimation unit estimates the central axis based on the voting vectors among the voting vectors estimated by the first estimation unit that satisfy a predetermined condition in terms of estimation accuracy. The object recognition device according to claim 1.

9. An object recognition method for recognizing the position and orientation of an object, comprising: A data acquisition step of acquiring a plurality of local data from a direction intersecting the central axis of the object; A first estimation step of estimating the voting vector corresponding to each of the plurality of local data by referring to an estimation model that defines the relationship between the local data of the object and the voting vector extending toward the central axis of the object based on the plurality of local data acquired in the data acquisition step; A second estimation step of estimating the central axis of the object based on the voting vectors corresponding to each of the plurality of local data estimated in the first estimation step; comprising; The second estimation step includes estimating the position of the object in at least one direction intersecting the central axis and the posture of the object around the axis extending in the direction. An object recognition method.

10. Causing a computer to perform a data acquisition process of acquiring a plurality of local data from a direction intersecting the central axis of the object; A first estimation process of estimating the voting vector corresponding to each of the plurality of local data by referring to an estimation model that defines the relationship between the local data of the object and the voting vector extending toward the central axis of the object based on the plurality of local data acquired in the data acquisition process; A second estimation process of estimating the central axis of the object based on the voting vectors corresponding to each of the plurality of local data estimated in the first estimation process; and The second estimation process includes estimating the position of the object in at least one direction intersecting the central axis and the posture of the object around the axis extending in the direction. A program.

Images