Object attitude estimation device and object attitude estimation method

Abstract

To improve estimation accuracy and a calculation load in attitude estimation of an object.SOLUTION: Provided is a device for estimating an attitude of an object, the object attitude determination device including: a feature point measurement unit that acquires a feature point of an object; a CAD information extraction unit that acquires shape information of the object from CAD data of a CAD database; a motion definition information extraction unit that extracts motion definition information of a mechanism including the object from the CAD data; a motion restriction information generation unit that generates from the motion definition information, motion restriction information defining a movable range determined, with another object on the mechanism of the object defined as a starting point; and an attitude estimation unit that estimates an attitude of the object from the feature point, the shape information and the motion restriction information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for estimating the pose of an object. [Background technology]

[0002] Background art of the present invention includes, for example, Patent Document 1. Patent Document 1 discloses a method for generating a heat map that identifies the probability of the positions of feature points of an object from a two-dimensional image and comparing it with a three-dimensional virtual CAD model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-163503 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 makes it possible to estimate the pose of an object by comparing the measured feature points of the object with CAD data related to the shape of the object. However, further improvements are required in terms of the estimation accuracy and the computational load in estimating the pose of an object. [Means for solving the problem]

[0005] A preferred aspect of the present invention is an apparatus for estimating the posture of an object, comprising: a feature point measurement unit that acquires feature points of the object; a CAD information extraction unit that acquires shape information of the object from CAD data in a CAD database; an action definition information extraction unit that extracts action definition information of a mechanism including the object from the CAD data; a motion restriction information generation unit that generates, from the action definition information, motion restriction information that defines a range of motion of the object determined with respect to another object on the mechanism as a starting point; and a posture estimation unit that estimates the posture of the object from the feature points, the shape information, and the motion restriction information.

[0006] Another preferred aspect of the present invention is a method for estimating the posture of an object, comprising: a feature point measurement step of acquiring feature points of the object; a CAD information extraction step of acquiring shape information of the object from CAD data; an action definition information extraction step of extracting, from the CAD data, action definition information of a mechanism including the object; a action restriction information generation step of generating, from the action definition information, action restriction information that defines a range of motion of the object, determined with respect to another object on the mechanism as a starting point; and an attitude estimation step of estimating the posture of the object from the feature points, the shape information, and the action restriction information. [Effects of the Invention]

[0007] According to the present invention, it is possible to further improve the estimation accuracy and the calculation load in estimating the posture of an object. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of an object pose estimation device. [Figure 2] FIG. 1 is a perspective view illustrating a method for estimating the posture of an object. [Figure 3] FIG. 1 is a diagram illustrating a method for estimating the posture of an object. [Figure 4] 1 is a flowchart of a method for estimating the pose of an object. [Figure 5] FIG. 1 is a block diagram of an object pose estimation device. [Figure 6] 1 is a flowchart of a method for estimating the pose of an object. [Figure 7] FIG. 1 is a block diagram of an object pose estimation device. [Figure 8A] FIG. 1 is an explanatory diagram of an object posture estimation device. [Figure 8B] 1 is a table showing feature points and coordinates of an object pose estimation device. [Figure 9] FIG. 1 is a diagram illustrating a method for estimating the posture of an object. [Figure 10] 1 is a flowchart of a method for estimating the pose of an object. [Figure 11] 1 is a block diagram of an object pose estimation method. [Figure 12] 1 is a flowchart of a method for estimating the pose of an object. [Figure 13] FIG. 1 is a block diagram of an object pose estimation device. [Figure 14] 1 is a flowchart of a method for estimating the pose of an object. [Figure 15] FIG. 1 is a block diagram of an object pose estimation device. [Figure 16] 1 is a flowchart of a method for estimating the pose of an object. [Figure 17] 10 is a flowchart of a mechanism operation definition extraction method. [Figure 18A] FIG. 1 is a diagram illustrating a method for estimating the posture of an object. [Figure 18B] FIG. 1 is a diagram illustrating a method for estimating the posture of an object. DETAILED DESCRIPTION OF THE INVENTION

[0009] Examples of the present invention will be described below. However, the present invention should not be construed as being limited to the description of the following embodiments. Those skilled in the art will readily understand that the specific configuration can be modified within the scope of the concept and spirit of the present invention.

[0010] In the configurations of the embodiments described below, the same parts or parts having similar functions are denoted by the same reference numerals in different drawings, and redundant explanations may be omitted.

[0011] When there are multiple elements having the same or similar functions, they may be described using the same reference numeral with different subscripts. However, when there is no need to distinguish between multiple elements, the subscripts may be omitted.

[0012] The designations "first," "second," "third," etc. in this specification are used to identify components and do not necessarily limit the number, order, or content thereof. Furthermore, numbers used to identify components are used in different contexts, and numbers used in one context do not necessarily indicate the same configuration in another context. Furthermore, this does not prevent a component identified by a certain number from also serving the function of a component identified by another number.

[0013] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings etc. may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings etc.

[0014] All publications, patents, and patent applications cited herein are incorporated by reference in their entirety.

[0015] As used herein, elements referred to in the singular are intended to include the plural unless the context clearly indicates otherwise. [Example]

[0016] A first embodiment of an object pose estimation device will be described using Figures 1, 2, 3, and 4. According to this embodiment, when estimating the pose of an object, in addition to measured feature points of the object and CAD data related to the shape of the object, conditions of the object's range of motion derived based on the definition of the movement mechanism including the object are used, thereby making it possible to achieve high-precision and high-speed pose estimation.

[0017] FIG. 1 is a block diagram of an object pose estimation device 1 according to this embodiment. In FIG. 1, a feature point position information measurement unit 101 measures each feature point of an object for which pose estimation is to be performed from a captured image of the object, and a CAD database 102 is a database that pre-stores shape information and the like of CAD models related to each object, including the object for which pose estimation is to be performed. A CAD feature point position information and pose information extraction unit 103 extracts, from the CAD database 102, a set of combinations of feature point coordinates in a CAD model space of the object for which pose estimation is to be performed and pose information corresponding to the feature point position coordinates. Feature points can be obtained not only from captured images but also from measurement data obtained by measuring the target object using techniques such as TOF (Time of Flight) or LiDAR (Light Detection and Ranging).

[0018] Furthermore, the action definition information extraction unit 105 extracts the action definition of the mechanism including the object for which posture estimation is to be performed from the CAD database 102. The action restriction information generation unit 106 generates action restriction information indicating a range in which the object for which posture estimation is to be performed can move, from the action definition of the mechanism including the object for which posture estimation is to be performed extracted by the action definition information extraction unit 105, starting from other objects that make up the mechanism of the object for which posture estimation is to be performed.

[0019] Pose estimation unit 107 estimates the posture of the object from the feature point information input from feature point position information measurement unit 101, a set of posture information corresponding to feature point coordinates that highly match the CAD model-based measured feature points input from CAD feature point position information and posture information extraction unit 103, and the movement restriction information of the object whose posture is to be estimated, extracted by movement restriction information generation unit 106. Output terminal 108 outputs the posture estimation result.

[0020] The object posture estimation device 1 can be configured as a general information processing device, such as a server. The server generally includes a processing device, a storage device, an input device, and an output device. The CAD database 102 is configured by storing CAD data in a storage device, such as a magnetic disk device or semiconductor memory.

[0021] The feature point position information measurement unit 101, CAD feature point position information and posture information extraction unit 103, action definition information extraction unit 105, action restriction information generation unit 106, and posture estimation unit 107 are realized by, for example, a processing device executing software stored in a storage device. Functions equivalent to those configured by software can also be realized by hardware such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0022] The output terminal 108 is an example of an output device, and an image of an object captured by a camera or the like from outside the device is input to the feature point position information measurement unit 101 via an input device.

[0023] The above configuration may be configured by a single server, or any part of the input device, output device, processing device, and storage device may be configured by another server connected via a network.

[0024] Figure 2 shows an example of a motion mechanism including an object for which posture estimation is to be performed, and is an explanatory diagram of a method for estimating the posture of an object. In Figure 2, the mechanism is composed of a Body (301), Slide 1 (302) that performs a sliding motion relative to the Body (301), a Motor (303) fixed to Slide 1 (302), a Y_Robot (304) also fixed to Slide 1 (302), and Slide 2 (305) that performs a sliding motion relative to the Y_Robot (304). Body, Slide, and Robot are symbols that uniquely indicate each part of the object.

[0025] The CAD database 102 stores, in any data format, shape information (e.g., coordinates indicating the outer shape of the body) and positional relationship information (e.g., the positional relationship between the body and slide 1) of each part of the target object whose posture is to be estimated as CAD data, as well as posture information corresponding to feature point position coordinates as needed. The CAD data may also include information defining the operation of each part of the object (e.g., slide 2 slides in the longitudinal direction of the Y_Robot (direction of arrow 306), and Y_Robot and slide 1 are fixed).

[0026] The posture of an object is expressed by the (x coordinate, y coordinate, z coordinate, x rotation angle, y rotation angle, z rotation angle) of one or more points on the object. Therefore, the postures of Body (301), Slide1 (302), Motor (303), Y_Robot (304), and Slide2 (305) are expressed as follows:

[0027] Body;(Body_x, Body _y , Body _z, Body _Rx, Body _Ry, Body _Rz) …(Formula 1) Slide1;(Slide1_x, Slide1 _y , Slide1 _z, Slide1 _Rx, Slide1 _Ry, Slide1 _Rz) …(Formula 2) Motor;(Motor_x, Motor _y , Motor _z, Motor _Rx, Motor _Ry, Motor _Rz) …(Formula 3) Y_Robot;(Y_Robot_x, Y_Robot _y , Y_Robot _z, Y_Robot _Rx, Y_Robot _Ry, Y_Robot _Rz) …(Formula 4) Slide2;(Slide2_x, Slide2 _y , Slide2 _z, Slide2 _Rx, Slide2 _Ry, Slide2 _Rz) …(Formula 5)

[0028] For example, when estimating the posture of Slide2 (305), if Y_Robot (304) is facing in the direction of the arrow (306), for example, the x-axis direction, and taking into consideration the action definition information that "Slide2 slides in the longitudinal direction of Y_Robot," the action restriction information indicates that there are action restrictions such as "Slide2 (305) cannot move in the y-axis or z-axis directions, and cannot rotate on the x-axis, y-axis, or z-axis."

[0029] This is shown in a more simplified manner in Figure 3. Figure 3 is an explanatory diagram showing a method for estimating the posture of an object, and shows the movement restrictions imposed on the object for which posture estimation is to be performed. It shows that the movement restriction for Slide2 (305) in the direction of arrow 306 (x-axis direction) means that it can only assume a posture of positions 402, 403, or 404, or an interpolated or extrapolated posture. This movement restriction information can be shown, for example, as the range of possible coordinates of a specific point on Slide2 (305).

[0030] 2 includes a first sliding mechanism for the assembly of Slide1 (302), Motor (303), and Y_Robot (304) relative to Body (301), and a second sliding mechanism for Slide2 (305) relative to the assembly of Slide1 (302), Motor (303), and Y_Robot (304). The first sliding mechanism is movable in the longitudinal direction of Body (301), and the second sliding mechanism is movable in the longitudinal direction of Y_Robot (304) (arrow direction 306). However, to simplify the explanation, unless otherwise specified, we will proceed under the assumption that the first sliding mechanism for the assembly of Slide1 (302), Motor (303), and Y_Robot (304) relative to Body (301) is fixed and does not move.

[0031] As shown above, the movement constraints are incorporated into the posture representation of each element. If the movement constraints for Slide2 (305) starting from Y_Robot (304) are expressed as Δt, the posture of Slide2 is as shown in Equation 6 below. Slide2; (Y_Robot_x+Δx(t), Y_Robot _y+Δy(t), Y_Robot _z+Δz(t), Y_Robot _Rx+ΔRx(t), Y_Robot _Ry+ΔRy(t), Y_Robot _Rz+ΔRz(t)) …(Formula 6)

[0032] If we plug the fact that the arrow direction is the x-axis into the above equation, movement and rotation in directions other than the x-axis are restricted, resulting in the following. Slide2; (Y_Robot_x+Δx(t), Y_Robot _y , Y_Robot _z, Y_Robot _Rx, Y_Robot _Ry, Y_Robot _Rz) …(Formula 7)

[0033] If we show the posture of Slide 2 only by the difference of Y_Robot, we take the difference between Equation 6 and Equation 7, Slide2; (Δx(t),0,0,0,0) …(Formula 8) This can be shown as:

[0034] The flowchart showing the object pose estimation method shown in Fig. 4 shows the operation of the pose estimation device 1 shown in the block diagram of Fig. 1. In Fig. 4, step S201 indicates the start of the process.

[0035] Step S202 indicates a processing step in which the feature point position information measurement unit 101 measures each feature point of the object for which pose estimation is to be performed. Each feature point of the object can be measured by capturing an image of the object with a camera or the like, inputting the captured image from an input device to the feature point position information measurement unit 101, and extracting the feature points using a predetermined algorithm, but here a known technique will be used. Each feature point of the object can be represented using any coordinate system, but here it will be represented by (x coordinate, y coordinate, z coordinate).

[0036] Step S203 is a processing step following processing step S202 in which the action definition information extraction unit 105 extracts an action definition of a mechanism including an object whose posture is to be estimated from the CAD database 102. The action definition indicates how the objects constituting the mechanism move relative to each other.

[0037] Step S204 is a processing step following processing step S203 in which the movement restriction information generation unit 106 generates movement restriction information indicating the range of possible movements of the object whose posture is to be estimated, from the movement definition extracted by the movement definition information extraction unit 105, starting from other objects that constitute the mechanism of the object whose posture is to be estimated.

[0038] Step S205, following processing step S204, indicates a processing step in which the posture estimation unit 107 estimates the posture of the object from the feature point information input from the feature point position information measurement unit 101, a set of posture information corresponding to feature point coordinates that highly match the CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103, and the movement restriction information of the target object for posture estimation extracted by the movement restriction information generation unit 106.

[0039] Step S206 indicates that the process ends after the orientation of the object is estimated in step S205. Note that step S202 is not limited to the example in FIG. 4 and may be executed at any timing before step S205.

[0040] As described above, according to this embodiment, it is possible to use not only the feature points of the target object and a set of feature point information and orientation information of the 3D model of the target object, but also movement restrictions on the target object originating from an object other than the target object that constitutes a movement mechanism together with the target object. Therefore, it is possible to provide an object orientation estimation device and an object orientation estimation method that improve the accuracy of orientation determination by determining whether the movement restrictions are satisfied after estimating the orientation from the set of feature point information and orientation information of the 3D model of the target object, or that speed up the orientation estimation process by limiting the orientations that the target object can take.

[0041] Furthermore, by expressing the object's posture using differential data from the base point of the movement restriction as shown in (Equation 8), it is possible to achieve the effect of compressing and storing the amount of posture information data for the target object.By combining the posture data stored in this way with the movement restriction information, it is possible to easily restore (Equation 5), which indicates the normal posture. [Example]

[0042] Next, a second embodiment of the object pose estimation device will be described with reference to Figures 5 and 6. Figure 5 is a block diagram of an object pose estimation device 1-2 according to the second embodiment. Figure 6 is a flowchart showing a method for estimating the object pose.

[0043] In FIG. 5, the feature point position information measurement unit 101, CAD database 102, CAD feature point position information and posture information extraction unit 103, action definition information extraction unit 105, and action restriction information generation unit 106 are the same components as those shown in FIG. 1.

[0044] The orientation estimation calculation unit 501 estimates the orientation of an object from a set of feature point information input from the feature point position information measurement unit 101 and orientation information corresponding to feature point coordinates that have a high degree of agreement with CAD model-based measured feature points input from the CAD feature point position information and orientation information extraction unit 103.

[0045] The posture estimation accuracy calculation unit 502 calculates the degree of deviation between the feature points corresponding to the posture information of the CAD model and the feature points obtained by measurement, and outputs the calculated deviation amount as feedback to the posture estimation calculation unit 501 so that the posture estimation calculation unit 501 can perform posture estimation that minimizes the deviation amount.

[0046] Posture correction unit 503 determines whether the posture estimation result obtained by posture estimation calculation unit 501 satisfies the movement restrictions, and if not, calculates the difference between the range of motion of the conditions indicated in the movement restriction information and the estimated posture, i.e., posture accuracy information. If it is determined that the posture estimation result does not satisfy the movement restriction information, it performs correction based on the difference between the posture estimation result and the movement restriction information so that the difference becomes zero or falls within a predetermined allowable error range. The corrected posture estimation result is output from output terminal 108. Posture estimation calculation unit 501, posture estimation accuracy calculation unit 502, and posture correction unit 503 are an example of the configuration of posture estimation unit 107 in FIG. 1.

[0047] FIG. 6 is a flowchart showing a method for estimating the pose of an object, and shows the operation of the pose estimation device 1-2 shown in the block diagram of FIG.

[0048] 6, step S201 indicates the start of processing, and step S202 indicates a processing step in which each feature point of an object whose pose is to be estimated is measured by the feature point position information measurement unit 101. Processing step S202 is executed by the feature point position information measurement unit 101 in FIG.

[0049] Step S601 is a CAD feature point comparison step in which the posture estimation calculation unit 501 compares the feature points of the object measured in step S202 with the feature points of a CAD model corresponding to the object.

[0050] Step S602 is a CAD feature point corresponding posture calculation step in which the posture estimation calculation unit 501 obtains posture data corresponding to the CAD model feature points used in step S601.

[0051] Step S603 is processing in which the posture estimation accuracy calculation unit 502 calculates the degree of deviation between the measured feature points and the feature points of the CAD model.

[0052] In step S604, the posture estimation accuracy calculation unit 502 determines whether the error value calculated in step S603 is minimum (or equal to or less than the allowable value). If it is determined that the error value is not minimum (or equal to or less than the allowable value), the posture estimation calculation unit 501 reselects a set of posture and feature points for the posture corrected on the CAD model (S605).

[0053] The orientation that minimizes the error between the measured feature points and the positions of the feature points on the CAD model is found. The processes from step S601 to step S605 are executed by the orientation estimation calculation unit 501 and the orientation estimation accuracy calculation unit 502 in FIG.

[0054] Step S203 indicates a processing step in which the action definition information extraction unit 105 extracts, from the CAD database 102, the action definition of the mechanism including the object for which posture estimation is to be performed.

[0055] Step S204 is a processing step following processing step S203 in which the motion restriction information generation unit 106 generates motion restriction information indicating the range of possible motion of the object whose posture is to be estimated, from the motion definition of the mechanism including the object whose posture is to be estimated extracted by the motion definition information extraction unit 105, for the target object whose posture is to be estimated, starting from other objects that make up the mechanism.

[0056] In step S606, the posture correction unit 503 determines whether or not the posture of the object obtained in the processes of steps S601 to S605 violates the object's movement restriction information obtained in processing step S204. If there is no violation of the movement restriction, the process ends (S206).

[0057] If there is a violation of the movement restrictions, the posture correction unit 503 executes a process (S607) to correct the posture estimation so that the movement restrictions are satisfied or the difference with respect to the movement restrictions is less than a predetermined allowable error, and after satisfying the movement restrictions, terminates the process (S206).

[0058] If necessary, the accuracy of the attitude corrected by the attitude correction unit 503 is again determined by the attitude estimation accuracy calculation unit 502 .

[0059] As described above, according to this embodiment, after determining the posture of an object using a set of feature points of the target object and feature point information and posture information of the 3D model of the target object, it is possible to check whether the movement constraints of the target object, which are based on objects other than the target object that form the movement mechanism together with the target object, are satisfied, and if not, corrections can be made, thereby specifically demonstrating that it is possible to improve the accuracy of posture determination.

[0060] According to this embodiment, in the process of estimating posture using a set of measured feature points of the target object and feature point information and posture information of a 3D model of the target object, a set of feature point information of the 3D model and corresponding posture information with the highest degree of match is selected through sequential comparison. As another example, even if there are not a sufficient number of sets of feature point information and posture information of the 3D model for posture estimation, the effect of improving accuracy can be obtained by combining with various reported posture estimation methods, such as using AI to compare feature points and estimate posture using feature points on a 2D image of the target object and a 3D model of the target object. [Example]

[0061] Fig. 7 is a block diagram of an object posture estimation device 1-3 according to a third embodiment. In Fig. 7, the feature point position information measurement unit 101, CAD database 102, CAD feature point position information and posture information extraction unit 103, action definition information extraction unit 105, and action restriction information generation unit 106 are the same components as those shown in Fig. 1. However, in this embodiment, the feature point position information measurement unit 101 and the CAD feature point position information and posture information extraction unit 103 perform the same processing for both object A, the posture estimation target, and object B, which serves as the base point for the movement restriction of the posture estimation target object.

[0062] The posture estimation calculation unit 501-3 estimates the postures of both object A, the posture of which is the object for posture estimation, and object B, which constitutes a movement mechanism together with the posture estimation object and serves as the base point for restricting the movement of the posture estimation object, from a set of posture information corresponding to feature point information input from the feature point position information measurement unit 101 and feature point coordinates that highly match the CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103.

[0063] The target attitude estimation accuracy calculation unit 701 is a means for extracting the accuracy of the result of the attitude estimation process performed by the attitude estimation calculation unit 501 of the object A. Here, the accuracy of the attitude estimation process result will be described.

[0064] 8A and 8B are diagrams illustrating the reliability of the pose estimation results. Fig. 8A shows feature points R1 to R7 of the object to be measured and points V1 to V7 corresponding to R1 to R7 on the CAD model of the object. The coordinates of each point are shown in Fig. 8B.

[0065] If the posture is found ideally, the coordinates of each point will match perfectly, and the error will be 0. In reality, errors occur due to measurement errors at each feature point. The larger the error, the greater the dispersion of the measured values, and the lower the accuracy of the posture estimation result obtained from this. The size of the posture estimation error can be expressed as the sum of the squares of the errors Δ at each point, as shown in (Equation 9), and the posture estimation accuracy is the reciprocal Δ -2 This becomes: Δ 2 =(R1x-V1x) 2 +(R1y-V1y) 2 +(R1z-V1z) 2 +(R2x-V2x) 2 +(R2y-V2y) 2 + (R2z-V2z) 2 + (R3x-V3x) 2 +(R3y-V3y) 2 +(R3z-V3z) 2 +(R4x-V4x) 2 +(R4y-V4y) 2 + (R4z-V4z) 2 + +(R7x-V7x) 2 +(R7y-V7y) 2 + (R7z-V7z) 2 …(Formula 9)

[0066] The base attitude estimation accuracy calculation unit 702 is a means for extracting the accuracy of the result of the attitude estimation process performed by the attitude estimation calculation unit 501 of the object B, and similarly to the target attitude estimation accuracy calculation unit 701, calculates Δ -2 Therefore, the likelihood of object A is Δ -2 is ΔA -2 , the likelihood of object B is Δ -2 is ΔB -2 It is expressed as follows.

[0067] Correction weighting calculation unit 703 performs correction weighting when correcting the posture estimation results, based on the accuracy of posture estimation of objects A and B calculated by target posture estimation accuracy calculation unit 701 and base posture estimation accuracy calculation unit 702. Posture correction unit 704 corrects the posture estimation of object A in accordance with the correction weighting calculated by correction weighting calculation unit 703. Next, the posture estimation correction process of posture correction unit 704 will be further described.

[0068] 9 is a diagram illustrating a method for correcting posture estimation described in this embodiment. The posture estimation is corrected so as to satisfy the movement restriction information. Simply put, the posture estimation result of object A should be corrected so that the posture estimation result of object B, which is the base point of the movement mechanism, satisfies the movement restriction information. However, since the posture estimation result of object B cannot be obtained with 100% certainty, the correction is performed by correcting both object A and object B based on the certainty.

[0069] If object B corresponds to Y_Robot (304) in Figure 2 and object A corresponds to Slide2 (305) in Figure 2, the center point of object A is only allowed to move on the object A's central movement axis due to movement restrictions. Therefore, if the accuracy of object B's posture estimation is 100%, it is sufficient to correct only the posture estimation of object A and make the deviation on the diagram (the shortest distance between a point and a line) zero, but the accuracy of object B's posture estimation is not 100% either. Therefore, object B also needs to be corrected according to its posture accuracy.

[0070] That is, the attitude accuracy of object B ΔB -2 When normalized to 100, the normalization process results in ΔA -2 is expressed as 50, correction is made according to the likelihood by applying a weight of 100 / (100+50)=0.66 from object A and 50 / (100+50)=0.33 from object B.

[0071] Figure 10 is a flowchart showing the operation of the posture estimation device shown in the block diagram of Figure 7. Processes with the same reference numerals as those in Figure 6 are the same as those in Figure 6, and therefore a description thereof will be omitted. However, as mentioned above, in this embodiment, up until step S801, the same process is performed for both object A, the posture estimation target, and object B, which is the base point for restricting the movement of the posture estimation target object.

[0072] In step S604, the target attitude estimation accuracy calculation unit 701 and the base attitude estimation accuracy calculation unit 702 determine whether the error value calculated in step S603 is the minimum (or an acceptable value), and if it is determined that the error value is not the minimum (or an acceptable value), the attitude estimation calculation unit 501-3 may reselect a set of attitude and feature points for the attitude corrected on the CAD model (S605).

[0073] Step S801 is a target object element attitude accuracy extraction step, in which the target attitude estimation accuracy calculation unit 701 in FIG. 7 estimates the attitude of object A, that is, calculates the attitude estimation accuracy of the target object to be attitude-estimated.

[0074] 7 calculates the accuracy of estimating the posture of object B, i.e., the posture estimation accuracy of the object that serves as the base point of the motion restriction specification. The target object A and the base object B can be specified automatically from the motion definition or by using an appropriate GUI for operation.

[0075] If it is determined in processing step S606 that a motion restriction violation has occurred, the correction weighting calculation unit 703 performs accuracy weighting processing step S803. Processing step S803 is a processing step for calculating correction weights in accordance with the pose estimation accuracy of object A calculated in processing step S801 and the pose estimation accuracy of object B calculated in processing step S802.

[0076] In processing step S804, the posture estimation is corrected in accordance with the weighting calculated in S803. Processing step S803 is performed by the correction weighting calculation unit 703 in Fig. 7, and processing step S804 is performed by the posture correction unit 704 in Fig. 7. If the motion restrictions are satisfied in processing step S804, the processing ends (S206).

[0077] As described above, according to this embodiment, the posture of object A can be corrected by weighting the posture estimation results of object A, which is the target of posture estimation, and object B, which is the base point for restricting the movement of object A, based on the likelihood of the posture estimation results. This makes it possible to realize a correction process for improving the accuracy of posture determination.

[0078] Furthermore, according to this embodiment, correction is performed by weighting, so that the correction process can be performed at high speed without repeating the process.

[0079] Furthermore, although the present embodiment shows a case where there is one object that serves as a base point for the movement restriction, it is possible to deal with a case where there are multiple objects by repeatedly carrying out the operation restriction for each base point.

[0080] In the explanation of this embodiment, correction of posture estimation at a specific moment has been described. However, in cases where object A moves continuously, weighting based on likelihood comparison at specific timings such as the beginning or end of the movement may be used to correct the posture at each timing, or correction may be made based on the average value of the likelihood comparison.

[0081] Furthermore, even when various reported posture estimation methods are used, such as using AI to compare feature points on a 2D image of the target object and a 3D model of the target object to perform feature point comparison and posture estimation, the definition is different from that of Equation 9, but a similar effect can be obtained by performing correction according to the error weighting defined in accordance with the posture estimation method. [Example]

[0082] Next, a fourth embodiment of the object posture estimation device will be described with reference to Figures 11 and 12. In this embodiment, correction is repeated to minimize the sum of the errors between the accuracy information of the posture of the target object obtained by the first process and the accuracy information of the posture of another object serving as a base point obtained by the second process.

[0083] Fig. 11 is a block diagram of an object pose estimation device according to a fourth embodiment. Fig. 12 is a flowchart showing an object pose estimation method. In Fig. 11, components with the same reference numerals as in Fig. 7 have the same functions as those in Fig. 7, and their explanations will be omitted.

[0084] The attitude error accumulator 1101 calculates the accumulated value of the errors calculated by the target attitude estimation accuracy calculator 701 and the base attitude estimation accuracy calculator 702, i.e., ΔA 2 +ΔB 2 Ask for.

[0085] Attitude correction section 1102 corrects the attitude error accumulated value calculated by attitude error accumulator 1101 in a direction that reduces the accumulated value so as to satisfy the movement restrictions input from movement restriction information generator 106. The correction result is fed back to attitude estimation calculation section 501-4, and furthermore, based on the corrected attitude, the likelihood calculated by target attitude estimation accuracy calculation section 701 and base point attitude estimation accuracy calculation section 702 is recalculated, and attitude error accumulator 1101 recalculates the attitude estimation error accumulated value. In this way, correction is recursively repeated so as to minimize the sum of errors in the attitude accuracy information.

[0086] The posture estimation calculation unit 501 to target posture estimation accuracy calculation unit 701, base posture estimation accuracy calculation unit 702 to posture error accumulation unit 1101 to posture correction unit 1102 to posture estimation calculation unit 501 are repeated to correct the posture estimation results under conditions that minimize the posture error accumulation value while satisfying the operation restrictions input from the operation restriction information generation unit 106.

[0087] Fig. 12 shows a flowchart illustrating a method for estimating the pose of an object. Fig. 12 shows the operation of the pose estimation device shown in the block diagram of Fig. 11. In Fig. 12, processes denoted with the same reference numerals as in Fig. 10 are the same processes as in Fig. 10, and therefore descriptions thereof will be omitted.

[0088] If it is determined in processing step S606 that the movement constraints are violated, the posture correction unit 1102 executes posture processing correction step S1201. The posture processing correction step S1201 is for correcting the target object element posture error ΔA while satisfying the movement constraints extracted in processing step S204. 2 and base point object pose error ΔB 2 Attitude correction is performed to reduce the sum of

[0089] Step S1202 is a target object element attitude accuracy extraction processing step, in which the target attitude estimation accuracy calculation unit 701 calculates the target object element attitude accuracy ΔA- 2 Recalculate the above.

[0090] Step S1203 is a base element attitude accuracy extraction processing step, in which the base element attitude estimation accuracy calculation unit 702 calculates the target object element attitude accuracy ΔB- 2 Recalculate the above.

[0091] Step S1204 is a processing step for determining whether the attitude error is minimum, and the attitude error integrator 1101 calculates the target object element attitude error ΔA 2 and base point object pose error ΔB 2 is minimum. If it is determined that the posture error is not minimum, steps S1201 to S1203 are repeated. If it is determined that the posture error is minimum, the process ends. Therefore, correction can be performed to minimize the posture error while satisfying the operation restrictions.

[0092] As described above, according to this embodiment, the target object element attitude error ΔA 2 and base point object pose error ΔB 2 Since the posture determination correction process can be performed to minimize the sum of the above, it is possible to realize a correction process for improving the posture determination accuracy.

[0093] In addition, in the description of this embodiment, the explanation has been given assuming that there is one object serving as the base point for the motion restriction. However, if there are multiple base objects, for example, in FIG. 2, the slide mechanism of the group of Slide 1 (302), Motor (303), and Y_Robot (304) relative to Body (301) is not fixed (i.e., three or more objects form a mechanical relationship), the posture error Δ of each point of base object 1 (Body), base object 2 (group of Slide 1, Motor, and Y_Robot), and posture estimation target object (Slide 2) can be calculated. n This makes it possible to easily realize a correction process that can simultaneously minimize the sum of the above and satisfy the operation restrictions on the base point.

[0094] In addition, even when using various reported posture estimation methods, such as using AI to compare feature points on a 2D image of the target object and a 3D model of the target object to perform feature point comparison and posture estimation, a similar effect can be achieved by making corrections so that the sum of the target object element posture error and base object posture error, defined in accordance with the posture estimation method, is minimized. [Example]

[0095] Next, a fifth embodiment of the object posture estimation device will be described with reference to Figures 13 and 14. In this embodiment, the posture estimation calculation unit includes a first posture estimation calculation unit that calculates the posture of the object from feature points and object shape information, and a second posture estimation calculation unit that calculates the posture of the object from feature points, the object shape information, and object movement restriction information, and the processing efficiency is improved by switching between the first posture estimation calculation unit and the second posture estimation calculation unit.

[0096] The second orientation estimation calculation unit performs processing to narrow down comparison points between the measurement feature points and the shape information of the object in accordance with the movement restriction information, or processing to narrow down the existence range of the estimated orientation in accordance with the movement restriction information.

[0097] Fig. 13 is a block diagram of an object pose estimation device according to a fifth embodiment. Fig. 14 is a flowchart showing an object pose estimation method. In Fig. 13, components with the same reference numerals as in Fig. 11 are the same as those in Fig. 11, and their explanation will be omitted.

[0098] The first posture estimation calculation unit 1301 estimates the posture of object A, the posture of which is the posture estimation target, from a set of feature point information input from the feature point position information measurement unit 101 and posture information corresponding to feature point coordinates that have a high degree of agreement with CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103.

[0099] The second posture estimation calculation unit 1302 estimates the posture of the object from the feature point information input from the feature point position information measurement unit 101, the CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103, and the movement restriction information of the object whose posture is to be estimated, extracted by the movement restriction information generation unit 106.

[0100] First attitude estimation calculation section 1301 and second attitude estimation calculation section 1302 may have the function of attitude estimation accuracy calculation section 502, in which case error minimum determination steps S1405 and S604 become possible.

[0101] The use of the movement restriction information in the posture estimation process by the second posture estimation calculation unit 1302 will be described with reference to Figure 2. When estimating the posture of Slide 2 (305) in the movement mechanism of Figure 2, if Y_Robot (304) is facing the arrow direction (306), for example, the x-axis direction, Slide 2 (305) is restricted in its movement such that it cannot move in the y-axis or z-axis direction, and cannot rotate around the x-axis, y-axis, or z-axis. By utilizing this movement restriction, the posture of Slide 2 (305) can be determined by comparing, for example, only one of the measured feature points of Slide 2 (305) with the CAD model-based feature point coordinates and a set of posture information.

[0102] Furthermore, even without reducing the number of feature points to be compared, the posture can be estimated by comparing only postures in which only the x-axis coordinate has changed among the sets of feature point coordinates and posture information.

[0103] When movement restriction information generation unit 106 extracts movement restriction information, calculation processing switching unit 1303 selects first posture estimation calculation unit 1301 to perform posture estimation, and when movement restriction information generation unit 106 cannot generate movement restriction information, calculation processing switching unit 1303 selects second posture estimation calculation unit 1302 to perform posture estimation. Therefore, when movement restriction information is available, high-speed posture estimation processing can be performed using the movement restriction information.

[0104] Fig. 14 is a flowchart showing the operation of the posture estimation device shown in the block diagram of Fig. 13. In Fig. 14, the processes denoted by the same reference numerals as in Fig. 6 are the same processes as in Fig. 6, and the description thereof will be omitted.

[0105] Step S1400 is a processing step for determining whether or not there is motion restriction information for the object, and if it is determined that there is motion restriction information, the motion definition information extraction unit 105 performs a mechanism motion definition extraction process in step S203, and the motion restriction information generation unit 106 generates motion restrictions between CAD elements in step S204. The determination of whether or not there is motion restriction information can be made by the motion definition information extraction unit 105, a higher-level device (not shown), or by a user.

[0106] Step S1401 is a comparison feature point restriction or corresponding posture restriction processing step, in which processing is performed to limit the number of measured feature points and CAD model feature points to be compared, or to limit the posture information range within the set of feature point coordinates and posture information, based on the motion restriction information extracted in S204. This processing can be part of the functions of the second posture estimation calculation unit 1302.

[0107] Step S1402 is a CAD feature point comparison step in which second posture estimation calculation unit 1302 compares the feature points of the object measured in step S202 with the feature points of a CAD model corresponding to the object. If the number of feature points to be compared was limited in step S1401, step S1402 performs feature point comparison in accordance with the limit. If the processing of limiting the posture information range of the set of feature point coordinates and posture information was performed in step S1401, feature point comparison is performed within the limited posture information range.

[0108] Step S1403 is a CAD feature point corresponding posture calculation process in which the second posture estimation calculation unit 1302 determines posture data corresponding to the CAD model feature points. If the posture range is limited in S1401, the feature point corresponding posture calculation process is performed within the limited posture range.

[0109] In step S1404, the error between the measured feature points and the feature points of the selected CAD model is calculated. In step S1405, it is determined whether the calculated error is minimum. If the error is not determined to be minimum, the posture determination is corrected in step S1406, and steps S1402 to S1406 are repeated until the error is determined to be minimum.

[0110] As a result, the orientation that minimizes the error in the feature points can be quickly found by limiting it to the conditions specified in step S1401.

[0111] If it is determined in step S1400 that there is no movement restriction information, the processes in steps S601 to S605 are the same as those in steps S601 to S605 in FIG. 12, and posture estimation is performed without using movement restrictions.

[0112] As described above, according to this embodiment, when motion restriction information is available, it is possible to speed up the posture estimation process by reducing the number of feature points to be compared or by limiting the range of postures that can be taken by posture data corresponding to the CAD model feature points.

[0113] In this embodiment, the case where the feature point is limited to one has been described, but the number of feature points that can be limited is determined according to the range of possible motion restricted by the motion restriction. Also, even if the limit is not set to the maximum possible limit, the effect of speeding up can be obtained according to the limit. [Example]

[0114] Next, a sixth embodiment of the object posture estimation device will be described with reference to FIGS.

[0115] Fig. 15 is a block diagram of an object pose estimation device according to a sixth embodiment. Fig. 16 is a flowchart showing an object pose estimation method. In Fig. 15, components assigned the same reference numerals as in Fig. 11 are the same as those in Fig. 11, and their explanation will be omitted.

[0116] The process switching determination unit 1501 selects the process to be executed based on the amount of error between the movement restriction information and the posture estimation value input from the movement restriction difference calculation unit 1502 described later, and the accumulated value of the error input from the posture error accumulation unit 1101.

[0117] The processing switching determination unit 1501 selects the posture correction unit 1503 when the magnitude of the error information between the operation restriction information and the posture estimation value is equal to or less than a predetermined value α, selects the mechanism operation individual variation derivation unit 1505 when the magnitude of the error information is equal to or less than a predetermined value β and greater than α, and selects the failure information processing unit 1504 when the magnitude of the error information is greater than the predetermined value β.

[0118] Furthermore, the condition for the processing switching determination unit 1501 to select the mechanism operation individual variation derivation unit 1505 is set to be when the amount of error between the operation restriction information input from the operation restriction difference calculation unit 1502 and the posture estimation value is less than or equal to a predetermined value β and greater than α. However, by setting the amount of error between the operation restriction information input from the operation restriction difference calculation unit 1502 and the posture estimation value to be less than or equal to a predetermined value β and greater than α, and the accumulated value of the error input from the posture error accumulation unit 1101 to be less than or equal to a predetermined value, the processing switching determination unit 1501 can be performed when the accuracy of the determination condition for the amount of error between the operation restriction information and the posture estimation value is high, thereby increasing the accuracy of the switching determination.

[0119] Furthermore, the condition for the processing switching determination unit 1501 to select the failure information processing unit 1504 is set to be when the amount of error between the movement restriction information input from the movement restriction difference calculation unit 1502 and the posture estimation value is greater than a predetermined value β. However, by setting the amount of error between the movement restriction information input from the movement restriction difference calculation unit 1502 and the posture estimation value to be greater than the predetermined value β and the accumulated value of the error input from the posture error accumulation unit 1101 to be equal to or less than a predetermined value, the processing switching determination unit 1501 can be performed when the accuracy of the determination condition for the amount of error between the movement restriction information and the posture estimation value is high, thereby making it possible to increase the accuracy of the switching determination.

[0120] The movement restriction difference calculation unit 1502 generates information indicating the difference between the movement restriction information generated from the movement restriction information value input from the movement restriction information generation unit 106 and the estimated posture input from the posture estimation calculation unit 501, i.e., how much it deviates from the movement restriction information.

[0121] When selected by the processing switching determination unit 1501, the posture correction unit 1503 corrects the posture generated by the posture estimation calculation unit 501 based on the movement restriction information generated from the movement restriction information value input from the movement restriction information generation unit 106, and outputs the correction result from the output terminal 108.

[0122] When selected by the process switching determination unit 1501, the failure information processing unit 1504 performs information processing and the like to display a message indicating that a failure has occurred in the mechanism defined by the operation definition information extracted by the operation definition information extraction unit 105 (for example, that Y_Robot

[0304] and Slide2

[0305] in FIG. 3 are in a position that is significantly out of alignment with the slide mechanism, and that the slide mechanism appears to be damaged).

[0123] When selected by the process switching determination unit 1501, the mechanism operation individual variation derivation unit 1505 outputs the difference information input from the operation restriction difference calculation unit 1502 from the output terminal 1507 as the variation for each individual from the defined mechanism (for example, the variation of the individual Y_Robot

[0304] and Slide2

[0305] in Figure 3 measured the deviation amount from the X axis in the sliding direction of the sliding mechanism).

[0124] Fig. 16 is a flowchart showing the operation of the posture estimation device shown in the block diagram of Fig. 15. In Fig. 16, the processes denoted by the same reference numerals as in Fig. 12 are the same processes as in Fig. 12, and the description thereof will be omitted.

[0125] If it is determined in processing step S606 that a motion restriction violation has occurred, motion restriction condition versus posture error determination step S1601 is executed. Step S1601 of determining a motion restriction condition versus posture error indicates that the process switching determination unit 1501 in Fig. 15 switches processing in accordance with the amount of error between the motion restriction information input from the motion restriction difference calculation unit 1502 and the posture estimation value.

[0126] If it is determined in processing step S1601 that the amount of error between the movement restriction information and the posture estimation value is equal to or less than a predetermined value α, posture correction processing S1602 is selected. In posture correction processing S1602, posture correction unit 1503 in Fig. 15 corrects the posture generated by posture estimation calculation unit 501 based on movement restriction information generated from the movement restriction information value input from 106.

[0127] If it is determined in processing step S1601 that the amount of error between the motion constraint information and the posture estimated value is equal to or less than a predetermined value β and greater than α, then mechanism operation individual variance derivation step S1603 is selected. In mechanism operation individual variance derivation step S1603, mechanism operation individual variance derivation unit 1505 in Fig. 15 outputs the difference information from motion constraint difference calculation unit 1502 as the variance for each individual from the defined mechanism.

[0128] If it is determined in processing step S1601 that the error between the motion restriction information and the posture estimation value is greater than a predetermined value β, then mechanism failure determination processing S1604 is selected. In mechanism failure determination processing 1604, the failure information processing unit 1504 in Fig. 15 performs information processing to display a message indicating that a failure has occurred in the mechanism defined by the motion definition information extracted by the motion definition information extraction unit 105 (for example, that Y_Robot

[0304] and Slide2

[0305] in Fig. 2 are in a posture that is significantly out of alignment with the slide mechanism, and that the slide mechanism is likely to be damaged). If it is determined in S606 that there is no violation of the motion restriction, or after any of S1602, S1603, or S1604 has ended, then processing ends.

[0129] As described above, according to this embodiment, by switching the magnitude of the error amount between the operation restriction information and the posture estimation value as a processing switching condition, it is possible to not only correct the posture estimation, but also to determine that the operation mechanism has broken down and is no longer operating within the operation restrictions, and to measure the individual variation in the operation of the operation mechanism.By adding the integrated value of the posture estimation error to the processing switching condition in addition to the error amount between the operation restriction information and the posture estimation value, it is possible to improve the reliability of the condition determination.

[0130] Furthermore, when the target object for which posture estimation is to be performed is part of the robot or an object firmly grasped by the robot, a closed loop is configured so that the posture of the robot conforms to the control information of the robot, and therefore the control information can be used as movement restriction information. Treating the control information as movement restriction information is also valid in the first to sixth cases.

[0131] In addition, by using the function of deriving individual variations in mechanism operation in this embodiment and treating the control information as operation restrictions, it is also possible to monitor control abnormalities. Also, in some cases, robot manufacturers generate their own postures and trajectories to ensure smooth movement except for postures at key points in order to make the robot move smoothly, and in such cases, this function is also effective for users to check the deviations from the specifications. [Example]

[0132] A seventh embodiment of an object posture estimation device will be described with reference to Fig. 17. In this embodiment, when motion definition information is not provided and stored in a CAD database in advance, motion definition information is obtained based on data stored in the CAD database by a mechanism estimation process that includes at least one of a process for determining the number of connecting surfaces between objects constituting the mechanism and a process for determining whether or not there is a shared axis.

[0133] 17 is a flowchart showing a method for generating the action definition information from the arrangement conditions of each part of the action mechanism when the action definition information of the action mechanism that forms the basis of the action restriction information is not provided and stored in advance in the CAD database. This function can be added as an optional function of the action definition information extraction unit 105.

[0134] In Figure 17, S1701 indicates the start of processing, S1702 is a step for reading CAD information of an operation mechanism component object, S1703 is a step for determining whether operation mechanism definition information is present, S1704 is a step for reading operation mechanism definition information, S1705 is a step for generating connection relationships of operation mechanism components, S1706 is a step for analyzing the operation mechanism definition, and S1707 indicates the end of processing.

[0135] First, in step S1702, the CAD information reading step for the object CAD information of the operating mechanism components is performed, and the CAD information of the operating mechanism object including the target object for which posture estimation is to be performed (for example, the shape information and positional relationship information of each object 301 to 302 of the slide mechanism shown in Figure 2) is read from the CAD database. Next, it is determined whether the read CAD information of the operating mechanism object has been added with the operation definition information of the operating mechanism (for example, in Figure 2, information such as Y_Robot and Slide2 being slide mechanisms, and Y_Robot and Slide1 being fixed).

[0136] If it is determined in S1703 that the information has been added, the added operation mechanism definition information is read in operation mechanism definition information reading step S1704. If it is determined in S1703 that the information has not been added, the operation mechanism component connection relationship generation process S1705 generates the connection relationships (presence or absence of a shared center circle, number of shared axes, number of connection faces) of each object that makes up the operation mechanism.

[0137] The operation mechanism definition analysis process S1706 analyzes the operation mechanism definition according to the connection relationship generated in S1705. The analysis of the operation mechanism definition analyzes that if there is one shared axis with a shared circle at the center between the two objects in question, there is a rotation mechanism; if there is no connecting surface between the two objects, there is no contact; if there are one to three connecting surfaces between the two objects, there is a fixed mechanism; and if there are two connecting surfaces between the two objects, there is a sliding mechanism.

[0138] In step S1706, the mechanical relationships between two objects of each element of the operating mechanism are analyzed using graph theory to perform an analysis of the entire mechanism, and the analysis result is obtained as to whether the mechanism is a slide crank mechanism or a mechanism having two slide mechanisms as shown in Figure 2. When the above processing is completed, the processing ends (1707).

[0139] As described above, according to this embodiment, even if the operation definition information of the operating mechanism is not provided and stored in advance in the CAD database, the operation definition information of the operating mechanism can be automatically generated from the placement conditions of each part of the operating mechanism, so that it is possible to realize embodiments 1 to 6 even if the CAD database does not include definition information of the operating mechanism. [Example]

[0140] An eighth embodiment of an object pose estimation device will be described using FIGS. 18A and 18B. In this embodiment, shape information of an object that serves as the base point of the movable range is generated from a 3D point cloud measured from a captured image of the object. As described with reference to FIGS. 7, 11, and 15, using the pose of another object that serves as the base point of the movable range can improve the accuracy of estimating the pose of the target object. However, if the other object that serves as the base point is a floor, a workbench, or the like, there may be no data for it in the CAD data. In such cases, in the examples of FIGS. 7, 11, and 15, measured 3D point cloud information can be used as the shape information of the other object that serves as the base point.

[0141] 18A and 18B are explanatory diagrams of a simple movement mechanism consisting of a first part 1801 and a second part 1802. The first part 1801 is the object for which posture evaluation is performed, and the second part 1802 is the object that serves as the base point for imposing movement restrictions on the first part 1801. It is assumed that the second part 1802 is located at (x1, y1, z1, Rx1, Ry1, Rz1) = (0, 0, 0, 0, 0).

[0142] In this operating mechanism, as shown in Figure 18A, when the first part 1801 is within a range where it does not come into contact with the second part 1802, the orientation of the first part (x1, y1, z1, Rx1, Ry1, Rz1) can move freely without any movement restrictions on all coordinate axes and rotation axes, as shown by arrow 1803, and can take any orientation.

[0143] Also, when the first part 1801 is in contact with the second part 1802 as shown in Figure 18B, the orientation (x1, y1, z1, Rx1, Ry1, Rz1) can assume an orientation with movement restrictions in which only the x-axis, y-axis, and Rz rotation axis can move, as shown by arrow 1804.

[0144] 18A and 18B, an intermediate restriction between 1803 and 1804 is imposed. That is, in this movement mechanism, when z=0, the posture of the first part 1801 is restricted to movement only on the x-axis, y-axis, and Rz rotation axis, and posture estimation can be performed under this restriction.

[0145] Note that even if the shape information of the first part 1801 and the second part 1802 of the operating mechanism described in this embodiment is not stored in a CAD database, it is possible to measure the parts on site to obtain point cloud information, and extract operation restrictions by combining the extracted information of the operation definition information using Example 7 based on the point cloud information. However, since the CAD data of the first part is also information necessary for basic posture estimation, it is preferable to limit the use of the measured point cloud information to the second part as the measurement result.

[0146] As described above, according to this embodiment, if the movement of the object for which posture estimation is to be performed is restricted, it is possible to obtain the effects shown in embodiments 1 to 7 even with a very simple movement mechanism, which brings about the effect of widening the scope of application of the high accuracy and high speed posture estimation according to the present application. Furthermore, since a point cloud obtained by measuring the object that serves as the base point for the movement restriction can be used instead, the effects shown in embodiments 1 to 7 can be obtained even if CAD data of the object that serves as the base point for the movement restriction has not been prepared in advance.

[0147] According to the above embodiment, efficient attitude control can be realized, which reduces energy consumption, reduces carbon emissions, prevents global warming, and contributes to the realization of a sustainable society. [Explanation of symbols]

[0148] 101... feature point position information measurement unit, 102... CAD database, 103... CAD feature point position information and posture information extraction unit, 105... action definition information extraction unit, 106... action restriction information generation unit, 107... posture estimation unit

Claims

1. An apparatus for estimating the pose of an object, comprising: a feature point measurement unit that acquires feature points of the object; a CAD information extraction unit that acquires shape information of the object from CAD data in a CAD database; an operation definition information extraction unit that extracts operation definition information of a mechanism including the object from the CAD data; a motion restriction information generating unit that generates, from the motion definition information, motion restriction information that defines a range of motion of the object determined from another object on the mechanism as a starting point; An object posture estimation device, comprising: a posture estimation unit that estimates the posture of the object from the feature points, the shape information, and the movement restriction information.

2. The posture estimation unit a posture estimation calculation unit that estimates a posture of the object based on feature points of the object and feature points of a CAD model based on the shape information; a posture correction unit having a function of determining whether the posture calculated by the posture estimation calculation unit violates the range of motion defined by the movement restriction information, The object posture estimation device according to claim 1 .

3. The posture correction unit If the orientation violates the range of motion, the orientation of the object is corrected so that the difference between the orientation and the range of motion becomes smaller. The object posture estimation device according to claim 2.

4. a posture estimation accuracy calculation unit that calculates a deviation amount between the feature points of the object and the feature points of the CAD model base and feeds the deviation amount back to the posture estimation calculation unit; The object posture estimation device according to claim 2.

5. The feature point measurement unit acquiring feature points of the other object; The CAD information extraction unit acquiring shape information of the other object from the CAD data; a target pose estimation accuracy calculation unit that calculates a first deviation amount, which is a deviation amount between a feature point of the object and a feature point of the object based on a CAD model; a base point orientation estimation accuracy calculation unit that calculates a second deviation amount that is a deviation amount between the feature points of the other object and the feature points of the other object based on a CAD model, The posture correction unit correcting the attitude of the object using the first deviation amount and the second deviation amount; The object posture estimation device according to claim 2.

6. a correction weight calculation unit that calculates weights for correcting the attitude of the object and correcting the attitude of the other object, which are performed by the attitude correction unit, using the first deviation amount and the second deviation amount; The posture correction unit correcting the posture of the object using the calculation result of the correction weight calculation unit; The object posture estimation device according to claim 5.

7. The target attitude estimation accuracy calculation unit calculating a first attitude estimation error using the first deviation amount; The base point attitude estimation accuracy calculation unit calculating a second attitude estimation error using the second deviation amount; an attitude error accumulator that calculates an accumulated value of the first attitude estimation error and the second attitude estimation error; the posture correction unit recursively repeats the correction so as to minimize the integrated value. The object posture estimation device according to claim 5.

8. The posture estimation unit a first pose estimation calculation unit that estimates a pose of the object based on feature points of the object and feature points of a CAD model based on the shape information; a second orientation estimation calculation unit that estimates an orientation of the object based on feature points of the object, feature points of a CAD model based on the shape information, and the movement restriction information; a calculation processing switching unit that switches between the first attitude estimation calculation unit and the second attitude estimation calculation unit; The object posture estimation device according to claim 1 .

9. the second posture estimation calculation unit performs a process of narrowing down the number of feature points of the object and the number of feature points of the CAD model base in accordance with the movement restriction information. The object posture estimation device according to claim 8.

10. the second posture estimation calculation unit performs a process of narrowing down an existence range of a posture to be estimated in accordance with the movement restriction information. The object posture estimation device according to claim 8.

11. The posture estimation unit a posture estimation calculation unit that estimates a posture of the object based on feature points of the object and feature points of a CAD model based on the shape information; a movement restriction difference calculation unit that calculates an error between the posture estimated by the posture estimation calculation unit and a range of motion defined by the movement restriction information; a posture correction unit that corrects the posture estimated by the posture estimation calculation unit so as to reduce the error when the error calculated by the movement restriction difference calculation unit is equal to or smaller than a predetermined value α; The object posture estimation device according to claim 1 .

12. the attitude estimation unit includes a failure information processing unit, If the error calculated by the operational limit difference calculation unit exceeds a predetermined value β (where β>α), information notifying that a failure has occurred in the mechanism is output. The object pose estimation device according to claim 11.

13. the posture estimation unit includes a variation derivation unit, When the error calculated by the operational limit difference calculation unit is equal to or less than a predetermined value β (where β>α) and exceeds the value α, the error calculated by the operational limit difference calculation unit is output. The object pose estimation device according to claim 11.

14. The operation definition information extraction unit detecting at least one of the presence or absence of a shared center circle, the number of shared axes, and the number of connection surfaces of each object constituting a mechanism including the object as a connection relationship, and extracting the operation definition information based on the connection relationship; The object posture estimation device according to claim 1 .

15. The posture estimation unit generating shape information of other objects on the mechanism; The object posture estimation device according to claim 1 .

16. 1. A method for estimating a pose of an object, comprising: a feature point measuring step of acquiring feature points of the object; a CAD information extraction step of acquiring shape information of the object from CAD data; an operation definition information extraction step of extracting operation definition information of a mechanism including the object from the CAD data; a motion restriction information generating step of generating, from the motion definition information, motion restriction information that defines a range of motion of the object determined from another object on the mechanism as a starting point; A method for estimating a posture of an object, comprising: executing a posture estimation step of estimating a posture of the object from the feature points, the shape information, and the movement restriction information.

17. The posture estimation step includes: Estimating the posture of the object from the feature points and the shape information of the object; Estimating the pose of the other object; correcting the attitude of the object using the movement restriction information, the attitude of the object, and the attitude of the other object; The method for estimating the pose of an object according to claim 16.

18. The posture estimation step includes: narrowing down at least one of the comparison points between the feature points and the shape information and the existence range of the posture of the object to be estimated in accordance with the movement restriction information; The method for estimating an object pose according to claim 17.

Images