Object orientation estimation device and object orientation estimation method

Abstract

The present invention improves estimation accuracy and calculation load in estimating the orientation of an object.　This object orientation estimation device estimates the orientation of an object, and is characterized by comprising: a feature point measurement unit for acquiring a feature point of the object; a CAD information extraction unit for acquiring, from CAD data in a CAD database, shape information of the object; an operation definition information extraction unit for extracting, from the CAD data, operation definition information of a mechanism including the object; an operation restriction information generation unit for generating, from the operation definition information, operation restriction information defining, as a starting point, a movable range determined from another object on the mechanism of the object; and an orientation estimation unit for estimating the orientation of the object from the feature point, the shape information, and the operation restriction information.

Description

Object posture estimation device and object posture estimation method

[0001] The present invention relates to a technique for estimating the pose of an object.

[0002] Background of the present invention is, for example, Patent Document 1. Patent Document 1 discloses a method for generating a heat map that identifies the probability of the positions of feature points of an object from a two-dimensional image and comparing the heat map with a three-dimensional virtual CAD model.

[0003] Japanese Patent Application Laid-Open No. 2021-163503

[0004] Patent Document 1 makes it possible to estimate the pose of an object by comparing the measured feature points of the object with CAD data related to the shape of the object. However, further improvements are required in terms of the estimation accuracy and the calculation load in estimating the pose of an object.

[0005] A preferred aspect of the present invention is an apparatus for estimating the posture of an object, comprising: a feature point measurement unit that acquires feature points of the object; a CAD information extraction unit that acquires shape information of the object from CAD data in a CAD database; an action definition information extraction unit that extracts action definition information of a mechanism including the object from the CAD data; a action restriction information generation unit that generates, from the action definition information, action restriction information that defines a range of motion of the object defined with another object on the mechanism as a starting point; and an attitude estimation unit that estimates the posture of the object from the feature points, the shape information, and the action restriction information.

[0006] Another preferred aspect of the present invention is a method for estimating the posture of an object, comprising: a feature point measurement step of acquiring feature points of the object; a CAD information extraction step of acquiring shape information of the object from CAD data; an action definition information extraction step of extracting, from the CAD data, action definition information of a mechanism including the object; a action restriction information generation step of generating, from the action definition information, action restriction information that defines a range of motion of the object, the range of motion being determined with another object on the mechanism as a starting point; and an attitude estimation step of estimating the posture of the object from the feature points, the shape information, and the action restriction information.

[0007] According to the present invention, it is possible to further improve the estimation accuracy and the calculation load in estimating the posture of an object.

[0008] Block diagram of an object pose estimation device. Perspective view explaining an object pose estimation method. Explanatory diagram of an object pose estimation method. Flowchart of an object pose estimation method. Block diagram of an object pose estimation device. Flowchart of an object pose estimation method. Block diagram of an object pose estimation device. Explanatory diagram of an object pose estimation device. Table showing feature points and coordinates of the object pose estimation device. Explanatory diagram of an object pose estimation method. Flowchart of an object pose estimation method. Block diagram of an object pose estimation device. Flowchart of an object pose estimation method. Block diagram of an object pose estimation device. Flowchart of an object pose estimation method. Flowchart of an object pose estimation method. Flowchart of a mechanism operation definition extraction method. Explanatory diagram of an object pose estimation method. Explanatory diagram of an object pose estimation method.

[0009] Examples of the present invention will be described below. However, the present invention should not be construed as being limited to the description of the following embodiments. Those skilled in the art will readily understand that the specific configuration can be modified within the scope of the concept and spirit of the present invention.

[0010] In the configurations of the embodiments described below, the same parts or parts having similar functions are denoted by the same reference numerals in different drawings, and redundant explanations may be omitted.

[0011] When there are multiple elements having the same or similar functions, they may be described using the same reference numeral with different subscripts. However, when there is no need to distinguish between multiple elements, the subscripts may be omitted.

[0012] The terms "first," "second," "third," and the like used in this specification are used to identify components and do not necessarily limit the number, order, or content of the components. Furthermore, numbers used to identify components are used in different contexts, and a number used in one context does not necessarily indicate the same configuration in another context. Furthermore, this does not prevent a component identified by a certain number from also serving the function of a component identified by another number.

[0013] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings etc. may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings etc.

[0014] All publications, patents, and patent applications cited herein are incorporated by reference in their entirety.

[0015] As used herein, elements referred to in the singular are intended to include the plural unless the context clearly indicates otherwise.

[0016] A first embodiment of an object pose estimation device will be described using Figures 1, 2, 3, and 4. According to this embodiment, when estimating the pose of an object, in addition to measured feature points of the object and CAD data related to the shape of the object, conditions for the object's range of motion derived based on the definition of a movement mechanism including the object are used, thereby making it possible to achieve high-precision and high-speed pose estimation.

[0017] FIG. 1 is a block diagram of an object pose estimation device 1 according to this embodiment. In FIG. 1, a feature point position information measurement unit 101 measures each feature point of an object for which pose estimation is to be performed from a captured image of the object. A CAD database 102 is a database that pre-stores shape information and the like of CAD models related to each object, including the object for which pose estimation is to be performed. A CAD feature point position information and pose information extraction unit 103 extracts, from the CAD database 102, a set of combinations of feature point coordinates in CAD model space of the object for which pose estimation is to be performed and pose information corresponding to the feature point position coordinates. Feature points can be obtained not only from captured images but also from measurement data obtained by measuring the target object using techniques such as TOF (Time of Flight) or LiDAR (Light Detection and Ranging).

[0018] Furthermore, the action definition information extraction unit 105 extracts action definitions of a mechanism including an object for which posture estimation is to be performed from the CAD database 102. The action restriction information generation unit 106 generates action restriction information indicating a range in which the object for which posture estimation is to be performed can move, from the action definitions of the mechanism including the object for which posture estimation is to be performed extracted by the action definition information extraction unit 105, starting from other objects that constitute the mechanism of the object for which posture estimation is to be performed.

[0019] The posture estimation unit 107 estimates the posture of the object from the feature point information input from the feature point position information measurement unit 101, a set of posture information corresponding to feature point coordinates that highly match the CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103, and the movement restriction information of the object whose posture is to be estimated, extracted by the movement restriction information generation unit 106. An output terminal 108 outputs the posture estimation result.

[0020] The object posture estimation device 1 can be configured as a general information processing device, such as a server. The server generally includes a processing device, a storage device, an input device, and an output device. The CAD database 102 is configured by storing CAD data in a storage device, such as a magnetic disk device or semiconductor memory.

[0021] The feature point position information measurement unit 101, the CAD feature point position information and posture information extraction unit 103, the action definition information extraction unit 105, the action restriction information generation unit 106, and the posture estimation unit 107 are realized by, for example, a processing device executing software stored in a storage device. Functions equivalent to those configured by software can also be realized by hardware such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0022] The output terminal 108 is an example of an output device, and an image of an object captured by a camera or the like from outside the device is input to the feature point position information measurement unit 101 via an input device.

[0023] The above configuration may be configured by a single server, or any part of the input device, output device, processing device, and storage device may be configured by another server connected via a network.

[0024] 2 shows an example of a motion mechanism including an object for which posture estimation is to be performed, and is an explanatory diagram of a method for estimating the posture of an object. In FIG. 2, the mechanism is composed of a Body (301), Slide 1 (302) that performs a sliding motion relative to the Body (301), a Motor (303) fixed to Slide 1 (302), a Y_Robot (304) also fixed to Slide 1 (302), and Slide 2 (305) that performs a sliding motion relative to the Y_Robot (304). Body, Slide, and Robot are symbols that uniquely indicate each part of the object.

[0025] The CAD database 102 stores, in any data format, shape information (e.g., coordinates indicating the outer shape of the body) and positional relationship information (e.g., the positional relationship between the body and slide 1) of each part of a target object whose posture is to be estimated, as CAD data, and, if necessary, posture information corresponding to feature point position coordinates. The CAD data may also include information defining the operation of each part of the object (e.g., slide 2 slides in the longitudinal direction of the Y_Robot (direction of arrow 306), while Y_Robot and slide 1 are fixed).

[0026] The posture of an object is expressed by the (x coordinate, y coordinate, z coordinate, x rotation angle, y rotation angle, z rotation angle) of one or more points on the object. Therefore, the postures of Body (301), Slide1 (302), Motor (303), Y_Robot (304), and Slide2 (305) are expressed as follows:

[0027] Body; (Body_x, Body _y , Body _z, Body _Rx, Body _Ry, Body _Rz) … (Formula 1) Slide1; (Slide1_x, Slide1 _y , Slide1 _z, Slide1 _Rx, Slide1 _Ry, Slide1 _Rz) … (Formula 2) Motor; (Motor_x, Motor _y , Motor _z, Motor _Rx, Motor _Ry, Motor _Rz) …(Formula 3) Y_Robot;(Y_Robot_x, Y_Robot _y , Y_Robot _z, Y_Robot _Rx, Y_Robot _Ry, Y_Robot _Rz) …(Formula 4) Slide2;(Slide2_x, Slide2 _y , Slide2 _z, Slide2 _Rx, Slide2 _Ry, Slide2 _Rz) …(Formula 5) For example, when estimating the posture of Slide2 (305), if Y_Robot (304) is facing in the direction of the arrow (306), for example, the x-axis direction, and taking into consideration the action definition information that "Slide2 slides in the longitudinal direction of Y_Robot," the action restriction information indicates that there are action restrictions such as "Slide2 (305) cannot move in the y-axis or z-axis directions, and cannot rotate on the x-axis, y-axis, or z-axis."

[0028] This is shown in a more simplified manner in Figure 3. Figure 3 is an explanatory diagram illustrating a method for estimating the posture of an object, showing the movement restrictions imposed on the object for which posture estimation is to be performed. It shows that the movement restriction for Slide2 (305) in the direction of arrow 306 (x-axis direction) restricts it to only be able to assume a posture of positions 402, 403, or 404, or an interpolated or extrapolated posture. This movement restriction information can be shown, for example, as the range of possible coordinates of a predetermined point on Slide2 (305).

[0029] 2 includes a first sliding mechanism for the assembly of Slide 1 (302), Motor (303), and Y_Robot (304) relative to Body (301), and a second sliding mechanism for Slide 2 (305) relative to the assembly of Slide 1 (302), Motor (303), and Y_Robot (304). The first sliding mechanism is movable in the longitudinal direction of Body (301), and the second sliding mechanism is movable in the longitudinal direction of Y_Robot (304) (arrow direction 306). However, to simplify the explanation, unless otherwise specified, it will be assumed that the first sliding mechanism for the assembly of Slide 1 (302), Motor (303), and Y_Robot (304) relative to Body (301) is fixed and does not move.

[0030] The movement restrictions are incorporated into the posture representation of each element shown above. If the movement restrictions for Slide2 (305) starting from Y_Robot (304) are expressed as Δt, the posture of Slide2 is as shown in Equation 6 below. Slide2; (Y_Robot_x+Δx(t), Y_Robot _y+Δy(t), Y_Robot _z+Δz(t), Y_Robot _Rx+ΔRx(t), Y_Robot _Ry+ΔRy(t), Y_Robot _Rz+ΔRz(t)) ... (Equation 6) If we plug the fact that the arrow direction is the x-axis into the above equation, movement and rotation in directions other than the x-axis are restricted, resulting in the following. Slide2; (Y_Robot_x+Δx(t), Y_Robot _y , Y_Robot _z, Y_Robot _Rx, Y_Robot _Ry, Y_Robot _Rz) ... (Equation 7) If the posture of Slide2 is expressed using only the difference in Y_Robot, then by taking the difference between Equation 6 and Equation 7, it can be expressed as Slide2; (Δx(t),0,0,0,0) ... (Equation 8)

[0031] The flowchart showing the object pose estimation method shown in Fig. 4 shows the operation of the pose estimation device 1 shown in the block diagram of Fig. 1. In Fig. 4, step S201 indicates the start of the process.

[0032] Step S202 indicates a processing step in which the feature point position information measurement unit 101 measures each feature point of the object for which pose estimation is to be performed. Each feature point of the object can be measured by capturing an image of the object with a camera or the like, inputting the captured image from an input device to the feature point position information measurement unit 101, and extracting the feature points using a predetermined algorithm, but here a known technique will be used. Each feature point of the object can be represented using any coordinate system, but here it will be represented by (x coordinate, y coordinate, z coordinate).

[0033] Step S203 is a processing step following processing step S202 in which the action definition information extraction unit 105 extracts an action definition of a mechanism including an object for which posture estimation is to be performed from the CAD database 102. The action definition indicates how the objects constituting the mechanism move relative to each other.

[0034] Step S204 is a processing step following processing step S203 in which the movement restriction information generation unit 106 generates movement restriction information indicating the range of possible movements of the object for which posture estimation is to be performed, from the movement definition extracted by the movement definition information extraction unit 105, starting from other objects that constitute the mechanism of the object for which posture estimation is to be performed.

[0035] Step S205 is a processing step following processing step S204 in which the posture estimation unit 107 estimates the posture of the object from the feature point information input from the feature point position information measurement unit 101, a set of posture information corresponding to feature point coordinates that have a high degree of coincidence with the CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103, and the movement restriction information of the target object for posture estimation extracted by the movement restriction information generation unit 106.

[0036] Step S206 indicates that the process ends after the orientation of the object is estimated in step S205. Note that step S202 is not limited to the example of FIG. 4 and may be executed at any timing before step S205.

[0037] As described above, according to this embodiment, it is possible to use not only the feature points of the target object and a set of feature point information and orientation information of the 3D model of the target object, but also movement restrictions on the target object that are based on an object other than the target object that constitutes a movement mechanism together with the target object as a starting point. Therefore, it is possible to provide an object orientation estimation device and an object orientation estimation method that improve the accuracy of orientation determination by determining whether the movement restrictions are satisfied after estimating the orientation from the set of feature point information and orientation information of the 3D model of the target object, or that speed up the orientation estimation process by limiting the orientations that the target object can take.

[0038] Furthermore, by expressing the object's posture using differential data from the base point of the movement restriction as shown in (Equation 8), it is possible to obtain the effect of compressing and storing the data amount of posture information of the target object. By combining the posture data stored in this way with the movement restriction information, it is possible to easily restore (Equation 5) which indicates the normal posture.

[0039] Next, a second embodiment of the object pose estimation device will be described with reference to Figures 5 and 6. Figure 5 is a block diagram of an object pose estimation device 1-2 according to the second embodiment. Figure 6 is a flowchart showing a method for estimating the object pose.

[0040] 5, the feature point position information measurement unit 101, CAD database 102, CAD feature point position information and posture information extraction unit 103, action definition information extraction unit 105, and action restriction information generation unit 106 are the same components as those shown in FIG.

[0041] The posture estimation calculation unit 501 estimates the posture of an object from a set of feature point information input from the feature point position information measurement unit 101 and posture information corresponding to feature point coordinates that have a high degree of agreement with CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103.

[0042] The posture estimation accuracy calculation unit 502 calculates the degree of deviation between the feature points corresponding to the posture information of the CAD model and the feature points obtained by measurement, and outputs the calculated deviation amount as feedback to the posture estimation calculation unit 501 so that the posture estimation calculation unit 501 can perform posture estimation that minimizes the deviation amount.

[0043] The posture correction unit 503 determines whether the posture estimation result obtained by the posture estimation calculation unit 501 satisfies the movement restrictions, and if not, calculates the difference between the range of motion of the conditions indicated in the movement restriction information and the estimated posture, i.e., posture accuracy information. If it is determined that the posture estimation result does not satisfy the movement restriction information, it performs correction based on the difference between the posture estimation result and the movement restriction information so that the difference becomes zero or falls within a predetermined allowable error range. The corrected posture estimation result is output from the output terminal 108. The posture estimation calculation unit 501, the posture estimation accuracy calculation unit 502, and the posture correction unit 503 are an example of the configuration of the posture estimation unit 107 in FIG. 1 .

[0044] Flowchart showing a method for estimating the pose of an object FIG. 6 shows the operation of the pose estimation device 1-2 shown in the block diagram of FIG.

[0045] 6, step S201 indicates the start of processing, and step S202 indicates a processing step in which each feature point of an object whose pose is to be estimated is measured by the feature point position information measurement unit 101. Processing step S202 is executed by the feature point position information measurement unit 101 in FIG.

[0046] Step S601 is a CAD feature point comparison step in which the orientation estimation calculation unit 501 compares the feature points of the object measured in step S202 with the feature points of a CAD model corresponding to the object.

[0047] Step S602 is a CAD feature point corresponding posture calculation step in which the posture estimation calculation unit 501 obtains posture data corresponding to the CAD model feature points used in step S601.

[0048] Step S603 is processing in which the posture estimation accuracy calculation unit 502 calculates the degree of deviation between the measured feature points and the feature points of the CAD model.

[0049] In step S604, the posture estimation accuracy calculation unit 502 determines whether the error value calculated in step S603 is minimum (or equal to or less than an allowable value). If it is determined that the error value is not minimum (or equal to or less than an allowable value), the posture estimation calculation unit 501 reselects a set of posture and feature points for the posture corrected on the CAD model (S605).

[0050] In this way, an orientation that minimizes the error between the measured feature points and the positions of the feature points on the CAD model is obtained. The processes of steps S601 to S605 are executed by the orientation estimation calculation unit 501 and the orientation estimation accuracy calculation unit 502 in FIG.

[0051] Step S203 indicates a processing step in which the action definition information extraction unit 105 extracts, from the CAD database 102, the action definition of the mechanism including the object whose posture is to be estimated.

[0052] Step S204 is a processing step following processing step S203, in which the motion restriction information generation unit 106 generates motion restriction information indicating the range of possible motion of the object whose posture is to be estimated, from the motion definition of the mechanism including the object whose posture is to be estimated extracted by the motion definition information extraction unit 105, starting from other objects that make up the mechanism of the target object whose posture is to be estimated.

[0053] In step S606, the posture correction unit 503 determines whether or not the posture of the object obtained in the processes of steps S601 to S605 violates the object's movement restriction information obtained in processing step S204. If there is no violation of the movement restriction, the process ends (S206).

[0054] If there is a violation of the movement restrictions, the posture correction unit 503 executes a process (S607) to correct the posture estimation so that the movement restrictions are satisfied or the difference with respect to the movement restrictions is less than a predetermined allowable error, and then terminates the process (S206) after satisfying the movement restrictions.

[0055] If necessary, the accuracy of the attitude corrected by the attitude correction unit 503 is again determined by the attitude estimation accuracy calculation unit 502 .

[0056] As described above, according to this embodiment, after determining the posture of an object using a set of feature points of the target object and feature point information and posture information of a 3D model of the target object, it is possible to check whether the movement restrictions of the target object originating from an object other than the target object that constitutes the movement mechanism together with the target object are satisfied, and if not, to make corrections, thereby specifically demonstrating that it is possible to improve the accuracy of posture determination.

[0057] According to this embodiment, in the process of estimating posture using a set of measured feature points of the target object and feature point information and posture information of a 3D model of the target object, a set of feature point information of the 3D model and corresponding posture information with the highest degree of match is selected through sequential comparison. As another example, even when combined with various reported posture estimation methods, such as using AI to compare feature points and estimate posture using feature points on a 2D image of the target object and a 3D model of the target object without having a sufficiently large number of sets of feature point information and posture information of the 3D model for posture estimation, the effect of improving accuracy can be obtained.

[0058] Fig. 7 is a block diagram of an object posture estimation device 1-3 according to a third embodiment. In Fig. 7, the feature point position information measurement unit 101, CAD database 102, CAD feature point position information and posture information extraction unit 103, action definition information extraction unit 105, and action restriction information generation unit 106 are the same components as those shown in Fig. 1. However, in this embodiment, the feature point position information measurement unit 101 and the CAD feature point position information and posture information extraction unit 103 perform the same processing for both object A, the posture estimation target, and object B, which serves as the base point for the movement restriction of the posture estimation target object.

[0059] The posture estimation calculation unit 501-3 estimates the posture of both object A, the object for which posture estimation is to be performed, and object B, which, together with the object for which posture estimation is to be performed, constitutes a movement mechanism and serves as the base point for restricting the movement of the object for which posture estimation is to be performed, from a set of posture information corresponding to feature point coordinates that highly match the feature point information input from the feature point position information measurement unit 101 and the CAD feature point position information and CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103.

[0060] The target attitude estimation accuracy calculation unit 701 is a means for extracting the accuracy of the result of the attitude estimation process performed by the attitude estimation calculation unit 501 of the object A. Here, the accuracy of the attitude estimation process result will be described.

[0061] Figures 8A and 8B are diagrams illustrating the reliability of pose estimation results. Figure 8A shows feature points R1 to R7 of the object to be measured and points V1 to V7 corresponding to R1 to R7 on the CAD model of the object. The coordinates of each point are shown in Figure 8B.

[0062] If the posture is ideally determined, the coordinates of each point will be perfectly consistent, and the error will be zero. In reality, errors occur due to measurement errors at each feature point. The larger the error, the greater the dispersion of the measured values, and the lower the accuracy of the posture estimation result obtained from this. The magnitude of the posture estimation error can be expressed as the sum of the squares of the errors Δ at each point, as shown in (Equation 9), and the posture estimation accuracy is the reciprocal Δ -2 Δ ２ = (R1x-V1x) 2 +(R1y-V1y) 2 +(R1z-V1z) 2 +(R2x-V2x) 2 +(R2y-V2y) 2 + (R2z-V2z) 2 + (R3x-V3x) 2 +(R3y-V3y) 2 +(R3z-V3z) 2 +(R4x-V4x) 2 +(R4y-V4y) 2 + (R4z-V4z) 2 + ・・・・+(R7x-V7x) 2 +(R7y-V7y) 2 + (R7z-V7z) 2 ...(Equation 9) The base attitude estimation accuracy calculation unit 702 is a means for extracting the likelihood of the result of the attitude estimation process performed by the attitude estimation calculation unit 501 of the object B, and similarly to the target attitude estimation accuracy calculation unit 701, calculates Δ -2 Therefore, the likelihood of object A is Δ -2 is ΔA -2 , the likelihood of object B Δ -2 is ΔB -2 It is expressed as follows.

[0063] Correction weighting calculation unit 703 performs correction weighting when correcting the posture estimation results, based on the posture estimation accuracy of objects A and B calculated by target posture estimation accuracy calculation unit 701 and base posture estimation accuracy calculation unit 702. Posture correction unit 704 corrects the posture estimation of object A in accordance with the correction weighting calculated by correction weighting calculation unit 703. Next, the posture estimation correction process of posture correction unit 704 will be further described.

[0064] 9 is a diagram illustrating a method for correcting posture estimation described in this embodiment. The posture estimation is corrected so as to satisfy the movement restriction information. Simply put, the posture estimation result of object A may be corrected so that the posture estimation result of object B, which is the base point of the movement mechanism, satisfies the movement restriction information. However, since the posture estimation result of object B cannot be obtained with 100% accuracy, the correction is performed by correcting both object A and object B based on the accuracy.

[0065] If object B corresponds to Y_Robot (304) in Figure 2 and object A hits Slide2 (305) in Figure 2, the center point of object A is only allowed to move on the object A center movement axis due to movement restrictions. Therefore, if the accuracy of the orientation estimation of object B is 100%, it is sufficient to correct only the orientation estimation of object A and make the deviation on the diagram (the shortest distance between a point and a line) zero, but the accuracy of the orientation estimation of object B is not 100% either. Therefore, object B also needs to be corrected according to its orientation accuracy.

[0066] That is, the attitude accuracy of object B ΔB -2 When normalized to 100, the normalization process results in ΔA -2 is expressed as 50, correction is performed according to the likelihood by applying a weight of 100 / (100+50)=0.66 to object A and 50 / (100+50)=0.33 to object B.

[0067] Fig. 10 is a flowchart showing the operation of the posture estimation device shown in the block diagram of Fig. 7. Processes denoted by the same reference numerals as in Fig. 6 are the same as those in Fig. 6, and therefore description thereof will be omitted. However, as mentioned above, in this embodiment, up to step S801, the same process is performed for both object A, the posture estimation target, and object B, which is the base point for restricting the movement of the posture estimation target object.

[0068] In step S604, the target attitude estimation accuracy calculation unit 701 and the base attitude estimation accuracy calculation unit 702 determine whether the error value calculated in step S603 is the minimum (or an acceptable value), and if it is determined that the error value is not the minimum (or an acceptable value), the attitude estimation calculation unit 501-3 may reselect a set of attitude and feature points for the attitude corrected on the CAD model (S605).

[0069] Step S801 is a target object element attitude accuracy extraction step, in which the target attitude estimation accuracy calculation unit 701 in FIG. 7 estimates the attitude of object A, that is, calculates the attitude estimation accuracy of the target object for which attitude estimation is to be performed.

[0070] 7 calculates the accuracy of estimating the posture of object B, i.e., the posture estimation accuracy of the object that serves as the base point of the motion restriction specification. The target object A and the base object B can be specified automatically from the motion definition or by using an appropriate GUI for the operator.

[0071] If it is determined in processing step S606 that a motion restriction violation has occurred, the correction weight calculation unit 703 performs accuracy weighting processing step S803. Processing step S803 is a processing step for calculating correction weights in accordance with the pose estimation accuracy of object A calculated in processing step S801 and the pose estimation accuracy of object B calculated in processing step S802.

[0072] In processing step S804, the posture estimation is corrected in accordance with the weighting calculated in S803. Processing step S803 is performed by the correction weighting calculation unit 703 in Fig. 7, and processing step S804 is performed by the posture correction unit 704 in Fig. 7. If the motion restrictions are satisfied in processing step S804, the processing ends (S206).

[0073] As described above, according to this embodiment, the posture of object A can be corrected by weighting the results of posture estimation of object A, which is the object for which posture estimation is performed, and object B, which is the base point for restricting the movement of object A, based on the likelihood of the results. This makes it possible to realize a correction process for improving the accuracy of posture determination.

[0074] Furthermore, according to this embodiment, correction is performed by weighting, so that the correction process can be performed at high speed without repeating the process.

[0075] Furthermore, although the present embodiment shows a case where there is one object that serves as a base point for the movement restriction, it is possible to deal with a case where there are multiple objects by repeatedly carrying out the operation restriction for each base point.

[0076] In the explanation of this embodiment, correction of posture estimation at a specific moment has been described, but in cases where object A moves continuously, weighting based on likelihood comparison at specific timings such as the beginning or end of the movement may be used to correct the posture at each timing, or correction may be made using the average value of the likelihood comparison.

[0077] Furthermore, even when various posture estimation methods that have been reported are used, such as using AI to compare feature points on a 2D image of the target object and a 3D model of the target object to perform feature point comparison and posture estimation, a similar effect can be obtained by performing correction according to the error weighting defined in accordance with the posture estimation method, although the definition is different from that of Equation 9.

[0078] Next, a fourth embodiment of the object pose estimation device will be described with reference to Figures 11 and 12. In this embodiment, correction is repeated to minimize the sum of the errors between the accuracy information of the pose of the target object obtained by the first process and the accuracy information of the pose of another object serving as a base point obtained by the second process.

[0079] Fig. 11 is a block diagram of an object pose estimation device according to a fourth embodiment. Fig. 12 is a flowchart showing an object pose estimation method. In Fig. 11, components with the same reference numerals as in Fig. 7 have the same functions as those in Fig. 7, and their explanations will be omitted.

[0080] The attitude error accumulator 1101 calculates the accumulated value of the errors calculated by the target attitude estimation accuracy calculator 701 and the base attitude estimation accuracy calculator 702, i.e., ΔA 2 +ΔB 2 Ask for.

[0081] The attitude correction unit 1102 corrects the attitude error integrated value calculated by the attitude error accumulator 1101 in a direction to reduce it so as to satisfy the movement restrictions input from the movement restriction information generator 106. The correction result is fed back to the attitude estimation calculation unit 501-4, and further, the likelihood calculated by the target attitude estimation accuracy calculation unit 701 and the base point attitude estimation accuracy calculation unit 702 is recalculated based on the corrected attitude, and the attitude error accumulator 1101 recalculates the attitude estimation error integrated value. In this way, correction is repeated recursively so as to minimize the sum of errors in the attitude accuracy information.

[0082] The posture estimation calculation unit 501 to target posture estimation accuracy calculation unit 701, base posture estimation accuracy calculation unit 702 to posture error accumulation unit 1101 to posture correction unit 1102 to posture estimation calculation unit 501 are repeated to correct the posture estimation results under conditions that minimize the posture error accumulation value while satisfying the operation restrictions input from the operation restriction information generation unit 106.

[0083] Fig. 12 shows a flowchart illustrating a method for estimating the pose of an object. Fig. 12 shows the operation of the pose estimation device shown in the block diagram of Fig. 11. In Fig. 12, processes denoted by the same reference numerals as in Fig. 10 are the same processes as in Fig. 10, and therefore descriptions thereof will be omitted.

[0084] If it is determined in processing step S606 that there is a violation of the movement constraints, an attitude processing correction step S1201 is performed by the attitude correction unit 1102. The attitude processing correction step S1201 is a step for correcting the target object element attitude error ΔA 2 and base point object pose error ΔB 2 Attitude correction is performed to reduce the sum of

[0085] Step S1202 is a target object element attitude accuracy extraction processing step, in which the target attitude estimation accuracy calculation unit 701 calculates the target object element attitude accuracy ΔA- 2 Recalculate the above.

[0086] Step S1203 is a base element attitude accuracy extraction processing step, in which the base element attitude estimation accuracy calculation unit 702 calculates the target object element attitude accuracy ΔB- 2 Recalculate the above.

[0087] Step S1204 is a processing step for determining whether the attitude error is minimum, and the attitude error integrator 1101 calculates the target object element attitude error ΔA 2 and base point object pose error ΔB 2is the minimum. If it is determined that the posture error is not the minimum, steps S1201 to S1203 are repeated. If it is determined that the posture error is the minimum, the process ends. Therefore, correction can be performed to minimize the posture error while satisfying the operation restrictions.

[0088] As described above, according to this embodiment, the target object element attitude error ΔA 2 and base point object pose error ΔB 2 Since the posture determination correction process can be performed to minimize the sum of the above, it is possible to realize a correction process for improving the posture determination accuracy.

[0089] In addition, in the description of this embodiment, the explanation has been given assuming that there is one object serving as the base point for the motion restriction. However, if there are multiple base objects, for example, in FIG. 2, the slide mechanism of the group of Slide 1 (302), Motor (303), and Y_Robot (304) relative to the Body (301) is not fixed (i.e., three or more objects form a mechanical relationship), the posture error Δ of each point of the base object 1 (Body), the base object 2 (group of Slide 1, Motor, and Y_Robot), and the posture estimation target object (Slide 2) can be calculated. n This makes it possible to easily realize a correction process that can simultaneously minimize the sum of the above and satisfy the operation restrictions on the base point.

[0090] Furthermore, even when various reported posture estimation methods are used, such as using AI to compare feature points on a 2D image of the target object and a 3D model of the target object to perform feature point comparison and posture estimation, a similar effect can be achieved by making corrections so that the sum of the target object element posture error and base object posture error defined in accordance with the posture estimation method is minimized.

[0091] Next, a fifth embodiment of an object posture estimation device will be described with reference to Figures 13 and 14. In this embodiment, the posture estimation calculation unit includes a first posture estimation calculation unit that calculates the posture of the object from feature points and object shape information, and a second posture estimation calculation unit that calculates the posture of the object from feature points, the object shape information, and object movement restriction information, and processing efficiency is improved by switching between the first posture estimation calculation unit and the second posture estimation calculation unit.

[0092] The second orientation estimation calculation unit performs processing to narrow down comparison points between the measurement feature points and the shape information of the object in accordance with the movement restriction information, or processing to narrow down the existence range of the estimated orientation in accordance with the movement restriction information.

[0093] Fig. 13 is a block diagram of an object pose estimation device according to a fifth embodiment. Fig. 14 is a flowchart showing an object pose estimation method. In Fig. 13, components with the same reference numerals as in Fig. 11 are the same as those in Fig. 11, and their explanation will be omitted.

[0094] The first posture estimation calculation unit 1301 estimates the posture of object A, the posture of which is the posture estimation target, from a set of feature point information input from the feature point position information measurement unit 101 and posture information corresponding to feature point coordinates that have a high degree of agreement with CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103.

[0095] The second posture estimation calculation unit 1302 estimates the posture of the object from the feature point information input from the feature point position information measurement unit 101, the CAD model-based measured feature points input from the CAD feature point position information and posture information extraction unit 103, and the movement restriction information of the object whose posture is to be estimated, extracted by the movement restriction information generation unit 106.

[0096] The first attitude estimation calculation unit 1301 and the second attitude estimation calculation unit 1302 may have the function of the attitude estimation accuracy calculation unit 502, in which case the error minimum determination steps S1405 and S604 become possible.

[0097] The use of the movement restriction information in the posture estimation process by the second posture estimation calculation unit 1302 will be described with reference to Figure 2. When estimating the posture of Slide 2 (305) in the movement mechanism of Figure 2, if Y_Robot (304) is facing the arrow direction (306), for example, the x-axis direction, Slide 2 (305) is restricted in its movement such that it cannot move in the y-axis or z-axis direction, and cannot rotate around the x-axis, y-axis, or z-axis. By utilizing this movement restriction, the posture of Slide 2 (305) can be determined by comparing, for example, only one of the measured feature points of Slide 2 (305) with the CAD model-based feature point coordinates and a set of posture information.

[0098] Furthermore, even without reducing the number of feature points to be compared, the posture can be estimated by comparing only postures in which only the x-axis coordinate has changed among the sets of feature point coordinates and posture information.

[0099] When movement restriction information generation unit 106 extracts movement restriction information, calculation processing switching unit 1303 selects first posture estimation calculation unit 1301 to perform posture estimation, and when movement restriction information generation unit 106 cannot generate movement restriction information, calculation processing switching unit 1303 selects second posture estimation calculation unit 1302 to perform posture estimation. Therefore, when movement restriction information is available, high-speed posture estimation processing can be performed using the movement restriction information.

[0100] Fig. 14 is a flowchart showing the operation of the posture estimation device shown in the block diagram of Fig. 13. In Fig. 14, the processes denoted by the same reference numerals as in Fig. 6 are the same processes as in Fig. 6, and the description thereof will be omitted.

[0101] Step S1400 is a processing step for determining whether or not there is motion restriction information for the object. If it is determined that there is motion restriction information, the motion definition information extraction unit 105 performs a mechanism motion definition extraction process in step S203, and the motion restriction information generation unit 106 generates motion restrictions between CAD elements in step S204. The determination of whether or not there is motion restriction information can be made by the motion definition information extraction unit 105, a higher-level device (not shown), or by a user.

[0102] Step S1401 is a comparison feature point restriction or corresponding posture restriction processing step, in which the number of measured feature points to be compared with the number of feature points of the CAD model is restricted based on the motion restriction information extracted in S204, or the posture information range is restricted within the set of feature point coordinates and posture information. This processing can be included as part of the functions of the second posture estimation calculation unit 1302.

[0103] Step S1402 is a CAD feature point comparison step in which second posture estimation calculation unit 1302 compares the feature points of the object measured in step S202 with the feature points of a CAD model corresponding to the object. If the number of feature points to be compared was limited in step S1401, step S1402 performs feature point comparison in accordance with the limit. If the processing of limiting the posture information range of the set of feature point coordinates and posture information was performed in step S1401, feature point comparison is performed within the limited posture information range.

[0104] Step S1403 is a CAD feature point corresponding posture calculation process in which the second posture estimation calculation unit 1302 obtains posture data corresponding to the CAD model feature points. If the posture range is limited in S1401, the feature point corresponding posture calculation process is performed within the limited posture range.

[0105] In step S1404, the error between the measured feature points and the feature points of the selected CAD model is calculated. In step S1405, it is determined whether the calculated error is minimum. If it is not determined that the error is minimum, the posture determination is corrected in step S1406, and steps S1402 to S1406 are repeated until it is determined that the error is minimum.

[0106] As a result, the orientation that minimizes the error of the feature points can be quickly found by limiting it to the conditions specified in step S1401.

[0107] If it is determined in step S1400 that there is no movement restriction information, the processes in steps S601 to S605 are the same as those in steps S601 to S605 in FIG. 12, and posture estimation is performed without using movement restrictions.

[0108] As described above, according to this embodiment, when motion restriction information is available, it is possible to speed up the posture estimation process by reducing the number of feature points to be compared or by limiting the range of postures that can be taken by posture data corresponding to CAD model feature points.

[0109] In this embodiment, the case where the feature point is limited to one has been described, but the number of feature points that can be limited is determined according to the range of possible motion restricted by the motion restriction. Also, even if the limit is not set to the maximum possible limit, the effect of speeding up can be obtained according to the limit.

[0110] Next, a sixth embodiment of the object posture estimation device will be described with reference to FIGS.

[0111] Fig. 15 is a block diagram of an object pose estimation apparatus according to a sixth embodiment. Fig. 16 is a flowchart showing an object pose estimation method. In Fig. 15, components denoted by the same reference numerals as in Fig. 11 are the same as those in Fig. 11, and their explanation will be omitted.

[0112] The process switching determination unit 1501 selects the process to be executed based on the amount of error between the movement restriction information and the posture estimation value input from the movement restriction difference calculation unit 1502 described later, and the accumulated value of the error input from the posture error accumulation unit 1101.

[0113] The processing switching determination unit 1501 selects the posture correction unit 1503 when the magnitude of the error information between the operation restriction information and the posture estimation value is equal to or less than a predetermined value α, selects the mechanism operation individual variation derivation unit 1505 when the magnitude of the error information is equal to or less than a predetermined value β and greater than α, and selects the failure information processing unit 1504 when the magnitude of the error information is greater than the predetermined value β.

[0114] Furthermore, the condition for the processing switching determination unit 1501 to select the mechanism operation individual variation derivation unit 1505 is set to be when the amount of error between the operation restriction information input from the operation restriction difference calculation unit 1502 and the posture estimation value is less than or equal to a predetermined value β and greater than α. However, by setting the amount of error between the operation restriction information input from the operation restriction difference calculation unit 1502 and the posture estimation value to be less than or equal to a predetermined value β and greater than α, and the accumulated value of the error input from the posture error accumulation unit 1101 to be less than or equal to a predetermined value, the processing switching determination unit 1501 can be performed when the accuracy of the determination condition for the amount of error between the operation restriction information and the posture estimation value is high, thereby making it possible to increase the accuracy of the switching determination.

[0115] Furthermore, the condition for the processing switching determination unit 1501 to select the failure information processing unit 1504 is when the amount of error between the movement restriction information input from the movement restriction difference calculation unit 1502 and the posture estimation value is greater than a predetermined value β. However, by setting the amount of error between the movement restriction information input from the movement restriction difference calculation unit 1502 and the posture estimation value to be greater than the predetermined value β and the accumulated value of the error input from the posture error accumulation unit 1101 to be less than or equal to a predetermined value, the processing switching determination unit 1501 can be performed when the accuracy of the determination condition for the amount of error between the movement restriction information and the posture estimation value is high, thereby making it possible to increase the accuracy of the switching determination.

[0116] The movement restriction difference calculation unit 1502 generates information indicating the difference between the movement restriction information generated from the movement restriction information value input from the movement restriction information generation unit 106 and the estimated posture input from the posture estimation calculation unit 501, i.e., how much it deviates from the movement restriction information.

[0117] When selected by the processing switching determination unit 1501, the posture correction unit 1503 corrects the posture generated by the posture estimation calculation unit 501 based on the movement restriction information generated from the movement restriction information value input from the movement restriction information generation unit 106, and outputs the correction result from the output terminal 108.

[0118] When selected by the process switching determination unit 1501, the failure information processing unit 1504 performs information processing, etc., to display a message indicating that a failure has occurred in the mechanism defined by the operation definition information extracted by the operation definition information extraction unit 105 (for example, that Y_Robot

[304] and Slide2

[305] in Figure 3 are in a position that is significantly out of alignment with the slide mechanism, and that the slide mechanism is thought to be damaged).

[0119] When selected by the process switching determination unit 1501, the mechanism operation individual variance derivation unit 1505 outputs the difference information input from the operation restriction difference calculation unit 1502 from the output terminal 1507 as the variance for each individual from the defined mechanism (for example, the variance of the individual when Y_Robot

[304] and Slide2

[305] in Figure 3 measure the amount of deviation from the X axis in the sliding direction of the sliding mechanism).

[0120] Fig. 16 is a flowchart showing the operation of the posture estimation device shown in the block diagram of Fig. 15. In Fig. 16, the processes denoted by the same reference numerals as in Fig. 12 are the same processes as in Fig. 12, and the description thereof will be omitted.

[0121] If it is determined in processing step S606 that a motion restriction violation has occurred, a motion restriction condition versus posture error determination step S1601 is performed. In the motion restriction condition versus posture error determination step S1601, the process switching determination unit 1501 in Fig. 15 switches processing in accordance with the amount of error between the motion restriction information input from the motion restriction difference calculation unit 1502 and the posture estimation value.

[0122] If it is determined in processing step S1601 that the amount of error between the movement restriction information and the posture estimation value is equal to or less than a predetermined value α, posture correction processing S1602 is selected. In posture correction processing S1602, the posture correction unit 1503 in Fig. 15 corrects the posture generated by the posture estimation calculation unit 501 based on the movement restriction information generated from the movement restriction information value input from 106.

[0123] If it is determined in processing step S1601 that the amount of error between the motion restriction information and the posture estimation value is equal to or less than a predetermined value β and greater than α, then a mechanism operation individual variance derivation step S1603 is selected. In mechanism operation individual variance derivation step S1603, the mechanism operation individual variance derivation unit 1505 in Fig. 15 outputs the difference information from the motion restriction difference calculation unit 1502 as the variance for each individual from the defined mechanism.

[0124] If it is determined in processing step S1601 that the error between the operation restriction information and the posture estimation value is greater than a predetermined value β, then mechanism failure determination processing S1604 is selected. In mechanism failure determination processing 1604, the failure information processing unit 1504 in FIG. 15 performs information processing, etc., to display a message indicating that a failure has occurred in the mechanism defined by the operation definition information extracted by the operation definition information extraction unit 105 (for example, that Y_Robot

[304] and Slide2

[305] in FIG. 2 are in a posture that is significantly out of alignment with the slide mechanism, and that the slide mechanism is likely to be damaged). If it is determined in S606 that there is no violation of the operation restriction, or after any of S1602, S1603, and S1604 has ended, processing ends.

[0125] As described above, according to this embodiment, by switching the magnitude of the error amount between the operation restriction information and the posture estimation value as a processing switching condition, it is possible to not only correct the posture estimation, but also to determine that the operation mechanism has broken down and is no longer operating within the operation restrictions, and to measure the individual variation in the operation of the operation mechanism.By adding the integrated value of the posture estimation error to the processing switching condition in addition to the error amount between the operation restriction information and the posture estimation value, it is possible to improve the reliability of the condition determination.

[0126] Furthermore, when the target object for which posture estimation is to be performed is part of the robot or an object firmly grasped by the robot, a closed loop is configured so that the posture of the robot conforms to the control information of the robot, and therefore it is possible to use the control information as movement restriction information. Treating the control information as movement restriction information is also valid in the first to sixth cases.

[0127] In addition, by using the function of deriving individual variations in mechanism operation in this embodiment and treating the control information as operation restrictions, it is also possible to monitor control abnormalities. Also, in some cases, robot manufacturers generate their own postures and trajectories to ensure smooth movement except for postures at key points in order to make the robot move smoothly, and in such cases, this function is also effective for users to check the deviations from the specifications.

[0128] A seventh embodiment of an object posture estimation device will be described with reference to Fig. 17. In this embodiment, when motion definition information is not provided and stored in advance in a CAD database, motion definition information is obtained by a mechanism estimation process based on data stored in the CAD database, the process including at least one of a process for determining the number of connecting surfaces between objects constituting the mechanism and a process for determining whether or not there is a shared axis.

[0129] 17 is a flowchart showing a method for generating action definition information from the placement conditions of each part of a movement mechanism when the action definition information of the movement mechanism that forms the basis of the action restriction information is not provided and stored in advance in the CAD database. This function can be added as an optional function of the action definition information extraction unit 105.

[0130] In FIG. 17, S1701 indicates the start of processing, S1702 is a step for reading CAD information of operation mechanism component objects, S1703 is a step for determining whether operation mechanism definition information is present, S1704 is a step for reading operation mechanism definition information, S1705 is a step for generating connection relationships of operation mechanism components, S1706 is a step for analyzing operation mechanism definitions, and S1707 indicates the end of processing.

[0131] First, in step S1702 for reading CAD information of movement mechanism components objects, the movement mechanism object CAD information (for example, shape information and positional relationship information of each object 301 to 302 of the slide mechanism shown in FIG. 2) including the target object for which posture estimation is to be performed is read from the CAD database, and then it is determined whether movement definition information of the movement mechanism (for example, information that in FIG. 2, Y_Robot and Slide2 are slide mechanisms, and Y_Robot and Slide1 are fixed) has been added to the read movement mechanism object CAD information.

[0132] If it is determined in S1703 that the definition information has been added, the added definition information for the movement mechanism is read in step S1704. If it is determined in S1703 that the definition information has not been added, the connection relationships of the objects that make up the movement mechanism (presence or absence of a shared center circle, the number of shared axes, and the number of connection faces) are generated in operation mechanism component connection relationship generation process S1705.

[0133] The operation mechanism definition analysis process S1706 analyzes the operation mechanism definition according to the connection relationship generated in S1705. The analysis of the operation mechanism definition is performed by analyzing that if there is one shared axis having a shared circle center between the two objects in question, there is a rotation mechanism; if there is no connecting surface between the two objects, there is no contact; if there are one to three connecting surfaces between the two objects, there is a fixed mechanism; and if there are two connecting surfaces between the two objects, there is a sliding mechanism.

[0134] In step S1706, the mechanical relationships between two objects of each element of the operating mechanism are analyzed using graph theory to perform an analysis of the entire mechanism, and an analysis result is obtained, such as whether it is a slide crank mechanism or a mechanism having two slide mechanisms as shown in Figure 2. When the above processing is completed, the processing ends (1707).

[0135] As described above, according to this embodiment, even if the operation definition information of the operation mechanism is not provided and stored in advance in the CAD database, the operation definition information of the operation mechanism can be automatically generated from the placement conditions of each part of the operation mechanism, so that it is possible to realize embodiments 1 to 6 even if the CAD database does not include definition information of the operation mechanism.

[0136] An eighth embodiment of an object pose estimation device will be described using Figures 18A and 18B. In this embodiment, shape information of an object that serves as the base point of the movable range is generated from a 3D point cloud measured from a captured image of the object. As described with reference to Figures 7, 11, and 15, using the pose of another object that serves as the base point of the movable range can improve the accuracy of estimating the pose of the target object. However, if the other object that serves as the base point is a floor, a workbench, or the like, there may be no data available in the CAD data. In such cases, in the examples of Figures 7, 11, and 15, measured 3D point cloud information can be used as shape information of the other object that serves as the base point.

[0137] 18A and 18B are explanatory diagrams of a simple movement mechanism consisting of a first part 1801 and a second part 1802. The first part 1801 is a target object for posture evaluation, and the second part 1802 is a base object that imposes movement restrictions on the first part 1801. It is assumed that the second part 1802 is located at (x1, y1, z1, Rx1, Ry1, Rz1) = (0, 0, 0, 0, 0).

[0138] In this operating mechanism, when the first part 1801 is within a range where it does not come into contact with the second part 1802, as shown in Figure 18A, the posture of the first part (x1, y1, z1, Rx1, Ry1, Rz1) can move freely without any movement restrictions about all coordinate axes and rotation axes, as shown by arrow 1803, and can take any posture.

[0139] Furthermore, when the first part 1801 is in contact with the second part 1802 as shown in FIG. 18B, the orientation (x1, y1, z1, Rx1, Ry1, Rz1) can assume an orientation with movement restrictions in which only the x-axis, y-axis, and Rz rotation axis can move, as shown by arrow 1804.

[0140] 18A and 18B, an intermediate restriction between 1803 and 1804 is imposed. That is, in this movement mechanism, when z=0, the posture of the first part 1801 is restricted to movement only along the x-axis, y-axis, and Rz rotation axis, and posture estimation can be performed under this restriction.

[0141] Note that even if the shape information of the first part 1801 and the second part 1802 of the operating mechanism described in this embodiment is not stored in the CAD database, it is possible to measure the parts on site to obtain point cloud information, and to extract operation restrictions by combining the extracted information of the operation definition information using Example 7 based on the point cloud information. However, because the CAD data of the first part is also information necessary for basic posture estimation, it is preferable to limit the use of the measured point cloud information to the second part as the measurement result.

[0142] As described above, according to this embodiment, if the movement of the object for which pose estimation is to be performed is restricted, it is possible to obtain the effects shown in Examples 1 to 7 even with a very simple movement mechanism, thereby widening the scope of application of the high accuracy and high speed of pose estimation according to the present application. Furthermore, since a point cloud obtained by measuring an object that serves as the base point for movement restriction can be used instead, the effects shown in Examples 1 to 7 can be obtained even if CAD data of the object that serves as the base point for movement restriction has not been prepared in advance.

[0143] According to the above embodiment, efficient attitude control can be realized, which reduces energy consumption, reduces carbon emissions, prevents global warming, and contributes to the realization of a sustainable society.

[0144] 101... feature point position information measurement unit, 102... CAD database, 103... CAD feature point position information and posture information extraction unit, 105... action definition information extraction unit, 106... action restriction information generation unit, 107... posture estimation unit

Claims

1. An apparatus for estimating the posture of an object, comprising: a feature point measurement unit that acquires feature points of the object; a CAD information extraction unit that acquires shape information of the object from CAD data in a CAD database; an action definition information extraction unit that extracts action definition information of a mechanism including the object from the CAD data; a action restriction information generation unit that generates, from the action definition information, action restriction information that defines a range of motion of the object, determined with another object on the mechanism as a starting point; and a posture estimation unit that estimates the posture of the object from the feature points, the shape information, and the action restriction information.

2. The object posture estimation device according to claim 1, wherein the posture estimation unit comprises: a posture estimation calculation unit that estimates the posture of the object based on feature points of the object and feature points of a CAD model obtained from the shape information; and a posture correction unit that includes a function for determining whether the posture determined by the posture estimation calculation unit violates the range of motion defined by the movement restriction information.

3. The object posture estimation device according to claim 2, wherein the posture correction unit corrects the posture of the object so as to reduce the difference between the posture and the movable range if the posture violates the movable range.

4. The object pose estimation device according to claim 2, further comprising a pose estimation accuracy calculation unit that calculates the amount of deviation between the feature points of the object and the feature points of the CAD model base, and feeds back the amount of deviation to the pose estimation calculation unit.

5. The object posture estimation device according to claim 2, wherein the feature point measurement unit acquires feature points of the other object, the CAD information extraction unit acquires shape information of the other object from the CAD data, and the device comprises: a target posture estimation accuracy calculation unit that calculates a first deviation amount that is the deviation amount between the feature points of the object and the feature points of the CAD model-based object; and a base posture estimation accuracy calculation unit that calculates a second deviation amount that is the deviation amount between the feature points of the other object and the feature points of the CAD model-based other object, and the posture correction unit corrects the posture of the object using the first deviation amount and the second deviation amount.

6. An object posture estimation device as described in claim 5, further comprising a correction weighting calculation unit that uses the first deviation amount and the second deviation amount to calculate weightings for the correction of the posture of the object and the correction of the posture of the other object performed by the posture correction unit, wherein the posture correction unit corrects the posture of the object using the calculation results of the correction weighting calculation unit.

7. The object attitude estimation device according to claim 5, further comprising: an attitude error accumulator that calculates a first attitude estimation error using the first deviation amount; an attitude correction unit that calculates a second attitude estimation error using the second deviation amount; and an attitude error accumulator that calculates an integrated value of the first attitude estimation error and the second attitude estimation error; and a posture correction unit that recursively repeats correction so that the integrated value is minimized.

8. The object posture estimation device according to claim 1, wherein the posture estimation unit comprises: a first posture estimation calculation unit that estimates the posture of the object based on feature points of the object and CAD model-based feature points obtained from the shape information; and a second posture estimation calculation unit that estimates the posture of the object based on feature points of the object, CAD model-based feature points obtained from the shape information, and the movement restriction information; and a calculation processing switching unit that switches between the first posture estimation calculation unit and the second posture estimation calculation unit.

9. The object posture estimation device according to claim 8, wherein the second posture estimation calculation unit performs processing to narrow down the number of feature points of the object and the number of feature points of the CAD model base in accordance with the movement restriction information.

10. The object posture estimation device according to claim 8, wherein the second posture estimation calculation unit performs processing to narrow down the range of the posture to be estimated in accordance with the movement restriction information.

11. The object posture estimation device according to claim 1, wherein the posture estimation unit comprises: a posture estimation calculation unit that estimates the posture of the object based on feature points of the object and feature points of a CAD model based on the shape information; a movement restriction difference calculation unit that calculates the error between the posture estimated by the posture estimation calculation unit and a range of motion defined by the movement restriction information; and a posture correction unit that corrects the posture estimated by the posture estimation calculation unit so as to reduce the error when the error calculated by the movement restriction difference calculation unit is equal to or less than a predetermined value α.

12. The object posture estimation device according to claim 11, wherein the posture estimation unit includes a failure information processing unit, and when the error calculated by the operational restriction difference calculation unit exceeds a predetermined value β (where β > α), outputs information notifying that a failure has occurred in the mechanism.

13. The object posture estimation device according to claim 11, wherein the posture estimation unit includes a variation derivation unit, and when the error calculated by the movement restriction difference calculation unit is equal to or less than a predetermined value β (where β > α) and exceeds the value α, the error calculated by the movement restriction difference calculation unit is output.

14. The object posture estimation device according to claim 1, wherein the action definition information extraction unit detects at least one of the presence or absence of a central shared circle, the number of shared axes, and the number of connecting surfaces of each object constituting a mechanism including the object as a connection relationship, and extracts the action definition information based on the connection relationship.

15. The object posture estimation device according to claim 1, wherein the posture estimation unit generates shape information of other objects on the mechanism.

16. A method for estimating the posture of an object, comprising: a feature point measurement step of acquiring feature points of the object; a CAD information extraction step of acquiring shape information of the object from CAD data; an action definition information extraction step of extracting action definition information of a mechanism including the object from the CAD data; an action restriction information generation step of generating, from the action definition information, action restriction information that defines a range of motion of the object defined with another object on the mechanism as a starting point; and an attitude estimation step of estimating the posture of the object from the feature points, the shape information, and the action restriction information.

17. The method for estimating the posture of an object according to claim 16, wherein the posture estimation step estimates the posture of the object from the feature points of the object and the shape information, estimates the posture of the other object, and corrects the posture of the object using the movement restriction information, the posture of the object, and the posture of the other object.

18. The method for estimating the posture of an object according to claim 17, wherein the posture estimation step narrows down at least one of the comparison points between the feature points and the shape information and the existence range of the posture of the object to be estimated in accordance with the movement restriction information.

Images