Target object pose determination method and device, electronic equipment and readable storage medium
By determining the conversion data between the camera and the pattern coordinate system and the world coordinate system, the real-time and high-precision demand for target position information in the medical field is solved, and efficient position determination is achieved.
Patent Information
- Application Number
- CN202310536612.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-12
AI Technical Summary
In the medical field, the real-time, high precision and high stability of the position, distance and posture information of the target object in the space is difficult to meet.
By determining the first conversion data between the camera coordinate system and the pattern coordinate system and the second conversion data between the world coordinate system and the camera coordinate system, the position data of the target object is determined based on the data and the spatial data of the pattern. The specific steps include selecting feature points, converting coordinate system data of feature points, and determining the position and attitude information of the target object through matrix operations.
Real-time, accurate and stable determination of the target position is achieved, and the real-time and efficiency of the operation process is improved.
Smart Images

Figure CN118941624A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method for determining a position and posture of a target object, a device for determining a position and posture of a target object, an electronic device, and a readable storage medium. Background Art
[0002] With the development of intelligent technology, machine vision plays an increasingly important role in modern life and production. Through machine vision technology, we can obtain the position and distance information of objects in space, and then make reasonable plans for the movement of mechanical moving parts (such as robot arms, robots, unmanned vehicles, aircraft, etc.). At present, machine vision has been widely used in various fields such as unmanned vehicles and robots.
[0003] However, in fields such as medicine, there are higher requirements for the position, distance and posture information of targets in space, such as real-time, high precision, high stability, etc. Summary of the invention
[0004] In order to solve at least one aspect of the above-mentioned problems and defects existing in the prior art, an embodiment of the present invention provides a method for determining the posture of a target object, a device for determining the posture of a target object, an electronic device and a readable storage medium to solve the problem of how to achieve real-time determination of the posture of the target object.
[0005] An object of the present invention is to provide a method for determining the position and posture of a target object.
[0006] Another object of the present invention is to provide a device for determining the position and posture of a target object.
[0007] Another object of the present invention is to provide an electronic device.
[0008] Another object of the present invention is to provide a readable storage medium.
[0009] According to one aspect of the present invention, a method for determining a position and posture of a target object is provided, comprising:
[0010] Determine first conversion data for mutual conversion between a camera coordinate system and a pattern coordinate system of a pattern, wherein the pattern is a pattern on a target object;
[0011] Determine second conversion data for mutual conversion between the world coordinate system and the camera coordinate system;
[0012] The position and posture data of the target object are determined based on the first conversion data, the second conversion data, and the spatial data of the pattern.
[0013] In some embodiments, determining first conversion data for converting between the camera coordinate system and the pattern coordinate system comprises:
[0014] Select feature points on the pattern;
[0015] The first conversion data is determined based on the spatial data of the feature point in the pattern coordinate system and the intrinsic parameter data of the camera.
[0016] In some embodiments, determining the first conversion data based on the spatial data of the feature point in the pattern coordinate system and the intrinsic parameter data of the camera includes:
[0017] Determine the spatial data of the feature point in the pattern coordinate system;
[0018] Determine the spatial data of the feature point image in the image captured by the camera in the image coordinate system;
[0019] Determine the spatial data of the feature point image in the pattern coordinate system based on the spatial data of the feature point in the pattern coordinate system and the spatial data of the feature point image in the image coordinate system;
[0020] Based on the camera's intrinsic parameter data, the spatial data of the feature point image in the image coordinate system is converted to the camera coordinate system;
[0021] The first conversion data is determined based on the spatial data of the feature point image in the pattern coordinate system and the spatial data of the feature point image in the camera coordinate system.
[0022] In some embodiments, determining the spatial data of a feature point image in an image captured by a camera in an image coordinate system includes:
[0023] Converting the collected image into a grayscale image, and performing binarization processing on the grayscale image to obtain a binarized image;
[0024] Extracting the contour of the binarized image from the binarized image;
[0025] Feature points are determined in the extracted contour based on the position data of the selected feature points.
[0026] In some embodiments, extracting a contour of the binarized image from the binarized image includes:
[0027] determining a convex hull of the contour;
[0028] Compare the convex hull with a preset image of the pattern and remove contours that are different from the preset image,
[0029] Determining the feature points in the extracted contour based on the position data of the selected feature points includes:
[0030] Determining a contour corresponding to the pattern from the contour of the binary image based on the attribute information of the pattern;
[0031] Feature points are determined from the determined contour corresponding to the pattern based on the position data of the selected feature points.
[0032] In some embodiments, the attribute information of the pattern includes at least one of the size of the pattern, the shape of the pattern, and the arrangement regularity of the pattern.
[0033] The shape of the pattern includes at least one of a regular shape and an irregular shape,
[0034] The regular graphics include at least one of a circle, an ellipse, a rectangle, a polygon, a triangle, and ChArUco arranged in a regular pattern;
[0035] The irregular pattern includes logos arranged in a regular pattern.
[0036] In some embodiments, the feature points include one or more feature points;
[0037] The feature point is located at least one of a corner of the pattern, a center of the pattern, and a center of a portion of the pattern.
[0038] In some embodiments, the spatial data of the pattern includes spatial data of feature points in a pattern coordinate system.
[0039] Determining the position and posture data of the target object based on the first conversion data, the second conversion data, and the spatial data of the pattern includes:
[0040] Based on the first conversion data, the spatial data of the feature point in the pattern coordinate system is converted into the camera coordinate system;
[0041] The spatial data of the feature point in the camera coordinate system is converted to the world coordinate system based on the second conversion data to determine the position information of the target object.
[0042] In some embodiments, the first transformation data comprises a first transformation matrix,
[0043] The second transformation data includes a second transformation matrix,
[0044] Determining the position and posture data of the target object based on the first conversion data and the second conversion data includes:
[0045] The coordinate data of the feature point in the pattern coordinate system is converted to the camera coordinate system by multiplying the coordinate data of the feature point in the pattern coordinate system by the first conversion matrix;
[0046] The coordinate data of the feature points in the camera coordinate system are multiplied by the second transformation matrix to transform the coordinate data of the feature points into the world coordinate system.
[0047] In some embodiments, the first transformation matrix includes a first translation matrix and a first rotation matrix.
[0048] The second transformation matrix includes a second translation matrix and a second rotation matrix.
[0049] In some embodiments, determining the position and posture data of the target object based on the first conversion data and the second conversion data further includes:
[0050] Determine the posture information of the target object based on the first rotation matrix and the second rotation matrix,
[0051] The posture information of the target object is obtained by multiplying the first rotation matrix by the second rotation matrix.
[0052] In some embodiments, the camera coordinate system is determined as the world coordinate system, the second translation matrix is a zero vector, and the second rotation matrix is an identity matrix.
[0053] According to another aspect of the present invention, a device for determining a position and posture of a target object is provided, the device comprising:
[0054] a first determining module, the first determining module being configured to determine first conversion data for mutual conversion between a camera coordinate system and a pattern coordinate system of a pattern, the pattern being a pattern on a target object;
[0055] a second determination module, the second determination module being configured to determine second conversion data for mutual conversion between a world coordinate system and a camera coordinate system;
[0056] A control module is respectively connected to the first determination module and the second determination module in communication, and the control module is configured to determine the position and posture data of the target object based on the first conversion data, the second conversion data and the spatial data of the pattern.
[0057] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:
[0058] A memory and a processor, wherein the memory stores a program, wherein the processor implements the method for determining the position and posture of a target object according to any of the aforementioned embodiments when executing the program on the memory.
[0059] According to another aspect of the present invention, a readable storage medium is provided, in which a computer-readable program or instruction is stored. When the computer-readable program or instruction is executed by a processor, the method for determining the posture of the target object described in any of the aforementioned embodiments is implemented.
[0060] The method for determining the position and posture of a target object, the device for determining the position and posture of a target object, the electronic device, and the readable storage medium according to the present invention have at least one of the following advantages:
[0061] (1) The method for determining the position and posture of a target object, the device for determining the position and posture of a target object, the electronic device and the readable storage medium of the present invention can determine the position and posture information of the target object in real time through the image information collected by the camera, thereby improving the real-time performance of the operation process;
[0062] (2) The target object posture determination method, target object posture determination device, electronic device and readable storage medium of the present invention determine the target object posture information by setting a pattern on the target object, thereby achieving high precision, high stability and high efficiency in the posture determination process. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] These and / or other aspects and advantages of the present invention will become apparent and readily understood from the following description of the preferred embodiments in conjunction with the accompanying drawings, in which:
[0064] Figure 1 A method for determining the position and posture of a target object according to an embodiment of the present invention is shown;
[0065] Figure 2 A flowchart of determining first conversion data according to an embodiment of the present invention is shown;
[0066] Figure 3-7 Shows Figure 1 Patterns that can be used in the pose determination method shown;
[0067] Figure 8 Shows Figure 7 An enlarged view of part A in FIG.
[0068] Fig. 9 A device for determining the position and posture of a target object according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0069] The technical solution of the present invention is further specifically described below by examples and in conjunction with the accompanying drawings. In the specification, the same or similar reference numerals indicate the same or similar components. The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention and should not be construed as a limitation of the present invention.
[0070] In an embodiment of the present invention, a method for determining the posture of a target object is provided. The method can determine the posture information of the target object based on image information collected by a camera (for example, image information of a pattern on the target object), thereby simplifying the operation process.
[0071] The target object herein includes an object that is desired to be tracked, such as a body part of a subject. For example, the body part of the subject may be a head, a foot, and the like.
[0072] The pose in this paper includes both position and attitude.
[0073] The camera herein includes a camera capable of obtaining a color image or an infrared image, such as an RGB camera, an infrared camera, etc.
[0074] like Figure 1 As shown, the posture determination method includes:
[0075] Determine first conversion data for mutual conversion between a camera coordinate system and a pattern coordinate system of a pattern, wherein the pattern is a pattern on a target object;
[0076] Determine second conversion data for mutual conversion between the world coordinate system and the camera coordinate system;
[0077] The position and posture data of the target object are determined based on the first conversion data, the second conversion data, and the spatial data of the pattern.
[0078] The first conversion data of the embodiment of the present invention can be determined in real time through the image information collected by the camera, and the position and posture data of the target object can be determined by means of the determined second conversion data and the spatial data of the pattern on the target object. Since the determination process of the first conversion data is carried out in real time, the position and posture determination process of the embodiment of the present invention is also carried out in real time, which improves the real-time nature of the operation process.
[0079] The embodiment of the present invention determines the first conversion data for mutual conversion between the camera coordinate system and the pattern coordinate system, which can determine the transformation relationship between the camera and an external reference (such as a pattern on the target object), simplifying the operation process. That is, the embodiment of the present invention obtains the process of the position information of the target object through machine vision.
[0080] The spatial data of a pattern includes the spatial data of a point on the pattern in the pattern coordinate system. For example, the spatial data of a pattern is the coordinates of a point on the pattern in the pattern coordinate system.
[0081] In one example, the pattern is fixed to the target object to ensure that the relative relationship between the pattern and the target object (and the target image) is determined. The fixation can be achieved by laser engraving, pasting (for example, printing into a picture), etc.
[0082] The pattern plane fixed to the target object is flat, so as to obtain accurate posture data. The position of the pattern on the target object can be set as needed, as long as it can be captured by the camera, and the embodiments of the present invention are not limited thereto.
[0083] The pattern can be in the form of a calibration plate, or a distinctive pattern can be set as required to increase the freedom and recognition of product design. The shape of the pattern includes at least one of a regular pattern and an irregular pattern.
[0084] The regular graphics include at least one of a circle, an ellipse, a rectangle, a polygon, a triangle, and ChArUco arranged in a regular pattern, such as Figure 3-6 shown. Figure 3 An embodiment of the present invention is a checkerboard pattern. Figure 4 and Figure 5 The embodiment of the present invention is a circular pattern, and the difference between the two is that the arrangement rules of the circles are different. Figure 6 An embodiment of is ChArUco.
[0085] The irregular graphics include asymmetric polygons or asymmetric curved edges arranged in a regular pattern. For example, the irregular graphics include logos arranged in a regular pattern, such as Figure 7 shown. Figure 7 The pattern includes a left portion, an upper right portion, and a lower right portion. The left portion is similar to a lowercase n. The upper right portion is a ring with a solid circle connected by a rectangular segment in the upper right portion. The lower right portion is similar to a u turned 90 degrees to the left, but the two side ends of the left side of the figure are on an oblique line, and the length of the upper half of the figure is longer than the length of the lower half.
[0086] The embodiments of the present invention do not limit the specific pattern form, and those skilled in the art can set it as needed.
[0087] Specifically, Figure 2 As shown, determining the first conversion data for mutual conversion between the camera coordinate system and the pattern coordinate system of the pattern includes:
[0088] Selecting feature points on the pattern of the target object;
[0089] The first conversion data is determined based on the spatial data of the feature point in the pattern coordinate system and the intrinsic parameter data of the camera.
[0090] The setting of feature points can facilitate the determination of the pose information of the pattern (and the corresponding target object). In one example, one or more feature points can be selected on the pattern of the target object. The embodiments of the present disclosure do not limit the specific number of feature points, and those skilled in the art can set them as needed.
[0091] The position of the feature point can be set based on the specific pattern shape. For example, the position of the feature point includes at least one of the corners of the pattern, the center of the pattern, and the center of a portion of the pattern. Setting the position of the feature point at the corners of the pattern, the center of the pattern, and the center of a portion of the pattern can improve the accuracy of determining the position information of the feature point.
[0092] exist Figure 3In an embodiment of the present invention, the position of the feature point includes at least one of a corner point of the rectangle, a midpoint of each side, a center of the white rectangle, and a center of the black rectangle. Figure 4 and Figure 5 In an embodiment of the invention, the position of the feature point includes at least one of the center of a circle, the midpoint of the centers of two adjacent circles in the horizontal direction, and the midpoint of the centers of two adjacent circles in the vertical direction. Figure 6 In an embodiment of the present invention, the position of the feature point includes at least one of the corner points of each rectangle, the center of each rectangle, and the center of the rectangle with the special pattern. Figure 7 In the embodiment of the invention, the positions of the special points include the two corner points of the left part, the center of the circular part in the upper right part, and at least one of the four corner points of the lower right part (see Figure 8 ).
[0093] In one example, determining the first conversion data based on the spatial data of the feature point in the pattern coordinate system and the intrinsic parameter data of the camera includes:
[0094] Determine the spatial data of the feature point in the pattern coordinate system;
[0095] Determine the spatial data of the feature point image in the image captured by the camera in the image coordinate system;
[0096] Determine the spatial data of the feature point image in the pattern coordinate system based on the spatial data of the feature point in the pattern coordinate system and the spatial data of the feature point image in the image coordinate system;
[0097] Based on the camera's intrinsic parameter data, the spatial data of the feature point image in the image coordinate system is converted to the camera coordinate system;
[0098] The first conversion data is determined based on the spatial data of the feature point image in the pattern coordinate system and the spatial data of the feature point image in the camera coordinate system.
[0099] After the shape of the pattern is determined, the pattern coordinate system can be determined. The pattern coordinate system can be a three-dimensional Cartesian coordinate system. For example, a feature point (such as the centroid of the figure or the corner point of the figure) is selected as the origin of the three-dimensional Cartesian coordinate system, and the x-axis and y-axis of the three-dimensional Cartesian coordinate system are in the plane of the pattern, and the z-axis is perpendicular to the outward direction of the pattern.
[0100] The three-dimensional coordinates of the feature points in the pattern coordinate system can be determined. After selecting the feature points on the pattern, the three-dimensional coordinates of the feature points in the pattern coordinate system can be determined, for example, by measuring with a three-dimensional coordinate meter, or by using three-dimensional modeling software, or by manual measurement. Since the feature points are located in the plane of the pattern, the coordinate values of the feature points on the z-axis are all 0.
[0101] In one example, the image captured by the camera needs to capture a complete pattern image. Figure 3-5 The pattern in the embodiment requires a complete pattern image to determine the position information of the target object.
[0102] For patterns with unique coding information, embodiments of the present invention allow the camera to capture images of partial patterns. For example, for ChArUco patterns, it is possible to allow processing based on images of partial ChArUco patterns (i.e., images of the complete ChArUco pattern are not captured). ChArUco patterns have unique coding information. Specifically, each QR code in the ChArUco pattern is a unique code and is identifiable. When the captured image is a partial pattern of the ChArUco pattern, the captured QR code can be used to determine the position of the captured image of the partial pattern in the complete pattern image, thereby allowing the position information of the target object to be determined based on the image of this partial pattern.
[0103] The process of the camera collecting images can be carried out in real time or in a single shot. The camera collecting process can be triggered manually or automatically through program settings.
[0104] Determine the spatial data of the feature point image in the image captured by the camera in the image coordinate system includes:
[0105] Converting a captured image (such as a color image or an infrared image) into a grayscale image, and performing binarization processing on the grayscale image to obtain a binarized image;
[0106] Extracting the contour of the binarized image from the binarized image;
[0107] Feature points are determined in the extracted contour based on the position data of the selected feature points.
[0108] The captured image can be converted into a grayscale image based on existing methods. For example, an RGB image can be converted into a grayscale image by averaging. A grayscale image is an image with only one sampled color per pixel. Such images are usually displayed as grayscale from the darkest black to the brightest white.
[0109] The grayscale image can be binarized by, for example, a local average adaptive thresholding method to obtain a binary image. Unlike grayscale images, in the field of computer graphics, binary images only have two colors, black and white, while grayscale images have many levels of color depth between black and white.
[0110] Extracting the contour of the binary image from the binary image also includes:
[0111] Determining the convex hull of the contour, for example, extracting contours of the binary image and calculating the convex hull of each contour;
[0112] The convex hull is compared with a preset image of the pattern, and contours different from the preset image are removed.
[0113] The preset image of the pattern on the target object can be determined when the pattern is obtained. For example, in the form of a calibration plate, the preset image is an image of the calibration plate, which is a pattern that can be determined when the calibration plate is designed.
[0114] Determining the feature points in the extracted contour based on the position data of the selected feature points includes:
[0115] Determining a contour corresponding to the pattern from the contour of the binary image based on the attribute information of the pattern;
[0116] Feature points are determined from the determined contour corresponding to the pattern based on the position data of the selected feature points.
[0117] The attribute information of the pattern includes at least one of the size of the pattern, the shape of the pattern, and the arrangement regularity of the pattern.
[0118] The position data of the feature points are predetermined, such as corner points or the center point of a circle. With the known position data of the feature points, the feature points can be obtained from the determined contour corresponding to the pattern (i.e., the retained contour), and the position coordinates of the feature points in the image coordinate system can be recorded. The image coordinate system can be selected as a Cartesian rectangular coordinate system. For example, the upper left corner of the image can be selected as the origin of the image coordinate system, the horizontal right direction is the x-axis direction, and the vertical downward direction is the y-axis direction.
[0119] exist Figure 3 In an embodiment, the attribute information of the pattern includes the length and width of the rectangle of the pattern and the arrangement regularity of the rectangle. Figure 3 The pattern of the embodiment has the characteristics of high precision in the post-processing process, but it is necessary to ensure that the camera obtains a complete pattern image. Moreover, as the distance from the camera increases, the stability during the processing will decrease.
[0120] exist Figure 4 and Figure 5 In an embodiment of the present invention, the attributes of the pattern include the diameter of the circle, the position of the center of the circle, and the arrangement pattern of the circle. Figure 3 Compared to the example using Figure 4 and Figure 5 The pattern in the embodiment can improve the stability during the processing. Figure 4 and Figure 5 When you are viewing a pattern in the image, you also need to ensure that the camera obtains a complete image of the pattern.
[0121] exist Figure 6 In the embodiment of the present invention, the attributes of the pattern include the size and arrangement of the pattern. Figure 3-5 Compared with the pattern in the embodiment, Figure 6 The pattern of the embodiment allows processing based on a partial image of the pattern to achieve recognition of the position information of the target object.
[0122] exist Figure 7 In the embodiment of the present invention, the attributes of the pattern include the size and arrangement of the pattern. Figure 3-6 Compared with the patterns of the embodiments, Figure 7 The pattern in the embodiment is more recognizable, which is conducive to quickly identifying the pattern image.
[0123] The embodiments of the present invention can achieve high precision, high stability and high efficiency in the posture determination process based on the pattern design on the target object.
[0124] In one example, by training a model using a machine learning algorithm, the spatial data of a feature point image in an image coordinate system can be determined in an image captured by a camera. For example, a convolutional neural network can be selected as a training model, appropriate convolutional layers and pooling layers can be set, and the best accuracy can be obtained by adjusting the convolutional layers and pooling layers.
[0125] When the pattern coordinate system and the image coordinate system are known, the third conversion data for mutual conversion between the pattern coordinate system and the image coordinate system can be determined. Based on the spatial data of the feature point in the pattern coordinate system, the spatial data of the feature point image in the image coordinate system and the third conversion data, the spatial data of the feature point image in the pattern coordinate system can be determined.
[0126] The intrinsic parameter data of the camera can determine the mutual conversion relationship between the camera coordinate system and the image coordinate system. Based on the intrinsic parameter data of the camera, the spatial data of the feature point image in the image coordinate system can be converted to the camera coordinate system, thereby obtaining the spatial data of the feature point image in the camera coordinate system. The intrinsic parameter data of the camera can be obtained through the camera manual or through camera calibration.
[0127] After determining the spatial data of the feature point image in the pattern coordinate system and the spatial data in the camera coordinate system, the first conversion data can be determined based on the spatial data of the feature point image in the pattern coordinate system and the spatial data in the camera coordinate system. In one example, the coordinates of the feature point image in the pattern coordinate system and the coordinates of the camera coordinate system can be solved by a PNP (Perspective-n-Point) algorithm, a direct linear transformation (DLT, Direct Linear Transform) algorithm, a P3P (Perspective-3-Point) algorithm, an EPnP algorithm or a bundle adjustment (BA, Bundle Adjustment) algorithm to obtain the first conversion data.
[0128] The embodiments of the present invention obtain the conversion relationship between the camera coordinate system and the pattern coordinate system by means of the pattern on the target object, thereby allowing the transformation relationship between the camera and an external parameter to be determined in real time through machine vision, and further allowing the position information of the target object to be determined in real time.
[0129] The first conversion data includes a first conversion matrix. The first conversion matrix includes a first translation matrix T1 and a first rotation matrix R2.
[0130] The second conversion data is used to determine the conversion relationship between the world coordinate system and the camera coordinate system. The second conversion data can be determined by, for example, a calibration plate. The world coordinate system can be set as needed.
[0131] The second conversion data includes a second conversion matrix. The second conversion matrix includes a second translation matrix T2 and a second rotation matrix R2.
[0132] In an example, the camera coordinate system may be determined as the world coordinate system, the second translation matrix T2 may be a zero vector, and the second rotation matrix R2 may be a unit matrix.
[0133] Further, determining the position and posture data of the target object based on the first conversion data, the second conversion data and the spatial data of the pattern includes:
[0134] Based on the first conversion data, the spatial data of the feature point in the pattern coordinate system is converted into the camera coordinate system;
[0135] The spatial data of the feature point in the camera coordinate system is converted to the world coordinate system based on the second conversion data to determine the position information of the target object.
[0136] The embodiment of the present invention converts the spatial data of the feature point in the pattern coordinate system to the camera coordinate system based on the first conversion data, and then converts the spatial data of the feature point in the camera coordinate system to the world coordinate system based on the second conversion data, thereby obtaining the position information of the target object. The target object position determination process of the present invention can be performed in real time based on machine vision (image data collected by the camera), which simplifies the operation process and is conducive to efficiently obtaining the position information of the target object.
[0137] For example, determining the position and posture data of the target object based on the first conversion data and the second conversion data includes:
[0138] The coordinate data of the feature point in the pattern coordinate system is converted to the camera coordinate system by multiplying the coordinate data of the feature point in the pattern coordinate system by the first conversion matrix;
[0139] The coordinate data of the feature points in the camera coordinate system are multiplied by the second transformation matrix to transform the coordinate data of the feature points into the world coordinate system.
[0140] Furthermore, determining the position and posture data of the target object based on the first conversion data and the second conversion data also includes: determining the posture information of the target object based on the first rotation matrix R1 and the second rotation matrix R2.
[0141] For example, the posture information of the target object is obtained by multiplying the first rotation matrix R1 by the second rotation matrix R2.
[0142] The embodiment of the present invention determines the posture information of the target object through the first conversion data (such as the first rotation matrix) and the second conversion data (such as the second rotation matrix). The first conversion data is obtained in real time, so the embodiment of the present invention can obtain the posture information of the target object in real time.
[0143] In another embodiment of the present invention, a device for determining the position and posture of a target object is also provided. Fig. 9 As shown, the posture determination device 100 includes a first determination module 10 , a second determination module 20 and a control module 30 .
[0144] The first determination module 10 is configured to determine first conversion data for mutual conversion between a camera coordinate system and a pattern coordinate system of a pattern. The pattern is a pattern on a target object. The first determination module 10 can determine the first conversion data using the method described in any of the above embodiments.
[0145] The second determination module 20 is configured to determine second conversion data for mutual conversion between the world coordinate system and the camera coordinate system. The second determination module 20 may determine the second conversion data using the method described in any of the above embodiments.
[0146] The control module 30 is respectively connected to the first determination module 10 and the second determination module 20 in communication. The control module 30 is configured to determine the position and posture data of the target object based on the first conversion data, the second conversion data and the spatial data of the pattern. The control module 30 can determine the position and posture data of the target object using the method described in any of the above embodiments.
[0147] In an embodiment of the present invention, a readable storage medium is provided, wherein a program or instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the method for determining the position and posture of a target object described in any of the above embodiments is implemented.
[0148] The "readable storage medium" of an embodiment of the present invention refers to any medium that participates in providing a program or instruction to a processor for execution. The medium can take a variety of forms, including but not limited to non-volatile media, volatile media and transmission media. Non-volatile media include, for example, optical disks or disks, such as storage devices. Volatile media include dynamic memory, such as main memory. Transmission media include coaxial cables, copper wires and optical fibers, including wires containing buses. Transmission media can also take the form of sound waves or light waves, such as sound waves or light waves generated during radio frequency (RF) and infrared (IR) data communications. Common forms of readable storage media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs, any other optical media, punch cards, paper tapes, any other physical media with hole patterns, RAMs, PROMs and EPROMs, FLASH-EPROMs, any other memory chips or boxes, carriers as described below, or any other media from which a computer can read.
[0149] In an embodiment of the present invention, an electronic device is also provided. The electronic device (not shown) includes a processor (not shown) and a memory (not shown). A program is stored in the memory, and when the program is executed by the processor, the method for determining the position and posture of the target object in any of the above examples can be implemented.
[0150] In one example, the processor may be a microprocessor, such as a general-purpose processor such as a graphics processing unit (GPU), a central processing unit (CPU), a digital signal processor (DSP), etc. In one example, the processor may also be a microprocessor core implemented by a hardware circuit, such as a microprocessor core implemented in a hardware logic component by a reconfigurable logic, and the hardware logic component includes a field programmable gate array (FPGA), a complex programmable logic device (CPLD), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system on a chip (SOC), etc.
[0151] In one example, the processor may also be a virtual processor, which may be a virtual processor with the characteristics of an Intel x86 processor, or a virtual processor with the characteristics of a PowerPC processor. Preferably, the processor is a graphics processor. In one example, the processor may be a single-core processor or a multi-core processor.
[0152] In one example, the memory includes a volatile memory (i.e., a random access memory) and a non-volatile memory. The volatile memory includes a main memory, a cache, etc., and the non-volatile memory includes an auxiliary memory, etc. In one example, the memory can be set as a remote memory, and the remote memory can be connected to the processor via a network (wired network or wireless network). The network includes, but is not limited to, a wide area network, a local area network, a metropolitan area network, a personal area network, the Internet, a satellite communication network, and any combination thereof.
[0153] In one example, the processor executes a program based on the program obtained from the memory to create a corresponding task thread and execute the thread. In one example, the processor obtains a program from the external memory based on the read instruction in the memory to create a corresponding task thread and execute the thread. The above program is used to implement the method for determining the position and posture of the target object.
[0154] Although the subject matter described herein is provided in the general context of being executed in conjunction with the execution of an operating system and an application program on a computer system, it will be appreciated by those skilled in the art that it can also be implemented in conjunction with other types of program modules. Generally speaking, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types. It will be appreciated by those skilled in the art that the method steps described in conjunction with any one of the examples herein can all be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether to perform in hardware or software mode depends mainly on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0155] When the method steps are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Therefore, the technical solution of the present invention, or the part that contributes to the original technology, or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each example of the present invention.
[0156] The method for determining the position and posture of a target object, the device for determining the position and posture of a target object, the electronic device, and the readable storage medium according to the present invention have at least one of the following advantages:
[0157] (1) The method for determining the position and posture of a target object, the device for determining the position and posture of a target object, the electronic device and the readable storage medium of the present invention can determine the position and posture information of the target object in real time through the image information collected by the camera, thereby improving the real-time performance of the operation process;
[0158] (2) The target object posture determination method, target object posture determination device, electronic device and readable storage medium of the present invention can determine the target object's posture information by setting a pattern on the target object, thereby achieving high precision, high stability and high efficiency in the posture determination process.
[0159] Although some embodiments of the present general inventive concept have been shown and described, it will be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the present general inventive concept, the scope of which is defined in the claims and their equivalents.
Claims
1. A method for determining the position and posture of a target object, include: Determine first conversion data for mutual conversion between a camera coordinate system and a pattern coordinate system, wherein the pattern is a pattern on a target object; Determine second conversion data for mutual conversion between the world coordinate system and the camera coordinate system; The position and posture data of the target object are determined based on the first conversion data, the second conversion data, and the spatial data of the pattern.
2. The method for determining the position and posture of a target object according to claim 1, in, Determining first conversion data for mutual conversion between the camera coordinate system and the pattern coordinate system includes: Select feature points on the pattern; The first conversion data is determined based on the spatial data of the feature point in the pattern coordinate system and the intrinsic parameter data of the camera.
3. The method for determining the position and posture of a target object according to claim 2, in, Determining the first conversion data based on the spatial data of the feature point in the pattern coordinate system and the intrinsic parameter data of the camera includes: Determine the spatial data of the feature point in the pattern coordinate system; Determine the spatial data of the feature point image in the image captured by the camera in the image coordinate system; Determine the spatial data of the feature point image in the pattern coordinate system based on the spatial data of the feature point in the pattern coordinate system and the spatial data of the feature point image in the image coordinate system; Based on the camera's intrinsic parameter data, the spatial data of the feature point image in the image coordinate system is converted to the camera coordinate system; The first conversion data is determined based on the spatial data of the feature point image in the pattern coordinate system and the spatial data of the feature point image in the camera coordinate system.
4. The method for determining the position and posture of a target object according to claim 3, in, Determine the spatial data of the feature point image in the image captured by the camera in the image coordinate system includes: Converting the collected image into a grayscale image, and performing binarization processing on the grayscale image to obtain a binarized image; Extracting the contour of the binarized image from the binarized image; Feature points are determined in the extracted contour based on the position data of the selected feature points.
5. The method for determining the position and posture of a target object according to claim 4, in, Extracting the contour of a binary image from a binary image includes: determining a convex hull of the contour; Compare the convex hull with a preset image of the pattern and remove contours that are different from the preset image, Determining the feature points in the extracted contour based on the position data of the selected feature points includes: Determining a contour corresponding to the pattern from the contour of the binary image based on the attribute information of the pattern; Feature points are determined from the determined contour corresponding to the pattern based on the position data of the selected feature points.
6. The method for determining the position and posture of a target object according to claim 5, in, The attribute information of the pattern includes at least one of the size of the pattern, the shape of the pattern, and the arrangement regularity of the pattern. The shape of the pattern includes at least one of a regular shape and an irregular shape, The regular graphics include at least one of a circle, an ellipse, a rectangle, a polygon, a triangle, and ChArUco arranged in a regular pattern; The irregular graphics include logos arranged in a regular pattern.
7. The method for determining the position and posture of a target object according to claim 6, in, The feature points include one or more feature points; The feature point is located at least one of a corner of the pattern, a center of the pattern, and a center of a portion of the pattern.
8. The method for determining the position and posture of a target object according to any one of claims 2 to 7, in, The spatial data of the pattern includes spatial data of the feature points in the pattern coordinate system, Determining the position and posture data of the target object based on the first conversion data, the second conversion data, and the spatial data of the pattern includes: Based on the first conversion data, the spatial data of the feature point in the pattern coordinate system is converted into the camera coordinate system; The spatial data of the feature point in the camera coordinate system is converted to the world coordinate system based on the second conversion data to determine the position information of the target object.
9. The method for determining the position and posture of a target object according to claim 8, in, The first conversion data includes a first conversion matrix, The second transformation data includes a second transformation matrix, Determining the position and posture data of the target object based on the first conversion data and the second conversion data includes: The coordinate data of the feature point in the pattern coordinate system is converted to the camera coordinate system by multiplying the coordinate data of the feature point in the pattern coordinate system by the first conversion matrix; The coordinate data of the feature points in the camera coordinate system are multiplied by the second transformation matrix to transform the coordinate data of the feature points into the world coordinate system.
10. The method for determining the position and posture of a target object according to claim 9, in, The first transformation matrix includes a first translation matrix and a first rotation matrix. The second transformation matrix includes a second translation matrix and a second rotation matrix.
11. The method for determining the position and posture of a target object according to claim 10, in, Determining the position and posture data of the target object based on the first conversion data and the second conversion data further includes: Determine the posture information of the target object based on the first rotation matrix and the second rotation matrix, The posture information of the target object is obtained by multiplying the first rotation matrix by the second rotation matrix.
12. The method for determining the position and posture of a target object according to claim 10, in, The camera coordinate system is determined as the world coordinate system, the second translation matrix is a zero vector, and the second rotation matrix is a unit matrix.
13. A device for determining the position and posture of a target object, It is characterized in that The posture determination device comprises: a first determining module, the first determining module being configured to determine first conversion data for mutual conversion between a camera coordinate system and a pattern coordinate system of a pattern, the pattern being a pattern on a target object; a second determination module, the second determination module being configured to determine second conversion data for mutual conversion between a world coordinate system and a camera coordinate system; A control module is respectively connected to the first determination module and the second determination module in communication, and the control module is configured to determine the position and posture data of the target object based on the first conversion data, the second conversion data and the spatial data of the pattern.
14. An electronic device, It is characterized in that The electronic device comprises: A memory and a processor, wherein the memory stores a program, wherein the processor implements the method for determining the position and posture of a target object according to any one of claims 1 to 12 when executing the program on the memory.
15. A readable storage medium, It is characterized in that The readable storage medium stores a computer-readable program or instruction, and when the computer-readable program or instruction is executed by a processor, the method for determining the position and posture of a target object according to any one of claims 1 to 12 is implemented.
Citation Information
Patent Citations
Plane coding target for visual system and real-time pose measurement method thereof
CN114299172A
Device pose determination method, device and system, electronic device and medium
CN114862970A
Vehicle camera pose determination method and device, equipment, medium and program product
CN119228910A
Position and posture detection device, robot controller, and robot
JP2017219365A
