Method, device, electronic device and storage medium for processing posture data

Abstract

The present disclosure relates to a method, device, electronic device and storage medium for processing posture data. The method comprises: determining a first transformation matrix of the light capture coordinate system relative to the camera coordinate system according to the first spatial coordinates of the annotation object in the light capture coordinate system and the pixel coordinates of the annotation object in the camera coordinate system; determining a second transformation matrix of the annotation object coordinate system relative to the light capture coordinate system according to the first spatial coordinates of the annotation object; obtaining the second spatial coordinates of the annotation object in the object coordinate system, and determining a third transformation matrix of the object coordinate system relative to the annotation object coordinate system according to the first spatial coordinates and the second spatial coordinates; determining a target transformation matrix according to the first transformation matrix, the second transformation matrix and the third transformation matrix, and converting the initial posture parameters of the target object into target posture parameters according to the target transformation matrix; the present disclosure reduces the error in posture data acquisition and improves the efficiency of posture data processing by mutual conversion of multiple transformation matrices.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for processing posture data. Background Art

[0002] As the application scope of technologies such as augmented reality (AR), visual SLAM (simultaneous localization and mapping), video projection, virtual-real registration and pose measurement becomes wider and wider, the processing requirements for pose data required to realize the above technologies are also getting higher and higher.

[0003] In related technologies, objects are usually arranged in the scene first, and RGB or RGB-D video is recorded; the scene is densely reconstructed using the SfM (Structure From Motion, 3D reconstruction) method to obtain inter-frame camera pose data. However, the SfM method is mainly used in outdoor scenes, and the accuracy is mostly at the centimeter level. The error is still large when preparing pose data of small objects, and the efficiency of pose data processing is still low. Summary of the invention

[0004] The present disclosure provides a posture data processing method, device, electronic device and storage medium to at least solve the problem that the efficiency of posture data processing in the related art is still low. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a posture data processing method is provided, comprising:

[0006] Determine a first transformation matrix of the light capture coordinate system relative to the camera coordinate system according to the first spatial coordinates of the marked object in the light capture coordinate system and the pixel coordinates of the marked object in the camera coordinate system;

[0007] Determine a second transformation matrix of the coordinate system of the marked object relative to the light capture coordinate system according to the first spatial coordinates of the marked object;

[0008] Acquire the second spatial coordinates of the annotated object in the object coordinate system, and determine a third transformation matrix of the object coordinate system relative to the annotated object coordinate system according to the first spatial coordinates and the second spatial coordinates; the object coordinate system is the coordinate system of the target object marked by the annotated object;

[0009] A target transformation matrix is ​​determined according to the first transformation matrix, the second transformation matrix and the third transformation matrix, and the initial posture parameters of the target object are converted into target posture parameters according to the target transformation matrix.

[0010] In an exemplary embodiment, before determining a first transformation matrix of the light capture coordinate system relative to the camera coordinate system, the method further includes:

[0011] Acquire the position information of the marking points pre-set on the chessboard through the light capture system;

[0012] The first spatial coordinates of the marked object in the light capture coordinate system are determined according to the position information.

[0013] In an exemplary embodiment, the pixel coordinates of the marker in the camera coordinate system are determined by the following method, including:

[0014] Collecting a checkerboard image containing the marked object by an infrared camera;

[0015] Performing infrared filtering on the chessboard image to determine coordinate information of the marked object on the chessboard;

[0016] The pixel coordinates of the marked object in the camera coordinate system are determined according to the coordinate information.

[0017] In an exemplary embodiment, determining a second transformation matrix of the marker coordinate system relative to the light capture coordinate system according to the first spatial coordinates of the marker includes:

[0018] Determine, according to the first spatial coordinates of the annotated object, a coordinate transformation relationship between the coordinates of the annotated object and an annotated object system; the annotated object system is composed of all the annotated objects;

[0019] Determining the light capture measurement value of each of the marked objects according to the coordinate transformation relationship;

[0020] A second transformation matrix of the marker coordinate system relative to the light capture coordinate system is obtained according to each of the light capture measurement values.

[0021] In an exemplary embodiment, determining a third transformation matrix of the object coordinate system relative to the annotation object coordinate system according to the first space coordinate and the second space coordinate includes:

[0022] Constructing a first point cloud corresponding to the labeled object according to the first spatial coordinates, and determining a coordinate system of the first point cloud; constructing a second point cloud corresponding to the labeled object according to the second spatial coordinates, and determining a coordinate system of the second point cloud;

[0023] A third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined according to the coordinate system of the first point cloud and the coordinate system of the second point cloud.

[0024] In an exemplary embodiment, determining a third transformation matrix of the object coordinate system relative to the annotation object coordinate system according to the coordinate system of the first point cloud and the coordinate system of the second point cloud includes:

[0025] Acquire a first transformation relationship of the first spatial coordinate from the light capture coordinate system to the object coordinate system, and determine an initial value of the posture transformation of the marked object coordinate system relative to the light capture coordinate system according to the first transformation relationship;

[0026] The initial value of the posture transformation is used as a constraint condition to align the coordinate system of the first point cloud and the coordinate system of the second point cloud, and based on the alignment result, a third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined.

[0027] In an exemplary embodiment, determining a target transformation matrix according to the first transformation matrix, the second transformation matrix, and the third transformation matrix includes:

[0028] Obtaining the product of the first transformation matrix, the second transformation matrix and the third transformation matrix to obtain the target transformation matrix;

[0029] The converting the initial pose parameters of the target object into target pose parameters according to the target transformation matrix includes:

[0030] The product of the target transformation matrix and the initial pose parameter is obtained as the target pose parameter.

[0031] According to a second aspect of an embodiment of the present disclosure, there is provided a posture data processing device, comprising:

[0032] A first transformation matrix determining unit is configured to determine a first transformation matrix of the light capture coordinate system relative to the camera coordinate system according to a first spatial coordinate of the labeled object in the light capture coordinate system and a pixel coordinate of the labeled object in the camera coordinate system;

[0033] A second transformation matrix determining unit is configured to determine a second transformation matrix of a marker coordinate system relative to the light capture coordinate system according to the first spatial coordinates of the marker;

[0034] A third transformation matrix determining unit is configured to obtain a second spatial coordinate of the annotated object in an object coordinate system, and determine a third transformation matrix of the object coordinate system relative to the annotated object coordinate system according to the first spatial coordinate and the second spatial coordinate; the object coordinate system is a coordinate system of a target object marked by the annotated object;

[0035] The target posture parameter determination unit is configured to determine the target transformation matrix according to the first transformation matrix, the second transformation matrix and the third transformation matrix, and convert the initial posture parameters of the target object into target posture parameters according to the target transformation matrix.

[0036] In an exemplary embodiment, the first transformation matrix determining unit is further configured to perform:

[0037] Acquire the position information of the marking points pre-set on the chessboard through the light capture system;

[0038] The first spatial coordinates of the marked object in the light capture coordinate system are determined according to the position information.

[0039] In an exemplary embodiment, the first transformation matrix determining unit is further configured to perform:

[0040] Collecting a checkerboard image containing the marked object by an infrared camera;

[0041] Performing infrared filtering on the chessboard image to determine coordinate information of the marked object on the chessboard;

[0042] The pixel coordinates of the marked object in the camera coordinate system are determined according to the coordinate information.

[0043] In an exemplary embodiment, the second transformation matrix determination unit is further configured to perform:

[0044] Determine, according to the first spatial coordinates of the annotated object, a coordinate transformation relationship between the coordinates of the annotated object and an annotated object system; the annotated object system is composed of all the annotated objects;

[0045] Determining the light capture measurement value of each of the marked objects according to the coordinate transformation relationship;

[0046] A second transformation matrix of the marker coordinate system relative to the light capture coordinate system is obtained according to each of the light capture measurement values.

[0047] In an exemplary embodiment, the third transformation matrix determination unit is further configured to perform:

[0048] Constructing a first point cloud corresponding to the labeled object according to the first spatial coordinates, and determining a coordinate system of the first point cloud; constructing a second point cloud corresponding to the labeled object according to the second spatial coordinates, and determining a coordinate system of the second point cloud;

[0049] A third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined according to the coordinate system of the first point cloud and the coordinate system of the second point cloud.

[0050] In an exemplary embodiment, the third transformation matrix determination unit is further configured to perform:

[0051] Acquire a first transformation relationship of the first spatial coordinate from the light capture coordinate system to the object coordinate system, and determine an initial value of the posture transformation of the marked object coordinate system relative to the light capture coordinate system according to the first transformation relationship;

[0052] The initial value of the posture transformation is used as a constraint condition to align the coordinate system of the first point cloud and the coordinate system of the second point cloud, and based on the alignment result, a third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined.

[0053] In an exemplary embodiment, the target pose parameter determination unit is further configured to execute:

[0054] Obtaining the product of the first transformation matrix, the second transformation matrix and the third transformation matrix to obtain the target transformation matrix;

[0055] The target posture parameter determination unit is further configured to execute:

[0056] The product of the target transformation matrix and the initial pose parameter is obtained as the target pose parameter.

[0057] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0058] processor;

[0059] a memory for storing instructions executable by the processor;

[0060] Wherein, the processor is configured to execute the instructions to implement the posture data processing method as described in any one of the embodiments of the first aspect.

[0061] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the posture data processing method as described in any one of the embodiments of the first aspect.

[0062] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, wherein the computer program product includes instructions, and when the instructions are executed by a processor of an electronic device, the electronic device is able to perform the posture data processing method as described in any one of the embodiments of the first aspect.

[0063] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0064] The first transformation matrix of the light capture coordinate system relative to the camera coordinate system is determined according to the first spatial coordinates of the labeled object in the light capture coordinate system and the pixel coordinates of the labeled object in the camera coordinate system; secondly, the second transformation matrix of the labeled object coordinate system relative to the light capture coordinate system is determined according to the first spatial coordinates of the labeled object; the second spatial coordinates of the labeled object in the object coordinate system are obtained again, and the third transformation matrix of the object coordinate system relative to the labeled object coordinate system is determined according to the first spatial coordinates and the second spatial coordinates; the object coordinate system is the coordinate system of the target object marked by the labeled object; finally, the target transformation matrix is ​​determined according to the first transformation matrix, the second transformation matrix and the third transformation matrix, and the initial posture parameters of the target object are converted into target posture parameters according to the target transformation matrix; the present invention obtains the corresponding target transformation matrix through the first transformation matrix, the second transformation matrix and the third transformation matrix, converts the initial posture parameters of the target object into target posture parameters, realizes the acquisition of posture data of any object relative to the camera, reduces the error in posture data acquisition through the mutual conversion of multiple transformation matrices, and improves the efficiency of posture data processing.

[0065] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0067] Figure 1 It is an application environment diagram of a posture data processing method according to an exemplary embodiment.

[0068] Figure 2 It is a flowchart of a method for processing posture data according to an exemplary embodiment.

[0069] Figure 3 The diagram is a chessboard diagram according to an exemplary embodiment.

[0070] Figure 4 It is a flowchart of another method for processing posture data according to an exemplary embodiment.

[0071] Figure 5 It is a block diagram of a posture data processing device according to an exemplary embodiment.

[0072] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0073] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0074] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0075] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0076] The posture data processing method provided in the present disclosure can be applied to Figure 1 In the application environment shown. Among them, the terminal 110 interacts with the electronic device 120 through the network, and the electronic device 120 determines the first transformation matrix of the light capture coordinate system relative to the camera coordinate system according to the first spatial coordinates of the annotated object in the light capture coordinate system and the pixel coordinates of the annotated object in the camera coordinate system; the electronic device 120 determines the second transformation matrix of the annotated object coordinate system relative to the light capture coordinate system according to the first spatial coordinates of the annotated object; the electronic device 120 obtains the second spatial coordinates of the annotated object in the object coordinate system, and determines the third transformation matrix of the object coordinate system relative to the annotated object coordinate system according to the first spatial coordinates and the second spatial coordinates; the object coordinate system is the coordinate system of the target object marked by the annotated object; the electronic device 120 determines the target transformation matrix according to the first transformation matrix, the second transformation matrix and the third transformation matrix, and converts the initial pose parameters of the target object sent by the terminal 110 into the target pose parameters according to the target transformation matrix, and returns the target pose parameters to the terminal 110. Among them, the terminal 110 can be but not limited to various smart phones, tablet computers or laptop computers, etc., and the electronic device 120 can be implemented by an independent server or a server cluster composed of multiple servers.

[0077] Figure 2 is a flowchart of a method for processing posture data according to an exemplary embodiment. Figure 2 As shown, the posture data processing method is used in the electronic device 120, and includes the following steps.

[0078] In step S210, a first transformation matrix of the light capture coordinate system relative to the camera coordinate system is determined according to the first spatial coordinates of the marked object in the light capture coordinate system and the pixel coordinates of the marked object in the camera coordinate system.

[0079] Among them, the optical capture coordinate system (motion capture system, which can be abbreviated as MC) is the coordinate system of the data collected by the optical motion capture system; the optical motion capture system is composed of at least motion capture lens, motion capture software, calibration frame, marker points / markers and other components; the marker (marker) is attached to the target object (object) to be marked, and its rotation amount can be obtained after initialization by the optical capture system software. The essence of the optical motion capture system can be understood as a positioning system, which obtains various kinematic data such as speed and acceleration related to it through calculation and analysis; the optical motion capture lens is the core of the motion capture system. The optical motion capture lens can use infrared light as the principle to obtain the specific spatial position of the marker point through observation and positioning of specific marker points by multiple lenses. Usually, the LED light on the panel of the optical motion capture lens emits infrared light of a specific wavelength to illuminate the marker, and the reflective material on the surface of the marker reflects the infrared light back to the lens. After the reflected infrared light is processed by signal processing, image capture and algorithm processing, the two-dimensional coordinates of the marker in the lens are obtained; after calibration processing, multiple motion capture lenses can obtain the specific position of each lens, thereby calculating the corresponding three-dimensional coordinates. The motion capture software can combine the two-dimensional coordinates obtained by multiple lenses in the system and the position information of each lens in space, and obtain six degrees of freedom (6Dof), yaw angle (Yaw), roll angle (Roll), pitch angle (Pitch), Euler angle and other posture data through calculations on the software side.

[0080] Among them, the camera coordinate system (camera, abbreviated as cam) is a coordinate system determined by the camera optical center and viewing angle; the camera can be a depth image (RGB-D) camera, which is used to capture and obtain a depth image data set; Depth Map is an image or image channel containing information related to the distance of the surface of the scene object at the viewpoint; Depth Map is similar to a grayscale image, except that each of its pixel values ​​is the actual distance between the sensor and the target object; usually RGB images and Depth images are registered, so there is a one-to-one correspondence between pixels.

[0081] Specifically, the first transformation matrix of the light capture coordinate system relative to the camera coordinate system can be recorded as H mc→cam , which represents the coordinate transformation matrix from the light capture coordinate system to the camera coordinate system, that is, the camera extrinsic parameter labeling, so as to obtain the calibration of the extrinsic parameters between the light capture system and the camera coordinate system.

[0082] During the marking process, an optical marking stick or a checkerboard can be used for marking. The optical capture system will obtain the first three-dimensional spatial coordinates of the marker marker in the optical capture coordinate system through the marker set on the optical marking stick or the marker set on the checkerboard. i}; Under the camera's perspective, the infrared reflection characteristics of the labeled object can be used, and the infrared filtering algorithm and manual auxiliary labeling can be used to determine the position of the labeled object, and the two-dimensional pixel coordinates of the labeled object on the image can be obtained. i}, and use the PnP algorithm (Perspective-n-Point) to calculate the first spatial coordinate {P i} and pixel coordinates {p i} to obtain the first transformation matrix H of the light capture coordinate system relative to the camera coordinate system mc→cam .

[0083] In step S220, a second transformation matrix of the marker coordinate system relative to the light capture coordinate system is determined according to the first spatial coordinates of the marker.

[0084] The second transformation matrix of the marker coordinate system relative to the light capture coordinate system can be expressed as H marker→mc , represents the coordinate transformation matrix of the marker relative to the light capture coordinate system. The purpose of the second transformation matrix is ​​to obtain the transformation relationship of the marker coordinate system on the target object over time during the data capture process, that is, H marker→mc A function that represents time.

[0085] Specifically, the optical capture system can directly process the input optical capture data through the motion capture software, that is, obtain the coordinates of the labeled objects, and calculate the coordinate transformation relationship between the labeled objects and the rigid body system composed of all the labeled objects through the optical capture built-in algorithm of the motion capture software, obtain the measurement value and determine H. marker→mc The time synchronization in this step can be achieved by a pre-set soft trigger device, that is, the soft trigger device can control the start of the camera and the start of the light capture system in sequence during the running time of the code segment through the code development interface of the light capture system, so that the error is reduced to an acceptable range; it can also be manually assisted in time alignment (alignment is performed by observing the changes in the annotations between the light capture running frame and the camera shooting frame) to further reduce the error generated when determining the second transformation matrix.

[0086] In step S230, the second spatial coordinates of the annotated object in the object coordinate system are obtained, and the third transformation matrix of the object coordinate system relative to the annotated object coordinate system is determined according to the first spatial coordinates and the second spatial coordinates; the object coordinate system is the coordinate system of the target object marked by the annotated object.

[0087] The third transformation matrix of the object coordinate system relative to the annotation object coordinate system can be recorded as H obj→marker , represents the coordinate transformation matrix of the target object itself relative to the annotated object; obj represents the coordinate system of the target object annotated by the annotated object, which is initialized by the CAD model or obtained after manual initialization during the reconstruction scan process; the third transformation matrix can remove the bias error of the annotated object coordinate system relative to the object coordinate system.

[0088] Specifically, the target object to be marked can be reconstructed by a laser scanner, and the marked object is modeled as a sphere, and the position of the marked object in the object coordinate system is determined to obtain the second space coordinate {Q i}; Set the first space coordinate {P i} and the second space coordinate {Q i} Using the preset algorithm to process, the third transformation matrix H of the object coordinate system relative to the annotation object coordinate system can be obtained obj→marker .

[0089] In step S240, a target transformation matrix is ​​determined according to the first transformation matrix, the second transformation matrix, and the third transformation matrix, and the initial posture parameters of the target object are converted into target posture parameters according to the target transformation matrix.

[0090] Among them, the target transformation matrix is ​​expressed as H obj→cam , that is, the coordinate transformation matrix of the target object relative to the camera; the initial pose parameter can be the 6-DOF pose of the target object, and the target pose parameter is expressed as H according to the target transformation matrix obj→cam The pose parameters obtained by converting the 6-DOF pose of the target object;

[0091] Target transformation matrix H obj→cam In the formula, H = [R is equal to ∈ SE (3), where the rotation R ∈ SO (3) and the translation in, represents the real number domain, indicating that the translation t is a real vector of length 3; R represents rotation; t represents translation; SE(3) represents Special Euclidean Group in 3 dimensions; SO(3) represents Special Orthogonal Group in 3 dimensions.

[0092] Specifically, after obtaining the first transformation matrix, the second transformation matrix and the third transformation matrix, the electronic device can determine the target transformation matrix based on the first transformation matrix, the second transformation matrix and the third transformation matrix. Through the target transformation matrix, the initial posture parameters of the target object to be processed input by the terminal can be converted into target posture parameters, that is, the posture data of the target object relative to the camera is realized through the target transformation matrix, and the processing of the posture data is completed.

[0093] In the above-mentioned posture data processing method, the first transformation matrix of the light capture coordinate system relative to the camera coordinate system is determined according to the first spatial coordinates of the labeled object in the light capture coordinate system and the pixel coordinates of the labeled object in the camera coordinate system; secondly, the second transformation matrix of the labeled object coordinate system relative to the light capture coordinate system is determined according to the first spatial coordinates of the labeled object; the second spatial coordinates of the labeled object in the object coordinate system are obtained again, and the third transformation matrix of the object coordinate system relative to the labeled object coordinate system is determined according to the first spatial coordinates and the second spatial coordinates; the object coordinate system is the coordinate system of the target object marked by the labeled object; finally, the target transformation matrix is ​​determined according to the first transformation matrix, the second transformation matrix and the third transformation matrix, and the initial posture parameters of the target object are converted into target posture parameters according to the target transformation matrix; the present disclosure obtains the corresponding target transformation matrix through the first transformation matrix, the second transformation matrix and the third transformation matrix, converts the initial posture parameters of the target object into target posture parameters, realizes the acquisition of posture data of any object relative to the camera, reduces the error in posture data acquisition through the mutual conversion of multiple transformation matrices, and improves the efficiency of posture data processing.

[0094] In an exemplary embodiment, in step S210, before determining the first transformation matrix of the light capture coordinate system relative to the camera coordinate system, it also includes: acquiring position information of marking points pre-set on the chessboard through the light capture system; and determining the first spatial coordinates of the marked object in the light capture coordinate system based on the position information.

[0095] Specifically, Figure 3 The figure shows a schematic diagram of arranging markers on a chessboard. The light capture system determines the position information of each marker through an infrared camera based on the infrared reflective characteristics of the markers on the chessboard. The first spatial coordinates of the marker in the light capture coordinate system are further obtained through the algorithm of the light capture system based on the position information.

[0096] The technical solution provided by the embodiment of the present disclosure determines the position information of the marked object by capturing the infrared light reflected by the marked object through the light capture system, and obtains the first spatial coordinates of the marked object in the light capture coordinate system based on the position information, thereby realizing the spatial positioning of each marked object, obtaining the position of each marked object in space, and improving the efficiency of obtaining the first spatial coordinates of the marked object.

[0097] In an exemplary embodiment, the step of determining the pixel coordinates of the labeled object in the camera coordinate system can be specifically implemented by the following steps: acquiring a checkerboard image containing the labeled object through an infrared camera; performing infrared filtering on the checkerboard image to determine the coordinate information of the labeled object on the checkerboard; and determining the pixel coordinates of the labeled object in the camera coordinate system based on the coordinate information.

[0098] Specifically, a checkerboard image containing annotated objects is obtained through a camera, and the color of each grid in the checkerboard can be used to determine the annotated objects and the relative positions between the annotated objects to realize the positioning of the annotated objects in a plane perspective; the camera perspective uses the infrared reflection characteristics of the annotated objects, and through an infrared filtering algorithm combined with a manual auxiliary annotation method, the position of the annotated objects is determined to obtain the two-dimensional pixel coordinates of the annotated objects on the image {p i}; Get the two-dimensional pixel coordinates of the annotation object on the image {p i}, the PnP algorithm (Perspective-n-Point) can be used to calculate the first spatial coordinate {P i} and pixel coordinates {p i} to obtain the first transformation matrix H of the light capture coordinate system relative to the camera coordinate system mc→cam .

[0099] The technical solution provided by the embodiments of the present disclosure utilizes the infrared reflection characteristics of the marked object and the characteristics of the checkerboard that can locate the marked object, determines the coordinate information of the marked object under the camera's viewing angle, and further obtains the pixel coordinates of the marked object in the camera coordinate system based on the coordinate information. The pixel coordinates can be further used to determine the first spatial coordinates of the marked object, thereby improving the accuracy and efficiency of obtaining the first spatial coordinates of the marked object.

[0100] In an exemplary embodiment, a second transformation matrix of the labeled object coordinate system relative to the optical capture coordinate system is determined based on the first spatial coordinates of the labeled object, which can be specifically achieved through the following steps: based on the first spatial coordinates of the labeled object, a coordinate transformation relationship of the labeled object coordinate system relative to the labeled object system is determined; the labeled object system is composed of all the labeled objects; based on the coordinate transformation relationship, the optical capture measurement value of each labeled object is determined; based on each optical capture measurement value, the second transformation matrix of the labeled object coordinate system relative to the optical capture coordinate system is obtained.

[0101] Specifically, each marker attached to the target object constitutes a marker system; the light capture system can obtain the coordinates of the marker and then calculate the measurement value through the coordinate transformation relationship of the marker coordinates relative to the marker system through the built-in algorithm of the light capture system, that is, the second transformation matrix of the marker coordinate system relative to the light capture coordinate system. In this process, in order to ensure that the error remains within an acceptable range, time synchronization is required; the camera and the light capture system can be controlled by a preset soft trigger device, and the error will usually be reduced to 200 milliseconds; on this basis, the error can be further reduced manually, usually further reduced to 1 / 30 second; the manual assistance method is to observe the changes in the light capture markers between the light capture running frame and the camera shooting frame through human eyes for alignment.

[0102] The technical solution provided by the embodiment of the present disclosure utilizes the light capture system to obtain the light capture measurement value corresponding to the marked object according to the first spatial coordinates of the marked object, and finally determines the second transformation matrix of the marked object coordinate system relative to the light capture coordinate system, thereby realizing the transformation of the marked object coordinate system relative to the light capture coordinate system and improving the efficiency of determining the second transformation matrix of the light capture coordinate system.

[0103] In an exemplary embodiment, a third transformation matrix of the object coordinate system relative to the annotation object coordinate system is determined based on the first spatial coordinates and the second spatial coordinates. This can be specifically achieved through the following steps: construct a first point cloud corresponding to the annotation object based on the first spatial coordinates, and determine the coordinate system of the first point cloud; construct a second point cloud corresponding to the annotation object based on the second spatial coordinates, and determine the coordinate system of the second point cloud; determine the third transformation matrix of the object coordinate system relative to the annotation object coordinate system based on the coordinate system of the first point cloud and the coordinate system of the second point cloud.

[0104] Specifically, first, a reconstructed model of the target object is obtained by a laser scanner; then, the labeled objects are modeled into spheres according to their diameters, and the positions of the labeled objects in the object coordinate system are determined, and the point clouds of the labeled objects in the optical capture coordinate system are constructed as the first point cloud according to the positions, and the point clouds of the labeled objects in the object coordinate system of the target object are constructed as the second point cloud; then, at least the Kabsch algorithm can be used to determine the transformation relationship from the optical capture coordinate system to the object coordinate system according to the first point cloud of the labeled objects in the optical capture coordinate system and the second point cloud of the labeled objects in the object coordinate system, so as to determine the third transformation matrix of the object coordinate system relative to the labeled object coordinate system.

[0105] The technical solution provided by the embodiment of the present disclosure constructs a first point cloud and a second point cloud respectively according to the position of the marked object in the object coordinate system, and determines a transformation matrix according to the conversion relationship between the two point clouds, thereby eliminating the bias error from the marked object coordinate system to the object coordinate system to a certain extent, and improving the accuracy and acquisition efficiency of the third transformation matrix.

[0106] In an exemplary embodiment, the third transformation matrix of the object coordinate system relative to the annotation object coordinate system is determined based on the coordinate system of the first point cloud and the coordinate system of the second point cloud. This can be specifically achieved through the following steps: obtaining a first transformation relationship of the first spatial coordinate from the light capture coordinate system to the object coordinate system, and determining an initial value of the posture transformation of the annotation object coordinate system relative to the light capture coordinate system based on the first transformation relationship; using the initial value of the posture transformation as a constraint condition, aligning the coordinate system of the first point cloud and the coordinate system of the second point cloud, and determining the third transformation matrix of the object coordinate system relative to the annotation object coordinate system based on the alignment result.

[0107] Specifically, the first transformation relationship of the first spatial coordinate from the light capture coordinate system to the object coordinate system can be determined at least by the Kabsch algorithm; the Kabsch algorithm is an algorithm for calculating the optimal rotation matrix, so the first spatial coordinate of the labeled object in the light capture coordinate system {P i} and the position of the annotation object in the object coordinate system to obtain the second spatial coordinate {Q i} A first conversion relationship of the first spatial coordinate from the light capture coordinate system to the object coordinate system is obtained. Based on the initial value of the posture transformation of the annotated object coordinate system relative to the light capture coordinate system determined by the first conversion relationship, the ICP algorithm (an algorithm based on data registration method, using the nearest point search method to solve the problem based on free-form surfaces) can be used to perform registration processing of the coordinate system of the first point cloud and the coordinate system of the second point cloud with the initial value of the posture transformation as a constraint condition to determine the accurate third transformation matrix H of the object coordinate system relative to the annotated object coordinate system obj→marker .

[0108] The technical solution provided by the embodiment of the present disclosure determines the initial value of the posture transformation of the marked object coordinate system relative to the light capture coordinate system according to the first transformation relationship, and aligns the coordinate system of the first point cloud and the coordinate system of the second point cloud based on the initial value of the posture transformation as a constraint, thereby achieving accurate estimation of the transformation relationship between the point clouds, eliminating the bias error of the marked object coordinate system relative to the object coordinate system, and improving the accuracy and acquisition efficiency in the process of determining the third transformation matrix.

[0109] In an exemplary embodiment, a target transformation matrix is ​​determined according to the first transformation matrix, the second transformation matrix and the third transformation matrix, which can be specifically achieved by the following steps: obtaining the product of the first transformation matrix, the second transformation matrix and the third transformation matrix to obtain the target transformation matrix.

[0110] Specifically, the target transformation matrix H obj→cam It can be obtained by:

[0111] H obj→cam =H mc→camH marker→mc H obj→marker ;

[0112] That is, the product of the first transformation matrix, the second transformation matrix and the third transformation matrix is ​​the target transformation matrix.

[0113] The technical solution provided by the embodiment of the present disclosure uses the pre-acquired first transformation matrix, second transformation matrix and third transformation matrix to obtain the product of the above transformation matrices as the target transformation matrix; the pre-acquired transformation matrices have eliminated some errors in the determination process, so the obtained target transformation matrix is ​​also relatively accurate.

[0114] In an exemplary embodiment, the initial pose parameters of the target object are converted into target pose parameters according to the target transformation matrix, which can be specifically achieved by the following steps: obtaining the product of the target transformation matrix and the initial pose parameters as the target pose parameters.

[0115] Specifically, after obtaining the initial pose parameters attached to the target object through the light capture system, the determined target transformation matrix can be multiplied by the initial pose parameters, and the product of the multiplication is the target pose parameters, thereby realizing the conversion of the initial pose parameters and obtaining the pose data of the target object relative to the camera, thereby improving the efficiency and accuracy of acquiring the pose data.

[0116] In one embodiment, Figure 4 As shown, another posture data processing is provided, which specifically includes the following steps:

[0117] In step S410, the position information of the marking points pre-set on the chessboard is obtained through the light capture system; and the first spatial coordinates of the marked object in the light capture coordinate system are determined according to the position information.

[0118] In step S420, a checkerboard image containing the marked object is collected by an infrared camera; the checkerboard image is subjected to infrared filtering to determine the coordinate information of the marked object on the checkerboard; and the pixel coordinates of the marked object in the camera coordinate system are determined based on the coordinate information.

[0119] In step S430, a first transformation matrix of the light capture coordinate system relative to the camera coordinate system is determined according to the first spatial coordinates of the marked object in the light capture coordinate system and the pixel coordinates of the marked object in the camera coordinate system.

[0120] In step S440, the coordinate transformation relationship of the annotation object coordinates relative to the annotation object system is determined based on the first spatial coordinates of the annotation object; the annotation object system is composed of all the annotation objects; based on the coordinate transformation relationship, the light capture measurement value of each annotation object is determined; based on each light capture measurement value, the second transformation matrix of the annotation object coordinate system relative to the light capture coordinate system is obtained.

[0121] In step S450, a first point cloud corresponding to the marked object is constructed according to the first spatial coordinates, and a coordinate system of the first point cloud is determined; a second point cloud corresponding to the marked object is constructed according to the second spatial coordinates, and a coordinate system of the second point cloud is determined.

[0122] In step S460, a first transformation relationship of the first spatial coordinate from the light capture coordinate system to the object coordinate system is obtained, and an initial value of the posture transformation of the annotation object coordinate system relative to the light capture coordinate system is determined based on the first transformation relationship; the initial value of the posture transformation is used as a constraint condition to align the coordinate system of the first point cloud and the coordinate system of the second point cloud, and based on the alignment result, a third transformation matrix of the object coordinate system relative to the annotation object coordinate system is determined.

[0123] In step S470, the product of the first transformation matrix, the second transformation matrix and the third transformation matrix is ​​obtained to obtain the target transformation matrix; the product of the target transformation matrix and the initial posture parameters is obtained as the target posture parameters.

[0124] The technical solution provided by the embodiment of the present disclosure obtains the corresponding target transformation matrix through the first transformation matrix, the second transformation matrix and the third transformation matrix, converts the initial posture parameters of the target object into the target posture parameters, and realizes the acquisition of the posture data of any object relative to the camera. The mutual conversion of multiple transformation matrices reduces the error in the acquisition of posture data and improves the efficiency of posture data processing.

[0125] It should be understood that although Figure 2 , Figure 4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 , Figure 4 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0126] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0127] Figure 5 is a block diagram of a posture data processing device according to an exemplary embodiment. Figure 5The device includes a first transformation matrix determination unit 502, a second transformation matrix determination unit 504, a third transformation matrix determination unit 506 and a target posture parameter determination unit 508.

[0128] The first transformation matrix determining unit 502 is configured to determine a first transformation matrix of the light capture coordinate system relative to the camera coordinate system according to the first spatial coordinates of the labeled object in the light capture coordinate system and the pixel coordinates of the labeled object in the camera coordinate system;

[0129] The second transformation matrix determining unit 504 is configured to determine a second transformation matrix of the marker coordinate system relative to the light capture coordinate system according to the first spatial coordinates of the marker;

[0130] The third transformation matrix determining unit 506 is configured to obtain the second spatial coordinates of the annotation object in the object coordinate system, and determine the third transformation matrix of the object coordinate system relative to the annotation object coordinate system according to the first spatial coordinates and the second spatial coordinates; the object coordinate system is the coordinate system of the target object marked by the annotation object;

[0131] The target pose parameter determination unit 508 is configured to determine a target transformation matrix according to the first transformation matrix, the second transformation matrix and the third transformation matrix, and convert the initial pose parameters of the target object into target pose parameters according to the target transformation matrix.

[0132] In an exemplary embodiment, the first transformation matrix determining unit 502 is further configured to execute:

[0133] Acquire the position information of the marking points pre-set on the chessboard through the light capture system;

[0134] According to the position information, the first spatial coordinates of the marked object in the light capture coordinate system are determined.

[0135] In an exemplary embodiment, the first transformation matrix determining unit 502 is further configured to execute:

[0136] The checkerboard image containing the marked objects is collected by an infrared camera;

[0137] Perform infrared filtering on the chessboard image to determine the coordinate information of the marked object on the chessboard;

[0138] According to the coordinate information, the pixel coordinates of the marker in the camera coordinate system are determined.

[0139] In an exemplary embodiment, the second transformation matrix determining unit 504 is further configured to perform:

[0140] According to the first spatial coordinates of the annotated object, a coordinate transformation relationship between the coordinates of the annotated object and the annotated object system is determined; the annotated object system is composed of all the annotated objects;

[0141] According to the coordinate transformation relationship, the light capture measurement value of each marked object is determined;

[0142] According to each light capture measurement value, a second transformation matrix of the marker coordinate system relative to the light capture coordinate system is obtained.

[0143] In an exemplary embodiment, the third transformation matrix determining unit 506 is further configured to perform:

[0144] Constructing a first point cloud corresponding to the annotated object according to the first spatial coordinates, and determining a coordinate system of the first point cloud; constructing a second point cloud corresponding to the annotated object according to the second spatial coordinates, and determining a coordinate system of the second point cloud;

[0145] A third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined according to the coordinate system of the first point cloud and the coordinate system of the second point cloud.

[0146] In an exemplary embodiment, the third transformation matrix determining unit 506 is further configured to perform:

[0147] Acquire a first conversion relationship of the first spatial coordinate from the light capture coordinate system to the object coordinate system, and determine an initial value of the posture transformation of the marked object coordinate system relative to the light capture coordinate system according to the first conversion relationship;

[0148] The initial value of the posture transformation is used as a constraint condition, the coordinate system of the first point cloud and the coordinate system of the second point cloud are aligned, and the third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined according to the alignment result.

[0149] In an exemplary embodiment, the target pose parameter determination unit 508 is further configured to execute:

[0150] The product of the first transformation matrix, the second transformation matrix and the third transformation matrix is ​​obtained to obtain a target transformation matrix.

[0151] In an exemplary embodiment, the target pose parameter determination unit 508 is further configured to execute:

[0152] Get the product of the target transformation matrix and the initial pose parameters as the target pose parameters.

[0153] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0154] Figure 6is a block diagram of an electronic device 600 for executing a method for processing posture data according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0155] Reference Figure 6 , the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0156] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0157] The memory 604 is configured to store various types of data to support operations on the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, optical disk, or graphene memory.

[0158] The power supply component 606 provides power to the various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0159] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0160] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting an audio signal.

[0161] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0162] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of the components, such as the display and keypad of the electronic device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or the electronic device 600 component, the presence or absence of contact between the user and the electronic device 600, the device 600 orientation or acceleration / deceleration and the temperature change of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0163] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0164] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0165] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the above instructions can be executed by the processor 620 of the electronic device 600 to perform the above method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0166] In an exemplary embodiment, a computer program product is further provided. The computer program product includes instructions. The instructions can be executed by the processor 620 of the electronic device 600 to complete the above method.

[0167] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation methods according to the description of the method embodiments. The specific implementation methods can refer to the description of the relevant method embodiments, which will not be described one by one here.

[0168] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0169] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for processing posture data, characterized in that: include: Determine a first transformation matrix of the light capture coordinate system relative to the camera coordinate system according to the first spatial coordinates of the marked object in the light capture coordinate system and the pixel coordinates of the marked object in the camera coordinate system; According to the first spatial coordinates of the labeled object, a coordinate transformation relationship of the labeled object coordinates relative to the labeled object system is determined; the labeled object system is composed of all the labeled objects; according to the coordinate transformation relationship, a light capture measurement value of each of the labeled objects is determined; according to each of the light capture measurement values, a second transformation matrix of the labeled object coordinate system relative to the light capture coordinate system is obtained; Acquire the second spatial coordinates of the annotated object in the object coordinate system, and determine a third transformation matrix of the object coordinate system relative to the annotated object coordinate system according to the first spatial coordinates and the second spatial coordinates; the object coordinate system is the coordinate system of the target object marked by the annotated object; A target transformation matrix is ​​determined according to the first transformation matrix, the second transformation matrix and the third transformation matrix, and the initial posture parameters of the target object are converted into target posture parameters according to the target transformation matrix.

2. The method according to claim 1, characterized in that Before determining a first transformation matrix of the light capture coordinate system relative to the camera coordinate system, the method further includes: Acquire the position information of the marking points pre-set on the chessboard through the light capture system; The first spatial coordinates of the marked object in the light capture coordinate system are determined according to the position information.

3. The method according to claim 1, characterized in that The pixel coordinates of the annotation object in the camera coordinate system are determined by the following methods, including: Collecting a checkerboard image containing the marked object by an infrared camera; Performing infrared filtering on the chessboard image to determine coordinate information of the marked object on the chessboard; The pixel coordinates of the marked object in the camera coordinate system are determined according to the coordinate information.

4. The method according to claim 1, characterized in that: The step of determining a third transformation matrix of the object coordinate system relative to the annotation object coordinate system according to the first spatial coordinates and the second spatial coordinates includes: Constructing a first point cloud corresponding to the labeled object according to the first spatial coordinates, and determining a coordinate system of the first point cloud; constructing a second point cloud corresponding to the labeled object according to the second spatial coordinates, and determining a coordinate system of the second point cloud; A third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined according to the coordinate system of the first point cloud and the coordinate system of the second point cloud.

5. The method according to claim 4, characterized in that The determining, according to the coordinate system of the first point cloud and the coordinate system of the second point cloud, a third transformation matrix of the object coordinate system relative to the marked object coordinate system comprises: Acquire a first transformation relationship of the first spatial coordinate from the light capture coordinate system to the object coordinate system, and determine an initial value of the posture transformation of the marked object coordinate system relative to the light capture coordinate system according to the first transformation relationship; The initial value of the posture transformation is used as a constraint condition to align the coordinate system of the first point cloud and the coordinate system of the second point cloud, and based on the alignment result, a third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined.

6. The method according to claim 1, characterized in that The determining a target transformation matrix according to the first transformation matrix, the second transformation matrix, and the third transformation matrix includes: Obtaining the product of the first transformation matrix, the second transformation matrix and the third transformation matrix to obtain the target transformation matrix; The converting the initial pose parameters of the target object into target pose parameters according to the target transformation matrix includes: The product of the target transformation matrix and the initial pose parameter is obtained as the target pose parameter.

7. A posture data processing device, characterized in that: include: A first transformation matrix determining unit is configured to determine a first transformation matrix of the light capture coordinate system relative to the camera coordinate system according to a first spatial coordinate of the labeled object in the light capture coordinate system and a pixel coordinate of the labeled object in the camera coordinate system; The second transformation matrix determination unit is configured to determine the coordinate transformation relationship of the annotation object coordinates relative to the annotation object system according to the first spatial coordinates of the annotation object; the annotation object system is composed of all the annotation objects; determine the light capture measurement value of each of the annotation objects according to the coordinate transformation relationship; and obtain a second transformation matrix of the annotation object coordinate system relative to the light capture coordinate system according to each of the light capture measurement values; A third transformation matrix determining unit is configured to obtain a second spatial coordinate of the annotated object in an object coordinate system, and determine a third transformation matrix of the object coordinate system relative to the annotated object coordinate system according to the first spatial coordinate and the second spatial coordinate; the object coordinate system is a coordinate system of a target object marked by the annotated object; The target posture parameter determination unit is configured to determine the target transformation matrix according to the first transformation matrix, the second transformation matrix and the third transformation matrix, and convert the initial posture parameters of the target object into target posture parameters according to the target transformation matrix.

8. The device according to claim 7, characterized in that The first transformation matrix determining unit is further configured to execute: Acquire the position information of the marking points pre-set on the chessboard through the light capture system; The first spatial coordinates of the marked object in the light capture coordinate system are determined according to the position information.

9. The device according to claim 7, characterized in that The first transformation matrix determining unit is further configured to execute: Collecting a checkerboard image containing the marked object by an infrared camera; Performing infrared filtering on the chessboard image to determine coordinate information of the marked object on the chessboard; The pixel coordinates of the marked object in the camera coordinate system are determined according to the coordinate information.

10. The device according to claim 7, characterized in that The third transformation matrix determination unit is further configured to execute: Constructing a first point cloud corresponding to the labeled object according to the first spatial coordinates, and determining a coordinate system of the first point cloud; constructing a second point cloud corresponding to the labeled object according to the second spatial coordinates, and determining a coordinate system of the second point cloud; A third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined according to the coordinate system of the first point cloud and the coordinate system of the second point cloud.

11. The device according to claim 10, characterized in that The third transformation matrix determination unit is further configured to execute: Acquire a first transformation relationship of the first spatial coordinate from the light capture coordinate system to the object coordinate system, and determine an initial value of the posture transformation of the marked object coordinate system relative to the light capture coordinate system according to the first transformation relationship; The initial value of the posture transformation is used as a constraint condition to align the coordinate system of the first point cloud and the coordinate system of the second point cloud, and based on the alignment result, a third transformation matrix of the object coordinate system relative to the marked object coordinate system is determined.

12. The device according to claim 7, characterized in that The target posture parameter determination unit is further configured to execute: Obtaining the product of the first transformation matrix, the second transformation matrix and the third transformation matrix to obtain the target transformation matrix; The target posture parameter determination unit is further configured to execute: The product of the target transformation matrix and the initial pose parameter is obtained as the target pose parameter.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the posture data processing method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the posture data processing method as described in any one of claims 1 to 6.

15. A computer program product, comprising instructions, characterized in that: When the instruction is executed by a processor of an electronic device, the electronic device is enabled to execute the posture data processing method as described in any one of claims 1 to 6.

Images