A method and device for determining posture, electronic device, and storage medium

Abstract

The disclosed embodiment discloses a method and device for determining a posture, an electronic device, and a storage medium, which can improve the efficiency of determining a posture. The method includes: generating a scene point cloud map according to a real scene image, and extracting a scene coordinate point set from the scene point cloud map; obtaining a model point cloud map corresponding to the scene point cloud map, and selecting a model coordinate point set corresponding to the scene coordinate point set from the model point cloud map in parallel; based on the scene coordinate point and the model coordinate point set, obtaining the corresponding spatial transformation relationship between the scene point cloud map and the model point cloud map through parallel iterative calculation; and determining the relative posture between the scene point cloud map and the model point cloud map according to the spatial transformation relationship.

Description

Technical Field

[0001] The present disclosure relates to the field of software engineering, and in particular to a posture determination method and device, an electronic device, and a storage medium. Background Art

[0002] At present, with the development of three-dimensional (3D) vision, pose estimation plays an increasingly important role in medicine, object modeling, and robotics. Due to the limitations of existing sensors, most applications require extremely high precision and the combination of multiple images to obtain accurate 3D pose results of objects. Especially in mobile robot scenarios, whether 3D images can be obtained quickly and pose matching can be performed in a very short time has become the most important link that limits the robot's recognition accuracy and movement speed. However, the time cost of existing algorithms is high, which reduces the efficiency of determining pose. Summary of the invention

[0003] The embodiments of the present disclosure are intended to provide a method and device for determining a posture, an electronic device, and a storage medium, which can improve the efficiency of determining a posture.

[0004] The technical solution of the present disclosure is achieved as follows:

[0005] The present disclosure provides a method for determining a posture, the method comprising:

[0006] Generate a scene point cloud map according to the real scene image, and extract a scene coordinate point set from the scene point cloud map;

[0007] Acquire a model point cloud image corresponding to the scene point cloud image, and select a model coordinate point set corresponding to the scene coordinate point set from the model point cloud image in parallel;

[0008] Based on the scene coordinate points and the model coordinate point set, the corresponding spatial transformation relationship between the scene point cloud image and the model point cloud image is obtained through parallel iterative calculation;

[0009] According to the spatial transformation relationship, a relative pose between the scene point cloud image and the model point cloud image is determined.

[0010] The present disclosure provides a posture determination device, which includes: an extraction unit, a selection unit, an iteration unit and a determination unit, wherein:

[0011] The extraction unit is used to generate a scene point cloud map according to the real scene image, and extract a scene coordinate point set from the scene point cloud map;

[0012] The selection unit is used to obtain a model point cloud image corresponding to the scene point cloud image, and to select a model coordinate point set corresponding to the scene coordinate point set from the model point cloud image in parallel;

[0013] The iteration unit is used to obtain the corresponding spatial transformation relationship between the scene point cloud image and the model point cloud image through parallel iterative calculation based on the scene coordinate points and the model coordinate point set;

[0014] The determining unit is used to determine the relative pose between the scene point cloud image and the model point cloud image according to the spatial transformation relationship.

[0015] An embodiment of the present disclosure provides an electronic device, comprising: a processor, a memory, and a communication bus, wherein the memory communicates with the processor via the communication bus, and the memory stores one or more programs executable by the processor. When the one or more programs are executed, the processor executes any one of the above-mentioned posture determination methods.

[0016] An embodiment of the present disclosure provides a storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement any of the above-mentioned posture determination methods.

[0017] The disclosed embodiment provides a method and device for determining a posture, an electronic device, and a storage medium. The method includes: generating a scene point cloud map according to a real scene image, and extracting a scene coordinate point set from the scene point cloud map; obtaining a model point cloud map corresponding to the scene point cloud map, and selecting a model coordinate point set corresponding to the scene coordinate point set from the model point cloud map in parallel; obtaining the corresponding spatial transformation relationship between the scene point cloud map and the model point cloud map through parallel iterative calculation based on the scene coordinate point and the model coordinate point set; determining the relative posture between the scene point cloud map and the model point cloud map according to the spatial transformation relationship. By adopting the above scheme, the model coordinate point set corresponding to the scene coordinate point set can be quickly selected by parallel processing, which greatly improves the speed of selecting the model coordinate point set; further, when calculating the spatial transformation relationship that usually includes multiple parameters such as translation and rotation, each parameter can be calculated iteratively in parallel, and then the relative posture between the scene point cloud map and the model point cloud map is determined according to the calculated spatial transformation relationship, thereby improving the efficiency of determining the posture. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 An optional process diagram of a method for determining a posture provided in an embodiment of the present disclosure Figure 1 ;

[0019] Figure 2An optional process diagram of a method for determining a posture provided in an embodiment of the present disclosure Figure 2 ;

[0020] Figure 3 An optional process diagram of a method for determining a posture provided in an embodiment of the present disclosure Figure 3 ;

[0021] Figure 4 An optional process diagram of a method for determining a posture provided in an embodiment of the present disclosure Figure 4 ;

[0022] Figure 5 An optional process diagram of a method for determining a posture provided in an embodiment of the present disclosure Figure 5 ;

[0023] Figure 6 An optional process diagram of a method for determining a posture provided in an embodiment of the present disclosure Figure 6 ;

[0024] Figure 7 An optional process diagram of a method for determining a posture provided in an embodiment of the present disclosure Figure 7 ;

[0025] Figure 8 An optional process diagram of a method for determining a posture provided in an embodiment of the present disclosure Figure 8 ;

[0026] Fig. 9 A schematic diagram of a point-to-surface error measurement method provided in an embodiment of the present disclosure;

[0027] Fig.10 An optional process diagram of a method for determining a posture provided in an embodiment of the present disclosure Figure 9 ;

[0028] Fig.11 A schematic diagram of the structure of a posture determination device provided in an embodiment of the present disclosure;

[0029] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure.

[0031] Figure 1 An optional flow chart of the method provided in the embodiment of the present disclosure is combined with Figure 1 The steps shown are explained.

[0032] S101, generating a scene point cloud map according to a real scene image, and extracting a scene coordinate point set from the scene point cloud map.

[0033] In the embodiment of the present disclosure, the posture determination device first performs image acquisition on the scene to be positioned through an image acquisition device to obtain a real scene image of the scene to be positioned.

[0034] In the embodiments of the present disclosure, the image acquisition device may be a camera, a camcorder, a camera, or other device with an image acquisition function, and the embodiments of the present disclosure are not limited thereto.

[0035] In the disclosed embodiment, for a real scene image, the posture determination device can generate a scene point cloud map based on pixel points in the real scene image.

[0036] In the disclosed embodiment, a point cloud is a collection of a large number of points of surface features of an object or target, wherein each point in the point cloud includes the three-dimensional coordinates of the point and the normal vector corresponding to the point.

[0037] In the embodiment of the present disclosure, the posture determination device can obtain a scene point cloud map by performing laser measurement or photogrammetry on the real scene image, or can adopt other methods of generating a point cloud map. The specific selection is made according to the actual situation, and the embodiment of the present disclosure does not limit it.

[0038] In the disclosed embodiment, after obtaining the scene point cloud map, the posture determination device extracts a preset number of coordinate points from the scene point cloud map to obtain a scene coordinate point set.

[0039] In some embodiments, the posture determination device can import the scene point cloud map into a graphics processor, and select a set of scene coordinate points from the scene point cloud map through the graphics processor.

[0040] In the disclosed embodiment, the coordinate points in the scene coordinate point set are scene coordinate points.

[0041] In the embodiments of the present disclosure, in the scene point cloud map, the posture determination device can select a preset number of coordinate points by random selection; it can also extract object corner points in the scene point cloud map as a preset number of coordinate points; it can also select coordinate points within a specified range in the scene point cloud map as a preset number of coordinate points according to the needs of actual positioning, or it can also use uniform sampling, random sampling and normal vector sampling methods to obtain a preset number of coordinate points, etc. The specific selection is based on the actual situation, and the embodiments of the present disclosure are not limited.

[0042] In some embodiments, the preset number is usually greater than 3, that is, the posture determination device selects at least three coordinate points from the scene point cloud map to form a scene coordinate point set. The specific number is selected according to the point cloud density of the scene point cloud map and the actual positioning accuracy requirements, and is not limited in the embodiments of the present disclosure.

[0043] S102: Obtain a model point cloud image corresponding to the scene point cloud image, and select a model coordinate point set corresponding to the scene coordinate point set from the model point cloud image in parallel.

[0044] In the disclosed embodiment, the model point cloud map is a point cloud map corresponding to a pre-stored reference template that matches the scene point cloud map. After obtaining the scene coordinate point set, the posture determination device selects a model coordinate point set corresponding to the scene coordinate point set from the model point cloud map in parallel.

[0045] In the embodiment of the present disclosure, the coordinate system where each scene coordinate point in the scene point cloud map is located and the coordinate system where each model coordinate point in the model point cloud map is located are two different world coordinate systems. Among them, the model coordinate point is a coordinate point included in the model coordinate point set.

[0046] In the disclosed embodiment, in order to make the scene point cloud map and the model point cloud map have a good initial registration value, and avoid the algorithm falling into a local optimum due to unreasonable initial position matching between the scene point cloud map and the model point cloud map, before iteratively calculating the scene point cloud map and the model point cloud map to obtain the spatial transformation relationship, it is usually necessary to perform preliminary image matching on the scene point cloud map and the model point cloud map. Therefore, the posture determination device can pre-collect at least one image of a standard scene as at least one reference template, and generate at least one point cloud map corresponding to the reference template according to the at least one reference template, and store it in the storage unit of the electronic device. In this way, after obtaining the scene point cloud map, the posture determination device can first match the scene point cloud map with the point cloud map of at least one pre-stored reference template. Exemplarily, a neural network is used to perform image matching on the scene point cloud map and the point cloud map of at least one pre-stored reference template through an image feature matching algorithm, and the point cloud map of the reference template whose matching degree with the scene point cloud map is greater than a preset matching threshold is used as the model point cloud map.

[0047] In the disclosed embodiment, the graphics processor may be a GPU (Graphics Processing Unit), which is a microprocessor used to process image and graphics related computing tasks.

[0048] In the disclosed embodiment, the graphics processor can process data in a parallel manner. Parallel selection means that in the graphics processor, for all scene coordinate points pre-selected in the scene point cloud map, the corresponding model coordinate points are searched and determined in the model point cloud map at the same time, thereby obtaining a model coordinate point set corresponding to the scene coordinate point set.

[0049] In the embodiments of the present disclosure, according to each scene coordinate point in the scene coordinate point set, the posture determination device can adopt the nearest neighbor method, the normal direction nearest neighbor method, or the projection method to select the corresponding model coordinate point from the model point cloud map. The specific selection is made according to the actual situation and is not limited by the embodiments of the present disclosure.

[0050] In the embodiment of the present disclosure, by calling the model point cloud map through the graphics processor, the parallel processing capability of the graphics processor can be used to simultaneously search for the model coordinate point corresponding to each scene coordinate point in the scene coordinate point set in the model point cloud map, thereby effectively reducing the burden on the CPU and improving the speed of searching and selecting the model coordinate point set; the parallel selection of the model coordinate point set can also be achieved through other chips or processors with parallel processing capabilities. The specific selection will be made according to the actual situation, and the embodiment of the present disclosure is not limited.

[0051] S103, based on the scene coordinate points and the model coordinate point set, obtain the corresponding spatial transformation relationship between the scene point cloud image and the model point cloud image through parallel iterative calculation.

[0052] In the disclosed embodiment, after obtaining the set of corresponding point pairs, the posture determination device iteratively calculates the spatial position correspondence between the scene point cloud map and the model point cloud map based on the three-dimensional coordinate point pairs contained in the scene coordinate point set and the model coordinate point set, and solves the most accurate position relationship between the scene point cloud map and the model point cloud map as the corresponding spatial transformation relationship between the scene point cloud map and the model point cloud map.

[0053] In the disclosed embodiment, since the spatial position correspondence between the scene point cloud map and the model point cloud map is not clear at the beginning, it is necessary to first estimate the initial spatial transformation relationship between the scene point cloud map and the model point cloud map based on the initially obtained scene coordinate points and model coordinate point sets, and then align the scene point cloud map and the model point cloud map based on the initial spatial transformation relationship to obtain the alignment error, and with the goal of minimizing the alignment error, update the initial spatial transformation relationship through multiple iterative processes to gradually reduce the alignment error, until the iteration termination condition is finally met, and the corresponding spatial transformation relationship between the scene point cloud map and the model point cloud map is obtained.

[0054] In some embodiments, the spatial transformation relationship may be a transformation matrix corresponding to the scene point cloud image and the model point cloud image, and the transformation matrix is ​​used to characterize the positional relationship between the scene point cloud image and the model point cloud image. Through the transformation matrix, the pose determination device can map the point cloud data in the scene point cloud image coordinate system to the model point cloud image coordinate system through transformation methods such as translation and rotation, and then measure the positional relationship between the scene point cloud image and the model point cloud image in the same coordinate system, and perform pose determination based on the positional relationship between the scene point cloud image and the model point cloud image.

[0055] In some embodiments, the posture determination device can iteratively calculate the matrix parameters of a preset transformation matrix model based on the scene coordinate points and the model coordinate point set, and gradually align the scene point cloud map and the model point cloud map through the transformation matrix of the current iteration in each iteration until the alignment error meets the iteration termination condition, thereby obtaining the final transformation matrix that can accurately characterize the corresponding spatial transformation relationship between the scene point cloud map and the model point cloud map.

[0056] In the embodiment of the present disclosure, the posture determination device can iteratively obtain the corresponding transformation matrix between the scene point cloud map and the model point cloud map through an iterative closest point (ICP) algorithm, or other methods can be used, which are not limited in the embodiment of the present disclosure.

[0057] In some embodiments, the posture determination device can perform iterative calculations of spatial transformation relationships in a graphics processor. For example, when each iterative solution of the transformation matrix is ​​performed, the graphics processor can simultaneously calculate the matrix parameters of each position in the transformation matrix to achieve the effect of parallel computing; other chips or processors with parallel processing capabilities can also be used to achieve parallel iterative calculations of spatial transformation relationships. The specific selection is made based on actual conditions, and the embodiments of the present disclosure are not limited.

[0058] S104: Determine the relative position between the scene point cloud image and the model point cloud image according to the spatial transformation relationship.

[0059] In the embodiment of the present disclosure, after obtaining the transformation matrix, the posture determination device can restore the relative posture between the scene point cloud image and the model point cloud image according to the transformation matrix.

[0060] In the embodiment of the present disclosure, since the transformation matrix represents the positional relationship between the scene point cloud map and the model point cloud map, the posture determination device can restore the rotation and translation of the scene point cloud map relative to the model point cloud map through the transformation matrix, thereby determining the relative posture between the scene point cloud map and the model point cloud map.

[0061] In the embodiment of the present disclosure, the method for determining the posture can use the method in the prior art, which will not be described in detail in the embodiment of the present disclosure.

[0062] It can be understood that in the embodiments of the present disclosure, a model coordinate point set corresponding to a scene coordinate point set can be quickly selected through parallel processing, thereby greatly improving the speed of selecting the model coordinate point set; further, when calculating the spatial transformation relationship including multiple parameters such as translation and rotation, each parameter can be calculated iteratively in parallel, thereby overcoming the computational bottleneck of the traditional algorithm that requires serial search and selection of the model coordinate point set corresponding to the scene coordinate point set and serial calculation of the matrix parameters of the transformation matrix, thereby improving the efficiency of determining the posture.

[0063] In the disclosed embodiment, based on Figure 1 In S102, the model coordinate point set corresponding to the scene coordinate point set is selected from the model point cloud in parallel as follows: Figure 2 As shown, including S1021-S1022, as follows:

[0064] S1021. For each scene coordinate point in the scene coordinate point set, a parallel search is performed in the model point cloud map by a graphics processor, and a coordinate point closest to each scene coordinate point is synchronously selected.

[0065] In the disclosed embodiment, for a scene coordinate point in the scene coordinate point set, the posture determination device selects a coordinate point that is closest to the scene coordinate point from the model point cloud map through a graphics processor.

[0066] In the disclosed embodiment, the posture determination device uses a graphics processor to simultaneously search in the model point cloud map according to each scene coordinate point in the scene coordinate point set using the above method, and synchronously obtains the coordinate point closest to each scene coordinate point.

[0067] S1022: taking the coordinate point closest to each scene coordinate point as the model coordinate point corresponding to each scene coordinate point, thereby obtaining a model coordinate point set.

[0068] In the disclosed embodiment, the posture determination device uses the coordinate point closest to each scene coordinate point as the model coordinate point corresponding to each scene coordinate point, and uses the set composed of the model coordinate points corresponding to each scene coordinate point as the model coordinate point set.

[0069] It can be understood that in the embodiments of the present disclosure, the parallel processing characteristics of the graphics processor can be utilized to synchronously search and select the model coordinate point in the model point cloud map for each scene coordinate point in the scene coordinate point set for subsequent point cloud map alignment, thereby overcoming the serial processing method of the traditional method of finding the nearest point for one point and then starting the next point, thereby improving the efficiency of generating the model coordinate point set and the corresponding point pair set, and thereby improving the efficiency of posture determination.

[0070] In the disclosed embodiment, based on Figure 1 In S103, in the graphics processor, based on the scene coordinate point and the model coordinate point set, the corresponding spatial transformation relationship between the scene point cloud and the model point cloud is obtained through parallel iterative calculation. Figure 3 As shown, including S1031-S1034, as follows:

[0071] S1031. In one iteration process, the graphics processor calculates in parallel the matrix parameters in the preset transformation matrix model based on the spatial correspondence between the corresponding point pair set and the preset transformation matrix model to obtain the current transformation matrix; the spatial correspondence is the correspondence between the scene coordinate point set and the model coordinate point set in the corresponding point pair set established by the preset transformation matrix model; the corresponding point pair set is a point pair set consisting of each scene coordinate point in the scene coordinate point set and its corresponding model coordinate point in the model coordinate point set.

[0072] In the embodiment of the present disclosure, after obtaining the corresponding point pair set, the posture determination device can perform parallel calculations on the matrix parameters in the preset transformation matrix model based on the spatial correspondence between the corresponding point pair set and the preset transformation matrix model to obtain the current transformation matrix.

[0073] In the disclosed embodiment, the spatial correspondence relationship is established through a preset transformation matrix model, which is a correspondence relationship between a scene coordinate point set and a model coordinate point set in a corresponding point pair set; the preset transformation matrix model is a constraint relationship model for transforming from a scene point cloud coordinate system to a model point cloud coordinate system, and the matrix parameters in the preset transformation matrix model are unknown. The unknown matrix parameters need to be solved through the three-dimensional coordinates of the scene coordinate points and the model coordinate points of each corresponding point pair in the corresponding point pair set, according to the equation form of the spatial correspondence relationship satisfied by the corresponding point pair set and the preset transformation matrix model.

[0074] In the embodiment of the present disclosure, the posture determination device uses a point pair set consisting of each scene coordinate point in the scene coordinate point set and its corresponding model coordinate point in the model coordinate point set as a corresponding point pair set.

[0075] In the disclosed embodiment, the corresponding point pair set is composed of corresponding point pairs between the scene coordinate point set and the model coordinate point set, and each corresponding point pair includes a scene coordinate point and a corresponding model coordinate point.

[0076] In the disclosed embodiment, the preset transformation matrix model may be a rigid transformation matrix under different coordinate systems. For coordinate points in three-dimensional space, a 4x4 preset transformation matrix model may be constructed. For the matrix parameters of each position in the preset transformation matrix model, the posture determination device may utilize the parallel processing capability of the graphics processor to simultaneously perform parallel calculations on the matrix parameters of each position in the preset transformation matrix model based on the three-dimensional coordinate values ​​of each scene coordinate point in the corresponding point pair set and the corresponding three-dimensional coordinate values ​​of each model coordinate point, so as to simultaneously obtain the values ​​of all the matrix parameters, and substitute the solved matrix parameters into the preset transformation matrix model to obtain the current transformation matrix.

[0077] In the disclosed embodiment, the current transformation matrix represents the spatial transformation relationship between the scene point cloud coordinate system and the model point cloud coordinate system solved according to the corresponding point set.

[0078] In the embodiment of the present disclosure, the preset transformation matrix model includes rotation matrix parameters and translation matrix parameters. Figure 3 In S1031, based on the spatial correspondence between the corresponding point pair set and the preset transformation matrix model, the matrix parameters in the preset transformation matrix model are calculated in parallel to obtain the current transformation matrix. Figure 4 As shown, including S201-S203, as follows:

[0079] S201, approximate the preset transformation matrix model to obtain an approximate transformation matrix model; the approximate processing is to preset the rotation angle of the scene coordinate system corresponding to the model coordinate system to zero; the scene coordinate system is the coordinate system where the scene coordinate point is located, and the model coordinate system is the coordinate system where the model coordinate point is located.

[0080] In the embodiment of the present disclosure, the translation matrix model represents the unit coordinate amount that needs to be translated when aligning the scene point cloud coordinate system with the model point cloud coordinate system. The translation matrix parameters can be constructed by formula (1), as follows:

[0081]

[0082] In formula (1), T(t x ,t y ,t z ) is the translation matrix model in the preset transformation matrix model, t x ,t y ,t z They are the unit coordinates required to translate the x-axis, y-axis, and z-axis in the scene point cloud coordinate system to the model point cloud coordinate system.

[0083] In the embodiment of the present disclosure, the rotation matrix model represents the angle required to rotate when aligning the scene point cloud coordinate system with the model point cloud coordinate system. The rotation matrix model can be constructed by formula (2) and formula (3), as follows:

[0084]

[0085] in,

[0086]

[0087] In formula (3), R(α, β, γ) is the rotation matrix model, and α, β, γ are the rotation angles required to align the x-axis, y-axis, and z-axis in the scene point cloud coordinate system with the model point cloud coordinate system.

[0088] In the embodiment of the present disclosure, the spatial correspondence relationship between the corresponding point pair set and the preset transformation matrix model can be shown as formula (4), as follows:

[0089] d i =R(α,β,γ)·s i +T(t x ,t y ,t z ) (4)

[0090] It can be seen that in the embodiment of the present disclosure, the rotation matrix model R is nonlinear, which means that solving the rotation matrix parameters is essentially a nonlinear least squares problem. In practical applications, since the scene point cloud image and the model point cloud image have been pre-registered, the relative motion between the scene point cloud image and the model point cloud image is usually very small, so the rotation matrix parameters r can be calculated. 11 ,r 12 ,r 13 When , we can approximate the rotation angles α, β, and γ of the scene coordinate system corresponding to the model coordinate system, and consider that α, β, and γ are close to 0. Therefore, based on Figure 4 In S201, the preset transformation matrix model is approximated to obtain an approximate transformation matrix model. Figure 5 As shown, including S2011-S2012, as follows:

[0091] S2011. Approximate the rotation matrix model to obtain an approximate rotation matrix model.

[0092] In the embodiment of the present disclosure, the position and posture determination device performs approximate processing on the rotation matrix model, and considers that α, β, and γ are close to 0, so there is formula (5):

[0093] sinα=α, sinβ=β, sinγ=γ, cosα=1, cosβ=1, cosγ=1 (5)

[0094] Substituting formula (5) into formula (3) and then into formula (2) yields formula (6), as follows:

[0095]

[0096] In formula (6), It is the approximate rotation matrix model after approximate processing.

[0097] S2012. Use the approximate rotation matrix model to update the rotation matrix model in the preset transformation matrix model to obtain an approximate transformation matrix model.

[0098] In the disclosed embodiment, after obtaining the approximate rotation model, the posture determination device uses the approximate rotation model to replace the rotation matrix model in the preset transformation matrix model to obtain an approximate transformation matrix model including the approximate rotation matrix model and the translation matrix model.

[0099] S202, substituting the approximate transformation matrix model into the spatial correspondence relationship to obtain an approximate spatial correspondence relationship satisfied by the corresponding point pair set and the approximate transformation matrix.

[0100] In the disclosed embodiment, after obtaining the approximate rotation matrix model, the nonlinear least squares problem can be approximately converted into a linear least squares problem for solution.

[0101] In the embodiment of the present disclosure, the approximate rotation matrix model in the approximate transformation matrix model is And the translation matrix model T(t x ,t y ,t z ) is substituted into the spatial correspondence of formula (4), and formula (7) is obtained:

[0102] d i =R(α,β,γ)·s i +T(t x ,t y ,t z ) (7)

[0103] Multiply both ends of formula (7) by the normal vector of the model coordinate point and then deform it to obtain formula (8):

[0104] (R(α,β,γ)·s i +T(t x ,t y ,t z ))·n i -d i ·n i =0 (8)

[0105] In formula (8), the model coordinate point d i The three-dimensional coordinates and scene coordinate points s i The three-dimensional coordinates and the normal vector n of the model coordinate point i The three-dimensional vectors are expressed in the form of matrices, as shown in formulas (8-1), (8-2) and (8-3):

[0106]

[0107] Expand formula (8) according to formulas (8-1), (8-2) and (8-3) to obtain formula (9):

[0108]

[0109] By rearranging formula (9), we can obtain the approximate spatial correspondence between each pair of corresponding point pairs in the corresponding point pair set, as shown in formula (10):

[0110] Ax-b=0 (10)

[0111] In formula (10), x is the approximate transformation matrix, where the specific expansions of A, x and b are shown in formulas (10-1), (10-2), (10-3) and (10-4), respectively:

[0112]

[0113] x=(α β γ t x t y t z ) T (10-2)

[0114]

[0115]

[0116] S203: Based on the corresponding point pair set and the approximate spatial correspondence, the matrix parameters in the approximate transformation matrix model are calculated in parallel to obtain the current transformation matrix.

[0117] In the embodiment of the present disclosure, the posture determination device can parallelly calculate the coordinate values ​​and normal vectors of the scene coordinate points and the model coordinate points in each pair of corresponding point pairs in the corresponding point pair set, based on the formula of the approximate spatial correspondence relationship, on the matrix parameters in the approximate transformation matrix model to obtain the current transformation matrix.

[0118] In some embodiments of the present disclosure, based on Figure 4In S203, based on the corresponding point pair set and the approximate spatial correspondence, the matrix parameters in the approximate transformation matrix model are calculated in parallel, and the current transformation matrix can be obtained as follows: Figure 6 As shown, including S2031-S2035, as follows:

[0119] S2031. Obtain scene position information and model position information of each pair of corresponding point pairs in the corresponding point pair set; the scene position information is the position information of the scene coordinate point in each pair of corresponding point pairs; the model position information is the position information of the model coordinate point in each pair of corresponding point pairs.

[0120] In the disclosed embodiment, the posture determination device obtains scene position information and model position information of each pair of corresponding point pairs in the corresponding point pair set, wherein the scene position information is the position information of the scene coordinate points in each pair of corresponding point pairs, including the three-dimensional coordinates of the scene coordinate points in each pair of corresponding point pairs, and the model position information is the position information of the model coordinate points in each pair of corresponding point pairs, including the three-dimensional coordinates and normal vector of the model coordinate points in each pair of corresponding point pairs.

[0121] S2032. Substitute the scene position information and model position information of each pair of corresponding point pairs into the approximate spatial correspondence relationship, and simultaneously calculate the rotation matrix parameters and the translation matrix parameters; the rotation matrix parameters are the parameters in the approximate transformation matrix that represent the rotation angle transformation, and the translation matrix parameters are the parameters in the approximate transformation matrix that represent the translation transformation.

[0122] In the disclosed embodiment, the posture determination device substitutes the three-dimensional coordinates of the scene coordinate points and the three-dimensional coordinates of the model coordinate points and the normal vector of each pair of corresponding points into the formula (10) of the approximate spatial correspondence relationship, and performs synchronous calculations on the matrix parameters of each position in the approximate spatial change matrix in the graphics processor, and simultaneously obtains the rotation matrix parameters and the translation matrix parameters.

[0123] In the embodiment of the present disclosure, the rotation matrix parameters are the parameters in the approximate transformation matrix that characterize the rotation angle transformation, corresponding to α, β and γ in formula (10-2); the translation matrix parameters are the parameters in the approximate transformation matrix that characterize the translation transformation, corresponding to t in formula (10-2). x , t y and t z .

[0124] In the embodiment of the present disclosure, the method of solving quaternions can be used to calculate the translation matrix parameters in the translation matrix model and the rotation matrix parameters in the rotation matrix model. Other calculation methods can also be used. The specific selection is made based on actual conditions and will not be repeated in the embodiment of the present disclosure.

[0125] S2033. Use the rotation matrix parameters to update the approximate rotation matrix model to obtain the current rotation matrix.

[0126] In the disclosed embodiment, the posture determination device substitutes the calculated rotation matrix parameters α, β and γ into the approximate rotation matrix model to obtain the current rotation matrix.

[0127] S2034. Use the translation matrix parameters to update the translation matrix model to obtain the current translation matrix.

[0128] In the embodiment of the present disclosure, the position and posture determination device calculates the translation matrix parameter t x , t y and t z Substitute it into the translation matrix model to get the current translation matrix.

[0129] S2035. Multiply the current rotation matrix and the current translation matrix to obtain the current transformation matrix.

[0130] In the embodiment of the present disclosure, after obtaining the current rotation matrix and the current translation matrix, the posture determination device can multiply the current rotation matrix and the current translation matrix according to formula (11) to obtain the current transformation matrix.

[0131] M=T i (t x ,t y ,t z )·R i (α,β,γ) (11)

[0132] In formula (11), M is the current transformation matrix, T i (t x ,t y ,t z ) is the current rotation matrix, R i (α, β, γ) is the current translation matrix.

[0133] It can be understood that by approximating the rotation matrix model, the nonlinear least squares problem can be converted into a linear least squares problem for solution, which simplifies the calculation of the current transformation matrix and improves the efficiency of posture determination.

[0134] S1032. Perform coordinate transformation on each scene coordinate point in the corresponding point pair set according to the current transformation matrix to obtain an updated scene coordinate point corresponding to each scene coordinate point.

[0135] In the disclosed embodiment, after solving the current transformation matrix based on the corresponding point pair set, in order to verify the accuracy of the current transformation matrix, the posture determination device performs coordinate transformation on each scene coordinate point included in the corresponding point pair set according to the current transformation matrix to obtain the updated scene coordinate point corresponding to each scene coordinate point.

[0136] In the disclosed embodiment, the posture determination device may multiply the matrix composed of the three-dimensional coordinates of each scene coordinate point by the current transformation matrix to obtain an updated scene coordinate point corresponding to each scene coordinate point.

[0137] In the embodiment of the present disclosure, the posture determination device performs coordinate transformation on each scene coordinate point in the corresponding point pair set according to the current transformation matrix, and obtains the updated scene coordinate point corresponding to each scene coordinate point as shown in formula (12):

[0138] s' i =M·s i (12)

[0139] In formula (12), s i is the scene coordinate point, M is the current transformation matrix, and the scene coordinate point s i Substitute the current transformation matrix M into formula (12) and multiply them to get the updated scene coordinate point s' i .

[0140] S1033, performing the next iteration based on the updated scene coordinate point corresponding to each scene coordinate point, until the i-th iteration satisfies the preset iteration condition, and obtaining the i-th transformation matrix; wherein i is a positive integer greater than or equal to 1.

[0141] In the embodiment of the present disclosure, the posture determination device can align the scene point cloud map with the model point cloud map based on the updated scene coordinate points corresponding to each scene coordinate point, and perform the next iteration based on the alignment error until the i-th iteration meets the preset iteration conditions, and obtains the i-th transformation matrix, where i is a positive integer greater than or equal to 1.

[0142] In the disclosed embodiment, based on Figure 4 , S1033 can be Figure 7 As shown, it is implemented through S501-S504 as follows:

[0143] S501. Take the updated scene coordinate point corresponding to each scene coordinate point and the model coordinate point corresponding to each scene coordinate point as a pair of current corresponding point pairs, thereby obtaining a current corresponding point pair set.

[0144] In the disclosed embodiment, for an updated scene coordinate point corresponding to a scene coordinate point, the posture determination device uses a new corresponding point pair formed by the updated scene coordinate point and the model coordinate point corresponding to the scene coordinate point as a current corresponding point pair.

[0145] In the disclosed embodiment, the posture determination device performs the same processing on the updated scene coordinate point corresponding to each scene coordinate point and the model coordinate point corresponding to each scene coordinate point, thereby obtaining a current corresponding point pair set, wherein each pair of current corresponding point pairs in the current corresponding point pair set includes an updated scene coordinate point and a corresponding model coordinate point.

[0146] In some embodiments, the set of corresponding point pairs is {p i ,q i}, where each scene coordinate point p i The corresponding updated coordinate point is p' i , each scene coordinate point p i The corresponding model coordinate point is q i , then the position determination device will {p' i ,q i} as the current corresponding point pair set.

[0147] S502: Calculate the distance between each pair of points in the current corresponding point pair set to obtain the current distance set.

[0148] In the disclosed embodiment, each pair of current corresponding point pairs in the current corresponding point pair set includes an updated scene coordinate point and a corresponding model coordinate point. The posture determination device can calculate the distance between the points in each pair of current corresponding point pairs in the current corresponding point pair set based on the three-dimensional coordinate values ​​of the updated scene coordinate points in each pair of current corresponding point pairs and the three-dimensional coordinate values ​​and normal vectors of the corresponding model coordinate points, thereby taking the set consisting of the distances between the points in each pair of current corresponding point pairs as the current distance set.

[0149] In the embodiments of the present disclosure, the posture determination device can calculate the distance between the points in each pair of current corresponding point pairs in the current corresponding point pair set by a variety of measurement methods. Exemplarily, the posture determination device can calculate the point-to-point distance from the updated scene coordinate point to the corresponding model coordinate point in each pair of current corresponding point pairs, and can also calculate the distance from the tangent plane (point-to-plane) from the updated scene coordinate point to the corresponding model coordinate point, or can calculate the distance from the tangent plane (plane-to-plane) from the tangent plane of the updated scene coordinate point to the corresponding model coordinate point. The specific selection is made according to the actual situation, and the embodiments of the present disclosure are not limited.

[0150] In some embodiments, based on Figure 7 In S502, the distance between each pair of points in the current corresponding point pair set is calculated, and the current distance set can be obtained as follows: Figure 8 As shown, including S301-S302, as follows:

[0151] S301. Calculate the distance from the updated scene coordinate point in each pair of current corresponding point pairs to the tangent plane where the model coordinate point in each pair of current corresponding point pairs is located, and obtain the current distance of each pair of current corresponding point pairs.

[0152] In the embodiment of the present disclosure, a point-to-plane error measurement method may be used to calculate the distance between the updated scene coordinate point and the model coordinate point. For example, Fig. 9 As shown, the current corresponding point pair set includes the current corresponding point pair {(s'1, d1), (s'2, d2), (s'3, d3)}, where the updated scene coordinate point s'1 is located on the source surface, and the model coordinate point d1 is located on the destination surface. For the updated scene coordinate point s'1, the distance l1 from s'1 to the tangent plane of the surface position where d1 is located is calculated as the distance between the current corresponding point pair s'1 and d1. For the current corresponding point pairs (s'2, d2) and (s'3, d3), l2 and l3 are calculated in the same way, and l1, l2 and l3 are respectively used as the current distances corresponding to the current corresponding point pairs (s'1, d1), (s'2, d2), and (s'3, d3).

[0153] In the embodiment of the present disclosure, the distance between the updated scene coordinate point in each pair of current corresponding point pairs and the tangent plane where the model coordinate point in each pair of current corresponding point pairs is located can be calculated in S301 as shown in formula (13):

[0154] l=(s' i -d i )·n i (13)

[0155] In formula (13), d i is the model coordinate point, n i is the normal vector corresponding to the model coordinate point, and the updated scene coordinate point s' is obtained by formula (12) i After that, the scene coordinate point s' will be updated i The difference d with the corresponding model coordinate point i Multiply by the normal vector n of the model coordinate point i , we can calculate the updated scene coordinate point s' i To model coordinate point d i The distance l between the tangent planes.

[0156] S302: taking a set consisting of the current-time distances of each pair of current-time corresponding point pairs as a current-time distance set.

[0157] In the disclosed embodiment, after obtaining the current-time distance of each pair of current-time corresponding point pairs, the posture determination device uses the set consisting of the current-time distance of each pair of current-time corresponding point pairs as the current-time distance set.

[0158] In some embodiments, l1, l2 and l3 are the current distances corresponding to each pair of current corresponding point pairs (s'1, d1), (s'2, d2), (s'3, d3) in the current corresponding point pair set, and {l1, l2, l3} is the current distance set.

[0159] S503: Based on the current distance set, obtain the current distance error between the scene coordinate point set and the model coordinate point set.

[0160] In the disclosed embodiment, after obtaining the current distance set, the posture determination device evaluates the distance between the scene coordinate point and the model coordinate point based on all the current distances in the current distance set to obtain the current distance error between the scene coordinate point set and the model coordinate point set.

[0161] In some embodiments, the posture determination device can perform averaging processing on the current distance set to obtain the current distance error. The averaging method can be to take the average value of all current distances, or to take the sum of the squares of all current distances to measure the variation or deviation of the current distance, or other averaging methods, which are not limited in the embodiments of the present disclosure.

[0162] In some embodiments, the posture determination device can also obtain the current distance error based on the current distance set in other ways according to the needs of the actual application. For example, the maximum value is taken as the current distance error, and so on. The specific selection is made according to the actual situation, and the embodiments of the present disclosure are not limited.

[0163] In the disclosed embodiment, based on Figure 7 In S503, obtaining the current distance error between the scene coordinate point set and the model coordinate point set based on the current distance set may specifically include S401, as follows:

[0164] S401, summing up the squares of each current sub-distance in the current sub-distance set, and using the summation result as the current sub-distance error.

[0165] In the embodiment of the present disclosure, the current distance error can be obtained according to formula (14), as follows:

[0166]

[0167] In formula (14), e is the current distance error corresponding to the current distance set.

[0168] S504. In response to the situation that the current distance error does not meet the preset iteration condition, the current corresponding point pair set is updated based on the updated scene coordinate points corresponding to each scene coordinate point to obtain the next corresponding point pair set, and the next iteration is performed based on the next corresponding point pair set, until the i-th iteration meets the preset iteration condition, and the i-th transformation matrix is ​​obtained.

[0169] In the embodiment of the present disclosure, after obtaining the current distance error, the posture determination device can determine whether the preset iteration condition is met based on the current distance error. If the current distance error does not meet the preset iteration condition, the current corresponding point pair set is updated based on the updated scene coordinate point corresponding to each scene coordinate point to obtain the next corresponding point pair set, and then the next updated corresponding point pair set is iterated again until the distance error of the i-th iteration meets the preset iteration condition, and the i-th transformation matrix is ​​obtained.

[0170] In the disclosed embodiment, the preset iteration condition may be a preset threshold of the current distance error, or an upper limit on the number of iterations may be set. When the number of iterations reaches the upper limit, it is determined that the preset iteration condition is satisfied. The specific selection is made based on actual conditions and is not limited in the disclosed embodiment.

[0171] In the disclosed embodiment, when the current distance error does not meet the preset iteration conditions, it means that the error of the current transformation matrix calculated based on the corresponding point pair set is large and cannot accurately characterize the relationship between the scene coordinate points and the model coordinate points. Therefore, the posture determination device can select the coordinate points corresponding to each updated scene coordinate point in the model point cloud map as the updated model coordinate points on the updated scene coordinate points corresponding to each scene coordinate point obtained based on the current change matrix, and use each updated scene coordinate point and the corresponding updated model coordinate point as the next corresponding point pair set, and start the next iteration based on the next corresponding point pair set, until the i-th distance error corresponding to the i-th iteration meets the preset iteration conditions, then stop the iterative process and obtain the i-th transformation matrix.

[0172] In the disclosed embodiment, since the updated scene coordinate points are obtained by transforming the scene coordinate points through the current transformation matrix, that is, the updated scene coordinate point set formed by the updated scene coordinate points has a higher degree of alignment with the model point cloud map, the posture determination device selects the model coordinate point set in the model point cloud map based on the updated scene coordinate points corresponding to each scene coordinate point and performs the next iteration. The updated scene coordinate point set can be gradually aligned with the model coordinate point set to reduce the current distance error between the two. Therefore, when the i-th distance error calculated in the i-th iteration is the minimum distance error obtained in all i-iteration processes, the posture determination device obtains the i-th transformation matrix according to the matrix parameters of the preset matrix model calculated in the i-th iteration.

[0173] In some embodiments, the posture determination device may also take minimizing the current distance error in each iteration process as the iteration goal in each iteration process, and after all i iteration processes are completed, find the minimum distance error among all i distance errors obtained.

[0174] In the embodiment of the present disclosure, the posture determination device can also use formula (15) as the optimization function to optimize the distance error of the next iteration in each iteration process, and finally obtain the minimum distance error in all i iteration processes in the i-th iteration that meets the preset iteration conditions:

[0175]

[0176] In formula (15), M opt The optimal transformation matrix that minimizes the i-th distance error is also the transformation matrix ultimately required by the iterative process.

[0177] In the embodiment of the present disclosure, since the current transformation matrix model is calculated by approximating the preset transformation matrix model, the formula (10) is used to transform the formula (15), and the M in the formula (15) can be opt The solution of is converted into the solution of x in formula (16) opt To solve, x opt The transformation matrix required at the end of the iterative process is:

[0178] x opt =argmin x |Ax-b| 2 (16)

[0179] Perform pseudo inverse transformation on formula (16) to obtain formula (17), as follows:

[0180] x opt =A + b (17)

[0181] Perform SVD decomposition on formula (17) to obtain the corresponding transformation matrix x between the scene point cloud image and the model point cloud image: opt .

[0182] S1034. Use the i-th transformation matrix as the spatial transformation relationship.

[0183] In the disclosed embodiment, the minimum i-th distance error indicates that the i-th transformation matrix solved during the i-th iteration best conforms to the spatial position transformation relationship between the scene point cloud image and the model point cloud image. Therefore, the i-th matrix corresponding to the i-th distance error obtained during the iteration is used as the spatial transformation relationship.

[0184] It can be understood that in the embodiment of the present disclosure, during each iteration, the matrix parameters of each position in the preset transformation matrix model are calculated in parallel simultaneously through the physical characteristics of parallel processing of the graphics processor, thereby avoiding the one-to-one calculation of the matrix parameters of each position in the preset transformation matrix model, speeding up the solution of the current transformation matrix, and quickly obtaining the spatial transformation relationship between the scene coordinate point set and the model coordinate point set, thereby ultimately improving the efficiency of determining the posture.

[0185] In the disclosed embodiment, based on Figure 1 , after S104, you can also Fig.10 As shown, including S105, as follows:

[0186] S105: Based on the relative position and posture, perform scene positioning, three-dimensional modeling and / or virtual reality simulation process.

[0187] In the disclosed embodiment, the posture determination device determines the relative posture between the scenic spot cloud map and the model point cloud map, and can perform various graphic visualization processing based on the relative posture. For example, the posture determination device can obtain the rotation and translation of the scene point cloud map relative to the model point cloud map through the relative posture, and then determine the position of each scene coordinate point in the scene point cloud map in the coordinate system of the model point cloud map, and finally realize scene positioning.

[0188] In some embodiments, the posture determination device can also implement processes such as three-dimensional modeling and virtual reality simulation based on the determined relative position. The specific application process of the relative posture is not limited in the embodiments of the present disclosure.

[0189] The present disclosure provides a posture determination device, corresponding to a posture determination method, Fig.11 A schematic diagram of the structure of a posture determination device provided in an embodiment of the present disclosure is shown in FIG. Fig.11 As shown, the posture determination device 1 includes: an extraction unit 100, a selection unit 200, an iteration unit 300 and a determination unit 400, wherein:

[0190] The extraction unit 100 is used to generate a scene point cloud map according to the real scene image, and extract a scene coordinate point set from the scene point cloud map;

[0191] The selection unit 200 is used to obtain a model point cloud image corresponding to the scene point cloud image, and to select a model coordinate point set corresponding to the scene coordinate point set from the model point cloud image in parallel;

[0192] The iteration unit 300 is used to obtain the corresponding spatial transformation relationship between the scene point cloud image and the model point cloud image through parallel iterative calculation based on the scene coordinate points and the model coordinate point set;

[0193] The determining unit 400 is used to determine the relative pose between the scene point cloud image and the model point cloud image according to the spatial transformation relationship.

[0194] In some embodiments, the iteration unit 300 is also used to, during an iteration process, perform parallel calculations on matrix parameters in the preset transformation matrix model based on the spatial correspondence between the corresponding point pair set and the preset transformation matrix model through a graphics processor to obtain a current transformation matrix; the spatial correspondence is a correspondence between the scene coordinate point set and the model coordinate point set in the corresponding point pair set established through the preset transformation matrix model; the corresponding point pair set is a point pair set consisting of each scene coordinate point in the scene coordinate point set and its corresponding model coordinate point in the model coordinate point set; according to the current transformation matrix, perform coordinate transformation on each scene coordinate point in the corresponding point pair set to obtain an updated scene coordinate point corresponding to each scene coordinate point; perform the next iteration based on the updated scene coordinate point corresponding to each scene coordinate point, until the i-th iteration satisfies the preset iteration condition, and obtains the i-th transformation matrix; wherein i is a positive integer greater than or equal to 1; and use the i-th transformation matrix as the spatial transformation relationship.

[0195] In some embodiments, the iteration unit 300 is further used to take the updated scene coordinate point corresponding to each scene coordinate point and the model coordinate point corresponding to each scene coordinate point as a pair of current corresponding point pairs, thereby obtaining a current corresponding point pair set; calculate the distance between the points in each pair of current corresponding point pairs in the current corresponding point pair set to obtain a current distance set; based on the current distance set, obtain the current distance error between the scene coordinate point set and the model coordinate point set; in response to the current distance error not satisfying a preset iteration condition, update the current corresponding point pair set based on the updated scene coordinate point corresponding to each scene coordinate point to obtain a next corresponding point pair set, and perform the next iteration based on the next corresponding point pair set, until the i-th iteration satisfies the preset iteration condition, and obtains the i-th transformation matrix.

[0196] In some embodiments, the selection unit 200 is further configured to perform a parallel search in the model point cloud map for each scene coordinate point in the scene coordinate point set through a graphics processor, and synchronously select a coordinate point that is closest to each scene coordinate point;

[0197] The coordinate point closest to each scene coordinate point is used as the model coordinate point corresponding to each scene coordinate point, thereby obtaining the model coordinate point set.

[0198] In some embodiments, the iteration unit 300 is also used to calculate the distance from the updated scene coordinate point in each pair of current-time corresponding point pairs to the tangent plane where the model coordinate point in each pair of current-time corresponding point pairs is located, to obtain the current-time distance of each pair of current-time corresponding point pairs; and use the set consisting of the current-time distances of each pair of current-time corresponding point pairs as the current-time distance set.

[0199] In some embodiments, the iteration unit 300 is further configured to sum the squares of each current sub-distance in the current sub-distance set, and use the summation result as the current sub-distance error.

[0200] In some embodiments, the iteration unit 300 is also used to perform approximate processing on the preset transformation matrix model to obtain an approximate transformation matrix model; the approximate processing is to preset the rotation angle of the scene coordinate system corresponding to the model coordinate system to zero; the scene coordinate system is the coordinate system where the scene coordinate point is located, and the model coordinate system is the coordinate system where the model coordinate point is located; the approximate transformation matrix model is substituted into the spatial correspondence relationship to obtain the approximate spatial correspondence relationship satisfied by the corresponding point pair set and the approximate transformation matrix; based on the corresponding point pair set and the approximate spatial correspondence relationship, the matrix parameters in the approximate transformation matrix model are calculated in parallel to obtain the current transformation matrix.

[0201] In some embodiments, the preset transformation matrix model includes a rotation matrix model and a translation matrix model, and the iteration unit 300 is also used to approximate the rotation matrix model to obtain an approximate rotation matrix model; and use the approximate rotation matrix model to update the rotation matrix model in the preset transformation matrix model to obtain an approximate transformation matrix model.

[0202] In some embodiments, the iteration unit 300 is also used to obtain scene position information and model position information of each pair of corresponding point pairs in the corresponding point pair set; the scene position information is the position information of the scene coordinate points in each pair of corresponding point pairs; the model position information is the position information of the model coordinate points in each pair of corresponding point pairs; the scene position information and the model position information of each pair of corresponding point pairs are respectively substituted into the approximate spatial correspondence relationship, and the rotation matrix parameters and the translation matrix parameters are simultaneously calculated; the rotation matrix parameters are the parameters in the approximate transformation matrix that characterize the rotation angle transformation, and the translation matrix parameters are the parameters in the approximate transformation matrix that characterize the translation transformation; the rotation matrix parameters are used to update the rotation matrix model to obtain the current rotation matrix; the translation matrix parameters are used to update the translation matrix model to obtain the current translation matrix; the current rotation matrix is ​​multiplied by the current translation matrix to obtain the current transformation matrix.

[0203] In some embodiments, the iteration unit 300 is also used to, if the current distance error does not meet the preset iteration conditions, select in parallel from the model point cloud map, through the graphics processor, the updated model coordinate points corresponding to the updated scene coordinate points of each scene coordinate point, and use the set consisting of each updated scene coordinate point and each updated model coordinate point as the next corresponding point pair set, and perform the next iteration based on the next corresponding point pair set to obtain the next distance error corresponding to the next iteration, until the i-th distance error corresponding to the i-th iteration meets the preset iteration conditions, and obtain the i-th transformation matrix.

[0204] In some embodiments, the iteration unit 300 is also used to multiply the difference between an updated scene coordinate point in a pair of current corresponding point pairs and a model coordinate point in the current corresponding point pair by the normal vector of the model coordinate point to obtain the distance between the current corresponding point pairs, thereby obtaining the current distance of each pair of current corresponding point pairs.

[0205] In some embodiments, the posture determination device 1 further includes a positioning unit, and the positioning unit is used to perform scene positioning, three-dimensional modeling and / or virtual reality simulation processes based on the relative posture.

[0206] It should be noted that, in actual applications, the above-mentioned extraction unit 100, selection unit 200, iteration unit 300 and determination unit 400 can be implemented by a processor 54 located on the electronic device 2, specifically a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA).

[0207] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0208] The disclosed embodiment provides an electronic device corresponding to a method for determining a posture; Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown in FIG. Fig.12As shown, the electronic device 2 includes: a processor 54, a memory 55 and a communication bus 56. The memory 55 communicates with the processor 54 via the communication bus 56. The memory 55 stores one or more programs executable by the processor 54. When one or more programs are executed, any one of the posture determination methods in the aforementioned embodiments is executed by the processor 54.

[0209] An embodiment of the present disclosure provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors 54. When the programs are executed by the processor 54, the posture determination method of the embodiment of the present disclosure is implemented.

[0210] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0211] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0212] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0214] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the protection scope of the present disclosure.

Claims

1. A method for determining a posture, characterized in that: include: Generate a scene point cloud map according to the real scene image, and extract a scene coordinate point set from the scene point cloud map; Acquire a model point cloud image corresponding to the scene point cloud image, and select a model coordinate point set corresponding to the scene coordinate point set from the model point cloud image in parallel; Based on the scene coordinate point set and the model coordinate point set, a corresponding spatial transformation relationship between the scene point cloud image and the model point cloud image is obtained through parallel iterative calculation; Determining the relative position between the scene point cloud image and the model point cloud image according to the spatial transformation relationship; The step of obtaining the corresponding spatial transformation relationship between the scene point cloud image and the model point cloud image through parallel iterative calculation based on the scene coordinate point set and the model coordinate point set includes: In one iteration, based on the spatial correspondence between the corresponding point pair set and the preset transformation matrix model, the scene coordinate system of the preset transformation matrix corresponds to the model coordinate system with a rotation angle of zero; the matrix parameters in the preset transformation matrix model are calculated in parallel to obtain the current transformation matrix; the spatial correspondence is the correspondence between the scene coordinate point set and the model coordinate point set in the corresponding point pair set established by the preset transformation matrix model; the corresponding point pair set is a point pair set consisting of each scene coordinate point in the scene coordinate point set and its corresponding model coordinate point in the model coordinate point set; According to the current transformation matrix, coordinate transformation is performed on each scene coordinate point in the corresponding point pair set to obtain an updated scene coordinate point corresponding to each scene coordinate point; Perform the next iteration based on the updated scene coordinate point corresponding to each scene coordinate point, until the i-th iteration satisfies the preset iteration condition, and obtains the i-th transformation matrix; wherein i is a positive integer greater than or equal to 1; The i-th transformation matrix is ​​used as the spatial transformation relationship.

2. The method according to claim 1, characterized in that The step of performing the next iteration based on the updated scene coordinate point corresponding to each scene coordinate point until the i-th iteration satisfies the preset iteration condition, and using the i-th transformation matrix as the spatial transformation relationship includes: Taking the updated scene coordinate point corresponding to each scene coordinate point and the model coordinate point corresponding to each scene coordinate point as a pair of current corresponding point pairs, thereby obtaining a current corresponding point pair set; Calculate the distance between each pair of points in the current corresponding point pair set to obtain a current distance set; Based on the current distance set, obtaining a current distance error between the scene coordinate point set and the model coordinate point set; In response to the situation that the current distance error does not satisfy the preset iteration condition, the current corresponding point pair set is updated based on the updated scene coordinate points corresponding to each scene coordinate point to obtain the next corresponding point pair set, and the next iteration is performed based on the next corresponding point pair set until the i-th iteration satisfies the preset iteration condition, thereby obtaining the i-th transformation matrix.

3. The method according to claim 1, characterized in that The step of selecting a model coordinate point set corresponding to the scene coordinate point set from the model point cloud image in parallel includes: For each scene coordinate point in the scene coordinate point set, a graphics processor is used to perform a parallel search in the model point cloud map, and a coordinate point closest to each scene coordinate point is synchronously selected; The coordinate point closest to each scene coordinate point is used as the model coordinate point corresponding to each scene coordinate point, thereby obtaining the model coordinate point set.

4. The method according to claim 2, characterized in that: The step of calculating the distance between each pair of points in the current corresponding point pair set to obtain the current distance set includes: Calculate the distance between the updated scene coordinate point in each pair of current corresponding point pairs and the tangent plane where the model coordinate point in each pair of current corresponding point pairs is located, and obtain the current distance of each pair of current corresponding point pairs; The set consisting of the current-time distances of each pair of current-time corresponding point pairs is used as the current-time distance set.

5. The method according to claim 2 or 4, wherein obtaining the current distance error between the scene coordinate point set and the model coordinate point set based on the current distance set comprises: The squares of each current sub-distance in the current sub-distance set are summed, and the sum result is used as the current sub-distance error.

6. The method according to claim 1, characterized in that The step of performing parallel calculation on matrix parameters in the preset transformation matrix model based on the spatial correspondence between the corresponding point pair set and the preset transformation matrix model to obtain the current transformation matrix includes: The preset transformation matrix model is approximated to obtain an approximate transformation matrix model; the approximate processing is to preset the rotation angle of the scene coordinate system corresponding to the model coordinate system to zero; the scene coordinate system is the coordinate system where the scene coordinate point is located, and the model coordinate system is the coordinate system where the model coordinate point is located; Substituting the approximate transformation matrix model into the spatial correspondence relationship to obtain an approximate spatial correspondence relationship satisfied by the corresponding point pair set and the approximate transformation matrix; Based on the corresponding point pair set and the approximate spatial correspondence relationship, matrix parameters in the approximate transformation matrix model are calculated in parallel to obtain the current transformation matrix.

7. The method according to claim 6, characterized in that The preset transformation matrix model includes: a rotation matrix model and a translation matrix model. The approximate processing of the preset transformation matrix model to obtain an approximate transformation matrix model includes: Approximately processing the rotation matrix model to obtain an approximate rotation matrix model; The approximate rotation matrix model is used to update the rotation matrix model in the preset transformation matrix model to obtain an approximate transformation matrix model.

8. The method according to claim 6, characterized in that The step of performing parallel calculation on matrix parameters in the approximate transformation matrix model based on the corresponding point pair set and the approximate spatial corresponding relationship to obtain a current transformation matrix includes: Obtaining scene position information and model position information of each pair of corresponding point pairs in the corresponding point pair set; the scene position information is the position information of the scene coordinate points in each pair of corresponding point pairs; the model position information is the position information of the model coordinate points in each pair of corresponding point pairs; Substituting the scene position information and the model position information of each pair of corresponding point pairs into the approximate spatial correspondence relationship, respectively, and synchronously calculating the rotation matrix parameters and the translation matrix parameters; the rotation matrix parameters are the parameters in the approximate transformation matrix that characterize the rotation angle transformation, and the translation matrix parameters are the parameters in the approximate transformation matrix that characterize the translation transformation; The rotation matrix model is updated using the rotation matrix parameters to obtain the current rotation matrix; The translation matrix model is updated using the translation matrix parameters to obtain a current translation matrix; The current rotation matrix is ​​multiplied by the current translation matrix to obtain the current transformation matrix.

9. The method according to claim 2, characterized in that: In response to the current distance error not satisfying the preset iteration condition, updating the current corresponding point pair set based on the updated scene coordinate point of each scene coordinate point to obtain the next corresponding point pair set, and performing the next iteration based on the next corresponding point pair set until the i-th iteration satisfies the preset iteration condition, thereby obtaining the i-th transformation matrix, including: If the current distance error does not meet the preset iteration condition, the graphics processor selects updated model coordinate points corresponding to the updated scene coordinate points of each scene coordinate point in parallel from the model point cloud image, and takes the set consisting of each updated scene coordinate point and each updated model coordinate point as the next corresponding point pair set, and performs the next iteration based on the next corresponding point pair set to obtain the next distance error corresponding to the next iteration, until the i-th distance error corresponding to the i-th iteration meets the preset iteration condition, and the i-th transformation matrix is ​​obtained.

10. The method according to claim 4, characterized in that The calculating the distance from the updated scene coordinate point in each pair of current corresponding point pairs to the tangent plane where the model coordinate point in each pair of current corresponding point pairs is located to obtain the current distance of each pair of current corresponding point pairs includes: For a pair of current-time corresponding point pairs, the difference between the updated scene coordinate point in the current-time corresponding point pair and the model coordinate point in the current-time corresponding point pair is multiplied by the normal vector of the model coordinate point to obtain the distance between the current-time corresponding point pairs, thereby obtaining the current-time distance of each pair of current-time corresponding point pairs.

11. The method according to any one of claims 1 to 10, characterized in that: After determining the relative position and posture between the scene point cloud image and the model point cloud image according to the spatial transformation relationship, the method further includes: Based on the relative position and posture, scene positioning, three-dimensional modeling and / or virtual reality simulation processes are performed.

12. A posture determination device, characterized in that: The posture determination device comprises: an extraction unit, a selection unit, an iteration unit and a determination unit, wherein: The extraction unit is used to generate a scene point cloud map according to the real scene image, and extract a scene coordinate point set from the scene point cloud map; The selection unit is used to obtain a model point cloud image corresponding to the scene point cloud image, and to select a model coordinate point set corresponding to the scene coordinate point set from the model point cloud image in parallel; The iteration unit is used to obtain the corresponding spatial transformation relationship between the scene point cloud image and the model point cloud image through parallel iterative calculation based on the scene coordinate point set and the model coordinate point set; it is also used to, in one iteration process, through a graphics processor, based on the spatial correspondence satisfied by the corresponding point pair set and the preset transformation matrix model, the scene coordinate system of the preset transformation matrix corresponds to the model coordinate system with a rotation angle of zero; parallel calculation is performed on the matrix parameters in the preset transformation matrix model to obtain the current transformation matrix; the spatial correspondence is established through the preset transformation matrix model, and the scene coordinate point set in the corresponding point pair set and a model coordinate point set; the corresponding point pair set is a point pair set consisting of each scene coordinate point in the scene coordinate point set and its corresponding model coordinate point in the model coordinate point set; according to the current transformation matrix, coordinate transformation is performed on each scene coordinate point in the corresponding point pair set to obtain an updated scene coordinate point corresponding to each scene coordinate point; the next iteration is performed based on the updated scene coordinate point corresponding to each scene coordinate point, until the i-th iteration satisfies the preset iteration condition, and the i-th transformation matrix is ​​obtained; wherein i is a positive integer greater than or equal to 1; the i-th transformation matrix is ​​used as the spatial transformation relationship; The determining unit is used to determine the relative pose between the scene point cloud image and the model point cloud image according to the spatial transformation relationship.

13. An electronic device, characterized in that: The electronic device includes: a processor, a memory and a communication bus, the memory communicates with the processor via the communication bus, the memory stores one or more programs executable by the processor, and when the one or more programs are executed, the processor executes the method according to any one of claims 1-11.

14. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method according to any one of claims 1-11.

Images