Three-dimensional point cloud registration method and device based on feature descriptor and storage medium

Abstract

The present application relates to a three-dimensional point cloud registration method and device based on feature descriptors and a storage medium, applied to the technical field of dangerous chemical warehouse safety distance monitoring, comprising: using gray difference to optimize Harris corner points and fuse with SURF feature points to perform feature extraction and matching on an RGB image, improving the feature information extraction capability of weak texture areas, mapping the matched two-dimensional feature points to three-dimensional to construct H-SURF feature descriptors for point cloud coarse registration, and selecting the neighborhood point cloud centered on the feature point with the minimum error in the H-SURF feature descriptor as a feature region of interest as the input of an ICP algorithm for fine registration, solving the problem that the ICP algorithm and its related improved algorithms are prone to local optimal solution when the overlap degree of the view is too low, and improving the algorithm efficiency and accuracy, and using the feature region of interest as the input also solves the problem of large data processing amount and long time consumption of the ICP algorithm in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of dangerous goods warehouse safety distance monitoring, and particularly relates to a three-dimensional point cloud registration method and device based on a feature descriptor and a storage medium. BACKGROUND

[0002] A dangerous goods warehouse has a large range and a complex environment, and it is difficult for a traditional method to timely and accurately warn potential safety problems. "Five distances" are the key to dangerous goods warehouse safety monitoring. Point cloud is a collection of data points containing target three-dimensional space information, which can provide better environmental perception for dangerous goods warehouse safety monitoring. The point cloud information collected at a single position is less and the phenomenon of mutual occlusion of objects is prone to occur. The three-dimensional information of a target in a larger range can be obtained by registering and splicing point clouds collected from two positions. Therefore, a registration method for point clouds from two positions is a key in large scene three-dimensional reconstruction.

[0003] The core of point cloud registration from two positions is to find an accurate corresponding point set and calculate a corresponding rotation and translation relationship. The classic ICP (Iterative Closest Point) algorithm proposed by Besl has high registration accuracy and is widely used in point cloud registration. However, the algorithm has low timeliness, is too dependent on the initial pose of point cloud, and is prone to fall into a local optimal solution when the overlap rate of two pieces of point cloud is less than 70%. In addition, the existing three-dimensional feature extraction method needs to process a large amount of point cloud data, which will cause a large cumulative error due to too few overlapping point sets of two pieces of point cloud when the point cloud size is large, and the algorithm takes too long. SUMMARY

[0004] Therefore, the present application aims to provide a three-dimensional point cloud registration method and device based on a feature descriptor and a storage medium to solve the problems that the existing ICP algorithm is too dependent on the initial pose of point cloud and is prone to fall into a local optimal solution when the overlap rate of two pieces of point cloud is less than 70%, and the existing three-dimensional feature extraction method needs to process a large amount of point cloud data, which will cause a large cumulative error due to too few overlapping point sets of two pieces of point cloud when the point cloud size is large, and the algorithm takes too long.

[0005] According to a first aspect of an embodiment of the present application, a three-dimensional point cloud registration method based on a feature descriptor is provided, and the method comprises the following steps.

[0006] An RGB image, a depth image, and a point cloud image under double visual angles of a warehouse space are acquired by a depth camera;

[0007] After the RGB image is grayed, the gray difference between each pixel point in the RGB image and its field pixel point is obtained, and a candidate corner point is obtained by setting a gray difference threshold;

[0008] The Hessian matrix is introduced to calculate the shape descriptor of each candidate corner point, and the candidate corner points are further screened by setting a shape descriptor threshold to obtain the target corner point.

[0009] SURF feature points are extracted on the RGB image by the SURF algorithm, and the SURF feature points are combined with the target corner point to obtain H-SURF feature descriptors;

[0010] The H-SURF feature descriptors under the two views are paired, and the two-dimensional H-SURF feature descriptor pairs after successful pairing are mapped to a three-dimensional space to obtain two pieces of three-dimensional point clouds.

[0011] An initial rotation matrix R and a translation matrix T are obtained by coarsely registering the two pieces of three-dimensional point clouds. R t ;

[0012] The two pieces of three-dimensional point clouds are transformed into the coordinate system of the other piece of three-dimensional point cloud by the initial rotation matrix R and the translation matrix T. R t The two pieces of three-dimensional point clouds are transformed into the coordinate system of the other piece of three-dimensional point cloud by the initial rotation matrix R and the translation matrix T.

[0013] Preferably,

[0014] The initial rotation matrix R and the translation matrix T are obtained by coarsely registering the two pieces of three-dimensional point clouds. R t including:

[0015] The H-SURF feature descriptors in the two pieces of three-dimensional point clouds are denoted as sets R and T, respectively. X P The barycenters of each H-SURF feature descriptor in the sets R and T are calculated to obtain two new sets, i.e. two new pieces of three-dimensional point clouds. X P The matrix R*T is obtained by multiplying the two new pieces of three-dimensional point clouds. W The matrix R*T is decomposed by SVD to obtain the initial rotation matrix R and the translation matrix T. W R t .

[0016] Preferably,

[0017] ​​​​​​​The step of using two regions of interest as input to the ICP algorithm for fine registration, and then transforming the point cloud map under the dual-view perspective using the output of the ICP algorithm to obtain the target point cloud map of the warehouse space includes:

[0018] The two regions of interest are respectively used as target point clouds. P and Source Cloud Q ;

[0019] In the target point cloud P Take the middle point set pi ∈ P And find the source point cloud Q The corresponding point set in qi ∈ Q , making ||qi-pi|| Minimum;

[0020] Based on point set pi Point set qi Initial rotation matrix R and translation matrix t Calculate the error function F If the error value is less than or equal to a preset error threshold, then the initial rotation matrix... R and translation matrix t For the optimal rotation matrix R b and translation matrix t b ;

[0021] If the error value is greater than the preset error threshold, then... pi Using the initial rotation matrix R Translation matrix t Perform rotation and translation transformations to obtain a new target point cloud. P 1;

[0022] Obtain new target point cloud P The point set of 1 pi 1. And identify the source point cloud. Q The corresponding point set in qi ∈ Q , making ||qi 1 - pi|| Minimum;

[0023] New target point cloud P 1 and Source Cloud Q The feature points in the data are denoted as sets. X 1 and sets P 1. Calculate the sets respectively X 1 and sets Pthe center of gravity of each feature point in the first set of points, to obtain two new sets, and multiplying the two new sets to obtain a matrix W 1, performing SVD decomposition on the matrix W 1 to obtain a new rotation matrix R 1 and a translation matrix t 1;

[0024] based on the set of points pi 1, the set of points qi、 a new rotation matrix R 1 and a translation matrix t 1 to calculate an error value of an error function F , if the error value is less than or equal to a preset error threshold, the new rotation matrix R 1 and the translation matrix t 1 are the best rotation matrix R b and the translation matrix t b ;

[0025] If the error value is greater than the preset error threshold, repeat the above steps until the best rotation matrix R b and the translation matrix t b are obtained.

[0026] Through the best rotation matrix R b and the translation matrix t b , one of the point cloud graphs in the two views is transformed into the coordinate system of the other point cloud graph, and a target point cloud graph of the storage space is obtained.

[0027] Preferably,

[0028] After the RGB graph is grayed, the gray difference between each pixel point in the RGB graph and its domain pixel point is obtained, and a candidate corner point is obtained by setting a gray difference threshold, including:

[0029] By calculating the gray difference between each pixel point and its 8-neighborhood pixel point after the RGB graph is grayed, 8 gray differences are obtained.

[0030] The size of each gray difference and a preset gray difference threshold is judged respectively, if a certain gray difference is less than the preset gray difference threshold, it is considered that the pixel point is similar to a certain neighborhood pixel point in the periphery; if a certain gray difference is greater than or equal to the preset gray difference threshold, it is considered that the pixel point is not similar to a certain neighborhood in the periphery, the number of similar pixel points of a certain pixel point and 8-neighborhood pixel points is obtained, if the number of similar pixel points meets a preset number, the pixel point is taken as a candidate corner point.

[0031] Preferably,

[0032] The introduced Hessian matrix calculates the shape descriptor of each candidate corner point, further screens the candidate corner points by setting a shape descriptor threshold, and obtains the target corner point, including:

[0033] respectively calculate the second-order partial derivatives of the image gray function I ( x , y ) to x and y , wherein x and y are the coordinates of the candidate corner point;

[0034] According to the two eigenvalues of the second-order Hessian matrix solved from the second-order partial derivatives of each candidate corner point, the shape descriptor is obtained by multiplying the two eigenvalues, and the shape descriptor of each candidate corner point is calculated respectively;

[0035] The shape descriptor of each candidate corner point is compared with the preset shape descriptor threshold respectively, and if the shape descriptor of the candidate corner point is less than the preset shape descriptor threshold, the candidate corner point is taken as the target corner point.

[0036] Preferably,

[0037] The SURF feature points are combined with the target corner points to obtain H-SURF feature descriptors, including:

[0038] The target corner points and the SURF feature points are combined to form a new feature point set A ;

[0039] Any target corner point or SURF feature point in the feature point set A is taken as the center, and a fixed range is obtained through a preset radius;

[0040] The fixed range is equally divided into a plurality of sub-blocks, the haar wavelet response values of all pixel points in each sub-block range along the X direction and the Y direction are calculated and summed to obtain the H-SURF feature descriptor of any target corner point or SURF feature point in the feature point set A ;

[0041] The H-SURF feature descriptors of all target corner points or SURF feature points in the feature point set A are calculated respectively.

[0042] Preferably,

[0043] The H-SURF feature descriptors under the double-view angle are paired, and the two-dimensional H-SURF feature descriptor pair after successful pairing is mapped to a three-dimensional space to obtain two pieces of three-dimensional point clouds, including:

[0044] calculating the Euclidean distance of the H-SURF feature descriptor under the dual view, judging the similarity of the feature points under the dual view through the Euclidean distance, and obtaining the matched feature point pair by setting the Euclidean distance threshold;

[0045] screening the feature point pair through the RANSAC algorithm to obtain a target feature point pair;

[0046] mapping the two-dimensional target feature point pair to a three-dimensional space through the depth value of each on the depth map and the parameter of the depth camera to obtain two pieces of three-dimensional point clouds;

[0047] the mapping of the two-dimensional target feature point pair to the three-dimensional space through the depth value of each on the depth map and the parameter of the depth camera to obtain two pieces of three-dimensional point clouds comprises:

[0048] obtaining the intrinsic matrix of the depth camera through the camera calibration technology;

[0049] obtaining the depth value of the target feature point pair on the respective depth map through the depth map;

[0050] mapping the two-dimensional target feature point pair to a three-dimensional space based on the depth value of each feature point and the intrinsic matrix of the depth camera to obtain two pieces of three-dimensional point clouds.

[0051] According to the second aspect of the embodiment of the application, a three-dimensional point cloud registration device based on a feature descriptor is provided, and the device comprises:

[0052] an image acquisition module: configured to acquire an RGB image, a depth map and a point cloud map under a dual view of a storage space through a depth camera;

[0053] a candidate corner point acquisition module: configured to acquire the gray difference between each pixel point and its field pixel point in the RGB image after the RGB image is grayed, and obtain a candidate corner point by setting a gray difference threshold;

[0054] a target corner point acquisition module: configured to introduce a Hessian matrix to calculate the shape descriptor of each candidate corner point, further screen the candidate corner point by setting a shape descriptor threshold, and obtain a target corner point;

[0055] a fusion module: configured to extract a SURF feature point on the RGB image through a SURF algorithm, combine the SURF feature point with the target corner point, and obtain an H-SURF feature descriptor;

[0056] a three-dimensional mapping module: configured to pair the H-SURF feature descriptor under the dual view, and map the two-dimensional H-SURF feature descriptor pair after the pairing to a three-dimensional space to obtain two pieces of three-dimensional point clouds;

[0057] coarse registration module: for obtaining an initial rotation matrix by coarsely registering the two pieces of three-dimensional point clouds R and a translation matrix t ;

[0058] fine registration module: for obtaining a fine registration matrix by the initial rotation matrix R and a translation matrix t transforming one piece of three-dimensional point clouds in the two pieces of three-dimensional point clouds into the coordinate system of the other piece of three-dimensional point clouds, selecting a pair of three-dimensional H-SURF feature descriptors with the smallest error in the two pieces of three-dimensional point clouds, obtaining a feature region of interest of the two pieces of three-dimensional point clouds with the point as the center and a certain distance as the radius, taking the two feature regions of interest as the input of the ICP algorithm to perform fine registration, and transforming the point cloud graphs under the double perspectives through the output of the ICP algorithm to obtain the target point cloud graph of the storage space.

[0059] According to a third aspect of the embodiments of the present application, a storage medium is provided, which stores a computer program, and when the computer program is executed by a host, each step of the above method is implemented.

[0060] The technical solutions provided by the embodiments of the present application can include the following beneficial effects:

[0061] The present application improves the feature information extraction capability of weak texture areas by using gray difference optimized Harris corner points and fusing SURF feature points to perform feature extraction and matching on the RGB image in the point cloud coarse registration stage, maps the matched feature points to three-dimensional through the depth map and camera parameters, constructs H-SURF feature descriptors, performs point cloud coarse registration, selects the neighborhood point cloud of the feature point with the smallest error of the H-SURF feature descriptor as the feature region of interest, takes it as the input of the ICP algorithm to perform fine registration, solves the problem that the ICP algorithm and its related improved algorithms are easy to fall into local optimal solution when the perspective coincidence degree is too low, improves the algorithm efficiency and accuracy, and at the same time, only part of the entire three-dimensional region, that is, the feature region of interest is fine registered, which solves the problem that the entire three-dimensional point cloud is taken as the input of the ICP algorithm in the prior art, the data processing amount is large, and the time consumption is long.

[0062] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0063] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.

[0064] Figure 1is a flowchart of a feature descriptor-based three-dimensional point cloud registration method according to an exemplary embodiment;

[0065] Figure 2 is a flowchart of a fine registration method according to an exemplary embodiment;

[0066] Figure 3 is a coarse registration schematic diagram in a warehouse simulation scenario according to another exemplary embodiment;

[0067] Figure 4 is a point cloud registration test result schematic diagram in different scenarios according to an exemplary embodiment;

[0068] Figure 5 is a system schematic diagram of a feature descriptor-based three-dimensional point cloud registration device according to an exemplary embodiment;

[0069] In the drawings: 1 - image acquisition module, 2 - candidate corner point acquisition module, 3 - target corner point acquisition module, 4 - fusion module, 5 - three-dimensional mapping module, 6 - coarse registration module, 7 - fine registration module. DETAILED DESCRIPTION

[0070] The exemplary embodiments will be described in detail herein, with examples shown in the drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0071] Embodiment One

[0072] Figure 1 is a flowchart of a feature descriptor-based three-dimensional point cloud registration method according to an exemplary embodiment, as shown in Figure 1 the method comprises:

[0073] S1, acquiring an RGB image, a depth image, and a point cloud image under a dual-view of a warehouse space through a depth camera;

[0074] S2, after the RGB image is grayed, the gray difference between each pixel point in the RGB image and its field pixel point is obtained, and candidate corner points are obtained by setting a gray difference threshold;

[0075] S3, introducing a Hessian matrix to calculate the shape descriptor of each candidate corner point, further screening the candidate corner points by setting a shape descriptor threshold, and obtaining target corner points;

[0076] S4, extracting SURF feature points on the RGB image by SURF algorithm, combining the SURF feature points with the target corner points to obtain H-SURF feature descriptors;

[0077] S5, pairing the H-SURF feature descriptors under the double view angles, and mapping the two-dimensional H-SURF feature descriptor pairs after successful pairing to the three-dimensional space to obtain two pieces of three-dimensional point clouds;

[0078] S6, obtaining an initial rotation matrix R and a translation matrix T by coarse registration of the two pieces of three-dimensional point clouds R t ;

[0079] S7, transforming one piece of three-dimensional point cloud to the coordinate system of the other piece of three-dimensional point cloud by the initial rotation matrix R and the translation matrix T R t , selecting a pair of three-dimensional H-SURF feature descriptors with the minimum error from the two pieces of three-dimensional point clouds, taking the point as the center and a certain distance as the radius to obtain the feature regions of interest of the two pieces of three-dimensional point clouds, taking the two feature regions of interest as the input of the ICP algorithm to perform fine registration, and transforming the point cloud graphs under the double view angles by the output of the ICP algorithm to obtain the target point cloud graph of the storage space;

[0080] It can be understood that the stacks in the hazardous chemical warehouse, such as wooden boxes, are mostly cuboids, the stack surface texture is weak but has obvious corner point features, the existing corner point acquisition method is simple to calculate, and the direction and X Y ​​​The gradient of the direction reflects the change of the gray value of the image to identify the corner point, but in large scale image, the timeliness is poor and the corner points are easy to stack, and the proportion of the number of corner points in the whole image is very small, aiming at the above problems, the application proposes a new corner point acquisition method, calculates the gray difference between each pixel point and the neighborhood pixel point to obtain the candidate corner point for preliminary screening, replaces the original gradient calculation with addition and subtraction to improve the operation efficiency, and then introduces the Hessian matrix to calculate the shape descriptor of each candidate corner point, further screens the candidate corner point by setting the shape descriptor threshold, obtains the target corner point, the SIFT feature descriptor has scale invariance, and has certain robustness to noise, light intensity and the like, but the calculation amount is too large and the effect is not ideal in smooth and weak texture scene, while the SURF feature inherits the characteristics of SIFT, and improves the operation efficiency, extracts the SURF feature point on the RGB image through the SURF algorithm, it is worth emphasizing that the extraction of the SURF feature point through the SURF algorithm is a mature prior art, and no more description is made here, then the SURF feature point is combined with the target corner point to obtain the H-SURF feature descriptor, the H-SURF feature descriptor under the double visual angle is matched to obtain the two-dimensional H-SURF feature descriptor pair, the application constructs the mapping relationship by means of the camera parameter, maps the two-dimensional H-SURF feature point pair which has been matched to the three-dimensional space to form the three-dimensional and one-to-one H-SURF feature descriptor, these points are the final result sought by mainstream point cloud feature extraction algorithms such as SIFT-3D, and then the point cloud initial pose is adjusted after the coarse registration to enhance the robustness of the ICP algorithm in low coincidence rate registration, improve the registration speed and accuracy of the point cloud, and at the same time, the coarse registration gives an initial rotation matrix R and the translation matrix t , through the initial rotation matrix R and the translation matrix ttransforming one of the two pieces of three-dimensional point clouds into the coordinate system of the other piece of three-dimensional point clouds, and selecting a pair of three-dimensional H-SURF feature descriptors with the smallest error in the two pieces of three-dimensional point clouds, obtaining a feature region of interest of the two pieces of three-dimensional point clouds with the point as the center and 30 cm as the radius, and performing fine registration on the two feature regions of interest as the input of the ICP algorithm, and obtaining the target point cloud graph of the warehouse space through the output of the ICP algorithm; the application uses gray difference optimized Harris corner points and fuses SURF feature points to perform feature extraction and matching on the RGB image in the point cloud coarse registration stage, improves the feature information extraction capability of the weak texture area, maps the matched feature points to the three-dimensional H-SURF feature descriptor through the depth map and the camera parameter, performs point cloud coarse registration, selects the neighborhood point cloud of the H-SURF feature descriptor with the smallest error as the feature region of interest, and performs fine registration as the input of the ICP algorithm, solves the problem that the ICP algorithm and its related improved algorithms are easy to fall into local optimal solution when the overlap degree of the view is too low, and improves the algorithm efficiency and accuracy, and at the same time, only part of the entire three-dimensional region, that is, the feature region of interest, is fine registered, which solves the problem of large data processing amount and long time consumption in the prior art that the entire three-dimensional point cloud is taken as the input of the ICP algorithm.

[0081] Preferably,

[0082] The initial rotation matrix obtained by coarse registration of the two pieces of three-dimensional point clouds R and the translation matrix t comprise:

[0083] The H-SURF feature descriptors in the two pieces of three-dimensional point clouds are denoted as sets X and P , respectively, the barycenters of each H-SURF feature descriptor in the sets X and P are calculated, two new sets, i.e. two new pieces of three-dimensional point clouds, are obtained, and the matrix W is obtained by multiplying the two new pieces of three-dimensional point clouds; W SVD decomposition is performed on the matrix R to obtain the initial rotation matrix t and the translation matrix ;

[0084] It can be understood that, in obtaining the 3D H-SURF feature descriptor, the n feature descriptor sets in the two pieces of point clouds are denoted as sets X and P , as follows:

[0085]

[0086] set of points X with the center of gravity of the set of points P and form a new point cloud formula:

[0087]

[0088] multiply the new two pieces of point cloud to obtain a matrix W , and perform SVD decomposition on the matrix W , as shown below:

[0089]

[0090] wherein, represents a 3x3 diagonal matrix, which has no actual meaning, T is a mathematical transpose symbol, represents the transpose matrix of the matrix V , U and V is a 1x3 three-dimensional vector, so we can obtain the initial rotation matrix U and the translation matrix V by R and t , and the specific calculation formula is as shown below:

[0091] the initial rotation matrix R :

[0092]

[0093] the initial translation matrix t :

[0094]

[0095] Preferably,

[0096] the two feature regions of interest are taken as inputs of the ICP algorithm for fine registration, and the target point cloud graph of the warehouse space is obtained by transforming the point cloud graphs under the double visual angles through the output of the ICP algorithm.

[0097] S901, two feature regions of interest are taken as target point cloud P and source point cloud Q respectively;

[0098] S902, a point set P ∈ pi is taken in the target point cloud P , and a corresponding point set Q ∈ qi is found in the source point cloud Q , so that ||qi-pi|| minimum;

[0099] S903, based on the point set pi , the point set qi , the initial rotation matrix R and the translation matrix t , the error value of the error function F , if the error value is less than or equal to the preset error threshold, the initial rotation matrix R and the translation matrix t are the optimal rotation matrix R b and the translation matrix t b ;

[0100] S904, if the error value is greater than the preset error threshold, the initial rotation matrix pi and the translation matrix R are used for rotation and translation transformation to obtain a new target point cloud t 1; P

[0101] S905, the point set P 1 of the new target point cloud pi 1 is obtained, and the corresponding point set Q in the source point cloud qi ∈ Q is found, so that || qi 1 -pi|| is minimum;

[0102] S906, the feature points in the new target point cloud P 1 and the source point cloud Q are respectively recorded as set X 1 and set P 1, the center of gravity of each feature point in set X 1 and set P 1 is calculated respectively to obtain two new sets, and the multiplication of the two new sets obtains a matrix W 1, the SVD decomposition of the matrix W 1 is performed to obtain a new rotation matrix R 1 and a translation matrix t 1 respectively;

[0103] S907, based on the point set pi 1, the point set qi、 , the new rotation matrix R 1 and the translation matrix t 1, the error value of the error function F is calculated, if the error value is less than or equal to the preset error threshold, the new rotation matrix R is the optimal rotation matrix1 and translation matrix t 1 is the optimal rotation matrix R b and translation matrix t b ;

[0104] S908, If the error value is greater than the preset error threshold, repeat the above steps until the optimal rotation matrix is ​​obtained. R b and translation matrix t b ;

[0105] S909, through the optimal rotation matrix R b and translation matrix t b Transform one point cloud image from the dual-view point cloud image into the coordinate system of the other point cloud image to obtain the target point cloud image of the warehouse space.

[0106] Understandably, as shown in the attached document Figure 2 As shown, the core idea of ​​the ICP algorithm is to minimize the error function through continuous iteration. In each iteration, it optimizes the target point cloud... P One point pi According to a certain correspondence, in the template point cloud Q Find the point with the closest Euclidean distance. qi As its corresponding point, the optimal matching parameters are calculated. R and t This makes the error function F ( R , t By obtaining the minimum value, a set of optimal rigid body transformation parameters is finally solved, so that the transformed target point cloud corresponds to the template point cloud. Specifically:

[0107] The two regions of interest obtained above are respectively used as target point clouds. P and Source Cloud Q ;

[0108] In the target point cloud P Take the middle point set pi ∈ P And find the source point cloud Q The corresponding point set in qi ∈ Q , making ||qi-pi|| Minimum, since the coarse registration process has already provided the initial rotation matrix to the ICP algorithm. R and translation matrix t， Therefore, the error function can be calculated directly. F Error value, error function FThe expression is:

[0109]

[0110] Calculate the error function F After determining the error value, compare it with a preset error threshold. If the error value is less than or equal to the preset error threshold, the fine registration ends, and the initial rotation matrix is... R and translation matrix t As the optimal rotation matrix R b and translation matrix t b However, if the error value exceeds the preset error threshold, iterative calculation is required again. pi Using the initial rotation matrix R Translation matrix t Perform rotation and translation transformations to obtain a new target point cloud. P 1. Create a new target point cloud. P 1 and Source Cloud Q The feature points in the data are denoted as sets. X 1 and sets P 1. Based on the initial rotation matrix described above R and translation matrix t The calculation formula can be used to calculate the new rotation matrix. R 1 and translation matrix t 1. Based on point sets pi 1. Point set qi、 New rotation matrix R 1 and translation matrix t 1. Calculate the error function F If the error value is less than or equal to the preset error threshold, then the new rotation matrix... R 1 and translation matrix t 1 is the optimal rotation matrix R b and translation matrix t b If the error exceeds the preset error threshold, then a new rotation matrix will be used. R 1 and translation matrix t 1 pair pi 1. Perform rotation and translation transformations to obtain a new target point cloud. P 2. Until a certain iteration, if the calculated error value is less than or equal to a preset error threshold, then the rotation matrix and translation matrix of this iteration are taken as the optimal rotation matrix. R b and translation matrix t b Then through the optimal rotation matrix Rb and a translation matrix t b Transform one point cloud in the initial double-view point cloud into the coordinate system in another point cloud to obtain a target point cloud of the storage space, which contains fused point cloud information in double views and can provide better environmental perception for safety monitoring of hazardous chemical storage.

[0111] Preferably,

[0112] After the RGB image is grayed, the gray difference between each pixel point in the RGB image and its domain pixel point is obtained, and a candidate corner point is obtained by setting a gray difference threshold, including:

[0113] By calculating the gray difference between each pixel point and its 8-neighborhood pixel point after the RGB image is grayed, 8 gray differences are obtained.

[0114] The size of the 8 gray differences and the preset gray difference threshold is judged respectively, if a certain gray difference is less than the preset gray difference threshold, it is considered that the pixel point is similar to a certain neighborhood pixel point; if a certain gray difference is greater than or equal to the preset gray difference threshold, it is considered that the pixel point is not similar to a certain neighborhood, the number of similar pixel points of a certain pixel point and 8 neighborhood pixel points is obtained, if the number of similar pixel points meets the preset number, the pixel point is taken as a candidate corner point.

[0115] It can be understood that, let I(x,y) be any pixel point after the RGB image is grayed, I(i,j) be any point in the 8-neighborhood with I(x,y) as the center, the candidate corner point discrimination method is as follows:

[0116]

[0117] In the formula, N is the threshold value of the gray difference, M is the number of similar points to the center point I(x,y) , that is, the number of gray differences less than the preset gray difference threshold, when M takes the value of 0, it indicates that the center point may be an independent noise point, when M takes the value of 8, it indicates that the gray value near the center point in the 8-neighborhood changes little, and it is impossible to be a corner point, and M the value of which is between 5 and 7, the situation is complex, and it may be mixed with edge points, etc., so it is discarded, and when the value of M is between 2 and 4, the center point is considered as a candidate corner point.

[0118] Preferably,

[0119] The introduced Hessian matrix calculates a shape descriptor of each candidate corner point, the candidate corner points are further screened by setting a shape descriptor threshold, and the target corner point is obtained.

[0120] The second-order partial derivatives of the image gray scale functions I x y x and y are calculated respectively, wherein x and y are the coordinates of the candidate corner point;

[0121] Two eigenvalues of a second-order Hessian matrix are solved according to the second-order partial derivatives of each candidate corner point, the shape descriptor is obtained by multiplying the two eigenvalues, and the shape descriptor of each candidate corner point is calculated respectively;

[0122] The shape descriptor of each candidate corner point is compared with a preset shape descriptor threshold respectively, if the shape descriptor of the candidate corner point is less than the preset shape descriptor threshold, the candidate corner point is taken as the target corner point.

[0123] It can be understood that after the candidate corner points are discriminated by using the Harris algorithm, the Hessian matrix is introduced:

[0124]

[0125] wherein, H is a second-order Hessian matrix, , and are the second-order partial derivatives of the image gray scale functions I(x,y) x and y , two eigenvalues H and of the matrix are solved, and the calculation formula is as follows:

[0126]

[0127]

[0128] In the above formula, is a shape descriptor, the closer the shape descriptor of a point to the negative maximum value of , the more likely the point is a corner point, therefore, the shape descriptor of each candidate corner point is compared with a preset shape descriptor threshold respectively by the shape descriptor threshold, if the shape descriptor of the candidate corner point is less than the preset shape descriptor threshold, the candidate corner point is taken as the target corner point.

[0129] Preferably,​​​​

[0130] The combination of the SURF feature points and the target corner points to obtain the H-SURF feature descriptor includes:

[0131] The target corner points and the SURF feature points are combined to form a new feature point set A ;

[0132] Any target corner point or SURF feature point in the feature point set A is taken as the center, and a fixed range is obtained through a preset radius;

[0133] The fixed range is equally divided into a plurality of sub-blocks, the haar wavelet response values of all pixel points in each sub-block along the X direction and the Y direction are calculated and summed to obtain the H-SURF feature descriptor of any target corner point or SURF feature point in the feature point set A ;

[0134] The H-SURF feature descriptors of all target corner points or SURF feature points in the feature point set A are calculated respectively;

[0135] It can be understood that the target corner points and the SURF feature points are combined to form a new feature point set A(x,y) , each point in A(x,y) is taken as the center, and a circle with a radius of 20 S is divided, wherein, S is the scale factor of the SURF feature point, the circle is equally divided into 16 sub-blocks, the haar wavelet response values of all pixel points in each sub-block along the X direction and the Y direction are calculated and summed to obtain the H-SURF feature descriptor, and the expression is as follows:

[0136]

[0137] In the formula, and are the haar wavelet response values of all pixel points in the sub-block along the X direction and the Y direction, respectively.

[0138] Preferably,

[0139] The H-SURF feature descriptors under the double-view angles are paired, and the two-dimensional H-SURF feature descriptor pair after successful pairing is mapped to a three-dimensional space to obtain two pieces of three-dimensional point clouds, which include:

[0140] Calculate the Euclidean distance of H-SURF feature descriptors under dual-view perspectives, use the Euclidean distance to determine the similarity of feature points under dual-view perspectives, and obtain matching feature point pairs by setting an Euclidean distance threshold;

[0141] The RANSAC algorithm is used to filter feature point pairs to obtain target feature point pairs.

[0142] Two-dimensional target feature point pairs are mapped to three-dimensional space using their respective depth values ​​on the depth map and the parameters of the depth camera, resulting in two three-dimensional point clouds.

[0143] The step of mapping two-dimensional target feature point pairs to three-dimensional space using their respective depth values ​​on the depth map and the parameters of the depth camera to obtain two three-dimensional point clouds includes:

[0144] The intrinsic parameter matrix of the depth camera is obtained through camera calibration technology;

[0145] The depth values ​​of the target feature point pairs on their respective depth maps are obtained through the depth maps.

[0146] Based on the depth value of each feature point and the intrinsic parameter matrix of the depth camera, the two-dimensional target feature point pairs are mapped to three-dimensional space to obtain two three-dimensional point clouds.

[0147] It is understandable that the H-SURF feature descriptors for the target image and the image to be matched (either viewpoint is the target image and the other viewpoint is the image to be matched) are calculated. and The Euclidean distance D is given by the following expression:

[0148]

[0149] In the formula, n represents the dimension of the feature vector. The similarity of feature points under two views is determined by the Euclidean distance D. Matching feature point pairs are obtained by setting an Euclidean distance threshold. After feature point matching, mismatched points are further eliminated by the RANSAC algorithm. It is worth noting that the RANSAC algorithm is also an existing algorithm, and there are existing technologies that use the RANSAC algorithm to filter matching points. This algorithm is mostly used for line fitting. The core idea is to assume that there are most normal data and a small amount of abnormal data in the given data, and to randomly sample. Assuming that the selected sample data is correct, a model is generated and other points are verified. This process is repeated iteratively until the error is minimized. The 3D point cloud is formed by the depth map with the help of camera parameters. After the camera parameters are corrected, the pixels of the RGB image and the depth map can be matched one-to-one. The depth value of the pixel on the RGB image can then be obtained. The intrinsic parameter matrix of the depth camera is obtained by camera calibration technology. Y，

[0150]

[0151] wherein, , are the focal length of the depth camera coordinate system x axis, y axis direction, , is the coordinate of the center of the camera optical axis, and after obtaining the intrinsic matrix Y and the depth value of the pixel point, three-dimensional transformation can be performed, and any point on the RGB image P(u,v) is mapped to the point in the three-dimensional space after the three-dimensional transformation Q(x,y,z) , as shown below:

[0152]

[0153] wherein, Z is the depth value of the point on the depth map with the coordinate (u,v) .

[0154] In order to verify the above scheme of the present application, the TUM RGB-D data set of the Technical University of Munich in Germany is also selected for robustness test, the TUM data set contains data generated by the Microsoft Kinect depth camera in 39 different indoor scenes, and is widely used in indoor three-dimensional reconstruction and Slam research. The three-dimensional reconstruction range of the indoor scene of the data set is from 3m to 8m, and the reconstructed scene has similar characteristics to the dangerous chemical warehouse environment, such as weak texture and mutual occlusion of targets. Table 1 is the average value of the number of feature points after the test of 10 scenes,

[0155] Table 1 Corner detection data

[0156]

[0157] Table 2 is the result of feature matching. The algorithm in this paper can obtain more feature point matching in the warehouse simulation environment. In a computer desk scene in the TUM data, the SIFT algorithm can obtain more feature point matching due to the rich texture of the computer desk scene, but the time consumption is too long. The smooth wooden box scene is similar to the dangerous chemical warehouse, which has characteristics such as smoothness, weak texture and obvious corner points. SIFT and its improved SURF involve normal and other geometric features in the calculation process, and the robustness is poor in the weak texture environment. The H-SURF in this paper mainly depends on the corner feature, and has good robustness in the scene with obvious corner feature,

[0158] Table 2 Feature matching data

[0159]

[0160] From Table 2, it can be seen that the present application can successfully match more feature points than other existing technologies, and takes less time; the results of mapping the warehouse simulation scenario in Table 2 to three dimensions to form 3D H-SURF descriptors and performing point cloud coarse registration are shown in Figs. 6A-6C. Figure 3 Figure 3 The right side of Figs. 6A-6C is the coarse registration result of 3D H-SURF feature descriptors, the 3D H-SURF feature descriptors in the left side of the to-be-matched point cloud are mapped to the target point cloud, and the coarse matching result on the right side is obtained, the higher the coincidence degree, the higher the coarse matching accuracy, and the specific coordinate transformation is listed in Table 3, wherein the coordinate unit is meter,

[0161] Table 3: Coordinates before and after coarse matching transformation of 3D H-SURF feature descriptors

[0162] Point Number First Group Second Group Coarse Registration Result (circle) 1 (1.069,-0.599,2.553) (0.507,-0.693,2.735) (1.074,-0.609,2.549) 2 (0.578,-0.319,2.221) (0.118,-0.388,2.345) (0.596, -0.331, 2.248) 3 (0.712,-0.239,2.210) (0.2480,-0.307,2.345) (0.718, -0.242, 2.221) 4 (0.764,-0.223,2.196) (0.305,-0.295,2.345) (0.772, -0.227, 2.211) 5 (0.789,-0.030,1.796) (0.414,-0.095,1.927) (0.779, -0.027, 1.779) 6 (-0.095,0.049,1.834) (-0.449,0.0378,1.769) (-0.103,0.056,1.805) 7 (0.149,0.157,1.561) (-0.157,0.1338,1.568) (0.135,0.166,1.547) 8 (0.245,0.276,1.462) (-0.034,0.250,1.501) (0.235,0.288,1.456) 9 (0.734,-0.228,2.210) (0.272,-0.295,2.345) (0.741,-0.229,2.216)

[0163] The rotation matrix obtained by the above coarse transformation is as follows: R and the translation vector is as follows: t

[0164] Let the point Q(x,y,z) be any point in the second group, and the following formula is the formula for mapping it to the coordinate system in which the first group P(x1, y1, z1) is located:

[0165]

[0166] The point cloud registration experiment part of the scene for testing the robustness of the algorithm in the TUM data set is as follows:

[0167] Among them, two frames in the cabinet scene in the TUM data set are selected, the three-dimensional reconstruction range of the scene is 8 meters, the environment is single, the texture is weak, and the average moving distance between each frame is 16.34 cm, the point cloud registration error of the scene is shown in Table 4, and the algorithm in this paper has good timeliness and small error in the 8m three-dimensional reconstruction range;

[0168] Table 4: TUM feature matching data

[0169]

[0170] ​​To verify the robustness of the algorithm in this paper under different degrees of coincidence of two pieces of point cloud, this experiment selects three kinds of scenes with coincidence degree below 80%, below 70% and below 50% to test the point cloud registration. To verify the registration effect of the algorithm in this paper, the K-D tree accelerated ICP algorithm, FPFH+ICP algorithm and NDT+ICP algorithm are selected for comparison test. The ICP algorithm has good registration accuracy under the conditions that the coincidence degree is above 80% and the initial poses of the two pieces of point cloud are in the same horizontal plane. When the coincidence degree is below 70%, the ICP algorithm and its improved algorithms need to select feature point sets from the two pieces of point cloud for brute force matching. When the coincidence degree is too low or the initial pose is poor after rough registration, it will fall into a local optimal solution. The error transformation of the ICP fine registration iteration after rough registration is shown in the accompanying Figure 4 Figure 4 Root Mean Square (RMS) is in centimeters, and the RMS of iteration number 0 is the rough registration error. The rough registration error of the ICP algorithm is the initial pose error of the point cloud. In the scene with a coincidence degree below 80%, the error of the algorithm in this paper and the NDT+ICP algorithm after rough registration is small, and only 2 iterations of ICP are needed for error convergence. When the coincidence degree is below 70%, the rough registration error of NDT+ICP and FPFH+ICP is smaller than the initial pose, but the point cloud pose is not in the same plane after rough registration, which leads to falling into a local optimal solution and the error convergence presents a fluctuating trend. The algorithm in this paper uses the feature region of interest after rough registration to replace the whole point cloud for ICP iteration. The overall coincidence degree of the two feature regions of interest after rough registration can reach more than 90%, the point cloud size is small and the pose is ideal, which can avoid falling into a local optimal solution and has good timeliness. The specific values are shown in Table 5,

[0171] Table 5 Experimental data in the range of 3.5 meters in the warehouse simulation

[0172]

[0173] In Table 5, the rough registration stage of FPFH+ICP and NDT+ICP algorithms needs to find feature point sets from unordered three-dimensional point clouds. The point cloud data size of the hazardous chemical warehouse can reach hundreds of thousands, which leads to poor timeliness and large cumulative error when the coincidence degree of the two pieces of point cloud is low. The algorithm in this paper finds feature points from two-dimensional RGB images and maps them to three-dimensional, which reduces the influence of point cloud data size on algorithm timeliness and cumulative error in the process of point cloud registration, thereby obtaining better test results.

[0174] Embodiment Two

[0175] Figure 5 is a system schematic diagram of a feature descriptor based three-dimensional point cloud registration device according to an exemplary embodiment, which comprises: ​

[0176] Image acquisition module 1: Used to acquire RGB images, depth maps, and point cloud maps of the warehouse space from dual perspectives using a depth camera;

[0177] Candidate Corner Acquisition Module 2: After converting the RGB image to grayscale, it acquires the grayscale difference between each pixel in the RGB image and its neighboring pixels, and obtains candidate corners by setting a grayscale difference threshold;

[0178] Target Corner Acquisition Module 3: Used to introduce the Hessian matrix to calculate the shape descriptor of each candidate corner point, and further filter the candidate corner points by setting the shape descriptor threshold to obtain the target corner point;

[0179] Fusion Module 4: Used to extract SURF feature points from the RGB image using the SURF algorithm, and combine the SURF feature points with the target corner points to obtain the H-SURF feature descriptor;

[0180] 3D mapping module 5: used to pair H-SURF feature descriptors under dual views, and map the successfully paired 2D H-SURF feature descriptor pairs to 3D space to obtain two 3D point clouds;

[0181] Coarse registration module 6: Used to obtain an initial rotation matrix by coarsely registering the two 3D point clouds. R With translation matrix t ;

[0182] Fine-tuning module 7: Used to match the initial rotation matrix R with the translation matrix R. t One of the two 3D point clouds is transformed into the coordinate system of the other 3D point cloud. The pair of 3D H-SURF feature descriptors with the smallest error between the two 3D point clouds is selected. The region of interest of the two 3D point clouds is obtained with this point as the center and a certain distance as the radius. The two regions of interest are used as the input of the ICP algorithm for fine registration. The output of the ICP algorithm is used to transform the point cloud map under the dual view to obtain the target point cloud map of the warehouse space.

[0183] Understandably, the image acquisition module 1 of this application is used to acquire RGB images, depth maps, and point cloud images of the warehouse space from dual perspectives using a depth camera; the candidate corner acquisition module 2 is used to obtain the grayscale difference between each pixel in the RGB image and its neighboring pixels after grayscale conversion, and obtain candidate corners by setting a grayscale difference threshold; the target corner acquisition module 3 is used to calculate the shape descriptor of each candidate corner by introducing a Hessian matrix, and further filter the candidate corners by setting a shape descriptor threshold to obtain target corners; the fusion module 4 is used to extract SURF feature points on the RGB image using the SURF algorithm, and combine the SURF feature points with the target corners to obtain H-SURF feature descriptors; the three-dimensional mapping module 5 is used to pair the H-SURF feature descriptors from dual perspectives, and map the successfully paired two-dimensional H-SURF feature descriptor pairs to three-dimensional space to obtain two three-dimensional point clouds; the coarse registration module 6 is used to obtain an initial rotation matrix by coarsely registering the two three-dimensional point clouds. R With translation matrix t The fine registration module 7 is used to match the initial rotation matrix R with the translation matrix R. t One of the two 3D point clouds is transformed into the coordinate system of the other 3D point cloud. The pair of 3D H-SURF feature descriptors with the smallest error between the two point clouds is selected. Using this point as the center and a certain distance as the radius, the regions of interest (ROIs) of the two point clouds are obtained. These ROIs are then used as input to the ICP algorithm for fine registration. The output of the ICP algorithm is used to transform the point cloud images under the dual-view perspective to obtain the target point cloud image of the warehouse space. This application improves the extraction of feature information in weakly textured areas by using grayscale difference optimization of Harris corner points and fusing them with SURF feature points in the coarse registration stage of the point cloud. This method enables coarse registration of point clouds by mapping matched feature points to 3D H-SURF feature descriptors using depth maps and camera parameters. It then selects the neighborhood point cloud of the feature point with the smallest H-SURF feature descriptor error as the region of interest (ROI) and uses it as input for fine registration of the ICP algorithm. This solves the problem of the ICP algorithm and its related improvements easily getting trapped in local optima when the viewpoint overlap is too low, and improves the algorithm's efficiency and accuracy. Furthermore, by performing fine registration only on a portion of the entire 3D region, i.e., the ROI, it not only improves viewpoint overlap but also addresses the issues of large data processing volume and long processing time in existing technologies that use the entire 3D point cloud as input for the ICP algorithm.

[0184] Example 3:

[0185] This embodiment provides a storage medium storing a computer program, which, when executed by a host controller, implements the various steps in the above method.

[0186] It can be understood that the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0187] It can be understood that the same or similar parts in the above embodiments can be mutually referred to, and the content not described in detail in some embodiments can refer to the same or similar content in other embodiments.

[0188] It should be noted that in the description of the present application, the terms "first", "second", etc. are only for the purpose of description, and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is at least two.

[0189] Any process or method descriptions in flow charts or otherwise described herein represents an example of embodiments of the present application that can be embodied in code means that include one or more steps for accomplishing a specified logical function or process. The scope of embodiments of the present application encompass hardware and software that, when properly configured and utilized, carry out the processes described in the embodiments of the present application. Embodiments of the present application as previously described are therefore best implemented when the computer does not require a human software writer to be directly involved in the generation of the code to perform such processes. It is therefore understood that the scope of the technical solutions of the present application also encompasses code means to perform the processes described in the embodiments of the present application.

[0190] It should be understood that parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized by hardware, and as in another embodiment, it can be realized by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logical functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0191] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, and when executed, include one or a combination of steps of the method embodiments.

[0192] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can exist physically separately, or two or more units can be integrated in one module. The above integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0193] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0194] In the description of the present specification, the description referring to the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in an appropriate manner.

[0195] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary, and are not to be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. A method for registering three-dimensional point clouds based on feature descriptors, characterized in that, The method comprises: obtaining an RGB image, a depth image and a point cloud image under a double view of a storage space through a depth camera; after the RGB image is grayscaled, obtaining a gray difference between each pixel point in the RGB image and its field pixel point, and obtaining a candidate corner point by setting a gray difference threshold value; introducing a Hessian matrix to calculate a shape descriptor of each candidate corner point, further screening the candidate corner point by setting a shape descriptor threshold value, and obtaining a target corner point; extracting a SURF feature point on the RGB image through a SURF algorithm, combining the SURF feature point with the target corner point, and obtaining an H-SURF feature descriptor; pairing the H-SURF feature descriptor under the double view, and mapping a two-dimensional H-SURF feature descriptor pair after pairing success to a three-dimensional space to obtain two pieces of three-dimensional point clouds; An initial rotation matrix is obtained by coarse registration of the two three-dimensional point clouds R and a translation matrix t ; by an initial rotation matrix R and a translation matrix t Transform one of the two pieces of three-dimensional point cloud into the coordinate system of the other piece of three-dimensional point cloud, and select a pair of three-dimensional H-SURF feature descriptors with the smallest error in the two pieces of three-dimensional point cloud. Obtain the feature region of interest of the two pieces of three-dimensional point cloud with the point as the center and a certain distance as the radius. Take the two feature regions of interest as the input of the ICP algorithm to perform fine registration. Transform the point cloud graph under the double view through the output of the ICP algorithm to obtain the target point cloud graph of the warehouse space.

2. The method of claim 1, wherein The initial rotation matrix is obtained by coarsely registering the two three-dimensional point clouds R and a translation matrix t comprising: Let the H-SURF feature descriptors in the two three-dimensional point clouds be denoted as sets X and P respectively. X The barycenters of each H-SURF feature descriptor in sets P and W are calculated respectively to obtain two new sets, i.e. two new three-dimensional point clouds, and the matrix W is obtained by multiplying the two new three-dimensional point clouds. R The SVD decomposition is performed on the matrix t to obtain the initial rotation matrix and the translation matrix .

3. The method of claim 2, wherein the precise registration of the two pieces of feature regions of interest as the input of the ICP algorithm, and the transformation of the point cloud images under the double view through the output of the ICP algorithm to obtain the target point cloud image of the storage space comprises: two feature regions of interest are respectively taken as target point cloud P and source point cloud Q ; In the target point cloud P take the point set pi ∈ P , and find the corresponding point set Q in the source point cloud qi ∈ Q such that ||qi-pi|| is minimal; Based on a point set pi , the point set qi , an initial rotation matrix R and a translation matrix t Calculating an error value of an error function F If the error value is less than or equal to a preset error threshold, the initial rotation matrix R and the translation matrix t are the optimal rotation matrix R b and the translation matrix t b ​ If the error value is greater than a preset error threshold, the target point cloud is rotated and translated pi using the initial rotation matrix R and the translation matrix t to obtain a new target point cloud P 1; acquiring a new target point cloud P 1 the point set of pi 1 and find the corresponding point set in the source point cloud Q 1 qi ∈ Q such that ||qi 1 -pi|| min The new target point cloud P 1 and the source point cloud Q The feature points in the new target point cloud X 1 and the source point cloud P 1 are respectively recorded as sets X 1 and P 1, respectively, the center of gravity of each feature point in the sets X 1 and P 1 is calculated to obtain two new sets, the two new sets are multiplied to obtain a matrix W 1, the matrix W 1 is subjected to SVD decomposition to obtain a new rotation matrix R 1 and a translation matrix t 1; Point set based pi 1. A point set qi、 1. A new rotation matrix R 1. A translation matrix t 1. An error function F 1. If the error value is less than or equal to a predetermined error threshold, the new rotation matrix R 1. A translation matrix t 1. Is the optimal rotation matrix R b and a translation matrix t b ; If the error value is greater than a preset error threshold, repeating the above steps until the optimal rotation matrix is obtained R b and the translation matrix t b ; by the best rotation matrix R b and translation matrix t b transform one of the point cloud graphs in the two perspectives into the coordinate system in the other point cloud graph, to obtain a target point cloud graph of the warehouse space.

4. The method of claim 1, wherein after the RGB image is grayscaled, obtaining a gray difference between each pixel point in the RGB image and its field pixel point, and obtaining a candidate corner point by setting a gray difference threshold value comprises: obtaining eight gray differences by calculating the gray difference values between each pixel point and its eight neighborhood pixel points after the RGB image is grayscaled; respectively judging the sizes of the eight gray differences and the preset gray difference threshold value, if a certain gray difference is less than the preset gray difference threshold value, regarding that the pixel point is similar to a certain neighborhood pixel point around it, if a certain gray difference is greater than or equal to the preset gray difference threshold value, regarding that the pixel point is not similar to a certain neighborhood pixel point around it, obtaining the similar number of a certain pixel point and its eight neighborhood pixel points, and if the similar number meets the preset number, regarding the pixel point as a candidate corner point.

5. The method of claim 4, wherein introducing a Hessian matrix to calculate a shape descriptor of each candidate corner point, further screening the candidate corner point by setting a shape descriptor threshold value, and obtaining a target corner point comprises: calculating the second order partial derivatives of the image gray level function I ( x , y ) with respect to x and y , where x and y are the coordinates of the candidate corner point; solving two eigenvalues of a second-order Hessian matrix according to the second-order partial derivatives of each candidate corner point, multiplying the two eigenvalues to obtain a shape descriptor, and respectively calculating the shape descriptor of each candidate corner point; respectively comparing the shape descriptor of each candidate corner point with the preset shape descriptor threshold value, if the shape descriptor of the candidate corner point is less than the preset shape descriptor threshold value, regarding the candidate corner point as a target corner point.

6. The method of claim 5, wherein combining the SURF feature point with the target corner point to obtain an H-SURF feature descriptor comprises: The target corner points and the SURF feature points are combined to form a new feature point set A ; With the feature point set A Any target corner point or SURF feature point as the center, a fixed range is obtained by a predetermined radius; Divide the fixed area into multiple sub-blocks, and calculate the distances of all pixels within each sub-block. X direction and Y The Haar wavelet response values ​​in the direction are summed to obtain the feature point set. A H-SURF feature descriptor for any target corner point or SURF feature point; calculating feature point sets A H-SURF feature descriptors for all target corner points or SURF feature points in the middle.

7. The method of claim 6, wherein The H-SURF feature descriptors under the dual view angles are paired, and the two-dimensional H-SURF feature descriptor pairs after successful pairing are mapped to a three-dimensional space, to obtain two pieces of three-dimensional point clouds including: The Euclidean distances of the H-SURF feature descriptors under the dual view angles are calculated, the similarity of the feature points under the dual view angles is judged through the Euclidean distances, and the matched feature point pairs are obtained through setting an Euclidean distance threshold; The feature point pairs are screened through the RANSAC algorithm, to obtain target feature point pairs; The two-dimensional target feature point pairs are mapped to a three-dimensional space through the depth values of the target feature point pairs on the depth maps and the parameters of the depth camera, to obtain two pieces of three-dimensional point clouds; The two-dimensional target feature point pairs are mapped to a three-dimensional space through the depth values of the target feature point pairs on the depth maps and the parameters of the depth camera, to obtain two pieces of three-dimensional point clouds including: The intrinsic matrix of the depth camera is obtained through a camera calibration technique; The depth values of the target feature point pairs on the respective depth maps are obtained through the depth maps; The two-dimensional target feature point pairs are mapped to a three-dimensional space based on the depth values of each feature point and the intrinsic matrix of the depth camera, to obtain two pieces of three-dimensional point clouds.

8. An apparatus for registering three-dimensional point clouds based on feature descriptors, characterized in that The device includes: An image acquisition module: configured to acquire an RGB map, a depth map and a point cloud map under dual view angles of a storage space through a depth camera; A candidate corner point acquisition module: configured to acquire the gray difference between each pixel point and the field pixel points in the RGB map after the RGB map is grayed, and obtain candidate corner points through setting a gray difference threshold; A target corner point acquisition module: configured to introduce a Hessian matrix to calculate the shape descriptor of each candidate corner point, and further screen the candidate corner points through setting a shape descriptor threshold, to obtain target corner points; A fusion module: configured to extract SURF feature points on the RGB map through a SURF algorithm, combine the SURF feature points with the target corner points, and obtain H-SURF feature descriptors; A three-dimensional mapping module: configured to pair the H-SURF feature descriptors under the dual view angles, and map the two-dimensional H-SURF feature descriptor pairs after successful pairing to a three-dimensional space, to obtain two pieces of three-dimensional point clouds; coarse registration module: for obtaining an initial rotation matrix by coarse registration of the two three-dimensional point clouds R and a translation matrix t ; Fine registration module: for transforming one of the two three-dimensional point clouds into the coordinate system of the other three-dimensional point cloud by means of an initial rotation matrix R and a translation matrix T t Transform one of the two three-dimensional point clouds into the coordinate system of the other three-dimensional point cloud, and select a pair of three-dimensional H-SURF feature descriptors with the smallest error in the two three-dimensional point clouds. Obtain the feature region of interest of the two three-dimensional point clouds with the point as the center and a certain distance as the radius. Take the two feature regions of interest as the input of the ICP algorithm for fine registration. Transform the point cloud graph under the double view through the output of the ICP algorithm to obtain the target point cloud graph of the warehouse space.

9. A storage medium, characterized by The storage medium stores a computer program, and the computer program is executed by the host controller to realize each step in the three-dimensional point cloud registration method based on feature descriptors according to any one of claims 1-7.

Images