Positioning method and device based on RGBD

Abstract

The present invention proposes a positioning method and device based on RGBD, wherein the method comprises: using a laser radar to obtain environmental point cloud data, and constructing the environmental point cloud data according to a laser mileage calculation method to obtain an offline point cloud map; extracting the offline point cloud map to obtain source feature information; using an RGBD camera sensor to obtain a target image of the corresponding environment, and obtaining three-dimensional point cloud data of the target image; extracting the three-dimensional point cloud data to obtain target feature information; matching the source feature information and the target feature information to perform posture recovery and six-degree-of-freedom estimation, and obtaining the positioning result of the RGBD camera sensor in the offline point cloud map; thus, the present invention constructs a map through a laser radar device, which can be reused by multiple devices equipped with RGBD cameras, thereby greatly improving the accuracy and robustness of indoor positioning while reducing the use cost.

Description

Technical Field

[0001] The present invention relates to the field of visual positioning technology, and in particular to a positioning method based on RGBD, a positioning device based on RGBD, a computer-readable storage medium and a computer device. Background Art

[0002] Among the related technologies, achieving precise positioning is crucial for self-driving cars and intelligent robots. Autonomous mobile robots such as self-driving cars need precise positioning to navigate safely. Although the existing global navigation satellite system (GNSS) can provide global positioning, its accuracy and reliability are not enough for robot navigation. For example, in urban environments, buildings usually block or reflect satellite signals, resulting in non-line-of-sight and multipath problems. To alleviate this problem, positioning methods using sensors on mobile devices are a feasible solution. Visual odometer technology fused with IMU has been well developed in pose state estimation. It not only perceives rich semantic information in the environment, but also provides more accurate position estimation. However, pose state estimation methods that only use image features are prone to failure due to changes in lighting or texture in the environment, and when the loop is closed, cumulative drift and pose jumps will inevitably occur. Unlike vision-based positioning solutions, lasers can accurately measure distances and are also robust to changes in lighting. They have higher accuracy and robustness than visual positioning, but the cost and weight of 3D lidar have largely hindered its widespread application and promotion. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems in the above-mentioned technology to a certain extent. To this end, one object of the present invention is to propose a positioning method based on RGBD, which performs real-time positioning in a priori laser point cloud map through an RGBD camera, greatly improving the accuracy and robustness of positioning while reducing costs.

[0004] A second object of the present invention is to provide a computer-readable storage medium.

[0005] A third object of the present invention is to provide a computer device.

[0006] The fourth objective of the present invention is to provide a positioning device based on RGBD.

[0007] To achieve the above-mentioned purpose, the first aspect of the present invention proposes a positioning method based on RGBD, which includes the following steps: using a laser radar to obtain environmental point cloud data, and constructing the environmental point cloud data according to a laser mileage calculation method to obtain an offline point cloud map; performing feature extraction on the offline point cloud map to obtain source feature information, wherein the source feature information includes point, line and surface features; using an RGBD camera sensor to obtain a target image of the corresponding environment, and obtaining three-dimensional point cloud data of the target image; performing feature extraction on the three-dimensional point cloud data to obtain target feature information, wherein the target feature information includes point, line and surface features; matching the source feature information and the target feature information to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining the positioning result of the RGBD camera sensor in the offline point cloud map.

[0008] According to the RGBD-based positioning method of the embodiment of the present invention, a laser radar is first used to obtain environmental point cloud data, and the environmental point cloud data is constructed according to the laser mileage calculation method to obtain an offline point cloud map; then the offline point cloud map is feature extracted to obtain source feature information, wherein the source feature information includes point, line and surface features; then an RGBD camera sensor is used to obtain a target image of the corresponding environment, and three-dimensional point cloud data of the target image is obtained; then the three-dimensional point cloud data is feature extracted to obtain target feature information, wherein the target feature information includes point, line and surface features; finally, the source feature information and the target feature information are matched to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining the positioning result of the RGBD camera sensor in the offline point cloud map. Therefore, the present invention constructs a map through an expensive laser radar device, which can be reused by multiple devices equipped with RGBD cameras, while reducing the cost of using the entire system, greatly improving the accuracy and robustness of indoor positioning.

[0009] In addition, the RGBD-based positioning method proposed in the above embodiment of the present invention may also have the following additional technical features:

[0010] Optionally, after constructing the environmental point cloud data according to the laser mileage calculation method to obtain an offline point cloud map, it also includes: using CloudCompare to denoise the offline point cloud map.

[0011] Optionally, feature extraction is performed on the offline point cloud map to obtain source feature information, including: using a 3D-LineDetection algorithm to extract line and surface features in the offline point cloud map, wherein the center point and normal vector information of the surface feature are stored, and the two endpoint information of the line feature are stored; using a Spinnet algorithm to extract descriptors of points in the offline point cloud map to obtain a 32-dimensional feature descriptor for each point cloud, and storing the descriptors.

[0012] Optionally, the source feature information and the target feature information are matched to perform posture recovery and six-degree-of-freedom estimation, including: using RANSAC iteration to obtain the coarse positioning of points in the source feature information between corresponding points in the target feature information; using ICP algorithm to obtain the rotation amount of points in the source feature information between corresponding points in the target feature information; establishing an optimization function based on the source feature information and the target feature information, so as to obtain the translation amount of the source feature information between corresponding target feature information through the optimization function.

[0013] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium on which a RGBD-based positioning program is stored. When the RGBD-based positioning program is executed by a processor, the RGBD-based positioning method as described above is implemented.

[0014] According to the computer-readable storage medium of an embodiment of the present invention, by storing an RGBD-based positioning program, the RGBD-based positioning program is executed by the processor to implement the RGBD-based positioning method as described above, thereby performing real-time positioning in the prior laser point cloud map through the RGBD camera, thereby greatly improving the accuracy and robustness of positioning while reducing costs.

[0015] To achieve the above objectives, the third aspect of the present invention proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the RGBD-based positioning method as described above is implemented.

[0016] According to the computer device of an embodiment of the present invention, a RGBD-based positioning program is stored in a memory, so that when the RGBD-based positioning program is executed by a processor, the RGBD-based positioning method as described above is implemented, thereby performing real-time positioning in a priori laser point cloud maps through an RGBD camera, thereby greatly improving the accuracy and robustness of positioning while reducing costs.

[0017] To achieve the above-mentioned purpose, the fourth aspect of the present invention proposes a positioning device based on RGBD, including: a laser radar map acquisition device for acquiring environmental point cloud data; a map construction module for constructing the environmental point cloud data according to a laser mileage calculation method to obtain an offline point cloud map; a first feature extraction module for performing feature extraction on the offline point cloud map to obtain source feature information, wherein the source feature information includes point, line and surface features; an RGBD camera sensor for acquiring a target image of the corresponding environment and acquiring three-dimensional point cloud data of the target image; a second feature extraction module for performing feature extraction on the three-dimensional point cloud data to obtain target feature information, wherein the target feature information includes point, line and surface features; a feature fusion matching module for matching the source feature information and the target feature information to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining the positioning result of the RGBD camera sensor in the offline point cloud map.

[0018] According to the RGBD-based positioning device provided by the embodiment of the present invention, the environmental point cloud data is obtained by the laser radar map acquisition device; the environmental point cloud data is then constructed by the map construction module according to the laser mileage calculation method to obtain an offline point cloud map; and the offline point cloud map is feature extracted by the first feature extraction module to obtain source feature information, wherein the source feature information includes point, line and surface features; then the target image of the corresponding environment is obtained by the RGBD camera sensor, and the three-dimensional point cloud data of the target image is obtained; and the three-dimensional point cloud data is feature extracted by the second feature extraction module to obtain target feature information, wherein the target feature information includes point, line and surface features; finally, the source feature information and the target feature information are matched by the feature fusion matching module to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining the positioning result of the RGBD camera sensor in the offline point cloud map. Therefore, the present invention constructs a map by an expensive laser radar device, which can be reused by multiple devices equipped with RGBD cameras, while reducing the use cost of the entire system, greatly improving the accuracy and robustness of indoor positioning.

[0019] In addition, the RGBD-based positioning device proposed in the above embodiment of the present invention may also have the following additional technical features:

[0020] Optionally, the map construction module is further used to perform denoising on the offline point cloud map using CloudCompare.

[0021] Optionally, the first feature extraction module is further used to extract line and surface features in the offline point cloud map using a 3D-LineDetection algorithm, wherein the center point and normal vector information of the surface feature are stored, and the two endpoint information of the line feature are stored; and the Spinnet algorithm is used to extract the descriptors of the points in the offline point cloud map to obtain a 32-dimensional feature descriptor for each point cloud, and the descriptors are stored.

[0022] Optionally, the feature fusion matching module is further used to use RANSAC iteration to obtain the coarse positioning of points in the source feature information between the corresponding points in the target feature information; use ICP algorithm to obtain the rotation amount of points in the source feature information between the corresponding points in the target feature information; establish an optimization function based on the source feature information and the target feature information, so as to obtain the translation amount of the source feature information between the corresponding target feature information through the optimization function. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flowchart of a positioning method based on RGBD according to an embodiment of the present invention;

[0024] Figure 2 FIG. 4 is a block diagram of a positioning device based on RGBD according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0026] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0027] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0028] Figure 1 Flow chart of the RGBD-based positioning method according to an embodiment of the present invention; Figure 1As shown, the RGBD-based positioning method of the embodiment of the present invention includes the following steps:

[0029] Step 101, using laser radar to obtain environmental point cloud data, and constructing the environmental point cloud data according to the laser mileage calculation method to obtain an offline point cloud map.

[0030] That is to say, a laser radar is used to collect environmental point cloud data, and then an offline point cloud is constructed based on the environmental point cloud data using a mature laser mileage calculation method.

[0031] As an example, a laser radar device is used to collect environmental laser data, and then the laser radar is constructed using the laser radar positioning algorithm such as NDT based on the collected laser radar data packets.

[0032] As a specific embodiment, after constructing the environmental point cloud data according to the laser mileage calculation method to obtain an offline point cloud map, CloudCompare is also used to perform denoising on the offline point cloud map.

[0033] It should be noted that during the process of LiDAR map collection, dynamic objects such as people or vehicles, as well as some noise from the sensor itself, will be recorded, so these noises on the map need to be eliminated before use. The specific operation can be done manually by using software such as CloudCompare to perform some specific processing of the point cloud, or by using algorithms for automatic noise removal.

[0034] Step 102: extract features from the offline point cloud map to obtain source feature information, wherein the source feature information includes point, line and surface features.

[0035] As a specific embodiment, feature extraction is performed on the offline point cloud map to obtain source feature information, including:

[0036] The 3D-LineDetection algorithm is used to extract line and surface features in the offline point cloud map, where the center point and normal vector information of the surface feature are stored, and the two endpoint information of the line feature is stored;

[0037] The Spinnet algorithm is used to extract the descriptors of the points in the offline point cloud map to obtain a 32-dimensional feature descriptor for each point cloud, and the descriptors are stored.

[0038] That is to say, different methods are used to extract features of points, lines, and surfaces; the feature information of lines and surfaces is extracted and stored through the point cloud algorithm using structural features such as normal vectors and curvature from offline point cloud maps; for example, the 3D-LineDetection algorithm can be used to extract plane and line segment feature data from point cloud maps offline, and then the center point and normal vector information of a plane and the two endpoints of a line segment are stored for subsequent matching work; the point cloud deep learning algorithm is used to extract the descriptors of points in the offline point cloud map; for example, the Spinnet algorithm can be used to obtain a 32-dimensional feature descriptor for each point cloud, and the descriptor is stored in the feature space for subsequent matching work.

[0039] Step 103: Use an RGBD camera sensor to acquire a target image of the corresponding environment, and acquire three-dimensional point cloud data of the target image.

[0040] Step 104 , extracting features from the three-dimensional point cloud data to obtain target feature information, wherein the target feature information includes point, line and surface features.

[0041] As an embodiment, the color image and depth image of the environment are collected by an RGBD camera sensor and converted into a three-dimensional point cloud, and the point, line and surface features in the environment are extracted using the same algorithm as the above-mentioned source feature information extraction.

[0042] It should be noted that the map constructed by the lidar device can be reused by multiple devices equipped with RGBD cameras. That is to say, the source feature information extracted from an offline point cloud map can be matched with multiple target feature information, thereby obtaining the corresponding positioning results of multiple devices equipped with RGBD cameras in the offline point cloud map.

[0043] Step 105 , matching the source feature information with the target feature information to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining a positioning result of the RGBD camera sensor in the offline point cloud map.

[0044] It should be noted that due to the many differences between the two point clouds such as density and viewing angle, the traditional point cloud matching algorithm cannot be used directly; for this reason, the present invention adopts two stages to perform rotation and translation calculations.

[0045] As an embodiment, for point features, in the feature space extracted in the previous step, a rough positioning between two point clouds is obtained through RANSAC iteration; this step can make the overlapping part between the two point clouds as close as possible; and then the principle of ICP is used to calculate the rotation amount R between the two.

[0046] After estimating the rotation, the translation is further estimated using points, lines, and planes.

[0047] First, the point cloud of the lidar map is projected into the field of view of RGBD, and then the optimization function is established through the corresponding point, line, and surface features of the extracted RGBD point cloud and the lidar point cloud, and finally the translation amount is calculated.

[0048] For point features, for example, the reprojection error is used to define the error function e of the j-th point in the point cloud P at the k-th frame data:

[0049]

[0050] in, is the reprojection function, R is the rotation obtained in the previous step, and t is the translation to be calculated.

[0051] That is, the estimated point cloud P of the jth frame is reprojected into the image of the kth frame using the reprojection error function, and the error function e is obtained by subtracting the actual measured value on the kth frame.

[0052] For the straight line feature, the straight line is first normalized to obtain the error function defined between the straight lines:

[0053]

[0054] in, is the normalized straight line, Represented as an endpoint of a straight line.

[0055] For surface features, in order to obtain the plane for optimization , the plane is defined using the following formula:

[0056]

[0057] The error function of the plane is defined as follows:

[0058]

[0059] in, and Expressed as the azimuth and elevation of the normal, is the transformation relationship between the camera and the world coordinate system, the normal vector of the plane is n=(nx, ny, nz), and d is the distance from the camera origin to the plane.

[0060] So far, the translation estimate is defined as follows:

[0061]

[0062] in, is the inverse covariance matrix of the points, , and They are respectively the robust Huber cost function.

[0063] Finally, the LM algorithm is used iteratively to obtain the final result.

[0064] Through the above steps, the final positioning of the RGBD camera in the prior LiDAR point cloud map is obtained from the three features of point, line and surface.

[0065] In summary, according to the RGBD-based positioning method of the embodiment of the present invention, a laser radar is first used to obtain environmental point cloud data, and the environmental point cloud data is constructed according to the laser mileage calculation method to obtain an offline point cloud map; then the offline point cloud map is feature extracted to obtain source feature information, wherein the source feature information includes point, line and surface features; then the RGBD camera sensor is used to obtain the target image of the corresponding environment, and the three-dimensional point cloud data of the target image is obtained; then the three-dimensional point cloud data is feature extracted to obtain target feature information, wherein the target feature information includes point, line and surface features; finally, the source feature information and the target feature information are matched to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining the positioning result of the RGBD camera sensor in the offline point cloud map. Therefore, the present invention constructs a map through an expensive laser radar device, which can be reused by multiple devices equipped with RGBD cameras, while reducing the cost of using the entire system, greatly improving the accuracy and robustness of indoor positioning.

[0066] In addition, an embodiment of the present invention further proposes a computer-readable storage medium on which a positioning program based on RGBD is stored. When the positioning program based on RGBD is executed by a processor, the positioning method based on RGBD as described above is implemented.

[0067] According to the computer-readable storage medium of an embodiment of the present invention, by storing an RGBD-based positioning program, the RGBD-based positioning program is executed by the processor to implement the RGBD-based positioning method as described above, thereby performing real-time positioning in the prior laser point cloud map through the RGBD camera, thereby greatly improving the accuracy and robustness of positioning while reducing costs.

[0068] In addition, an embodiment of the present invention further proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the RGBD-based positioning method as described above is implemented.

[0069] According to the computer device of an embodiment of the present invention, a RGBD-based positioning program is stored in a memory, so that when the RGBD-based positioning program is executed by a processor, the RGBD-based positioning method as described above is implemented, thereby performing real-time positioning in a priori laser point cloud maps through an RGBD camera, thereby greatly improving the accuracy and robustness of positioning while reducing costs.

[0070] Figure 2 FIG. 1 is a block diagram of a positioning device based on RGBD according to an embodiment of the present invention. Figure 2 As shown, the RGBD-based positioning device includes: a laser radar map acquisition device 201, a map construction module 202, a first feature extraction module 203, an RGBD camera sensor 204, a second feature extraction module 205 and a feature fusion matching module 206.

[0071] Among them, the optical radar map acquisition device 201 is used to obtain environmental point cloud data; the map construction module 202 is used to construct the environmental point cloud data according to the laser mileage calculation method to obtain an offline point cloud map; the first feature extraction module 203 is used to extract features from the offline point cloud map to obtain source feature information, wherein the source feature information includes point, line and surface features; the RGBD camera sensor 204 is used to obtain a target image of the corresponding environment and obtain three-dimensional point cloud data of the target image; the second feature extraction module 205 is used to extract features from the three-dimensional point cloud data to obtain target feature information, wherein the target feature information includes point, line and surface features; the feature fusion matching module 206 is used to match the source feature information and the target feature information to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining the positioning result of the RGBD camera sensor in the offline point cloud map.

[0072] As an embodiment, the map construction module 202 is further used to perform denoising on the offline point cloud map using CloudCompare.

[0073] As an embodiment, the first feature extraction module 203 is further used to extract line and surface features in the offline point cloud map using a 3D-LineDetection algorithm, wherein the center point and normal vector information of the surface feature are stored, and the two endpoint information of the line feature are stored; the Spinnet algorithm is used to extract the descriptor of the point in the offline point cloud map to obtain a 32-dimensional feature descriptor for each point cloud, and the descriptor is stored.

[0074] As an embodiment, the feature fusion matching module 206 is further used to use RANSAC iteration to obtain the coarse positioning of points in the source feature information between the points in the corresponding target feature information; use the ICP algorithm to obtain the rotation amount between the points in the source feature information and the points in the corresponding target feature information; establish an optimization function based on the source feature information and the target feature information, so as to obtain the translation amount of the source feature information between the corresponding target feature information through the optimization function.

[0075] It should be noted that the above explanations and descriptions of the embodiment of the RGBD-based positioning method are also applicable to the RGBD-based positioning device of this embodiment, and will not be repeated here.

[0076] In summary, according to the RGBD-based positioning device provided by the embodiment of the present invention, the environmental point cloud data is obtained by the laser radar map acquisition device; the environmental point cloud data is then constructed by the map construction module according to the laser mileage calculation method to obtain an offline point cloud map; and the offline point cloud map is feature extracted by the first feature extraction module to obtain source feature information, wherein the source feature information includes point, line and surface features; then the target image of the corresponding environment is obtained by the RGBD camera sensor, and the three-dimensional point cloud data of the target image is obtained; and the three-dimensional point cloud data is feature extracted by the second feature extraction module to obtain target feature information, wherein the target feature information includes point, line and surface features; finally, the source feature information and the target feature information are matched by the feature fusion matching module to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining the positioning result of the RGBD camera sensor in the offline point cloud map. Therefore, the present invention constructs a map by an expensive laser radar device, which can be reused by multiple devices equipped with RGBD cameras, while reducing the use cost of the entire system, greatly improving the accuracy and robustness of indoor positioning.

[0077] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0081] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0082] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0083] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0084] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0085] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0086] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms should not be understood as necessarily being directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0087] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A positioning method based on RGBD, It is characterized in that The following steps are involved: Using laser radar to obtain environmental point cloud data, and constructing the environmental point cloud data according to the laser mileage calculation method to obtain an offline point cloud map; Performing feature extraction on the offline point cloud map to obtain source feature information, wherein the source feature information includes point, line and surface features; Using an RGBD camera sensor to acquire a target image of a corresponding environment, and acquiring three-dimensional point cloud data of the target image; Performing feature extraction on the three-dimensional point cloud data to obtain target feature information, wherein the target feature information includes point, line and surface features; Matching the source feature information and the target feature information to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining a positioning result of the RGBD camera sensor in the offline point cloud map; The step of extracting features from the offline point cloud map to obtain source feature information includes: A 3D-LineDetection algorithm is used to extract line features and surface features in the offline point cloud map, wherein the center point and normal vector information of the surface feature are stored, and the two endpoint information of the line feature are stored; The Spinnet algorithm is used to extract the descriptors of the points in the offline point cloud map to obtain a 32-dimensional feature descriptor of each point cloud, and the descriptors are stored; The source feature information and the target feature information are matched to perform posture recovery and six-degree-of-freedom estimation, including: Using RANSAC iteration to obtain a rough positioning of points in the source feature information between corresponding points in the target feature information; Using the ICP algorithm to obtain the rotation amount between the points in the source feature information and the corresponding points in the target feature information; An optimization function is established according to the source feature information and the target feature information, so as to obtain the translation amount of the source feature information between the corresponding target feature information through the optimization function.

2. The RGBD-based positioning method according to claim 1, It is characterized in that After constructing the environmental point cloud data according to the laser mileage calculation method to obtain an offline point cloud map, the method further includes: using CloudCompare to perform denoising on the offline point cloud map.

3. A computer-readable storage medium, It is characterized in that An RGBD-based positioning program is stored thereon, and when the RGBD-based positioning program is executed by a processor, the RGBD-based positioning method as described in any one of claims 1-2 is implemented.

4. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the RGBD-based positioning method as described in any one of claims 1 to 2 is implemented.

5. A positioning device based on RGBD, It is characterized in that include: LiDAR map acquisition equipment, used to obtain environmental point cloud data; A map construction module, used to construct the environmental point cloud data according to the laser mileage calculation method to obtain an offline point cloud map; A first feature extraction module, used for performing feature extraction on the offline point cloud map to obtain source feature information, wherein the source feature information includes point, line and surface features; An RGBD camera sensor is used to obtain a target image of a corresponding environment and obtain three-dimensional point cloud data of the target image; A second feature extraction module, used to extract features from the three-dimensional point cloud data to obtain target feature information, wherein the target feature information includes point, line and surface features; A feature fusion matching module, used to match the source feature information with the target feature information to perform posture recovery and six-degree-of-freedom estimation, thereby obtaining a positioning result of the RGBD camera sensor in the offline point cloud map; The first feature extraction module is further used to extract line features and surface features in the offline point cloud map using a 3D-LineDetection algorithm, wherein the center point and normal vector information of the surface feature are stored, and the two endpoint information of the line feature are stored; the descriptor of the point in the offline point cloud map is extracted using a Spinnet algorithm to obtain a 32-dimensional feature descriptor for each point cloud, and the descriptor is stored; Among them, the feature fusion matching module is further used to use RANSAC iteration to obtain the coarse positioning of points in the source feature information between the corresponding points in the target feature information; use ICP algorithm to obtain the rotation amount of points in the source feature information between the corresponding points in the target feature information; establish an optimization function according to the source feature information and the target feature information, so as to obtain the translation amount of the source feature information between the corresponding target feature information through the optimization function.

6. The RGBD-based positioning device according to claim 5, It is characterized in that The map construction module is further used to perform denoising on the offline point cloud map using CloudCompare.

Images