Road surface point cloud data acquisition method, system and device based on depth image and SIFT operator, and storage medium

Abstract

The invention discloses a road surface point cloud data acquisition method, system and device based on a depth image and an SIFT operator, and a storage medium, and relates to the technical field of point cloud processing, and the method comprises the steps: obtaining an RGB image and a depth image; converting the depth image into a point cloud; and carrying out RGB image splicing and point cloud splicing. Compared with the prior art, the road surface point cloud data acquisition method based on the depth image and the SIFT operator provided by the invention has the advantages that the cost is lower, and the data acquisition is more convenient. The required data volume is small, and the requirements for calculation and storage performance are low. The environment adaptation effect is good, and weather interference conditions such as rain and snow can be dealt with more easily. The automation degree is high, less manual intervention is needed, and the technical threshold is low. Data acquisition is fast, and application scenarios are wide. The method can solve the problems that in the prior art, data acquisition equipment is high in cost, the acquired data size is large, and the requirement for data storage and calculation is high, and the calculation requirement and cost can be greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud processing, and in particular to a method, system, device and storage medium for acquiring road surface point cloud data based on a depth image and a SIFT operator. Background Art

[0002] As the global urbanization process accelerates, the size of cities continues to expand, and the demand for transportation grows rapidly. A large number of road construction and expansion have doubled the pressure on road maintenance. At the same time, the road infrastructure in many countries and regions has entered the aging stage. Long-term use and lack of maintenance have led to various problems on the road surface, such as cracks and potholes. In this context, timely and accurate acquisition of road surface data has become an important basis for ensuring road life and traffic safety. Most current methods are based on laser technology to obtain road surface point cloud data. Although this method performs well in terms of accuracy and range, it still has some defects and challenges:

[0003] 1. Real-time and convenience: Laser scanning equipment usually takes a long time to complete the entire scanning process, especially when high precision is required. The equipment is also usually large in size and not easy to carry and deploy.

[0004] 2. High cost: Laser scanning devices, such as LiDAR, are usually expensive, which limits their popularization and application in many scenarios, especially in the consumer market.

[0005] 3. Large amount of data: The point cloud data generated by laser scanning is very large, especially in the case of high resolution, and the data storage and processing requirements are very high, requiring powerful hardware resources and complex processing algorithms.

[0006] 4. Dynamic scene challenges: In dynamic environments, such as scanning moving objects or traffic scenes, the real-time acquisition and processing of laser point clouds is more complicated and may lead to data inaccuracy.

[0007] 5. Greatly affected by the environment: Laser technology performs poorly in adverse weather conditions (such as rain, snow, and fog). Laser signals will be affected by scattering or reflection, resulting in reduced data quality.

[0008] There are few methods that can be directly applied to the acquisition and stitching of road surface point cloud data, and the existing methods are relatively complex to implement, have poor scalability, and are costly and technically difficult. Summary of the invention

[0009] In view of the above-mentioned problems, the present invention is proposed.

[0010] Therefore, the technical problem solved by the present invention is: the existing technical laser acquisition method has the problems of high cost for acquiring point cloud data, high requirements for computing performance and storage performance, low degree of automation, and how to reduce the interference of environmental conditions on data collection results.

[0011] To solve the above technical problems, the present invention provides the following technical solutions: a method for acquiring road surface point cloud data based on depth image and SIFT operator, comprising acquiring RGB image and depth image; converting the depth image into point cloud; and performing RGB image stitching and point cloud stitching.

[0012] As a preferred solution of the method for acquiring road surface point cloud data based on depth image and SIFT operator described in the present invention, wherein: acquiring RGB image and depth image includes shooting road surface data with a depth camera, and the depth camera can obtain RGB image and depth image.

[0013] As a preferred solution of the method for acquiring road surface point cloud data based on depth image and SIFT operator of the present invention, wherein: the conversion of the depth image into point cloud includes converting the RGB image and the depth image into point cloud data, and calculating the mapping from the two-dimensional image coordinates to the three-dimensional space coordinates, which is expressed as: (1) (2) (3) Where (u, v) is the pixel coordinate in the image, (X, Y, Z) is the coordinate of the point in three-dimensional space, D(u, v) is the depth value of the pixel (u, v) in the depth map, cx, cy is the position of the optical center of the camera in the image coordinate system, fx, fy is the focal length of the camera, depth_scale is the unit scaling factor of the depth map, here let depth_scale = 10.

[0014] As a preferred solution of the method for acquiring road surface point cloud data based on depth image and SIFT operator described in the present invention, the conversion of depth image into point cloud includes first using Open3D library function to read RGB image and depth image, then creating an RGBD image object RGBDImage, combining the RGB image with the depth image, using depth_scale to convert the depth image from image unit to actual depth value, and setting the depth cutoff distance to limit the depth range, and then defining the projection model of the camera for projecting the RGBD image into three-dimensional space, realizing the conversion of RGBD image into point cloud data, and saving the generated point cloud as a file in PCD format.

[0015] As a preferred solution of the method for acquiring road surface point cloud data based on depth image and SIFT operator described in the present invention, the RGB image stitching also includes using a stitcher function to stitch the RGB image parts obtained by the depth camera as a whole, and stitching the RGB images into a whole image.

[0016] As a preferred solution of the road surface point cloud data acquisition method based on depth image and SIFT operator described in the present invention, the RGB image stitching includes using SIFT operator to extract and match features of each image with the overall image again to obtain the homography matrix required for the transformation of each image to the overall image.

[0017] As a preferred solution of the method for acquiring road surface point cloud data based on depth image and SIFT operator described in the present invention, the point cloud stitching includes using the point cloud generated corresponding to each picture and the homography matrix between each RGB image and the overall image to realize the planar transformation of the corresponding point cloud, and stitching each point cloud data into a complete road surface point cloud data.

[0018] Another object of the present invention is to provide a road surface point cloud data acquisition system based on depth images and SIFT operators, which can use a depth camera to capture road surface data through one of the schemes, thereby solving the problems of current laser acquisition technology, such as poor real-time performance and convenience, high cost, and great environmental impact.

[0019] As a preferred solution of the road surface point cloud data acquisition system based on depth image and SIFT operator described in the present invention, it includes: an image acquisition module, a conversion module, and an image stitching module. The image acquisition module is used to acquire RGB images and depth images. The conversion module is used to convert each depth image and its corresponding RGB image into a separate point cloud file. The image stitching module is used to apply the transformation matrix obtained when stitching the RGB images to the corresponding point cloud data, so as to synthesize each separated point cloud into the point cloud data of the entire road.

[0020] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of a method for acquiring road surface point cloud data based on a depth image and a SIFT operator.

[0021] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for acquiring road surface point cloud data based on a depth image and a SIFT operator.

[0022] Beneficial effects of the present invention: Compared with existing methods, the method for acquiring road surface point cloud data based on depth image and SIFT operator provided by the present invention has lower cost and is more convenient to acquire data. The required amount of data is small, and the requirements for computing and storage performance are low. It has good adaptability to the environment and is easier to deal with weather interference such as rain and snow. It has a high degree of automation, requires less manual intervention, and has a low technical threshold. It acquires data quickly and has a wide range of application scenarios. This method can solve the problems of high cost of data acquisition equipment, large amount of data acquired, and high requirements for data storage and computing in current technology, and can greatly reduce computing requirements and costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0024] Figure 1 An overall flow chart of a method for acquiring road surface point cloud data based on depth image and SIFT operator provided for the first embodiment of the present invention.

[0025] Figure 2 An example diagram of collected RGB images of a method for acquiring road surface point cloud data based on a depth image and a SIFT operator provided in the second embodiment of the present invention.

[0026] Figure 3 A stitching result diagram using a stitcher function of a method for acquiring road surface point cloud data based on a depth image and a SIFT operator provided in a second embodiment of the present invention.

[0027] Figure 4 A method for acquiring road surface point cloud data based on depth image and SIFT operator provided in the second embodiment of the present invention includes a stitching result image corresponding to a panoramic image using the SIFT operator.

[0028] Figure 5 A test stitching RGB result image of a method for acquiring road surface point cloud data based on a depth image and a SIFT operator provided in the second embodiment of the present invention.

[0029] Figure 6 A test point cloud stitching result diagram of a method for acquiring road surface point cloud data based on a depth image and a SIFT operator provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0031] Example 1, reference Figure 1 , as an embodiment of the present invention, provides a method for acquiring road surface point cloud data based on a depth image and a SIFT operator, comprising: S1: Get RGB image and depth image.

[0032] Furthermore, obtaining the RGB image and the depth image includes photographing road surface data using a depth camera, and the depth camera can obtain the RGB image and the depth image.

[0033] Specifically, the road surface data is captured by a depth camera, which can obtain RGB images and depth images. The depth camera is controlled by a program. To ensure that the acquired data is complete and suitable for subsequent image stitching and fusion, the shooting must be completely parallel to the ground to ensure that the RGB image data can be mapped to the point cloud data and the point cloud can be stitched. At the same time, it is also required to be at the same height when taking different photos to ensure that different images have the same depth when stitching, so that when stitching the point cloud, only the stitching on the same plane is considered. In addition, it is required that the distance between each two adjacent images should not be too long, and there should be a large overlap so that there are enough matching feature points when extracting features.

[0034] S2: Convert depth image to point cloud.

[0035] Furthermore, when converting the depth image to a point cloud, including converting the RGB image and the depth image to point cloud data, the mapping from the two-dimensional image coordinates to the three-dimensional space coordinates is mainly calculated. The calculation formula is: (1) (2) (3) Where (u, v) is the pixel coordinate in the image, (X, Y, Z) is the coordinate of the point in three-dimensional space, D(u, v) is the depth value of the pixel (u, v) in the depth map, cx, cy is the position of the optical center of the camera in the image coordinate system, fx, fy is the focal length of the camera, depth_scale is the unit scaling factor of the depth map, here let depth_scale = 10.

[0036] It should be noted that converting a depth image into a point cloud includes first using the Open3D library function to read the RGB image and the depth image, then creating an RGBD image object RGBDImage, combining the RGB image with the depth image, using depth_scale to convert the depth map from image units to actual depth values, and setting the depth cutoff distance to limit the depth range, and then defining the camera's projection model to project the RGBD image into three-dimensional space, converting the RGBD image into point cloud data, and saving the generated point cloud as a PCD format file.

[0037] S3: Perform RGB image stitching and point cloud stitching.

[0038] Furthermore, the existing SIFT algorithm for image stitching has a more obvious effect on feature extraction and stitching between two pictures. However, when this method is applied to multiple pictures, the previous picture needs to be translated, rotated, and other operations to make the feature points correspond. However, some small translation and rotation operations will also have a huge impact after the superposition of multiple pictures. After a certain number of pictures, there may be insufficient feature points that can match between the two pictures, resulting in interrupted stitching. Therefore, in the stitching part of RGB images, this method combines the stitcher function in Opencv with the SIFT operator to realize the stitching of RGB images and obtain the homography matrix applied to image transformation. This matrix prepares for the subsequent stitching of point clouds. It is difficult to obtain the transformation matrix between images using only the stitcher function.

[0039] RGB image stitching and point cloud stitching include using the stitcher function to stitch the RGB image parts captured by the depth camera as a whole, stitching the RGB images into a whole image.

[0040] RGB image stitching and point cloud stitching also include using the SIFT operator to extract and match features of each image with the overall image again, and obtain the homography matrix required for the transformation of each image to the overall image.

[0041] It should be noted that the stitcher function is first used to stitch the RGB image parts captured by the depth camera as a whole, and the RGB images are stitched into a whole image. Then the SIFT operator is used to extract and match the features of each image with the whole image again. This process obtains the homography matrix required for the transformation of each image to the whole image. The SIFT operator first performs Gaussian blur: (4) Where G(x,y,σ) represents the Gaussian blur value at point (x,y), and σ controls the degree of blur. This function is a two-dimensional Gaussian distribution. The larger the σ, the more obvious the blur effect. By applying different σ values, multiple Gaussian images are generated to detect features at different scales.

[0042] Then apply the difference Gaussian: (5) Where k is the scale factor, ,By calculating the difference between two Gaussian images at different scales, the edges and corners in the image can be effectively found, which are the candidate areas of feature points.

[0043] The key point response is evaluated by the Hessian matrix to evaluate the local extrema: (6)

[0044] The Hessian matrix is ​​a second-order derivative matrix used to describe the change in curvature in an image. The key point response can be calculated by the following formula: (7) Where det(H) is the determinant of the Hessian matrix, which reflects the strength of the key point. trace(H) is the trace of the Hessian matrix, that is, the sum of the diagonal elements, which reflects the smoothness of the curvature. k is a constant (usually 0.04) used to balance these two quantities so that more obvious feature points get higher response values.

[0045] After that, a main direction is assigned to each key point to ensure the rotation invariance of the descriptor and the gradient is calculated: (8)

[0046] Compute the gradient magnitude and direction: (9) (10)

[0047] Gradients are calculated around the keypoints, with the magnitude indicating the strength of the pixel change and the direction indicating the direction of the change. By calculating these values, a dominant direction can be assigned to each keypoint, making subsequent descriptors independent of image rotation.

[0048] Generate a feature descriptor in the neighborhood of the key point, and form a 128-dimensional descriptor by calculating the gradient direction histogram around the key point: (11)

[0049] The feature descriptor usually divides the area around the key point into 4×4 sub-regions, and calculates the gradient direction histogram in each sub-region. Each bin of the histogram corresponds to a specific direction. The final 128-dimensional descriptor is composed of the histograms of 16 sub-regions.

[0050] Use descriptors for feature matching and calculate the distance between descriptors: (12)

[0051] The difference between two descriptors is calculated by L2 distance (Euclidean distance), Represents the jth element of the two descriptors respectively. A smaller distance indicates that the two feature points are likely to be the same. The SIFT algorithm ensures its robustness under scale, rotation, and illumination changes and is suitable for various computer vision tasks, including image stitching.

[0052] After obtaining the feature points, use the K nearest neighbor algorithm to find the two best matching points for each feature point: (13) Where d1 and d2 are the distances between the two nearest neighbors, and then the ratio test is used to filter the matching points (14)

[0053] Assume there are n pairs of matching points ( )and( ), for each pair of matching points, construct the following equation (15)

[0054] The linear equations can be solved by the least square method, and finally the homography matrix M is obtained.

[0055] This method uses the RANSAC (Random Sampling Consensus) method to robustly estimate the homography matrix in the presence of noise and outliers. This method has lower time complexity than brute force matching. The RANSAC method randomly selects N pairs of matching points (at least 4) to calculate the preliminary homography matrix, and for each matching point, calculates the reprojection error: (16)

[0056] If the reprojection error is less than the preset threshold, the point is an inlier point. Repeat this process multiple times and record the model with the largest number of inliers as the final homography matrix.

[0057] The above steps, from feature extraction to matching, and then to the calculation of the homography matrix, finally obtain a robust image stitching result. Each formula and step is interrelated to form a complete RGB image stitching process and obtain the homography matrix.

[0058] It should be noted that point cloud stitching includes using the point cloud generated by each image and the homography matrix between each RGB image and the overall image to achieve the plane transformation of the corresponding point cloud, and stitching each point cloud data into a complete road surface point cloud data. Because it is parallel to the ground and photographed at the same height, only the stitching of the point cloud in the x, y plane direction is considered.

[0059] In order to obtain the road surface point cloud data, the point cloud generated corresponding to each picture and the homography matrix between each RGB image and the overall image are used to realize the plane transformation of the corresponding point cloud. Each point cloud data can be spliced ​​into a complete road surface point cloud data. Because it is parallel to the ground and photographed at the same height, only the splicing of the point cloud in the x, y plane direction is considered.

[0060] For each point cloud , using the calculated homography matrix To perform coordinate transformation, first convert each point (x, y) into homogeneous coordinates (x, y, 1), and then perform coordinate transformation. Homogeneous coordinates can use matrix multiplication to uniformly represent transformations such as translation, rotation, scaling, and perspective by introducing an extra dimension (usually 1). In particular, when calculating the perspective transformation (Homography matrix) between images, the introduction of homogeneous coordinates allows two-dimensional plane transformations to be completed using matrix operations. Coordinate transformation formula: (17)

[0061] Finally, all transformed point clouds are merged into an initial empty point cloud. After merging, there may be overlapping points, which need to be deduplicated. Define a distance threshold d. If the distance between two points is less than d, they are considered to overlap: (18)

[0062] After deduplication, the complete road surface point cloud data is obtained.

[0063] Example 2, reference Figure 2-Figure 6 , which is an embodiment of the present invention, provides a method for acquiring road surface point cloud data based on depth image and SIFT operator. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0064] First, a depth camera is used to capture RGB images and depth images of multiple road surface areas. During the capture process, the depth camera is set parallel to the ground and the shooting height is kept consistent to ensure that different images have the same depth when stitched together. In addition, there is enough overlap between adjacent images (the overlap area is not less than 30%) to ensure that the image features can match.

[0065] The captured depth image is converted into point cloud data using the Open3D library. Each pair of RGB image and depth image is first loaded into the program, and the depth map is converted into actual depth values ​​by setting the depth scaling factor. Then, the mapping from two-dimensional pixel coordinates to three-dimensional space coordinates is calculated based on the camera intrinsic parameters. All converted point cloud data are saved in PCD format for subsequent processing.

[0066] For the stitching of RGB images, we first use the Stitcher function in OpenCV for preliminary stitching. On this basis, we use the SIFT (Scale Invariant Feature Transform) algorithm to extract and match the image feature points, and calculate the homography matrix from each image to the overall image. The feature points obtained by the SIFT algorithm are matched, and the RANSAC algorithm is combined to remove abnormal matching points, and finally the accurate transformation matrix required for image stitching is obtained.

[0067] For point cloud stitching, coordinate transformation is performed using the point cloud data generated for each image and the homography matrix between the image and the overall image. Since the camera remains parallel to the ground during shooting, point cloud stitching only considers the transformation of the XY plane. During the point cloud stitching process, the point cloud of the overlapping parts is deduplicated and all point cloud data are merged to obtain a complete road surface point cloud.

[0068] Table 1 Experimental data comparison table Test subject name Image number RGB image stitching accuracy Point cloud stitching accuracy Feature matching points Calculation time (seconds) Point count after deduplication of point cloud Experiment 1 Image 1 0.98 0.95 1500 12 45000 Experiment 1 Image 2 0.97 0.94 1450 11 44000 Experiment 2 Image 3 0.99 0.96 1600 10 46000 Experiment 2 Image 4 0.96 0.92 1400 13 43000 Experiment 3 Image 5 0.98 0.95 1550 12 45000 Experiment 3 Image 6 0.97 0.93 1480 11 44000

[0069] From the table data, the RGB image stitching accuracy remained above 0.96 in most experiments, with the highest reaching 0.99, indicating that this method exhibits high accuracy in the process of stitching multiple images. The point cloud stitching accuracy is also between 0.92 and 0.96, indicating that the point cloud stitching method implemented by the homography matrix has good accuracy and robustness for the recovery of complex road surface data. Compared with the traditional single image stitching method, this method can better ensure the integrity and accuracy of point cloud data, especially in the accurate reconstruction of road surfaces.

[0070] The number of feature matching points reflects the complexity of matching between images and the quality of feature points. A higher number of matching points (such as 1600 feature points) usually means that there is more available information between images for splicing, which improves the accuracy of image fusion. From the experimental data, the number of feature matching points is basically stable between 1400 and 1600, without large fluctuations, indicating that this method can stably extract image feature points and perform effective matching.

[0071] In terms of computing time, the average computing time of the experiment is 11.5 seconds. Although the time of different experiments varies slightly, the overall time is within a reasonable range, ensuring the real-time nature of data processing. Compared with traditional image stitching methods, the computing efficiency of this method has been effectively improved, especially when processing large-scale image data, it has obvious advantages.

[0072] After deduplication, the final point cloud data points in all experiments were maintained between 43,000 and 46,000, and the amount of data after deduplication was relatively stable. The deduplication operation effectively eliminated overlapping points, ensuring the efficiency and accuracy of the point cloud data. Compared with the non-deduplicated data, the deduplicated point cloud data not only reduced the storage volume, but also improved the efficiency of subsequent processing.

[0073] Reference Figure 6 It can be seen that the upper part of the picture is the corresponding matrix obtained by using SIFT to stitch two-dimensional images. The separate and prepared point cloud data are stitched into a schematic diagram of the entire point cloud, but the point cloud is not deduplicated. The one below is after deduplication. The white part above is tested with A4 paper. Whether the original connected handwriting on the paper can be connected together after stitching, it approximately reflects the stitching effect of the road surface.

[0074] Compared with the existing technology, this method is significantly innovative in many aspects. First, in the image stitching part, by combining the advantages of the Stitcher function and the SIFT algorithm, the stitching accuracy can be effectively improved, especially when processing more images, the accuracy of the transformation matrix of each image is guaranteed, and the stitching interruption problem caused by small errors is reduced. Secondly, through the introduction of the homography matrix, point cloud stitching can accurately realize the XY plane stitching of the point cloud, solving the stitching problem when scanning large-scale road surfaces. Finally, by deduplicating the point cloud data, the storage and processing efficiency of the point cloud data is optimized, greatly improving the practical application value of the algorithm.

[0075] Embodiment 3, an embodiment of the present invention, provides a road surface point cloud data acquisition system based on depth image and SIFT operator, including an image acquisition module, a conversion module, and an image stitching module.

[0076] The image acquisition module is used to obtain RGB images and depth images. The conversion module is used to convert each depth image and its corresponding RGB image into a separate point cloud file. The image stitching module is used to apply the transformation matrix obtained when stitching the RGB images to the corresponding point cloud data, so as to synthesize each separated point cloud into the point cloud data of the entire road.

[0077] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0078] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0079] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk case (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0080] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logical function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0081] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for acquiring road surface point cloud data based on depth image and SIFT operator, characterized in that: include: Get RGB image and depth image; Convert depth image to point cloud; Perform RGB image stitching and point cloud stitching.

2. The method for acquiring road surface point cloud data based on depth image and SIFT operator as claimed in claim 1, characterized in that: The obtaining of the RGB image and the depth image includes photographing the road surface data using a depth camera, and the depth camera can obtain the RGB image and the depth image.

3. The method for acquiring road surface point cloud data based on depth image and SIFT operator as claimed in claim 1, characterized in that: The converting of the depth image into a point cloud includes converting the RGB image and the depth image into point cloud data, and calculating the mapping from the two-dimensional image coordinates to the three-dimensional space coordinates, which is expressed as: （1） （2） （3） Where (u, v) is the pixel coordinate in the image, (X, Y, Z) is the coordinate of the point in three-dimensional space, D(u, v) is the depth value of the pixel (u, v) in the depth map, cx, cy is the position of the optical center of the camera in the image coordinate system, fx, fy is the focal length of the camera, depth_scale is the unit scaling factor of the depth map, here let depth_scale = 10.

4. The method for acquiring road surface point cloud data based on depth image and SIFT operator as claimed in claim 3, characterized in that: The conversion of the depth image into a point cloud includes first using the Open3D library function to read the RGB image and the depth image, then creating an RGBD image object RGBDImage, combining the RGB image with the depth image, using depth_scale to convert the depth image from image units to actual depth values, and setting the depth cutoff distance to limit the depth range, and then defining the camera's projection model for projecting the RGBD image into three-dimensional space, converting the RGBD image into point cloud data, and saving the generated point cloud as a file in PCD format.

5. The method for acquiring road surface point cloud data based on depth image and SIFT operator as claimed in claim 4, characterized in that: The RGB image stitching includes using a stitcher function to stitch the RGB image parts captured by the depth camera as a whole, and stitching the RGB images into a whole image.

6. The method for acquiring road surface point cloud data based on depth image and SIFT operator according to claim 5, characterized in that: The RGB image stitching includes using the SIFT operator to extract and match features of each image with the overall image again, and obtaining the homography matrix required for transforming each image to the overall image.

7. The method for acquiring road surface point cloud data based on depth image and SIFT operator according to claim 6, characterized in that: The point cloud stitching includes using the point cloud generated corresponding to each image and the homography matrix between each RGB image and the overall image to realize the plane transformation of the corresponding point cloud, and stitching each point cloud data into a complete road surface point cloud data.

8. A system using the method for acquiring road surface point cloud data based on depth image and SIFT operator as claimed in any one of claims 1 to 7, characterized in that: Image acquisition module, conversion module, image stitching module; The image acquisition module is used to acquire RGB images and depth images; The conversion module is used to convert each depth image and its corresponding RGB image into a separate point cloud file; The image stitching module is used to apply the transformation matrix obtained when stitching the RGB images to the corresponding point cloud data, so as to synthesize each separated point cloud into the point cloud data of the entire road.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for acquiring road surface point cloud data based on depth image and SIFT operator according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for acquiring road surface point cloud data based on a depth image and a SIFT operator according to any one of claims 1 to 7 are implemented.

Images