Point cloud image data fusion method, system and device based on improved DBSCAN clustering algorithm and storage medium

Abstract

The invention discloses a point cloud image data fusion method, system and device based on an improved DBSCAN clustering algorithm and a storage medium, and belongs to the field of multi-sensor data fusion, the method uses an acquisition system composed of a laser radar and a camera to complete point cloud and image shooting and acquisition of a detected target scene, and the point cloud and image shooting and acquisition have different characteristics; the sensor fusion mode is used for target detection, so that advantage complementation can be realized; an improved DBSCAN algorithm is used for completing detection target point cloud clustering and generating a three-dimensional bounding box, finally, a point cloud cluster bounding box model is projected to a corresponding image pixel position, fusion of image data and point cloud data is completed, the DBSCAN point cloud clustering algorithm is improved, automatic calculation of a minimum neighborhood and an initial radius value is achieved, and finally, point cloud detection is completed. And the efficiency of detecting the target point cloud clustering is improved, so that the robustness and the accuracy of the detected target are improved, and the omission ratio and the false detection rate of target detection under the condition that the illumination condition is insufficient are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-sensor data fusion, and in particular relates to a point cloud image data fusion method, system, device and storage medium based on an improved DBSCAN clustering algorithm. Background Art

[0002] There are many kinds of sensors for sensing the environment, including laser radar, camera, inertial measurement unit (IMU), global positioning system (GPS), etc. Laser radar is a mechanism that senses the attitude and position of a target by emitting laser pulses based on the reflected echo. Laser radar can accurately detect related parameters such as spatial position, target size, shape, and running speed, and has the advantages of covering a wide range of detection. Visual camera is a device that uses optical principles to collect images and has high image stability. The visual camera controls the focus and propagation of light through the lens, and focuses the light onto the sensor through the optical lens to form an image. Inertial measurement unit (IMU) is a device that measures the three-axis attitude angle (or angular velocity) and acceleration of an object, and has very important application value in navigation. Global Positioning System (GPS) is a high-precision radio navigation positioning system based on artificial satellites.

[0003] A single visual sensor is greatly affected by the ambient light conditions. The outdoor environment is relatively complex. Changes in high-altitude lighting limit the target recognition of the visual sensor, making it easy to miss or misdetect targets. Summary of the invention

[0004] The purpose of the present invention is to provide a point cloud image data fusion method, system, device and storage medium based on an improved DBSCAN clustering algorithm to solve the problem that a single visual sensor is greatly affected by the environment, which limits target recognition and monitoring and is prone to missed detection and false detection.

[0005] The embodiment of the present application is implemented as follows.

[0006] Optionally, in some embodiments of the present application, a point cloud image data fusion method based on an improved DBSCAN clustering algorithm includes:

[0007] Input the initial radius ε, the minimum number of neighbors Minpts and the point cloud data D;

[0008] Select a point q that has not been visited from the point cloud data D, and find all points that meet the minimum number of neighbors Minpts in the area with q as the center and the initial radius of ε, that is, the points that can be reached by q point density;

[0009] Get the number P of point q in the neighborhood of eps i , if P i≥Minpts, q points are classified as core points. If P i ≤Minpt, the q point is classified as a noise point until the optimal minimum number of neighbors Minpts is found;

[0010] Continue to search for points that are directly reachable by point q. The points searched above are core clustering points.

[0011] Repeat the above steps until all point clouds belonging to the detection target in the point cloud data D are divided and searched, then it is determined that the point cloud improved DBSCAN clustering is completed, and the point cloud cluster after improved DBSCAN clustering is obtained;

[0012] Establish a 3D bounding model of the point cloud cluster after improving DBSCAN clustering;

[0013] The 3D enclosing model of the point cloud cluster established after the improved DBSCAN clustering is projected to the corresponding image pixel position to complete the fusion of point cloud data and image data.

[0014] Optionally, in some embodiments of the present application, obtaining the initial radius ε includes:

[0015] rang=Max{P k |0≤k≤m}

[0016] In the formula, the initial radius ε is the median value of the segment where rang is located, P k It is represented by the detection frequency of dist(i,j) in each segment after the random distance interval distance is divided into m segments. Dist(i,j) represents the Euclidean distance between points i and j, and rang represents the maximum frequency.

[0017] The acquisition of dist(i,j) includes:

[0018]

[0019] Where dist(i,j) represents the Euclidean distance between points i and j; i Indicates the distance between point i and the X coordinate axis; y i Indicates the distance between point i and the Y coordinate axis; z i Indicates the distance between point i and the Z coordinate axis; x j Indicates the distance between point j and the X coordinate axis; y j Indicates the distance between point j and the Y coordinate axis; z j Indicates the distance between point j and the Z coordinate axis.

[0020] The distance interval distance is obtained by:

[0021] maxdist=Max{dist(ij|0≤i≤n,0≤j≤D}

[0022] mindist=Min{dist(ij|0≤i≤n,0≤j≤D}

[0023] distance = maxdist-mindist

[0024] In the formula, maxdist represents the maximum distance between points i and j, mindist represents the minimum distance between points i and j, distance represents the distance interval between points i and j, n represents the number of points, and D represents the point cloud data.

[0025] Optionally, in some embodiments of the present application, the point cloud data D is obtained by downsampling the original point cloud data using a voxel grid method, removing outlier noise points in the point cloud data using a statistical filtering algorithm, and then performing ground segmentation using a random sampling consensus algorithm RANSAC to obtain the point cloud data D.

[0026] Optionally, in some embodiments of the present application, the number P of q points in the neighborhood of eps i The gains include:

[0027] P i =count{dist(i,j)<ε|0≤j≤D}

[0028] Where P i represents the number of neighboring points of point i, dist(i,j) represents the Euclidean distance between points i and j, ε represents the initial radius, and D represents the point cloud data.

[0029] Optionally, in some embodiments of the present application, the method for obtaining the 3D enclosing model includes: obtaining the maximum point cloud P in the point cloud cluster after improved DBSCAN clustering max and the minimum point cloud P min , then the calculation formula for the length l, width w, and height h of the 3D bounding box is:

[0030] l = x max -x min

[0031] w=y max -y min

[0032] h=z max -z min

[0033] In the formula, x max Indicates the maximum value within the range of the X-axis; x minIndicates the minimum value within the range of the X coordinate axis; y max Indicates the maximum value within the range of the Y coordinate axis; y min Indicates the minimum value within the range of the Y coordinate axis; z max Indicates the maximum value within the range of the Z coordinate axis; z min Indicates the minimum value within the range of the Z coordinate axis.

[0034] Accordingly, the embodiment of the present application also provides a point cloud image data fusion calculation system based on the improved DBSCAN clustering algorithm, including: a parameter module for inputting an initial radius ε, a minimum number of neighbors Minpts and point cloud data D;

[0035] The q-point density reachable point module is used to select a point q that has not been visited from the point cloud data D, and find all points that meet the minimum number of neighbors Minpts in the area with the q point as the center and the initial radius of ε, that is, the points reachable by the q-point density;

[0036] The ideal minimum number of neighbors Minpts module is used to obtain the number P of points q in the eps neighborhood i , if P i ≥Minpts, q points are classified as core points. If P i ≤Minpt, the q point is classified as a noise point until the optimal minimum number of neighbors Minpts is found;

[0037] The core clustering point module is used to continue searching for points that are directly reachable by the q point density. The points searched above are the core clustering points.

[0038] The point cloud cluster module after improved DBSCAN clustering is used to repeat the above steps until all point clouds belonging to the detection target in the point cloud data D are divided and searched, then it is determined that the point cloud improved DBSCAN clustering is completed, and the point cloud cluster after improved DBSCAN clustering is obtained;

[0039] The 3D enclosing model module of point cloud cluster is used to establish the 3D enclosing model of point cloud cluster after improved DBSCAN clustering;

[0040] The fusion module of point cloud data and image data is used to project the 3D enclosing model of the point cloud cluster established after the improved DBSCAN clustering to the corresponding image pixel position to complete the fusion of point cloud data and image data.

[0041] Accordingly, an embodiment of the present application also provides a computer device, including a storage device and a processor, wherein the storage device stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0042] Accordingly, an embodiment of the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the processor executes the steps of the above method.

[0043] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0044] This application uses a collection system composed of a laser radar and a camera to complete the point cloud and image capture of the detection target scene. The laser radar and the camera are two different environmental perception sensors with different characteristics. The sensor fusion method for target detection can achieve complementary advantages and make up for the shortcomings of single sensor target detection. The improved DBSCAN algorithm is used to complete the clustering of the detection target point cloud and generate a three-dimensional bounding box. Finally, the point cloud cluster bounding box model is projected to the corresponding image pixel position to complete the fusion of image data and point cloud data. The DBSCAN point cloud clustering algorithm is improved to automatically calculate the minimum neighborhood and initial radius value, thereby improving the efficiency of clustering the detection target point cloud. The laser radar and camera data fusion target detection method of this application improves the robustness and accuracy of the detected target, and reduces the missed detection rate and false detection rate of target detection under insufficient lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a scene data collection diagram of the present invention;

[0046] Figure 2 This is a comparison diagram of the point cloud before and after denoising of the present invention;

[0047] Figure 3 This is a comparison diagram before and after ground segmentation using the RANSAC algorithm of the present invention;

[0048] Figure 4 This is the simulation result of DBSCAN clustering point cloud;

[0049] Figure 5 Improve the DBSCAN point cloud clustering simulation results for the present invention;

[0050] Figure 6 It is the effect diagram of the bounding box model and the projection frame of the present invention;

[0051] Figure 7 This is a network structure diagram of the improved YOLOv5s feature fusion network and detection head of the present invention;

[0052] Figure 8 A GAM structure diagram provided for an embodiment of the present invention;

[0053] Fig. 9 A structural diagram of a GAM channel attention module provided by an embodiment of the present invention;

[0054] Fig.10 A structural diagram of a GAM spatial attention module provided by an embodiment of the present invention;

[0055] Fig.11 A schematic diagram of SIoU calculation provided in an embodiment of the present invention;

[0056] Fig.12 A comparison chart of the loss values ​​of YOLOv5s and improved YOLOv5s provided in an embodiment of the present invention;

[0057] Fig.13 A comparison chart of the accuracy, recall rate, and average precision of the YOLOv5s and the improved YOLOv5s provided in the embodiment of the present invention;

[0058] Fig.14 This is a target detection effect diagram of the laser radar-camera data fusion provided by an embodiment of the present invention;

[0059] Fig.15 These are some pictures of a test set with different lighting conditions provided by the embodiments of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0061] The technical solution of this application is as follows:

[0062] The present application embodiment provides a point cloud image data fusion method based on an improved DBSCAN clustering algorithm, including:

[0063] S1, input the initial radius ε, the minimum number of neighbors Minpts and the point cloud data D;

[0064] S2, select a point q that has not been visited from the point cloud data D, and find all points that meet the minimum number of neighbors Minpts in the area with q as the center and the initial radius ε, that is, the points that can be reached by q point density;

[0065] S3. Get the number P of point q in the neighborhood of eps i , if P i ≥Minpts, q points are classified as core points. If P i ≤Minpt, the q point is classified as a noise point until the optimal minimum number of neighbors Minpts is found;

[0066] S4, continue to search for points that are directly reachable by the density of point q, and the points searched above are the core clustering points;

[0067] S5, repeat the above steps until all point clouds belonging to the detection target in the point cloud data D are divided and searched, then it is determined that the point cloud improved DBSCAN clustering is completed, and the point cloud cluster after improved DBSCAN clustering is obtained;

[0068] S6, establishing a 3D enclosing model of the point cloud cluster after improved DBSCAN clustering;

[0069] S7. Project the 3D enclosing model of the point cloud cluster established after the improved DBSCAN clustering to the corresponding image pixel position to complete the fusion of the point cloud data and the image data.

[0070] This application uses a collection system composed of a laser radar and a camera to complete the point cloud and image capture of the detection target scene. The laser radar and the camera are two different environmental perception sensors with different characteristics. The sensor fusion method for target detection can achieve complementary advantages and make up for the shortcomings of single sensor target detection. The improved DBSCAN algorithm is used to complete the clustering of the detection target point cloud and generate a three-dimensional bounding box. Finally, the point cloud cluster bounding box model is projected to the corresponding image pixel position to complete the fusion of image data and point cloud data. The DBSCAN point cloud clustering algorithm is improved to automatically calculate the minimum neighborhood and initial radius value, thereby improving the efficiency of clustering the detection target point cloud. The laser radar and camera data fusion target detection method of this application improves the robustness and accuracy of the detected target, and reduces the missed detection rate and false detection rate of target detection under insufficient lighting conditions.

[0071] In S1:

[0072] In some embodiments, obtaining the initial radius ε includes:

[0073] rang=Max{P k |0≤k≤m}

[0074] In the formula, the initial radius ε is the median value of the segment where rang is located, P k It is expressed as the detection frequency of dist(i,j) in each segment after dividing the random distance interval distance into m segments. Dist(i,j) represents the Euclidean distance between points i and j, and rang represents the maximum frequency.

[0075] Furthermore, dist(i,j) is obtained by:

[0076]

[0077] Where dist(i,j) represents the Euclidean distance between points i and j; i Indicates the distance between point i and the X coordinate axis; y i Indicates the distance between point i and the Y coordinate axis; zi Indicates the distance between point i and the Z coordinate axis; x j Indicates the distance between point j and the X coordinate axis; y j Indicates the distance between point j and the Y coordinate axis; z j Indicates the distance between point j and the Z coordinate axis.

[0078] Furthermore, obtaining the distance interval distance includes:

[0079] maxdist=Max{dist(ij|0≤i≤n,0≤j≤D}

[0080] mindist=Min{dist(ij|0≤i≤n,0≤j≤D}

[0081] distance = maxdist-mindist

[0082] In the formula, maxdist represents the maximum distance between points i and j, mindist represents the minimum distance between points i and j, distance represents the distance interval between points i and j, n represents the number of points, and D represents the point cloud data.

[0083] In some embodiments, the original point cloud data is downsampled using a voxel grid method.

[0084] It can be understood that the original point cloud data obtained contains unnecessary point cloud data such as trees, ground, and walls. If the original point cloud data with high density and redundant information is used for subsequent data fusion, it will affect the computer processing speed and the accuracy of subsequent fusion and registration. This application uses the voxel grid method to downsample the original point cloud data and use a smaller amount of data to approximate the original point cloud data. It can also be understood that the voxel grid method downsampling can significantly reduce the amount of calculation of the original point cloud data while maintaining the spatial distribution characteristics of the original point cloud data.

[0085] Furthermore, the voxel grid method is used to downsample the original point cloud data, and then the statistical filtering algorithm is used to remove the outlier noise points in the point cloud data. It can be understood that when the laser radar collects data, it is affected by its own performance limitations and the complex external environment, resulting in certain errors in the original point cloud data obtained. Point cloud data is prone to deviation in three-dimensional space, and the deviated point cloud data is called outlier noise point. In order to eliminate the interference of outlier noise points on the anti-vibration hammer target detection, the original point cloud data set needs to be denoised.

[0086] Furthermore, the point cloud data after removing the outlier noise points in the point cloud data is segmented using the random sampling consensus algorithm RANSAC to obtain the point cloud data D.

[0087] It can be understood that after the original point cloud data is divided into regions and the ground point cloud is segmented, a series of independent ground point cloud data D are obtained.

[0088] It can be understood that the RANSAC algorithm can better eliminate the interference of outliers and outliers during fitting, making the fitted straight line or curve closer to the theoretical value. When performing ground segmentation on point cloud data, using the RANSAC algorithm can theoretically achieve more accurate and precise segmentation results.

[0089] In the S3:

[0090] In some embodiments, the number P of q points in the neighborhood of eps i The gains include:

[0091] P i =count{dist(i,j)<ε|0≤j≤D}

[0092] Where P i represents the number of neighboring points of point i, dist(i,j) represents the Euclidean distance between points i and j, ε represents the initial radius, and D represents the point cloud data.

[0093] It can be understood that the number P of points q in the eps neighborhood can be calculated according to the above formula i , if P i ≥Minpts, point p is a core point. If P i ≤Minpt, and q points are noise points until the most ideal minimum number of neighbors Minpts is found to make the clustering effect more ideal.

[0094] In S4:

[0095] It can be understood that the search for points that are directly reachable by the density of q points continues until no points that are directly reachable by the density of q points are found. The points searched above are core clustering points.

[0096] In S6:

[0097] It is understandable that it is difficult to distinguish the clustering effect after point cloud clustering, and it is necessary to use a 3D bounding box to present it more intuitively and output the image effect after point cloud clustering.

[0098] The method for obtaining the 3D enclosing model includes: obtaining the maximum point cloud P in the point cloud cluster after improved DBSCAN clustering max and the minimum point cloud P min , then the calculation formula for the length l, width w, and height h of the 3D bounding box is:

[0099] l = x max -x min

[0100] w=y max -y min

[0101] h=z max -z min

[0102] In the formula, x max Indicates the maximum value within the range of the X-axis; x min Indicates the minimum value within the range of the X coordinate axis; y max Indicates the maximum value within the range of the Y coordinate axis; y min Indicates the minimum value within the range of the Y coordinate axis; z max Indicates the maximum value within the range of the Z coordinate axis; z min Indicates the minimum value within the range of the Z coordinate axis.

[0103] It can be understood that by establishing a bounding model for various point cloud clusters after improved DBSCAN clustering, the point cloud clustering results can be effectively visualized, which is convenient for subsequent clustering effect evaluation. According to the laser radar and camera conversion matrix, MATLAB is used to project the clustered point cloud clusters onto the corresponding two-dimensional image plane to form a bounding box projection rectangular frame, completing the data-level fusion between sensors. The projection frame position is the detection target pixel position.

[0104] In this application, first, in a simulated power transmission line scene on campus, an acquisition system consisting of a laser radar and a camera is used to complete the image capture and point cloud acquisition of the anti-vibration hammer scene, and then the point cloud data is downsampled to reduce the number of point clouds and the amount of calculation; statistical filtering denoising and RANSAC ground removal are used to eliminate outlier point clouds, remove unnecessary point clouds, and use the improved DBSCAN algorithm to complete the clustering of the anti-vibration hammer point cloud and generate a three-dimensional bounding box. Finally, the point cloud cluster bounding box model is projected to the corresponding image pixel position to complete the fusion of image data and point cloud data; this application uses an improved DBSCAN point cloud clustering algorithm to automatically calculate the minimum neighborhood and initial radius value, thereby improving the efficiency of anti-vibration hammer point cloud clustering; the laser radar and camera data fusion target detection method is applied to the inspection robot to improve the robot's robustness and accuracy in detecting anti-vibration hammer targets, and reduce the missed detection rate and false detection rate of anti-vibration hammer target detection under insufficient lighting conditions.

[0105] In a second aspect, an embodiment of the present application provides a point cloud image data fusion system based on an improved DBSCAN clustering algorithm, comprising:

[0106] Parameter module, used to input the initial radius ε, the minimum number of neighbors Minpts and the point cloud data D;

[0107] The q-point density reachable point module is used to select a point q that has not been visited from the point cloud data D, and find all points that meet the minimum number of neighbors Minpts in the area with the q point as the center and the initial radius of ε, that is, the points reachable by the q-point density;

[0108] The ideal minimum number of neighbors Minpts module is used to obtain the number P of points q in the eps neighborhood i , if P i ≥Minpts, q points are classified as core points. If P i ≤Minpt, the q point is classified as a noise point until the optimal minimum number of neighbors Minpts is found;

[0109] The core clustering point module is used to continue searching for points that are directly reachable by the q point density. The points searched above are the core clustering points.

[0110] The point cloud cluster module after improved DBSCAN clustering is used to repeat the above steps until all point clouds belonging to the detection target in the point cloud data D are divided and searched, then it is determined that the point cloud improved DBSCAN clustering is completed, and the point cloud cluster after improved DBSCAN clustering is obtained;

[0111] The 3D enclosing model module of point cloud cluster is used to establish the 3D enclosing model of point cloud cluster after improved DBSCAN clustering;

[0112] The fusion module of point cloud data and image data is used to project the 3D enclosing model of the point cloud cluster established after the improved DBSCAN clustering to the corresponding image pixel position to complete the fusion of point cloud data and image data.

[0113] In the parameter module:

[0114] In some embodiments, obtaining the initial radius ε includes:

[0115] rang=Max{P k |0≤k≤m}

[0116] The initial radius ε is the median value of the segment where rang is located.

[0117] Where P k It is expressed as the detection frequency of dist(i,j) in each segment after dividing the random distance interval distance into m segments. Dist(i,j) represents the Euclidean distance between points i and j, and rang represents the maximum frequency.

[0118] Furthermore, dist(i,j) is obtained by:

[0119]

[0120] Where dist(i,j) represents the Euclidean distance between points i and j; i Indicates the distance between point i and the X coordinate axis; y i Indicates the distance between point i and the Y coordinate axis; z i Indicates the distance between point i and the Z coordinate axis; x j Indicates the distance between point j and the X coordinate axis; y j Indicates the distance between point j and the Y coordinate axis; z j Indicates the distance between point j and the Z coordinate axis.

[0121] Furthermore, obtaining the distance interval distance includes:

[0122] maxdist=Max{dist(ij|0≤i≤n,0≤j≤D}

[0123] mindist=Min{dist(ij|0≤i≤n,0≤j≤D}

[0124] distance = maxdist-mindist

[0125] In the formula, maxdist represents the maximum distance between points i and j, mindist represents the minimum distance between points i and j, distance represents the distance interval between points i and j, n represents the number of points, and D represents the point cloud data.

[0126] In some embodiments, the original point cloud data is downsampled using a voxel grid method.

[0127] It can be understood that the original point cloud data obtained contains unnecessary point cloud data such as trees, ground, and walls. If the original point cloud data with high density and redundant information is used for subsequent data fusion, it will affect the computer processing speed and the accuracy of subsequent fusion and registration. This application uses the voxel grid method to downsample the original point cloud data and use a smaller amount of data to approximate the original point cloud data. It can also be understood that the voxel grid method downsampling can significantly reduce the amount of calculation of the original point cloud data while maintaining the spatial distribution characteristics of the original point cloud data.

[0128] Furthermore, the voxel grid method is used to downsample the original point cloud data, and then the statistical filtering algorithm is used to remove the outlier noise points in the point cloud data. It can be understood that when the laser radar collects data, it is affected by its own performance limitations and the complex external environment, resulting in certain errors in the original point cloud data obtained. Point cloud data is prone to deviation in three-dimensional space, and the deviated point cloud data is called outlier noise point. In order to eliminate the interference of outlier noise points on the anti-vibration hammer target detection, the original point cloud data set needs to be denoised.

[0129] Furthermore, the point cloud data after removing the outlier noise points in the point cloud data is segmented using the random sampling consensus algorithm RANSAC to obtain the point cloud data D.

[0130] It can be understood that after the original point cloud data is divided into regions and the ground point cloud is segmented, a series of independent ground point cloud data D are obtained.

[0131] It can be understood that the RANSAC algorithm can better eliminate the interference of outliers and outliers during fitting, making the fitted straight line or curve closer to the theoretical value. When performing ground segmentation on point cloud data, using the RANSAC algorithm can theoretically achieve more accurate and precise segmentation results.

[0132] The ideal minimum number of neighbors Minpts module:

[0133] In some embodiments, the number P of q points in the neighborhood of eps i The gains include:

[0134] P i =count{dist(i,j)<ε|0≤j≤D}

[0135] Where P i represents the number of neighboring points of point i, dist(i,j) represents the Euclidean distance between points i and j, ε represents the initial radius, and D represents the point cloud data.

[0136] It can be understood that the number P of points q in the eps neighborhood can be calculated according to the above formula i , if P i ≥Minpts, point p is a core point. If P i ≤Minpt, and q points are noise points until the most ideal minimum number of neighbors Minpts is found to make the clustering effect more ideal.

[0137] In the core clustering point module:

[0138] It can be understood that the search for points that are directly reachable by the density of q points continues until no points that are directly reachable by the density of q points are found. The points searched above are core clustering points.

[0139] In the 3D enclosure model module of the point cloud cluster:

[0140] It is understandable that it is difficult to distinguish the clustering effect after point cloud clustering, and it is necessary to use a 3D bounding box to present it more intuitively and output the image effect after point cloud clustering.

[0141] The method for obtaining the 3D enclosing model includes: obtaining the maximum point cloud P in the point cloud cluster after improved DBSCAN clusteringmax and the minimum point cloud P min , then the calculation formula for the length l, width w, and height h of the 3D bounding box is:

[0142] l = x max -x min

[0143] w=y max -y min

[0144] h=z max -z min

[0145] In the formula, x max Indicates the maximum value within the range of the X-axis; x min Indicates the minimum value within the range of the X coordinate axis; y max Indicates the maximum value within the range of the Y coordinate axis; y min Indicates the minimum value within the range of the Y coordinate axis; z max Indicates the maximum value within the range of the Z coordinate axis; z min Indicates the minimum value within the range of the Z coordinate axis.

[0146] It can be understood that by establishing a bounding model for various point cloud clusters after improved DBSCAN clustering, the point cloud clustering results can be effectively visualized, which is convenient for subsequent clustering effect evaluation. According to the laser radar and camera conversion matrix, MATLAB is used to project the clustered point cloud clusters onto the corresponding two-dimensional image plane to form a bounding box projection rectangular frame, completing the data-level fusion between sensors. The projection frame position is the detection target pixel position.

[0147] In a third aspect, the present application provides a computer device comprising a storage device and a processor, wherein the storage device stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the point cloud image data fusion method based on the improved DBSCAN clustering algorithm as described above.

[0148] The computer device may be a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with the user through a keyboard, a mouse, a remote control, a touch pad, or a voice control device.

[0149] The memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory is often used to store the operating system and various application software installed on the computer device, such as the program code of the point cloud image data fusion method based on the improved DBSCAN clustering algorithm, etc. In addition, the memory can also be used to temporarily store various types of data that have been output or are to be output.

[0150] The processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data, such as running the program code of the point cloud image data fusion method based on the improved DBSCAN clustering algorithm.

[0151] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the point cloud image data fusion method based on the improved DBSCAN clustering algorithm as described above.

[0152] Among them, the computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor so that the at least one processor performs the steps of the point cloud image data fusion method based on the improved DBSCAN clustering algorithm as described above.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application can essentially or partly contribute to the prior art in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server or network device, etc.) to execute the point cloud image data fusion method based on the improved DBSCAN clustering algorithm described in the embodiment of the present application.

[0154] The embodiment of the present application provides a target detection method based on improved YOLOv5s, including:

[0155] S01. Provide an image dataset and manually mark the detection targets on the images in the dataset;

[0156] S02. Build a YOLOv5s network model. The YOLOv5s network model includes a feature fusion network and a detection head. Add a GAM attention mechanism to the feature fusion network, and use the SIoU position loss function for the detection head.

[0157] S03, train the YOLOv5s network model;

[0158] S04, test the YOLOv5s network model;

[0159] S05. Input a picture without a detection target label to obtain a detection target result.

[0160] In order to improve the small target detection capability of YOLOv5s, this application proposes a new improvement method based on the YOLOv5s algorithm. First, by adding the GAM attention mechanism in the feature fusion network, the model is more focused on the features that are beneficial to target detection, thereby improving the accuracy of detection. Secondly, the position loss function is replaced with SIoU, which improves the model convergence speed and reasoning accuracy. The improved YOLOv5s in this application reduces the missed detection rate and false detection rate of target detection to a certain extent.

[0161] In the S01:

[0162] In some embodiments, the image dataset includes images that are open source on the Internet and images taken by the user.

[0163] Furthermore, the image dataset includes open source anti-vibration hammer images on the Internet and self-photographed simulated transmission line images.

[0164] It is understandable that the images taken by yourself need to be manually labeled with the detection targets on the images using Labelimg labeling software.

[0165] Furthermore, the images in the image dataset are subjected to a series of image enhancement techniques, including mirroring, rotation, and scale transformation.

[0166] In some embodiments, the picture dataset includes a fused image dataset.

[0167] Furthermore, Labelimg annotation software is used to manually annotate the detection targets on the images in the fused image dataset.

[0168] In the S02:

[0169] like Figure 7 As shown, in some embodiments, the YOLOv5s network model also includes an input and a backbone network.

[0170] like Figure 8 As shown, in some embodiments, a GAM attention mechanism is added to the back end of the feature fusion network, and the network structure of the GAM attention mechanism includes a channel attention module and a spatial attention module.

[0171] As you can understand, the full name of GAM is Global Attention Mechanism.

[0172] It can be understood that the introduction of the attention module in YOLOv5s can better capture features at different scales, and the introduction of the GAM attention mechanism YOLOv5s model can optimize the feature fusion network. The model can accurately capture and fuse target features at different scales, significantly improve the algorithm detection accuracy, enhance the model's adaptability to complex scenes, and also bring more reliable performance to small and medium-sized target detection tasks.

[0173] Figure 8 In, M c represents channel attention, M s Represents spatial attention. For the input feature map F1∈R C×W×H (C is the number of feature map channels, W×H is the size of the feature map), the intermediate state F2 and the output feature map F3 are calculated as shown in the following formula.

[0174] in, Represents vector multiplication.

[0175]

[0176] In some embodiments, the calculation formula of the channel attention module is as follows:

[0177] Mc(F1)=σ(Reper(MLP(Per(F1))))

[0178] In the formula, σ represents the Sigmoid function, Per represents the dimension conversion; Reper represents the reverse recovery dimension; MLP represents two multi-layer perceptrons with hidden layers; F1 represents the input feature map, F1∈R C×W×H , where C is the number of feature map channels and W×H is the size of the feature map.

[0179] It is understandable that Fig. 9 As shown in the figure, during data processing, the GAM attention mechanism first inputs the feature map into the channel attention module, which uses 3D arrangement technology to process the three-dimensional information in the feature map. Inside the channel attention module, two multi-layer perceptrons (MLPs) with hidden layers are designed. In order to reduce parameter overhead and improve model efficiency, the activation function size of the hidden layer is carefully set to the decay rate r.

[0180] In some embodiments, the calculation formula of the spatial attention module is as follows:

[0181]

[0182] In the formula, σ represents the Sigmoid function, represents a 7×7 convolution operation for feature map compression on the input feature map, It represents the 7×7 convolution operation for increasing the dimension of the compressed image, and F2 represents the input feature map.

[0183] It is understandable that Fig.10 As shown in Figure 1, in the spatial attention module, a 7×7 convolution is used to compress the feature map, and then the 7×7 convolution is used again to increase the dimension of the feature to generate a spatial attention map. The attention map is activated by the Sigmoid function to obtain a new feature map M. S (F2). The maximum pooling operation in the processing process may reduce the use of information and have adverse effects, so it is decided to remove the pooling step to retain and utilize the effective information in the feature map and improve the performance of the YOLOv5 model in the target detection task.

[0184] It is understandable that the traditional YOLOv5s model uses the bounding box loss function CIoU, but CIoU does not take into account the directional matching between the predicted box and the real box, which may cause the model to drift during the training process, and the convergence speed of the algorithm will be affected to a certain extent. In order to optimize the model performance and solve this problem, this application uses the SIoU loss function that includes the angle cost to replace the original CIoU loss function. By introducing directional matching, the SIoU loss function can more comprehensively describe the difference between the predicted box and the real box, and improve the convergence speed and overall performance of the model.

[0185] In some embodiments, the SIoU position loss function includes angle cost Λ, distance cost Δ, shape cost Ω and IoU cost, and the calculation formula of the SIoU position loss function is as follows:

[0186]

[0187] Where, L SIoU Represents the SIoU position loss function, Λ represents the angle cost, Δ represents the distance cost, Ω represents the shape cost and IoU is the intersection-over-union ratio, which represents the ratio of the intersection area of ​​two boxes to their union area.

[0188] Furthermore, the angle cost Λ is calculated as follows:

[0189]

[0190] In the formula, C h is the vertical distance between the center of the predicted box and the center of the true box, σ is the distance between the center of the predicted box and the center of the true box, and α is the angle between the horizontal axis and the line connecting the centers of each box.

[0191] It can be understood that the parameters involved in SIoU calculation are as follows Fig.11 As shown, where B is the predicted box, B GT is the real frame, C w and C h is the horizontal and vertical distance between the center point of the predicted box and the center point of the real box, and σ is the distance between the center point of the predicted box and the center point of the real box.

[0192] Furthermore, the distance cost △ is calculated as follows:

[0193]

[0194] γ 2 =2-Λ

[0195] In the formula, ρ x represents the normalized distance in the x direction; ρ y represents the normalized distance in the y direction; is the horizontal coordinate of the real frame; Indicates the ordinate of the real box; b cx Indicates the horizontal coordinate of the prediction box; b cy Represents the vertical coordinate of the prediction box; C w represents the horizontal distance from the center of the predicted box to the center of the real box, C h Represents the vertical distance from the center of the predicted box to the center of the true box.

[0196] It can be understood that the parameter γ is related to the angular cost.

[0197] It can be understood that when the angle between the center point of the real frame and the center point of the predicted frame is smaller, the distance cost is smaller. The distance cost increases.

[0198] Furthermore, the shape cost Ω is calculated as follows:

[0199]

[0200] In the formula, ω w Represents the normalization coefficient parameter in the horizontal direction; ω h represents the normalization coefficient in the vertical direction; w represents the width of the prediction box, w GT represents the width of the real box, h represents the height of the predicted box, and h GT represents the height of the true box; θ represents the coefficient controlling the shape loss.

[0201] Furthermore, the IoU cost calculation formula is as follows:

[0202]

[0203] In the formula, IoU represents the ratio of the area intersection and area union of the predicted box and the real box; B represents the area of ​​the predicted box, B GT is the area of ​​the real box.

[0204] In said S03:

[0205] When the image dataset includes open source anti-vibration hammer images on the Internet and self-photographed simulated transmission line images, training the YOLOv5s network model includes:

[0206] A dataset consisting of 1,600 images was created, of which three quarters (i.e. 1,200) were used as training sets to obtain the weight file for target recognition; the remaining quarter (i.e. 400) consisted of images of simulated working scenes and were used as a test set to verify the accuracy of the target recognition model.

[0207] It can be understood that in order to ensure the accuracy of the experiment, all samples were processed with a unified size and the dataset was created according to the standards of the COCO dataset.

[0208] When the image dataset includes a fused image dataset, the experimental group uses the fused image dataset and the control group uses the ordinary image dataset. The two sample sets, each with 750 images, are produced through a series of scale transformations and augmentation methods such as adding different lighting and shadows.

[0209] The anti-vibration hammer obstacles in 1500 pictures of the two training sets were manually labeled using Labelimg labeling software, and a one-to-one corresponding XML file was generated for each labeled picture.

[0210] Since the YOLOv5s model annotation format requires TXT format, the XML file is converted into TXT format.

[0211] In said S05:

[0212] In some embodiments, the picture without the detection target labeled includes an image taken by the user.

[0213] Furthermore, the pictures without labeled detection targets include images of simulated power transmission lines taken by the user.

[0214] In some embodiments, the picture without the detection target labeled includes a fused image.

[0215] The invention is further described below in conjunction with experimental examples.

[0216] Experimental Example 1

[0217] In this test case, the RS16 laser radar and the D435i camera are used to synchronously acquire point cloud image data. The data acquisition site is on the right side of the main entrance of the Mechanical and Electrical Building of Lanzhou University of Technology, and the simulated power transmission line is used as the data acquisition scene. The collected data is output to the generated time node folder, such as Figure 1 , Figure 1 The corresponding point cloud image examples in the folder.

[0218] The voxel grid method is used to downsample the original point cloud data.

[0219] Point cloud denoising, this experimental example uses conventional statistical filtering processing in this field, the purpose is to more accurately extract the target point cloud and enhance the stability and accuracy of the point cloud. After the original data is processed by statistical filtering and denoising by the open source software Cloud Compare, Matlab is used to realize the comparison and visualization of the point cloud before and after denoising. Figure 2 As shown, Figure 2 The left side shows the image before point cloud denoising, and the right side shows the image after point cloud denoising.

[0220] After downsampling the point cloud, the ground point cloud is segmented and the RANSAC algorithm is used to segment the ground point cloud. The results are as follows: Figure 3 As shown, Figure 3 The left side is the image before segmentation, and the right side is the image after segmentation.

[0221] Take the point cloud data after removing the ground, export the point cloud coordinates of the dense area in the data, use MATLAB software to simulate the dense area in the point cloud data, and verify the clustering effect of the improved DBSCAN clustering algorithm:

[0222] The initial radius and minimum neighborhood Minpts in the DBSCAN clustering algorithm were manually input and repeatedly tested to achieve the ideal clustering effect shown in the figure above. Figure 4 (a), (b), (c), and (d) are the point cloud clustering simulation results when ε=30, Minpts=10, ε=40, Minpts=10, ε=70, Minpts=20, and ε=60, Minpts=10, respectively. If you want to get a more ideal point cloud clustering effect, you still need to manually set these two parameters. The improved DBSCAN algorithm in this application can automatically calculate the initial radius and the minimum neighborhood value for the best clustering effect. Figure 5 It can be observed that when the initial radius ε = 80 and the minimum neighborhood Minpts = 20, the anti-vibration hammer point cloud clustering effect is the best.

[0223] By building a bounding model for various point cloud clusters after improved DBSCAN clustering, the point cloud clustering results can be effectively visualized, which is convenient for subsequent clustering effect evaluation. According to the laser radar and camera conversion matrix obtained in Chapter 2, MATLAB is used to project the clustered point cloud clusters onto the corresponding two-dimensional image plane to form a bounding box projection rectangular frame to complete the data-level fusion between sensors. The projection frame position is the anti-vibration hammer pixel position. The projection effect is as follows Figure 6 shown.

[0224] Test Example 2

[0225] When the image dataset includes open-source anti-vibration hammer images on the Internet and self-photographed simulated transmission line images, the training and testing results are as follows:

[0226] It can be understood that this application uses a high-performance computer with a CPU of AMD Ryzen 7 5800H 3.20GH and a GPU version of NVIDIA GeForce RTX 3060 for training. The official recommended parameter settings of YOLOv5 are followed during the training process. In the data input part, adaptive prior boxes, mosaic data enhancement and adaptive image scaling are used. The input image size is 640×640, the batchsize is set to 16, the epoch is set to 300, and the initialization learning rate is 0.01. Stochastic gradient descent is used to learn and update network parameters during the network model training process.

[0227] Fig.12 It shows that after 300 rounds of data set training and testing on the test set, the improved YOLOv5s model of this application has lower bounding box regression loss and target confidence loss than the original YOLOv5s, and has better performance.

[0228] Fig.12 (a) It can be seen that the bounding box loss of the improved YOLOv5s begins to be lower than that of YOLOv5s after 10 rounds of training iterations. In the subsequent training, the target confidence loss and bounding box loss values ​​of the improved YOLOv5s and YOLOv5s gradually converge. Compared with YOLOv5s, the improved YOLOv5s converges faster in bounding box regression loss and target confidence loss, and the values ​​are always smaller than YOLOv5s, and tend to be stable after 250 rounds of training.

[0229] Fig.12 In Fig. 2, (a) bounding box loss; (b) confidence loss.

[0230] These results show that the improved YOLOv5s algorithm has better generalization ability than the traditional YOLOv5s algorithm, and can more stably predict bounding boxes and target confidence during detection.

[0231] Fig.13 It shows that the improved YOLOv5s model of this application has higher performance indicators than the original YOLOv5s in terms of accuracy, recall rate and average precision mean, and has better performance.

[0232] Fig.13 In the figure, (a) precision value; (b) recall rate; (c) average precision value.

[0233] Fig.13 (a) It can be seen that after 200 rounds of training iterations, the accuracy of the improved YOLOv5s exceeds that of YOLOv5s, and the accuracy curve gradually stabilizes. The accuracy of the last training is 0.8654. Fig.13(b) It can be seen that after 250 training iterations, the recall rate of the improved YOLOv5s exceeds that of YOLOv5s, and the target recall rate of the last training is 0.7285. Fig.13 (c) It can be seen that after 50 rounds of training iterations, the average precision of the improved YOLOv5s exceeds that of YOLOv5s, and the average precision of the last training is 0.8325.

[0234] In order to eliminate the fluctuation of model performance during the 300 rounds of training, this application takes the model with the best comprehensive performance on the validation set for the YOLOv5s model before and after the improvement, and performs statistics on performance indicators. The results are shown in Table 1:

[0235] Table 1 Comparison of network performance indicators before and after improvement

[0236]

[0237] As can be seen from Table 1, by adding the GAM attention mechanism and using the SIoU position loss function, the improved YOLOv5s model improves the accuracy, recall rate and average precision by 1.47%, 2.65% and 2.17% respectively, and the FPS decreases. This is because the addition of the attention mechanism increases the amount of calculation, but the decrease is small.

[0238] There are 430 anti-vibration hammer targets in the 320 test sets. As shown in Table 2, the false detection rate and missed detection rate of the improved YOLOv5s in the anti-vibration hammer target detection verification are lower than those of the original YOLOv5s model.

[0239] Table 2 Statistics of target detection effect of anti-vibration hammer before and after improvement

[0240]

[0241] Test Example 3

[0242] When the image dataset includes a fused image dataset, the training and testing results are as follows:

[0243] In this experimental example, the sensor data fusion target detection method uses the fusion image set of two sensors to train the improved YOLOv5s model, and the visual target detection method uses the improved YOLOv5s model trained with a single camera captured image as the control group.

[0244] Visual target detection method: Use the scene images taken by the camera containing the anti-vibration hammer to create an image dataset and train the YOLOv5s model. Fusion target detection method: Use the images after the LiDAR point cloud and the camera image are fused to create a fusion dataset and train the YOLOv5s model.

[0245] To ensure the credibility of the experiment, the training sets of the two models were expanded using the same means, and the training parameters and experimental conditions were set consistently. The data set type was the only difference in the experiments.

[0246] (1) Accuracy comparison experiment: After training, the same image verification set is used to verify the anti-vibration hammer target detection effect, and the number of missed targets and falsely detected targets of the two models are counted.

[0247] (2) Lighting comparison experiment: After training, the same image verification set is used to verify the anti-vibration hammer target detection effect, and the number of missed targets and falsely detected targets of the two models under different lighting experiments is counted.

[0248] Dataset creation:

[0249] Target detection based on deep learning requires a large amount of sample data support, and different data sets will also produce different detection results. Since the anti-vibration hammer network data set does not contain the corresponding point cloud data, the data sets used in the experiment are all self-made from actual collection. In order to verify the effect of the research method of the present invention, the experimental group uses a fused image data set, and the control group uses a normal image data set. The two sample sets are made by a series of scale transformations and adding different lighting and shadows. The number of each set is 750.

[0250] The anti-vibration hammer obstacles in 1500 pictures of the two training sets were manually labeled using Labelimg labeling software, and a one-to-one corresponding XML file was generated for each labeled picture.

[0251] Since the YOLOv5s model annotation format requires TXT format, the XML file is converted into TXT format.

[0252] By establishing a good deep learning experimental environment, the storage and operation of a large amount of data can be effectively reduced, thereby greatly reducing the time cost required for anti-vibration hammer target detection and improving training efficiency. This application aims to establish a computer environment that can meet the needs of experimental operation, including hardware and software facilities, see software environment configuration table 3 for details.

[0253] Table 3 Training environment configuration

[0254]

[0255] When training and testing the algorithm, in order to ensure the fairness of the improved algorithm and other comparison algorithms, the experimental parameters should be set uniformly, where the input image size is 640×640, the batchsize parameter is set to 32, the epoch is set to 100, and the initialization learning rate is set to 0.01.

[0256] Accuracy comparison test results

[0257] In order to verify and compare the application effects of the two models in actual engineering, this application takes 150 images taken by the camera under normal lighting as the test set to evaluate the anti-vibration hammer target detection effect of the two models. The test set contains 480 anti-vibration hammer targets. The following is the statistics of missed detection and false detection targets in the two model verification sets:

[0258] Table 4 Accuracy comparison experimental results

[0259]

[0260] From Table 4, we can see that the improved YOLOv5s model trained with the fusion training set has a false detection rate of 1.25% and a missed detection rate of 0.63% in the accuracy comparison experiment, which are lower than the improved YOLOv5s model trained with the ordinary image set. It has been proved that in the detection task, the target detection method of the laser radar and camera data fusion has a higher correct recognition rate than the visual target detection method. The anti-vibration hammer target detection effect of the fusion target detection method is as follows Fig.14 shown.

[0261] Lighting comparison test results

[0262] The transmission line environment is complex, and the wire clamp inspection robot often works at different times. Light changes will also affect the anti-vibration hammer target detection effect of the visual sensor, so a light comparison experiment is designed to evaluate the robustness of the sensor data fusion target detection method. Different light brightness (L) and shadow effects (S) are added to the test set images, and 120 test images are selected to expand to 960 test sets. Each light and dark effect contains 320 anti-vibration hammer targets. The two models are tested, and the detection effects under different lighting are as follows: Fig.15 shown.

[0263] The target detection effect statistics of the two models under different lighting conditions are shown in Tables 5 and 6:

[0264] Table 5 Target detection effect of sensor data fusion target detection model under different illumination

[0265]

[0266] Table 6 Target detection effect of visual target detection model under different illumination

[0267]

[0268] From Table 5 and Table 6, we can see that the two target detection methods show certain regularities under different lighting conditions. When the lighting conditions are the worst (S=150) and the lighting intensity is the strongest (L=200), both target detection methods reach their peak values ​​at this moment. The number of false detections and missed detections is the lowest under normal lighting conditions. The number of false detections and missed detections of the fusion target detection method is lower than that of the single vision target detection method. Compared with single vision target detection, the effect of fusion target detection does not change significantly with lighting conditions and is more robust.

[0269] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A point cloud image data fusion method based on an improved DBSCAN clustering algorithm, characterized in that: include: Input the initial radius ε, the minimum number of neighbors Minpts and the point cloud data D; Select a point q that has not been visited from the point cloud data D, and find all points that meet the minimum number of neighbors Minpts in the area with q as the center and an initial radius of ε, that is, points that are reachable by q point density; Get the number P of point q in the neighborhood of eps i , if P i ≥Minpts, q points are classified as core points. If P i ≤Minpt, the q point is classified as a noise point until the optimal minimum number of neighbors Minpts is found; Continue to search for points that are directly reachable by point q. The points searched above are core clustering points. Repeat the above steps until all point clouds belonging to the detection target in the point cloud data D are divided and searched, then it is determined that the point cloud improved DBSCAN clustering is completed, and the point cloud cluster after improved DBSCAN clustering is obtained; Establish a 3D bounding model of the point cloud cluster after improving DBSCAN clustering; The 3D enclosing model of the point cloud cluster established after the improved DBSCAN clustering is projected to the corresponding image pixel position to complete the fusion of point cloud data and image data.

2. The point cloud image data fusion method based on the improved DBSCAN clustering algorithm according to claim 1, characterized in that: The initial radius ε is obtained by: rang=Max{P k |0≤k≤m} In the formula, the initial radius ε is the median value of the segment where rang is located, P k It is represented by the detection frequency of dist(i,j) in each segment after the random distance interval distance is divided into m segments. Dist(i,j) represents the Euclidean distance between points i and j, and rang represents the maximum frequency. The acquisition of dist(i,j) includes: Where dist(i,j) represents the Euclidean distance between points i and j; i Indicates the distance between point i and the X coordinate axis; y i Indicates the distance between point i and the Y coordinate axis; z i Indicates the distance between point i and the Z coordinate axis; x j Indicates the distance between point j and the X coordinate axis; y j Indicates the distance between point j and the Y coordinate axis; z j Indicates the distance between point j and the Z coordinate axis. The distance interval distance is obtained by: maxdist=Max{dist(ij|0≤i≤n,0≤j≤D} mindist=Min{dist(ij|0≤i≤n,0≤j≤D} distance = maxdist-mindist In the formula, maxdist represents the maximum distance between points i and j, mindist represents the minimum distance between points i and j, distance represents the distance interval between points i and j, n represents the number of points, and D represents the point cloud data.

3. The point cloud image data fusion method based on the improved DBSCAN clustering algorithm according to claim 1 or 2, characterized in that: The acquisition of the point cloud data D includes: downsampling the original point cloud data using a voxel grid method, removing outlier noise points in the point cloud data using a statistical filtering algorithm, and finally obtaining the point cloud data D by using a random sampling consensus algorithm RANSAC for ground segmentation.

4. The point cloud image data fusion method based on the improved DBSCAN clustering algorithm according to claim 1, characterized in that: The number of points q in the neighborhood of eps is P i The gains include: P i =count{dist(i,j)<ε|0≤j≤D} Where P i represents the number of neighboring points of point i, dist(i,j) represents the Euclidean distance between points i and j, ε represents the initial radius, and D represents the point cloud data.

5. The point cloud image data fusion method based on the improved DBSCAN clustering algorithm according to claim 1, characterized in that: The method for obtaining the 3D enclosing model includes: obtaining the maximum point cloud P in the point cloud cluster after improved DBSCAN clustering max and the minimum point cloud P min , then the calculation formula for the length l, width w, and height h of the 3D bounding box is: l=x max -x min w=y max -y min h=z max -z min In the formula, x max Indicates the maximum value within the range of the X-axis; x min Indicates the minimum value within the range of the X coordinate axis; y max Indicates the maximum value within the range of the Y coordinate axis; y min Indicates the minimum value within the range of the Y coordinate axis; z max Indicates the maximum value within the range of the Z coordinate axis; z min Indicates the minimum value within the range of the Z coordinate axis.

6. The point cloud image data fusion system based on the improved DBSCAN clustering algorithm is characterized by: include: Parameter module, used to input the initial radius ε, the minimum number of neighbors Minpts and the point cloud data D; The q-point density reachable point module is used to select a point q that has not been visited from the point cloud data D, and find all points that meet the minimum number of neighbors Minpts in the area with the q point as the center and the initial radius of ε, that is, the points reachable by the q-point density; The ideal minimum number of neighbors Minpts module is used to obtain the number P of points q in the eps neighborhood i , if P i ≥Minpts, q points are classified as core points. If P i ≤Minpt, the q point is classified as a noise point until the optimal minimum number of neighbors Minpts is found; The core clustering point module is used to continue searching for points that are directly reachable by the q point density. The points searched above are the core clustering points. The point cloud cluster module after improved DBSCAN clustering is used to repeat the above steps until all point clouds belonging to the detection target in the point cloud data D are divided and searched, then it is determined that the point cloud improved DBSCAN clustering is completed, and the point cloud cluster after improved DBSCAN clustering is obtained; The 3D enclosing model module of point cloud cluster is used to establish the 3D enclosing model of point cloud cluster after improved DBSCAN clustering; The fusion module of point cloud data and image data is used to project the 3D enclosing model of the point cloud cluster established after the improved DBSCAN clustering to the corresponding image pixel position to complete the fusion of point cloud data and image data.

7. Computer device, characterized in that The invention comprises a storage and a processor, wherein the storage stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that A computer program is stored, and when the computer program is executed by a processor, the processor is caused to execute the steps of the method according to any one of claims 1 to 5.

Images