Point cloud regression frame detection method and device based on picture detection, equipment and medium

Abstract

The embodiment of the invention provides a point cloud regression frame detection method and device based on picture detection, equipment and a medium, and the method comprises the following steps: obtaining an initial 3d regression frame of laser point cloud data, and obtaining expanded point cloud data of the laser point cloud data in an expanded range; mapping each point in the expanded point cloud data according to a top view, a front view and a left view to generate a point cloud three-view; performing enhancement processing on pixels in the point cloud three-view, generating an enhanced point cloud three-view, performing feature extraction on the enhanced point cloud three-view, generating corresponding picture features, converting the picture features into an N2-dimensional feature picture, and extracting a final 3d regression frame from the N2-dimensional feature picture; and performing weighted fusion on the initial 3d regression frame and the final 3d regression frame to generate a regression frame. According to the scheme, the advantages of an existing 3d detection network and an existing 2d detection network are utilized, and the detection precision of the regression frame of the point cloud detection algorithm is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a point cloud regression frame detection method, device, equipment and medium based on image detection. Background Art

[0002] Autonomous vehicles refer to motor vehicles that mainly rely on artificial intelligence, visual computing, radar, GPS positioning, and structured road vehicle-road collaboration technologies to enable the car to have the ability of environmental perception, path planning and autonomous control, so that the embedded edge computing terminal can automatically operate. Since 2020, the concept of L2 and L3 autonomous driving has begun to be popularized in the industry. 2021 is a very important node in the development of autonomous driving technology. Many industry insiders regard 2021 as the first year for autonomous driving technology to enter the L3 level. Different from traditional human-driven vehicles, the core feature of autonomous driving vehicles is the application and dominance of AI technology. Its driving process is also achieved by computers through uninterrupted collection of various types of information during driving, information analysis and self-learning through machine learning or deep learning, and then control of the vehicle, so as to achieve the system engineering of computer autonomous driving vehicles. For autonomous driving vehicles, what they pursue is safe driving under any road conditions and any environment and timely response to extreme situations to ensure the life safety of drivers, vehicle safety, and the safety of road participants.

[0003] At present, the field of autonomous driving is an important field for the deployment and implementation of deep learning algorithms. Autonomous driving algorithms are also in the process of rapid update and iteration. Various novel algorithms are constantly being proposed in the fields of target detection, lane line recognition, target tracking, semantic segmentation, dynamic target trajectory prediction, multi-task learning, etc. The core of the autonomous driving algorithm technology framework is divided into three parts: environmental perception, decision planning, and control execution. The environmental perception part converts sensor data into machine language of the scene where the vehicle is located, which can specifically include: object detection, recognition and tracking, environmental modeling, motion estimation, etc. In recent years, as high-level assisted driving solutions have been increasingly accepted by the market, the requirements for autonomous driving perception algorithms have also become higher and higher. However, the accuracy of the size, position, angle, category, etc. of the regression box of the current 3D point cloud detection algorithm needs to be further improved. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a point cloud regression frame detection method based on image detection to solve the technical problem of low regression frame accuracy of 3D point cloud detection algorithm in the prior art. The method includes:

[0005] Obtain laser point cloud data, obtain an initial 3D regression frame of the laser point cloud data through a 3D point cloud detection algorithm, set an enlargement ratio, enlarge the initial 3D regression frame to an enlarged range according to the enlargement ratio, and obtain enlarged point cloud data of the laser point cloud data in the enlarged range;

[0006] Map each point in the expanded point cloud data according to the top view, the front view and the left view to generate three-view point cloud, where the three-view point cloud includes three RBG images;

[0007] Enhance the pixels in the point cloud three-view image to generate the enhanced point cloud three-view image, extract features from the enhanced point cloud three-view image to generate corresponding image features, convert the image features into N2-dimensional feature images, and extract the final 3D regression frame from the N2-dimensional feature images;

[0008] The initial 3D regression frame and the final 3D regression frame are weighted fused to generate the regression frame of the laser point cloud data.

[0009] The embodiment of the present invention also provides a point cloud regression frame detection device based on image detection to solve the technical problem of low regression frame accuracy of 3D point cloud detection algorithm in the prior art. The device includes:

[0010] The initial regression frame extraction module is used to obtain laser point cloud data, obtain the initial 3D regression frame of the laser point cloud data through the 3D point cloud detection algorithm, set the expansion ratio, enlarge the initial 3D regression frame to the expansion range according to the expansion ratio, and obtain the expanded point cloud data of the laser point cloud data in the expanded range;

[0011] A three-view generation module, used to map each point in the expanded point cloud data according to the top view, the front view and the left view, respectively, to generate a point cloud three-view, wherein the point cloud three-view includes three RBG images;

[0012] The final regression frame extraction module is used to enhance the pixels in the point cloud three-view image to generate the enhanced point cloud three-view image, perform feature extraction on the enhanced point cloud three-view image, generate corresponding image features, convert the image features into N2-dimensional feature images, and extract the final 3D regression frame from the N2-dimensional feature images;

[0013] The regression frame fusion module is used to perform weighted fusion of the initial 3D regression frame and the final 3D regression frame to generate the regression frame of the laser point cloud data.

[0014] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned point cloud regression box detection methods based on image detection to solve the technical problem of low regression box accuracy of the 3D point cloud detection algorithm in the prior art.

[0015] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned point cloud regression box detection methods based on image detection, so as to solve the technical problem of low regression box accuracy of 3D point cloud detection algorithms in the prior art.

[0016] Compared with the prior art, the at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0017] The embodiment of the present invention performs regression correction on the point cloud detection frame from a completely new perspective, fully utilizing the advantages of the existing 3D detection network and 2D detection network, so as to improve the detection accuracy of the regression frame of the point cloud detection algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of a point cloud regression frame detection method based on image detection provided by an embodiment of the present invention;

[0020] Figure 2 is a flow chart of implementing the above-mentioned point cloud regression frame detection method based on image detection provided by an embodiment of the present invention;

[0021] Figure 3 is a structural block diagram of a computer device provided by an embodiment of the present invention;

[0022] Figure 4 It is a structural block diagram of a point cloud regression frame detection device based on image detection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0024] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0025] At present, the academic community has proposed that the size, position, category and angle of the regression frame of the 3D point cloud detection model are different from the true value. At the same time, the 2D image detection field is relatively mature. In order to improve the quality of the regression frame of the 3D point cloud detection model, the embodiment of the present invention designs a three-view based on the original 3D detection model. Figure 2 The two-stage detection method of d detection uses the advantage of 2d detection that the size, position, angle, category and other accuracy of the regression box of the image are relatively high to correct the 3d detection box and improve the detection accuracy of the model.

[0026] In an embodiment of the present invention, a point cloud regression frame detection method based on image detection is provided, such as Figure 1 and Figure 2 As shown, the method includes:

[0027] Step S101: acquiring laser point cloud data, acquiring an initial 3D regression frame of the laser point cloud data through a 3D point cloud detection algorithm, setting an enlargement ratio, enlarging the initial 3D regression frame to an enlarged range according to the enlargement ratio, and acquiring enlarged point cloud data of the laser point cloud data in the enlarged range;

[0028] Step S102: Map each point in the expanded point cloud data according to the top view, the front view and the left view to generate three point cloud views, wherein the three point cloud views include three RBG images;

[0029] Step S103: Enhance the pixels in the point cloud three-view image to generate the enhanced point cloud three-view image, extract features from the enhanced point cloud three-view image to generate corresponding image features, convert the image features into N2-dimensional feature images, and extract the final 3D regression frame from the N2-dimensional feature images;

[0030] Step S104: weighted fusion of the initial 3D regression frame and the final 3D regression frame to generate a regression frame of the laser point cloud data.

[0031] Specifically, according to the inspection results of the general detection algorithm, the range of the regression box is expanded in the original laser point cloud data according to a certain ratio, and the point cloud points within the range are taken out.

[0032] In the specific implementation, in order to use a special method to convert the point cloud data into a picture, the following steps are performed to map each point in the expanded point cloud data according to the top view, front view and left view to generate three views of the point cloud:

[0033] Map each point in the expanded point cloud data according to the top view, front view and left view respectively until the three views of the point cloud are generated: obtain the intensity value of each point in the expanded point cloud data, and use the intensity value as the first channel value, where the intensity value of the expanded point cloud data is 0 to the maximum intensity of the color channel; calculate the distance between each point in the expanded point cloud data and the origin of the vehicle, and use the distance as the second channel value, where the origin of the vehicle is the center point of the rear axle of the vehicle; calculate the height of each point in the expanded point cloud data relative to the origin of the vehicle, map the height geometrically to between 0 and the maximum intensity of the color channel, and use the mapping result as the third channel value; use the first channel value, the second channel value and the third channel value as the three channels of the RGB image to generate an RBG image.

[0034] Specifically, the points taken out are mapped according to the top view, front view, and left view, and the RGB three-channel values ​​(value range is 0-255) of the generated image correspond to: the first channel is the intensity value of the point cloud (the point cloud intensity value range is 0-255), the second channel is the distance of each point from the origin of the vehicle (that is, the center point of the rear axle of the vehicle) (the distance range is 0-255 (meters), because the detection range of the regression frame of the detection model is usually within 200 meters), and the third channel is the height of each point relative to the origin of the vehicle (because for autonomous driving detection, the height range of the detection target is approximately between -3 and 5 (meters), and then the height is proportionally mapped to 0-255).

[0035] In the specific implementation, in order to enhance the density of sparse point cloud data, the following steps are performed to enhance the pixels in the point cloud three-view image and generate the enhanced point cloud three-view image:

[0036] Set the connection radius, height difference parameter and intensity difference parameter; obtain the pixel points whose pixel values ​​are not zero in the three views of the point cloud, and generate a non-zero pixel point set; in the non-zero pixel point set, in a spherical area with the non-zero pixel point as the center and the connection radius as the radius, take the area whose height difference with the non-zero pixel point is less than the height difference parameter and whose intensity difference with the non-zero pixel point is less than the intensity difference parameter as a connected area, generate a first new pixel point in the connected area, and assign a pixel value to the first new pixel point, wherein the ratio of the pixel value of the first new pixel point to the pixel value of the non-zero pixel point is equal to the ratio of the distance between the first new pixel point and the non-zero pixel point.

[0037] Specifically, since the points in the expanded point cloud data are relatively sparse, the generated image is difficult to detect. Therefore, it is necessary to connect the points whose height difference is less than Threshold_H (height difference parameter) and whose intensity difference is less than Threshold_I (intensity difference parameter) in the spherical area with a radius of R (connection radius) where the pixel value is not 0. (That is, assign values ​​to the connecting area between points with high similarity, and the values ​​assigned are based on the ratio of the distance between the two).

[0038] In specific implementation, the following steps are performed to further enhance the density of sparse point cloud data:

[0039] An expansion radius is set, wherein the connected radius is more than ten times the expansion radius; in a non-zero pixel point set, a second new pixel point is generated in a spherical area with the non-zero pixel point as the center and the expansion radius as the radius, and the pixel value of the generated second new pixel point is assigned to the pixel value of the non-zero pixel point.

[0040] Specifically, a spherical area with a radius r (expansion radius) and a pixel point with a non-zero value as the center is expanded (that is, the pixel value corresponding to the pixel point with a non-zero value is assigned to the surrounding area).

[0041] In the specific implementation, in order to meet the needs of this single target regression detection, the following steps are used to extract features from the enhanced point cloud three views, generate corresponding image features, convert the image features into N2-dimensional feature images, and extract the final 3D regression frame from the N2-dimensional feature images:

[0042] The features of the enhanced point cloud three views are extracted through a 2D classification network to generate corresponding image features; the image features of the enhanced point cloud three views are converted into N2-dimensional feature images, and the N2-dimensional feature images are regressed to generate the final 3D regression frame.

[0043] In specific implementation, the following steps are used to convert the image features into N2-dimensional feature images, and then regress the N2-dimensional feature images to generate the final 3D regression frame:

[0044] The image features of the three views of the enhanced point cloud are flattened into N1 dimensions to generate N1-dimensional feature maps; the N1-dimensional feature maps are spliced ​​to generate N2-dimensional feature maps; the N2-dimensional feature maps are regressed using a multi-layer perceptron to generate the final 3D regression frame.

[0045] Specifically, the current 2D multi-target detection algorithm has weak global detection capabilities, and the current classification network with strong global detection capabilities does not have regression. Therefore, in order to meet the needs of this single target regression detection. Based on the 2D classification network, the features of the three views are extracted, and then the features of the three views are flattened into N1 dimensions and spliced ​​into N2 dimensions. Finally, the final 3D regression box is generated through multi-layer MLP layer regression.

[0046] In the specific implementation, in order to fuse the detection results of the two regression frames, the initial 3D regression frame and the final 3D regression frame are weightedly fused through the following steps to generate the regression frame of the laser point cloud data:

[0047] The confidence of the initial 3D regression frame is obtained as the first confidence C1, and the confidence of the final 3D regression frame is obtained as the second confidence C2; ​​the regression frame of the laser point cloud data Among them, B1 is the initial 3D regression box, and B2 is the final 3D regression box.

[0048] Specifically, the final 3D regression frame and the initial 3D regression frame are weightedly fused according to the confidence.

[0049] In this embodiment, a computer device is provided, such as Figure 3 As shown, it includes a memory 301, a processor 302, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned point cloud regression box detection methods based on image detection is implemented.

[0050] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0051] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned point cloud regression box detection methods based on image detection.

[0052] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0053] Based on the same inventive concept, a point cloud regression frame detection device based on image detection is also provided in an embodiment of the present invention, as described in the following embodiments. Since the principle of solving the problem by the point cloud regression frame detection device based on image detection is similar to that of the point cloud regression frame detection method based on image detection, the implementation of the point cloud regression frame detection device based on image detection can refer to the implementation of the point cloud regression frame detection method based on image detection, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0054] Figure 4 is a structural block diagram of a point cloud regression frame detection device based on image detection according to an embodiment of the present invention. Figure 4 As shown, it includes: an initial regression frame extraction module 401, a three-view image generation module 402, a final regression frame extraction module 403 and a regression frame fusion module 404. The structure is described below.

[0055] The initial regression frame extraction module 401 is used to obtain laser point cloud data, obtain the initial 3D regression frame of the laser point cloud data through the 3D point cloud detection algorithm, set the enlargement ratio, enlarge the initial 3D regression frame to the enlarged range according to the enlargement ratio, and obtain the enlarged point cloud data of the laser point cloud data in the enlarged range;

[0056] A three-view generating module 402 is used to map each point in the expanded point cloud data according to the top view, the front view and the left view, respectively, to generate a point cloud three-view, wherein the point cloud three-view includes three RBG images;

[0057] The final regression frame extraction module 403 is used to enhance the pixels in the point cloud three-view image to generate the enhanced point cloud three-view image, perform feature extraction on the enhanced point cloud three-view image to generate corresponding image features, convert the image features into N2-dimensional feature images, and extract the final 3D regression frame from the N2-dimensional feature images;

[0058] The regression frame fusion module 404 is used to perform weighted fusion on the initial 3D regression frame and the final 3D regression frame to generate a regression frame of the laser point cloud data.

[0059] In one embodiment, the three-view image generation module includes:

[0060] The three-view generating unit is used to map each point in the expanded point cloud data according to the top view, the front view and the left view respectively until the point cloud three-view is generated.

[0061] A top view channel generating unit is used to obtain the intensity value of each point in the expanded point cloud data and use the intensity value as the first channel value, wherein the intensity value of the expanded point cloud data is from 0 to the maximum intensity of the color channel;

[0062] A front view channel generation unit is used to calculate the distance between each point in the enlarged point cloud data and the origin of the vehicle, and use the distance as the second channel value, wherein the origin of the vehicle is the center point of the rear axle of the vehicle;

[0063] The left view channel generation unit is used to calculate the height of each point in the expanded point cloud data relative to the origin of the vehicle, map the height proportionally to between 0 and the maximum intensity of the color channel, and use the mapping result as the third channel value;

[0064] The RBG image generating unit is used to use the first channel value, the second channel value and the third channel value as three channels of the RGB image to generate an RBG image.

[0065] In one embodiment, the final regression box extraction module includes:

[0066] A parameter setting unit, used to set a connection radius, a height difference parameter and an intensity difference parameter;

[0067] A pixel set generating unit, used for obtaining pixel points whose pixel values ​​are not zero in the three-view image of the point cloud, and generating a non-zero pixel point set;

[0068] The first pixel filling unit is used to, in a non-zero pixel set, in a spherical area with the non-zero pixel as the center and the connected radius as the radius, take an area whose height difference with the non-zero pixel is less than a height difference parameter and whose intensity difference with the non-zero pixel is less than an intensity difference parameter as a connected area, generate a first new pixel in the connected area, and assign a pixel value to the first new pixel, wherein the ratio of the pixel value of the first new pixel to the pixel value of the non-zero pixel is equal to the ratio of the distance between the first new pixel and the non-zero pixel.

[0069] In one embodiment, the final regression box extraction module further includes:

[0070] An expansion radius setting unit, used to set the expansion radius, wherein the connection radius is more than ten times the expansion radius;

[0071] The second pixel filling unit is used to generate a second new pixel point in the non-zero pixel point set, in a spherical area with the non-zero pixel point as the center and the expansion radius as the radius, and assign the pixel value of the generated second new pixel point to the pixel value of the non-zero pixel point.

[0072] In one embodiment, the final regression box extraction module further includes:

[0073] A picture feature generation unit is used to extract features from the enhanced point cloud three-view image through a 2D classification network to generate corresponding picture features;

[0074] The final regression frame generation unit is used to convert the image features of the enhanced point cloud three-view image into an N2-dimensional feature image, and regress the N2-dimensional feature image to generate the final 3D regression frame.

[0075] In one embodiment, the final regression frame generation unit is used to flatten the image features of the three views of the enhanced point cloud into N1 dimensions respectively to generate N1-dimensional feature maps; after splicing the N1-dimensional feature maps, generate N2-dimensional feature maps; after regressing the N2-dimensional feature maps using a multi-layer perceptron, generate the final 3D regression frame

[0076] In one embodiment, the regression frame fusion module includes:

[0077] A confidence acquisition unit is used to acquire the confidence of the initial 3D regression frame as a first confidence C1, and acquire the confidence of the final 3D regression frame as a second confidence C2;

[0078] Final regression frame generation unit, used for the regression frame of laser point cloud data Among them, B1 is the initial 3D regression box, and B2 is the final 3D regression box.

[0079] The embodiments of the present invention achieve the following technical effects:

[0080] The target 3D detection frame of the 3D point cloud target detection algorithm is used to extract the corresponding point cloud data, and the point cloud three views of the target object are enhanced according to information such as intensity, height, and distance. Subsequently, the generated point cloud three views are subjected to a two-stage 2D target detection network process to improve the accuracy of 3D target detection; the generated sparse point cloud three views are connected and expanded to improve the accuracy of the subsequent two-stage 2D target detection; for the single target detection task, improvements are made based on the classification algorithm to support the simultaneous input of three views and return the 3D detection frame; the embodiment of the present invention regresses the point cloud detection frame from a completely new perspective, fully utilizing the advantages of the existing 3D detection network and the 2D detection network to make the detection accuracy of the regression frame of the point cloud detection algorithm higher.

[0081] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A point cloud regression box detection method based on image detection, characterized in that: include: Acquire laser point cloud data, acquire an initial 3D regression frame of the laser point cloud data through a 3D point cloud detection algorithm, set an enlargement ratio, enlarge the initial 3D regression frame to an enlarged range according to the enlargement ratio, and acquire enlarged point cloud data of the laser point cloud data in the enlarged range; Mapping each point in the enlarged point cloud data according to the top view, the front view and the left view respectively to generate three point cloud views, wherein the three point cloud views include three RBG images; Enhance the pixels in the point cloud three-view image to generate enhanced point cloud three-view image, extract features from the enhanced point cloud three-view image to generate corresponding image features, convert the image features into N2-dimensional feature images, and extract the final 3D regression frame from the N2-dimensional feature images; The initial 3D regression frame and the final 3D regression frame are weightedly fused to generate a regression frame of the laser point cloud data.

2. The point cloud regression frame detection method based on image detection according to claim 1, characterized in that: Map each point in the enlarged point cloud data according to the top view, the front view and the left view respectively to generate three point cloud views, including: Mapping each point in the expanded point cloud data according to the top view, the front view and the left view respectively until generating three point cloud views: obtaining the intensity value of each point in the expanded point cloud data, and using the intensity value as the first channel value, wherein the intensity value of the expanded point cloud data is 0 to the maximum intensity of the color channel; Calculate the distance between each point in the enlarged point cloud data and the origin of the vehicle, and use the distance as the second channel value, wherein the origin of the vehicle is the center point of the rear axle of the vehicle; Calculate the height of each point in the expanded point cloud data relative to the origin of the vehicle, map the height geometrically to a value between 0 and the maximum intensity of the color channel, and use the mapping result as the third channel value; The first channel value, the second channel value and the third channel value are used as three channels of an RGB image to generate an RBG image.

3. The point cloud regression frame detection method based on image detection according to claim 1, characterized in that: The pixels in the point cloud three-view image are enhanced to generate enhanced point cloud three-view images, including: Set the connectivity radius, height gap parameter and intensity gap parameter; Obtaining pixel points whose pixel values ​​are not zero in the three views of the point cloud to generate a non-zero pixel point set; In the non-zero pixel point set, in a spherical area with the non-zero pixel point as the center and the connected radius as the radius, an area whose height difference with the non-zero pixel point is less than the height difference parameter and whose intensity difference with the non-zero pixel point is less than the intensity difference parameter is taken as a connected area, a first new pixel point is generated in the connected area, and a pixel value of the first new pixel point is assigned, wherein the ratio of the pixel value of the first new pixel point to the pixel value of the non-zero pixel point is equal to the ratio of the distance between the first new pixel point and the non-zero pixel point.

4. The point cloud regression frame detection method based on image detection according to claim 3, characterized in that: Also includes: Setting an expansion radius, wherein the connection radius is more than ten times the expansion radius; In the non-zero pixel set, in a spherical area with the non-zero pixel as the center and the expansion radius as the radius, a second new pixel is generated, and the pixel value of the generated second new pixel is assigned to the pixel value of the non-zero pixel.

5. The point cloud regression frame detection method based on image detection according to claim 1, characterized in that: Performing feature extraction on the enhanced point cloud three-view image to generate corresponding image features, converting the image features into an N2-dimensional feature image, and extracting a final 3D regression frame from the N2-dimensional feature image, including: Extract features from the enhanced point cloud three-view image through a 2D classification network to generate corresponding image features; The image features of the enhanced point cloud three-view image are converted into an N2-dimensional feature image, and the N2-dimensional feature image is regressed to generate a final 3D regression frame.

6. The point cloud regression frame detection method based on image detection according to claim 5, characterized in that: Converting the image features into an N2-dimensional feature image, and regressing the N2-dimensional feature image to generate a final 3D regression frame, including: Flatten the image features of the enhanced point cloud three views into N1 dimensions respectively to generate N1-dimensional feature maps; After splicing the N1-dimensional feature maps, an N2-dimensional feature map is generated; After regressing the N2-dimensional feature map using a multi-layer perceptron, the final 3D regression frame is generated.

7. The point cloud regression frame detection method based on image detection according to claim 1, characterized in that: The initial 3D regression frame and the final 3D regression frame are weightedly fused to generate a regression frame of the laser point cloud data, including: Obtaining the confidence of the initial 3D regression frame as a first confidence C1, and obtaining the confidence of the final 3D regression frame as a second confidence C2; The regression frame of the laser point cloud data Among them, B1 is the initial 3D regression frame, and B2 is the final 3D regression frame.

8. A point cloud regression frame detection device based on image detection, characterized in that: include: An initial regression frame extraction module is used to obtain laser point cloud data, obtain an initial 3D regression frame of the laser point cloud data through a 3D point cloud detection algorithm, set an enlargement ratio, enlarge the initial 3D regression frame to an enlarged range according to the enlargement ratio, and obtain enlarged point cloud data of the laser point cloud data in the enlarged range; A three-view generating module, used for mapping each point in the expanded point cloud data according to a top view, a front view and a left view, respectively, to generate a point cloud three-view, wherein the point cloud three-view includes three RBG images; The final regression frame extraction module is used to enhance the pixels in the point cloud three-view image to generate enhanced point cloud three-view images, perform feature extraction on the enhanced point cloud three-view images to generate corresponding image features, convert the image features into N2-dimensional feature images, and extract the final 3D regression frame from the N2-dimensional feature images; The regression frame fusion module is used to perform weighted fusion on the initial 3D regression frame and the final 3D regression frame to generate a regression frame of the laser point cloud data.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the point cloud regression box detection method based on image detection according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the point cloud regression box detection method based on image detection according to any one of claims 1 to 7.

Images