A detection method and device for road fine targets during automatic driving

Abstract

The application relates to the technical field of target detection, and discloses a detection method and device for small targets on a road during automatic driving, which comprises the following steps: obtaining original laser data of an intelligent automobile during driving by using a laser module, and shooting original shooting pictures during driving by using a camera module; performing first target detection on the original shooting pictures to obtain a first target detection frame; obtaining a target to-be-corrected point set by taking each pixel point in the first target detection frame as a target to-be-corrected point; converting original laser data belonging to a laser coordinate system into a focal length coordinate system by using a camera coordinate system to obtain focal length laser data; cleaning the target to-be-corrected point set by using the focal length laser data to obtain a corrected detection frame; and replacing the first target detection frame with the corrected detection frame to obtain a target detection result during automatic driving of the intelligent automobile. The main purpose of the application is to solve the problem that a target detection frame generated by detecting small targets during automatic driving is inaccurate.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a method and apparatus for detecting small targets on roads during autonomous driving. Background Technology

[0002] With the continuous development of science and technology, intelligent driving technology is gradually becoming more widespread. Intelligent driving involves a variety of scientific and technological aspects, one of the core technologies being target detection. This means that when an intelligent vehicle is driving autonomously, it can detect surrounding objects, especially those in its direction of travel, in a timely manner to ensure the driving safety of the vehicle.

[0003] Currently, commonly used object detection methods for autonomous driving mainly rely on deep learning models. This involves using camera modules to capture real-time images of the scene in the direction the car is traveling, and then using deep learning models to identify objects within the scene. Examples of such deep learning models include YOLO, SSD, and VGG.

[0004] While deep learning-based target detection for autonomous driving can achieve target detection, it suffers from a low detection rate for small targets. This is because deep learning models typically rely on convolutional pooling operations. While convolutional pooling effectively extracts image features from large targets (such as vehicles and electric cars traveling in the direction of travel) during scene graph processing, it tends to lose image features of smaller targets during convolutional pooling. This manifests primarily as inaccurate bounding boxes for small targets; although the bounding boxes may identify the small target, they are often much larger than the actual small target.

[0005] Overcoming the challenge of improving the accuracy of small-sized target detection during autonomous driving is a pressing technical problem that urgently needs to be solved. Summary of the Invention

[0006] This invention provides a method and apparatus for detecting small targets on roads during autonomous driving, with the main purpose of solving the problem of inaccurate target detection boxes generated when detecting small targets during autonomous driving.

[0007] To achieve the above objectives, the present invention provides a method for detecting small road targets during autonomous driving, comprising:

[0008] When an intelligent vehicle activates autonomous driving, its camera module and laser module are activated. The front of the intelligent vehicle includes at least one camera module and one laser module.

[0009] The laser module is used to acquire the raw laser data of the intelligent vehicle while it is in motion, and the shooting focal length of the camera module is determined. Based on the shooting focal length, the raw images of the intelligent vehicle while it is in motion are acquired. The laser module and the raw laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the raw images belong to the focal length coordinate system.

[0010] Perform a first object detection on the original captured image to obtain a first object detection bounding box;

[0011] If the area of ​​the first target detection box is less than or equal to the box threshold, the target in the first target detection box is determined to be a small-volume target object. Each pixel in the first target detection box is obtained in sequence to obtain the target to be corrected point set.

[0012] The original laser data belonging to the laser coordinate system is transformed to the focal length coordinate system using the camera coordinate system to obtain the focal length laser data;

[0013] The target point set to be corrected is cleaned using focal length laser data to obtain the target cleaned point set, where the target cleaned point set corresponds to the corrected detection box;

[0014] The corrected detection box is used to replace the first target detection box to obtain the target detection result when the intelligent vehicle is driving autonomously.

[0015] Optionally, performing a first object detection on the original captured image to obtain a first object detection box includes:

[0016] The original captured image is input into the object detection model, which has the following structure: convolutional layer, normalization layer, ReLU activation layer, multi-level residual layer, pooling layer, linear transformation layer, and object detection box generation layer. The multi-level residual layer has four layers in the order of processing the original captured image. The first layer consists of two convolutional layers; the second layer consists of a pooling layer, a fully connected layer, a ReLU activation layer, a fully connected layer, and a Sigmoid activation layer, with the input data of the second layer being the output data of the first layer; the third layer is a fusion layer that performs a dot product operation between the output data of the first and second layers; the fourth layer is the same as the first layer, with the input data of the fourth layer being the output data of the third layer; and the object detection box generation layer is a YOLO model.

[0017] The original captured image is sequentially subjected to convolution, pooling, and activation operations using the convolutional layer, normalization layer, and ReLU activation layer to obtain the first activated image.

[0018] The first activated image is further convolved through two convolutional layers of the first layer structure to obtain the first convolutional image.

[0019] The first convolutional image is input into the second layer structure, and then passes through the pooling layer, fully connected layer, ReLU activation layer, fully connected layer, and Sigmoid activation layer of the second layer structure in sequence to obtain the second activated image;

[0020] Within the fusion layer of the third structure, the second activated image and the first convolutional image are multiplied by a dot to obtain the reconstructed image.

[0021] The reconstructed image is input into the fourth layer structure, where the data processing in the fourth layer structure is the same as in the first layer structure, to obtain the reconstructed convolutional image.

[0022] The reconstructed convolutional image is input into the YOLO model, and the YOLO model is used to detect the targets included in the reconstructed convolutional image to generate the first target detection box.

[0023] Optionally, the step of transforming the original laser data belonging to the laser coordinate system to the focal length coordinate system using the camera coordinate system to obtain focal length laser data includes:

[0024] Extract each set of laser data from the original laser data in sequence, and determine the coordinates of the laser data in the laser coordinate system as the original laser coordinates;

[0025] The original laser coordinates are transformed into the camera coordinate system to obtain the camera laser coordinates;

[0026] Transform the camera laser coordinates to the focal length coordinate system to obtain the focal length laser coordinates;

[0027] Having obtained the focal length laser coordinates corresponding to each set of laser data, the focal length laser coordinates corresponding to each set of laser data are summarized to obtain the focal length laser data.

[0028] Optionally, the step of transforming the original laser coordinates to the camera coordinate system to obtain the camera laser coordinates includes:

[0029] The camera laser coordinates are calculated using the following formula:

[0030]

[0031] Alternatively, the camera laser coordinates can be calculated using the following formula:

[0032]

[0033] Among them, A j-c A represents the camera laser coordinates of the j-th group of laser data in the original laser data within the camera coordinate system. j-L This represents the original laser coordinates corresponding to the j-th group of laser data in the original laser data. This represents the rotation matrix that transforms the coordinates from the laser coordinate system to the camera module's camera coordinate system. This represents the translation matrix used to transform the coordinates from the laser coordinate system to the camera module's camera coordinate system. This represents the transformation matrix from the laser coordinate system to the camera module's camera coordinate system.

[0034] Optionally, the step of transforming the camera laser coordinates to the focal length coordinate system to obtain the focal length laser coordinates includes:

[0035] The focal length laser coordinates are calculated using the following formula:

[0036]

[0037] Among them, A j-f A represents the focal length laser coordinates of the j-th group of laser data in the original laser data in the focal length coordinate system. c This represents the intrinsic parameter matrix of the cameras included in the camera module, z represents the z-axis coordinate of the j-th group of laser data in the original laser data in the camera coordinate system, and t x and t y f represents the translation factor of the focal length coordinate system. x and f y This represents the scaling factor of the shooting focal length.

[0038] Optionally, the step of using focal length laser data to clean the target point set to be corrected, to obtain a cleaned target point set, includes:

[0039] The coordinates of the first detection box of the first target detection box are determined from the set of target points to be corrected;

[0040] The laser target detection frame of the target object is determined using the focal length laser data, wherein the laser target detection frame includes laser detection frame coordinates;

[0041] Based on the coordinates of the first detection frame and the laser detection frame, the overlap between the first target detection frame and the laser target detection frame is determined;

[0042] If the overlap between the first target detection box and the laser target detection box is less than or equal to a preset overlap threshold, the driving speed of the intelligent vehicle at the current time is reduced, and the original laser data and original captured images are collected again at the next moment of the intelligent vehicle's driving, and the process of performing the first target detection on the original captured images to obtain the first target detection box is returned.

[0043] If the overlap between the first target detection box and the laser target detection box is greater than the overlap threshold, the first target detection box is modified using the laser target detection box to obtain a corrected detection box, wherein the coordinates included in the corrected detection box are used as the target cleaned point set.

[0044] Optionally, the method further includes:

[0045] Calculate the actual distance between the intelligent vehicle and the first target detection box at the current time to obtain the distance to the first target;

[0046] Determine the relationship between the first target distance and a pre-calculated target distance threshold, wherein the method for calculating the target distance threshold includes:

[0047] Obtain the current speed of the intelligent vehicle and calculate the braking distance based on the speed.

[0048] The braking distance is multiplied by a predetermined distance weight to obtain the target distance threshold, wherein the distance weight must be greater than or equal to 1;

[0049] If the distance to the first target is less than the target distance threshold, the intelligent vehicle will be stopped immediately.

[0050] Furthermore, reducing the speed of the intelligent vehicle at the current time includes:

[0051] If the distance to the first target is greater than or equal to the target distance threshold, reduce the speed of the intelligent vehicle at the current time.

[0052] Optionally, the coordinates of the first detection box of the first target detection box and the coordinates of the laser detection box of the laser target detection box are respectively:

[0053] m1=(q 1-l ,w 1-l ,q 1-r ,w 1-r )

[0054] m l =(q l-l ,w l-l ,q l-r ,w l-r )

[0055] Where m1 represents the coordinates of the first detection box corresponding to the first target detection box, m l q represents the coordinates of the laser target detection box. 1-l and w 1-l q represents the coordinates of the top-left corner of the first object detection box. 1-r and w 1-r q represents the coordinates of the bottom right corner of the first object detection box. l-l and w l-l q represents the coordinates of the top-left corner of the laser target detection box. l-r and w l-r This indicates the coordinates of the lower right corner of the laser target detection box.

[0056] Optionally, the step of using the laser target detection box to modify the first target detection box to obtain a modified detection box includes:

[0057] Based on the coordinates of the first detection frame and the laser detection frame, the overlapping area between the first target detection frame and the laser target detection frame is calculated, and the target overlapping detection frame is obtained based on the overlapping area.

[0058] Obtain the non-overlapping region between the target overlap detection box and the first target detection box;

[0059] The area ratio of the non-overlapping region to the area of ​​the first target detection box is calculated to obtain the non-overlapping area ratio.

[0060] If the non-overlapping area ratio is less than or equal to the preset non-overlapping threshold, the target overlapping detection box is determined to be the corrected detection box.

[0061] If the non-overlapping area ratio is greater than the non-overlapping threshold, multiple straight lines are constructed from the direction of the target overlapping detection box to the first target detection box. Each straight line contains multiple sets of coordinates, and each line is located within the non-overlapping area.

[0062] Calculate the pixel gradient values ​​of multiple sets of coordinates within each straight line, and determine the coordinates with the largest pixel gradient value in each straight line;

[0063] Connect the coordinates of the pixel gradient with the largest value in each straight line to obtain the corrected detection box.

[0064] To address the aforementioned problems, the present invention also provides a detection device for small road targets during autonomous driving, the device comprising:

[0065] The module startup module is used to activate the camera module and laser module of the intelligent vehicle when the intelligent vehicle starts autonomous driving. The front of the intelligent vehicle includes at least one camera module and one laser module. The laser module is used to acquire the raw laser data of the intelligent vehicle while it is in motion, and to determine the shooting focal length of the camera module. Based on the shooting focal length, the module acquires the raw images of the intelligent vehicle while it is in motion. The laser module and the raw laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the raw images belong to the focal length coordinate system.

[0066] The target detection box judgment module is used to perform a first target detection on the original captured image to obtain a first target detection box. If the area of ​​the first target detection box is less than or equal to the box threshold, the target in the first target detection box is determined to be a small-volume target object. Each pixel in the first target detection box is obtained in sequence to obtain a target to be corrected point set.

[0067] The coordinate system transformation module is used to transform the original laser data belonging to the laser coordinate system to the focal length coordinate system using the camera coordinate system to obtain focal length laser data.

[0068] The target detection box replacement module is used to clean the target point set to be corrected using focal length laser data to obtain a cleaned target point set, wherein the cleaned target point set corresponds to a corrected detection box. The corrected detection box is used to replace the first target detection box to obtain the target detection result when the intelligent vehicle is driving autonomously.

[0069] Compared to the problems described in the background art, the embodiments of the present invention first utilize a laser module to acquire the raw laser data of the intelligent vehicle while it is in motion, and determine the shooting focal length of the camera module. Based on the shooting focal length, the original captured images of the intelligent vehicle while it is in motion are obtained. The laser module and the raw laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the original captured images belong to the focal length coordinate system. It is evident that the embodiments of the present invention improve detection accuracy by combining the camera module and the laser module when performing target detection in autonomous driving. Furthermore, a first target detection is first performed on the original captured images to obtain the second target detection. A target detection box is used. If the area of ​​the first target detection box is greater than a preset threshold, the target within the first target detection box is determined to be a large-volume object. Since this embodiment of the invention is based on a deep learning model for target detection, it mainly selects targets by using target detection boxes. That is, the first target detection box will reasonably select targets based on their size. Therefore, the size of the target detection box can reflect the size of the target to a certain extent. Thus, in this embodiment of the invention, the size of the target within the first target detection box is determined by judging its area. If the area of ​​the first target detection box is greater than a preset threshold, the target size is determined. Let be a large-volume target. Since deep learning models perform well in detecting large-volume targets, there is no need to use a laser module to correct the target detection box. If the area of ​​the first target detection box is less than or equal to the box threshold, the target within the first target detection box is determined to be a small-volume target. Each pixel in the first target detection box is sequentially acquired to obtain the target point set to be corrected. The original laser data belonging to the laser coordinate system is transformed to the focal length coordinate system using the camera coordinate system to obtain focal length laser data. The focal length laser data is used to clean the target point set to be corrected to obtain the cleaned target point set, where the cleaned target point set corresponds to the corrected detection. The corrected detection box replaces the first target detection box to obtain the target detection result during autonomous driving of the intelligent vehicle. Obviously, the embodiment of the present invention converts the original laser data obtained by the laser module into the focal coordinate system of the image, and then corrects the target detection box of the target object through the original laser data to obtain the corrected detection box. Compared with the first target detection box, the corrected detection box fits the small target object better, which solves the problem of inaccurate target detection boxes generated when detecting small targets during autonomous driving, and realizes the effect of improving the target detection accuracy of intelligent vehicles for small targets in intelligent driving. Attached Figure Description

[0070] Figure 1 This is a flowchart illustrating a method for detecting small road targets during autonomous driving, provided in an embodiment of the present invention.

[0071] Figure 2This is a schematic diagram of a camera module and a laser module for detecting small road targets during autonomous driving, provided in an embodiment of the present invention.

[0072] Figure 3 This is a schematic diagram illustrating the pixel gradient value calculation of a method for detecting small road targets during autonomous driving, provided in an embodiment of the present invention.

[0073] Figure 4 A functional block diagram of a detection device for small road targets during autonomous driving provided in an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the method for detecting small road targets during autonomous driving, according to an embodiment of the present invention.

[0075] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0076] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0077] This application provides a method for detecting small road targets during autonomous driving. The execution entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method for detecting small road targets during autonomous driving can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0078] Example 1:

[0079] Reference Figure 1 The diagram shown is a flowchart illustrating a method for detecting small road targets during autonomous driving, according to an embodiment of the present invention. In this embodiment, the method for detecting small road targets during autonomous driving includes:

[0080] S1. When the intelligent vehicle activates autonomous driving, the camera module and laser module of the intelligent vehicle are activated. The front of the intelligent vehicle includes at least one camera module and one laser module.

[0081] It should be explained that, in order to improve the target detection accuracy of intelligent vehicles during autonomous driving, this embodiment of the invention utilizes laser data from a laser module to correct the targets detected by the camera module. Because various types of targets frequently appear in front of an intelligent vehicle during autonomous driving, and as the vehicle continues to move, the distance between the vehicle and these targets decreases, the detected targets by the camera module also increase with the vehicle's movement. Early detection of targets in front of the vehicle can effectively reduce the probability of traffic accidents.

[0082] The camera module typically includes a high-definition camera, which can capture a scene image in front of the intelligent vehicle and then identify the target object in the scene image based on the target detection algorithm; the laser module typically includes a lidar, which can emit a laser in front of the intelligent vehicle and then determine the target object in front based on the reflected laser.

[0083] However, object detection algorithms often perform poorly in recognizing small objects within a scene image. This is mainly because, although they can identify small objects, the bounding boxes generated by the algorithm are not accurate enough. The bounding boxes are often much larger than the small objects, which can affect the driving direction of the intelligent vehicle and even cause traffic accidents. Therefore, this invention uses LiDAR to correct the bounding boxes, thereby more accurately selecting small objects.

[0084] S2. Use the laser module to acquire the original laser data of the intelligent vehicle while it is driving, and determine the shooting focal length of the camera module. Based on the shooting focal length, acquire the original shooting image of the intelligent vehicle while it is driving. The laser module and the original laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the original shooting image belongs to the focal length coordinate system.

[0085] Understandably, since camera modules and laser modules belong to different technological directions, they both rely on different coordinate systems when performing their respective tasks. Specifically, the laser module and the raw laser data depend on the laser coordinate system. Generally, a three-dimensional laser coordinate system can be constructed with the laser module's location as the origin. When the laser module receives the reflected light from the emitted laser, the raw laser data can be obtained by determining the position of the reflected light in the laser coordinate system. Similarly, the camera module belongs to the camera coordinate system.

[0086] However, it should be emphasized that the original captured images are generally scene images of the direction in which the intelligent vehicle is driving, and the effect of the original captured images obtained by different shooting focal lengths is not the same. Therefore, in order to control the degree of distortion of the original captured images and the accuracy of subsequent target detection, the camera module belongs to the camera coordinate system, while the original captured images belong to the focal length coordinate system, which is generally constructed based on the focal position of the camera module.

[0087] S3. Perform a first target detection on the original captured image to obtain a first target detection box. If the area of ​​the first target detection box is greater than a preset box threshold, then the target in the first target detection box is determined to be a large-volume target object.

[0088] Specifically, performing a first object detection on the original captured image to obtain a first object detection box includes:

[0089] The original captured image is input into the object detection model, which has the following structure: convolutional layer, normalization layer, ReLU activation layer, multi-level residual layer, pooling layer, linear transformation layer, and object detection box generation layer. The multi-level residual layer has four layers in the order of processing the original captured image. The first layer consists of two convolutional layers; the second layer consists of a pooling layer, a fully connected layer, a ReLU activation layer, a fully connected layer, and a Sigmoid activation layer, with the input data of the second layer being the output data of the first layer; the third layer is a fusion layer that performs a dot product operation between the output data of the first and second layers; the fourth layer is the same as the first layer, with the input data of the fourth layer being the output data of the third layer; and the object detection box generation layer is a YOLO model.

[0090] The original captured image is sequentially subjected to convolution, pooling, and activation operations using the convolutional layer, normalization layer, and ReLU activation layer to obtain the first activated image.

[0091] The first activated image is further convolved through two convolutional layers of the first layer structure to obtain the first convolutional image.

[0092] The first convolutional image is input into the second layer structure, and then passes through the pooling layer, fully connected layer, ReLU activation layer, fully connected layer, and Sigmoid activation layer of the second layer structure in sequence to obtain the second activated image;

[0093] Within the fusion layer of the third structure, the second activated image and the first convolutional image are multiplied by a dot to obtain the reconstructed image.

[0094] The reconstructed image is input into the fourth layer structure to obtain the reconstructed convolutional image;

[0095] The reconstructed convolutional image is input into the YOLO model, and the YOLO model is used to detect the targets included in the reconstructed convolutional image to generate the first target detection box.

[0096] It should be explained that the object detection model is an object detection class model built based on deep learning. In order to improve the accuracy of object detection, this embodiment of the invention further constructs convolutional layers, normalization layers, ReLU activation layers, multi-level residual layers, pooling layers, and linear transformation layers in the front end of the YOLO model network. In order to prevent data loss from multiple convolutions and pooling, a multi-level residual layer is constructed as the core. The multi-level residual layer can extract important features in the image through multiple convolutions, pooling and mutual multiplication, thereby helping the YOLO model to quickly identify targets.

[0097] For example, if a smart car is driving on an open road, and the original image captured by the camera module shows a stone block on the road, the stone block can be identified using the aforementioned target detection model. It is understood that the target detection box will select the target object; that is, the first target detection box will reasonably select the stone block based on its size. Therefore, the size of the target detection box can reflect the size of the target object to a certain extent. Thus, in this embodiment of the invention, the size of the target within the first target detection box is determined by judging its area. If the area of ​​the first target detection box is greater than a threshold value, the stone block is confirmed as a large-volume target object.

[0098] It should be explained that deep learning-based object detection models, especially the object detection model built on the YOLO model mentioned above, have high accuracy in detecting large objects. Therefore, they can effectively and promptly identify large objects and feed back the location information of large objects to the intelligent vehicle. Thus, when a large object is identified, no further steps need to be performed in order to save resources.

[0099] S4. If the area of ​​the first target detection box is less than or equal to the box threshold, the target in the first target detection box is determined to be a small-volume target object. Each pixel in the first target detection box is obtained in sequence to obtain the target to be corrected point set.

[0100] However, further, if the first target detection box selects the stone pier based on its size, and the area of ​​the first target detection box is less than or equal to the box threshold, it indicates that the stone pier is small. Since deep learning-based target detection models are not good at detecting small objects, although they can detect small objects, the area of ​​the generated target detection box is very large, far exceeding the actual area of ​​the small object. An excessively large box area will directly affect the subsequent driving direction of the intelligent vehicle, and may even cause traffic accidents. Therefore, this embodiment of the invention needs to further adjust the area of ​​the target detection box.

[0101] For example, all pixels within the first target detection box corresponding to a small target object, such as the small stone block mentioned above, are obtained, which constitutes the target correction point set. It is understood that within the target correction point set, some pixels belong to the small stone block, while others correspond to the redundant area selected by the first target detection box.

[0102] S5. Transform the original laser data belonging to the laser coordinate system to the focal length coordinate system using the camera coordinate system to obtain the focal length laser data.

[0103] In detail, the process of projecting the raw laser data belonging to the laser coordinate system onto the focal length coordinate system using the camera coordinate system to obtain focal length laser data includes:

[0104] Extract each set of laser data from the original laser data sequentially, and perform the following operations on each set of laser data:

[0105] The coordinates of the laser data in the laser coordinate system are determined as the original laser coordinates;

[0106] The original laser coordinates are transformed into the camera coordinate system to obtain the camera laser coordinates;

[0107] Transform the camera laser coordinates to the focal length coordinate system to obtain the focal length laser coordinates;

[0108] Having obtained the focal length laser coordinates corresponding to each set of laser data, the focal length laser coordinates corresponding to each set of laser data are summarized to obtain the focal length laser data.

[0109] For example, if there are 100 sets of laser data in the original laser data, then each set of laser data corresponds to original laser coordinates in the laser coordinate system. It should be emphasized that the original laser coordinates consist of three-dimensional coordinates, namely the X, Y, and Z axes. Similarly, the camera coordinate system is also three-dimensional, meaning the camera laser coordinates are also three-dimensional. However, the focal length coordinate system to which the original captured image belongs is two-dimensional, meaning the original captured image is a two-dimensional image. The prerequisite for converting the three-dimensional original laser data to the focal length coordinate system is that the Z-axis of the camera laser coordinates is at the same distance from the focal length coordinate system to the camera coordinate system. In other words, after the original laser coordinates are converted to camera laser coordinates, if its Z-axis value is at the same distance from the focal length coordinate system to the camera coordinate system, then step S5 is executed.

[0110] Further, the transformation of the original laser coordinates to the camera coordinate system to obtain the camera laser coordinates includes:

[0111] The camera laser coordinates are calculated using the following formula:

[0112]

[0113] Alternatively, the camera laser coordinates can be calculated using the following formula:

[0114]

[0115] Among them, A j-c A represents the camera laser coordinates of the j-th group of laser data in the original laser data within the camera coordinate system. j-L This represents the original laser coordinates corresponding to the j-th group of laser data in the original laser data. This represents the rotation matrix that transforms the coordinates from the laser coordinate system to the camera module's camera coordinate system. This represents the translation matrix used to transform the coordinates from the laser coordinate system to the camera module's camera coordinate system. This represents the transformation matrix from the laser coordinate system to the camera module's camera coordinate system.

[0116] It should be explained that, within the driving space of an intelligent vehicle, the presence of camera and laser modules causes differences in the size and orientation of the same spatial point in the camera coordinate system and the laser coordinate system. To achieve coordinate transformation of the same spatial point in the two coordinate systems, translation, transformation, and rotation matrices are required. The definitions and relationships of these three matrices are all publicly available coordinate transformation techniques, and will not be elaborated upon further in this embodiment of the invention.

[0117] Further, the transformation of the camera laser coordinates to the focal length coordinate system to obtain the focal length laser coordinates includes:

[0118] The focal length laser coordinates are calculated using the following formula:

[0119]

[0120] Among them, A j-f A represents the focal length laser coordinates of the j-th group of laser data in the original laser data in the focal length coordinate system. c This represents the intrinsic parameter matrix of the cameras included in the camera module, z represents the z-axis coordinate of the j-th group of laser data in the original laser data in the camera coordinate system, and t x and t y f represents the translation factor of the focal length coordinate system. x and f y This represents the scaling factor of the shooting focal length.

[0121] It should be explained that several commonly used parameters are required in coordinate transformation. Besides the rotation matrix, these include translation factors. Translation factors effectively establish a one-to-one correspondence between one coordinate system and another. Since the focal length coordinate system in this embodiment is a two-dimensional coordinate system, its two corresponding translation factors are t... x and ty .

[0122] In summary, based on the coordinate transformation described above, the coordinates of the target object in front of the intelligent vehicle detected by the laser module (i.e., the laser data described in the embodiments of this invention) can be successfully transformed into the focal length coordinate system of the captured image obtained by the camera module, thereby allowing for further optimization of the detected target object within the same coordinate system.

[0123] S6. Use focal length laser data to clean the target point set to be corrected, and obtain the target cleaned point set.

[0124] Specifically, the step of using focal length laser data to clean the target point set to be corrected, and obtaining the cleaned target point set, includes:

[0125] The coordinates of the first detection box of the first target detection box are determined from the set of target points to be corrected;

[0126] The laser target detection frame of the target object is determined using the focal length laser data, wherein the laser target detection frame includes laser detection frame coordinates;

[0127] Based on the coordinates of the first detection frame and the laser detection frame, the overlap between the first target detection frame and the laser target detection frame is determined;

[0128] If the overlap between the first target detection box and the laser target detection box is less than or equal to a preset overlap threshold, the driving speed of the intelligent vehicle at the current time is reduced, and the original laser data and original captured images are collected again at the next moment of the intelligent vehicle's driving, and the process of performing the first target detection on the original captured images to obtain the first target detection box is returned.

[0129] If the overlap between the first target detection box and the laser target detection box is greater than the overlap threshold, the first target detection box is modified using the laser target detection box to obtain a corrected detection box, wherein the coordinates included in the corrected detection box are used as the target cleaned point set.

[0130] Understandably, the focal length laser data obtained by the laser module can be used to determine the target objects encountered by the intelligent vehicle while driving, from the perspective of the laser module. For example, the first target detection box selects small-volume targets, such as the small stone pier mentioned above. Logically, the laser module will also detect the small stone pier and generate a series of focal length laser data for it. Then, within the focal length coordinate system, the corresponding laser target detection box can be generated using this focal length laser data. Therefore, within the focal length coordinate system, the first target detection box for the small stone pier is generated using the camera module and a deep learning model, and the laser module generates the laser target detection box for the small stone pier.

[0131] Therefore, for the driving safety of intelligent vehicles, it is necessary to determine the overlap between the laser target detection box and the first target detection box. In principle, if both the laser target detection box and the first target detection box select the small stone block, the overlap between the laser target detection box and the first target detection box is relatively high. However, when the overlap between the laser target detection box and the first target detection box is very low, such as less than or equal to the above overlap threshold, it indicates that the laser target detection box and the first target detection box may not correspond to the same target object. It is very likely that the camera module and the laser module have identified different target objects. Therefore, in order to allow the intelligent vehicle sufficient detection time, it is necessary to reduce the driving speed of the intelligent vehicle at the current time, collect the original laser data and the original captured image again, and return to perform the first target detection on the original captured image to obtain the first target detection box and continue to perform target detection.

[0132] Furthermore, reducing the speed of the intelligent vehicle at the current time includes:

[0133] Calculate the actual distance between the intelligent vehicle and the first target detection box at the current time to obtain the distance to the first target;

[0134] Determine the relationship between the distance to the first target and the pre-calculated target distance threshold.

[0135] Furthermore, if the distance to the first target is less than the target distance threshold, the intelligent vehicle will be stopped directly.

[0136] If the distance to the first target is greater than or equal to the target distance threshold, reduce the speed of the intelligent vehicle at the current time.

[0137] It should be explained that intelligent vehicles traveling at different speeds correspond to different braking distances. For example, the braking distances corresponding to a driving speed of 30 km / h and 120 km / h are significantly different. In this embodiment of the invention, the method for calculating the pre-calculated target distance threshold includes:

[0138] Obtain the current speed of the intelligent vehicle and calculate the braking distance based on the speed.

[0139] The braking distance is multiplied by a predetermined distance weight to obtain the target distance threshold, wherein the distance weight must be greater than or equal to 1.

[0140] For example, the braking distance at a driving speed of 30 km / h is 3, and the braking distance at a driving speed of 120 km / h is 20. After multiplying each by a pre-set distance weight, the target distance threshold described in this embodiment of the invention can be obtained.

[0141] Furthermore, the coordinates of the first detection box of the first target detection box and the coordinates of the laser detection box of the laser target detection box are respectively:

[0142] m1=(q 1-l ,w 1-l ,q 1-r ,w 1-r )

[0143] m l =(q l-l ,w l-l ,q l-r ,w l-r )

[0144] Where m1 represents the coordinates of the first detection box corresponding to the first target detection box, m l q represents the coordinates of the laser target detection box. 1-l and w 1-l q represents the coordinates of the top-left corner of the first object detection box. 1-r and w 1-r q represents the coordinates of the bottom right corner of the first object detection box. l-l and w l-l q represents the coordinates of the top-left corner of the laser target detection box. l-r and w l-r The coordinates of the lower right corner of the laser target detection box are represented by q, x, and y.

[0145] Therefore, if the overlap between the first target detection box and the laser target detection box is greater than the overlap threshold, it indicates that the first target detection box and the laser target detection box have selected the same target object. If both have selected the aforementioned small stone piers, the next step is to adjust the target boxes of the small stone piers to ensure that the target boxes for selecting the small stone piers are more accurate. Further, the step of using the laser target detection box to modify the first target detection box to obtain the modified detection box includes:

[0146] Based on the coordinates of the first detection frame and the laser detection frame, the overlapping area between the first target detection frame and the laser target detection frame is calculated, and the target overlapping detection frame is obtained based on the overlapping area.

[0147] Obtain the non-overlapping area between the target overlap detection box and the first target detection box, and calculate the ratio of the area of ​​the non-overlapping area to the area of ​​the first target detection box to obtain the non-overlapping area ratio;

[0148] If the non-overlapping area ratio is less than or equal to the preset non-overlapping threshold, the target overlapping detection box is determined to be the corrected detection box.

[0149] If the non-overlapping area ratio is greater than the non-overlapping threshold, multiple straight lines are constructed from the direction of the target overlapping detection box to the first target detection box. Each straight line contains multiple sets of coordinates, and each straight line is located within the non-overlapping area.

[0150] Calculate the pixel gradient values ​​of multiple sets of coordinates within each straight line, and determine the coordinates with the largest pixel gradient value in each straight line;

[0151] Connect the coordinates of the pixel gradient with the largest value in each straight line to obtain the corrected detection box.

[0152] For example, see Figure 2 As shown, when both the first target detection box and the laser target detection box select the small stone pier, it is necessary to further adjust the target box of the small stone pier. Therefore, the overlapping area of ​​the first target detection box and the laser target detection box is calculated. Assuming that the first target detection box is a rectangle and the laser target detection box is a circle, the overlapping area of ​​the rectangle and the circle is the target overlap detection box. It can be understood that, under normal circumstances, the target overlap detection box still includes the small stone pier.

[0153] Further, see Figure 3 In this embodiment of the invention, the first target detection box needs to be corrected. Therefore, the non-overlapping region between the target overlapping detection box and the first target detection box (the non-overlapping region is the area where the arrow is located) is obtained. By calculating the area ratio of the non-overlapping region to the area of ​​the first target detection box, it is determined whether it is necessary to perform pixel gradient value calculation. It can be understood that if the non-overlapping area ratio is less than or equal to a preset non-overlapping threshold, it means that the target overlapping detection box and the first target detection box are similar, and the target overlapping detection box only slightly adjusts the first target detection box. Therefore, the target overlapping detection box can be used to directly replace the first target detection box to obtain the corrected detection box.

[0154] If the non-overlapping area ratio is greater than the non-overlapping threshold, it indicates that the target overlap detection box differs significantly from the first target detection box. In this case, directly confirming the target overlap detection box as the corrected detection box may result in the loss of some information about the target object. Therefore, please refer to [further details needed]. Figure 3 As shown, multiple straight lines are constructed from the target overlapping detection box to the first target detection box in the direction from the target overlapping detection box to the first target detection box. Each straight line is located within a non-overlapping region. The pixel gradient value on each straight line is calculated sequentially. The method for calculating the pixel gradient value is a publicly available technique and will not be elaborated here. Furthermore, the coordinates of the largest pixel gradient value are likely to be the segmentation point between the target object and the background. Therefore, connecting the coordinates of the largest pixel gradient change values ​​on each straight line yields the corrected detection box. It is understood that if the non-overlapping area ratio is greater than the non-overlapping threshold, the area of ​​the corrected detection box is greater than or equal to that of the target overlapping detection box.

[0155] S7. Replace the first target detection box with the corrected detection box to obtain the target detection result when the intelligent vehicle is driving autonomously.

[0156] Understandably, the detection box can be continuously adjusted according to the above technical steps to obtain a corrected detection box, and the corrected detection box can be used to replace the first target detection box to achieve more accurate target selection, thereby improving driving safety of intelligent vehicles during autonomous driving.

[0157] Compared to the problems described in the background art, the embodiments of the present invention first utilize a laser module to acquire the raw laser data of the intelligent vehicle while it is in motion, and determine the shooting focal length of the camera module. Based on the shooting focal length, the original captured images of the intelligent vehicle while it is in motion are obtained. The laser module and the raw laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the original captured images belong to the focal length coordinate system. It is evident that the embodiments of the present invention improve detection accuracy by combining the camera module and the laser module when performing target detection in autonomous driving. Furthermore, a first target detection is first performed on the original captured images to obtain the second target detection. A target detection box is used. If the area of ​​the first target detection box is greater than a preset threshold, the target within the first target detection box is determined to be a large-volume object. Since this embodiment of the invention is based on a deep learning model for target detection, it mainly selects targets by using target detection boxes. That is, the first target detection box will reasonably select targets based on their size. Therefore, the size of the target detection box can reflect the size of the target to a certain extent. Thus, in this embodiment of the invention, the size of the target within the first target detection box is determined by judging its area. If the area of ​​the first target detection box is greater than a preset threshold, the target size is determined. Let be a large-volume target. Since deep learning models perform well in detecting large-volume targets, there is no need to use a laser module to correct the target detection box. If the area of ​​the first target detection box is less than or equal to the box threshold, the target within the first target detection box is determined to be a small-volume target. Each pixel in the first target detection box is sequentially acquired to obtain the target point set to be corrected. The original laser data belonging to the laser coordinate system is transformed to the focal length coordinate system using the camera coordinate system to obtain focal length laser data. The focal length laser data is used to clean the target point set to be corrected to obtain the cleaned target point set, where the cleaned target point set corresponds to the corrected detection. The corrected detection box replaces the first target detection box to obtain the target detection result during autonomous driving of the intelligent vehicle. Obviously, the embodiment of the present invention converts the original laser data obtained by the laser module into the focal coordinate system of the image, and then corrects the target detection box of the target object through the original laser data to obtain the corrected detection box. Compared with the first target detection box, the corrected detection box fits the small target object better, which solves the problem of inaccurate target detection boxes generated when detecting small targets during autonomous driving, and realizes the effect of improving the target detection accuracy of intelligent vehicles for small targets in intelligent driving.

[0158] Example 2:

[0159] like Figure 4 The diagram shown is a functional block diagram of a detection device for small road targets during autonomous driving provided in an embodiment of the present invention.

[0160] The detection device 100 for small road targets during autonomous driving, as described in this invention, can be installed in an electronic device. Depending on the functions implemented, the detection device 100 may include a module activation module 101, a target detection box judgment module 102, a coordinate system transformation module 103, and a target detection box replacement module 104. The module described in this invention can also be called a unit, referring to a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0161] The module startup module 101 is used to start the camera module and laser module of the intelligent vehicle when the intelligent vehicle starts autonomous driving. The front of the intelligent vehicle includes at least one camera module and one laser module. The laser module is used to acquire the original laser data of the intelligent vehicle while it is driving, and to determine the shooting focal length of the camera module. Based on the shooting focal length, the original captured image of the intelligent vehicle while it is driving is acquired. The laser module and the original laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the original captured image belongs to the focal length coordinate system.

[0162] The target detection box judgment module 102 is used to perform a first target detection on the original captured image to obtain a first target detection box. If the area of ​​the first target detection box is less than or equal to the box threshold, the target in the first target detection box is determined to be a small-volume target object. Each pixel in the first target detection box is obtained in sequence to obtain a target to be corrected point set.

[0163] The coordinate system transformation module 103 is used to transform the original laser data belonging to the laser coordinate system to the focal length coordinate system using the camera coordinate system to obtain focal length laser data.

[0164] The target detection box replacement module 104 is used to clean the target point set to be corrected using focal length laser data to obtain a cleaned target point set, wherein the cleaned target point set corresponds to a corrected detection box. The corrected detection box is used to replace the first target detection box to obtain the target detection result when the intelligent vehicle is driving autonomously.

[0165] In detail, the modules in the road small target detection device 100 based on autonomous driving described in this embodiment of the invention employ the same methods as described above during use. Figure 1 The method described herein is the same as the method for detecting small targets on the road during autonomous driving, and can produce the same technical effect, so it will not be repeated here.

[0166] Example 3:

[0167] like Figure 5 The diagram shown is a schematic representation of an electronic device for implementing a method for detecting small road targets during autonomous driving, according to an embodiment of the present invention.

[0168] The electronic device 1 may include a processor 10, a memory 11, a bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a program for detecting small road targets during autonomous driving.

[0169] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 can include both internal and external storage units of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as code for a road small target detection program during autonomous driving, but also to temporarily store data that has been output or will be output.

[0170] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., a program for detecting small road targets during autonomous driving) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0171] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0172] Figure 5 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 5 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0173] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0174] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0175] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0176] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0177] The memory 11 in the electronic device 1 stores a program for detecting small road targets during autonomous driving, which is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0178] When an intelligent vehicle activates autonomous driving, its camera module and laser module are activated. The front of the intelligent vehicle includes at least one camera module and one laser module.

[0179] The laser module is used to acquire the raw laser data of the intelligent vehicle while it is in motion, and the shooting focal length of the camera module is determined. Based on the shooting focal length, the raw images of the intelligent vehicle while it is in motion are acquired. The laser module and the raw laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the raw images belong to the focal length coordinate system.

[0180] Perform a first object detection on the original captured image to obtain a first object detection bounding box;

[0181] If the area of ​​the first target detection box is less than or equal to the box threshold, the target in the first target detection box is determined to be a small-volume target object. Each pixel in the first target detection box is obtained in sequence to obtain the target to be corrected point set.

[0182] The original laser data belonging to the laser coordinate system is transformed to the focal length coordinate system using the camera coordinate system to obtain the focal length laser data;

[0183] The target point set to be corrected is cleaned using focal length laser data to obtain the target cleaned point set, where the target cleaned point set corresponds to the corrected detection box;

[0184] The corrected detection box is used to replace the first target detection box to obtain the target detection result when the intelligent vehicle is driving autonomously.

[0185] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 4 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0186] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0187] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0188] When an intelligent vehicle activates autonomous driving, its camera module and laser module are activated. The front of the intelligent vehicle includes at least one camera module and one laser module.

[0189] The laser module is used to acquire the raw laser data of the intelligent vehicle while it is in motion, and the shooting focal length of the camera module is determined. Based on the shooting focal length, the raw images of the intelligent vehicle while it is in motion are acquired. The laser module and the raw laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the raw images belong to the focal length coordinate system.

[0190] Perform a first object detection on the original captured image to obtain a first object detection bounding box;

[0191] If the area of ​​the first target detection box is less than or equal to the box threshold, the target in the first target detection box is determined to be a small-volume target object. Each pixel in the first target detection box is obtained in sequence to obtain the target to be corrected point set.

[0192] The original laser data belonging to the laser coordinate system is transformed to the focal length coordinate system using the camera coordinate system to obtain the focal length laser data;

[0193] The target point set to be corrected is cleaned using focal length laser data to obtain the target cleaned point set, where the target cleaned point set corresponds to the corrected detection box;

[0194] The corrected detection box is used to replace the first target detection box to obtain the target detection result when the intelligent vehicle is driving autonomously.

[0195] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0196] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0197] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting a small target on a road during automatic driving, characterized in that, The method comprises: When the intelligent automobile is started to drive automatically, start the camera module and the laser module of the intelligent automobile, wherein the position of the head of the intelligent automobile comprises at least one group of camera module and one group of laser module; Obtain the original laser data of the intelligent automobile when driving by using the laser module, and determine the shooting focal length of the camera module, and obtain the original shooting picture of the intelligent automobile when driving based on the shooting focal length, wherein the laser module and the original laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the original shooting picture belongs to the focal length coordinate system; Perform first target detection on the original shooting picture to obtain a first target detection frame, and if the frame area of the first target detection frame is greater than a preset frame threshold, determine that the target in the first target detection frame is a large-volume target object; If the frame area of the first target detection frame is less than or equal to the frame threshold, determine that the target in the first target detection frame is a small-volume target object, and sequentially obtain each pixel point in the first target detection frame to obtain a target to-be-corrected point set; Convert the original laser data belonging to the laser coordinate system to the focal length coordinate system by using the camera coordinate system to obtain focal length laser data; Cleanse the target to-be-corrected point set by using the focal length laser data to obtain a target cleaned point set, wherein the target cleaned point set corresponds to a corrected detection frame; Replace the first target detection frame with the corrected detection frame to obtain a target detection result when the intelligent automobile drives automatically; The step of cleansing the target to-be-corrected point set by using the focal length laser data to obtain a target cleaned point set comprises: Determine the first detection frame coordinates of the first target detection frame from the target to-be-corrected point set; Determine the laser target detection frame of the target object by using the focal length laser data, wherein the laser target detection frame comprises laser detection frame coordinates; Determine the coincidence degree of the first target detection frame and the laser target detection frame according to the first detection frame coordinates and the laser detection frame coordinates; If the coincidence degree of the first target detection frame and the laser target detection frame is less than or equal to a preset coincidence threshold, reduce the driving speed of the intelligent automobile at the current time, continue to collect the original laser data and the original shooting picture at the next time when the intelligent automobile drives, and return to the first target detection step; If the coincidence degree of the first target detection frame and the laser target detection frame is greater than the coincidence threshold, trim the first target detection frame by using the laser target detection frame to obtain a corrected detection frame, wherein the coordinates included in the corrected detection frame are the target cleaned point set; The step of trimming the first target detection frame by using the laser target detection frame to obtain a corrected detection frame comprises: According to the first detection frame coordinates and the laser detection frame coordinates, calculate the overlapping area of the first target detection frame and the laser target detection frame to obtain a target overlapping detection frame; Obtain the non-overlapping area of the target overlapping detection frame and the first target detection frame; And calculate the area ratio of the non-overlapping area to the area of the first target detection frame to obtain a non-overlapping area ratio; If the non-overlapping area ratio is less than or equal to a preset non-overlapping threshold, determine that the target overlapping detection frame is the corrected detection frame. If the non-coincidence area ratio is greater than the non-coincidence threshold, a plurality of straight lines are constructed from the target coincidence detection frame to the first target detection frame in a direction, wherein each straight line includes a plurality of groups of coordinates, and each straight line is located in the non-coincidence area; Pixel gradient values of the plurality of groups of coordinates in each straight line are calculated, and a coordinate with the maximum pixel gradient value is determined; The coordinates with the maximum pixel gradient value in each straight line are sequentially connected to obtain a corrected detection frame. 2.The method for detecting a small road target based on automatic driving time according to claim 1, wherein, The method further includes: The original photographed picture is input into a target detection model, wherein a model structure of the target detection model is: a convolution layer, a normalization layer, a RELU activation layer, a multi-level residual layer, a pooling layer, a linear change layer, and a target detection frame generation layer, and the multi-level residual layer has four layers in the order of processing the original photographed picture, a first layer structure is composed of two convolution layers; a second layer structure is composed of a pooling layer, a full connection layer, a RELU activation layer, a full connection layer, and a Sigmoid activation layer, input data of the second layer structure is output data of the first layer structure; a third layer structure is a fusion layer, the fusion layer performs a point multiplication operation on the output data of the first layer structure and the output data of the second layer structure; a fourth layer structure is the same as the first layer structure, and input data of the fourth layer structure is output data of the third layer structure; and the target detection frame generation layer is a YOLO model; The original photographed picture is sequentially subjected to convolution, pooling, and activation operations by using the convolution layer, the normalization layer, and the RELU activation layer to obtain a first activated picture; The first activated picture is subjected to a convolution operation by the two convolution layers of the first layer structure to obtain a first convolution picture; The first convolution picture is input into the second layer structure and sequentially subjected to a pooling operation, a full connection operation, a RELU activation operation, a full connection operation, and a Sigmoid activation operation by the second layer structure to obtain a second activated picture; In the fusion layer of the third layer structure, the second activated picture and the first convolution picture are subjected to a point multiplication operation to obtain a reorganized photographed picture; The reorganized photographed picture is input into the fourth layer structure, and data processing of the fourth layer structure is the same as that of the first layer structure to obtain a reorganized convolution picture; The reorganized convolution picture is input into the YOLO model, and the YOLO model detects targets included in the reorganized convolution picture to generate the first target detection frame. 3.The method for detecting small road targets based on automatic driving time according to claim 1, wherein, The method further includes: Each group of laser data in the original laser data is sequentially extracted, and each group of laser data is subjected to the following operations: Coordinates of each group of laser data in the laser coordinate system are obtained to obtain original laser coordinates; The original laser coordinates are converted into camera coordinates to obtain camera laser coordinates; The camera laser coordinates are converted into focal length coordinates to obtain focal length laser coordinates; The focal length laser coordinates corresponding to each group of laser data are summarized to obtain the focal length laser data. 4.The method for detecting small road targets based on automatic driving time according to claim 3, characterized in that, The method further includes: The camera laser coordinates are calculated according to the following formula: wherein, represents the original laser data in the first group of laser data in the camera laser coordinates of the camera coordinate system, represents the original laser data in the first group of laser data corresponding to the original laser coordinates, represents the rotation matrix of the laser coordinate system to the camera coordinates of the camera module camera coordinate system, represents the translation matrix of the laser coordinate system to the camera coordinates of the camera module camera coordinate system, represents the conversion matrix of the laser coordinate system to the camera coordinates of the camera module camera coordinate system. 5.The method for detecting small road targets based on automatic driving time according to claim 4, characterized in that, The converting the camera laser coordinates to the focal length coordinate system to obtain focal length laser coordinates comprises: The focal length laser coordinates are calculated according to the following formula: wherein, represents the focal length coordinate of the i-th group of laser data in the focal length coordinate system, represents the focal length coordinate of the i-th group of laser data in the focal length coordinate system, represents the intrinsic matrix of the camera included in the camera module, represents the focal length coordinate of the i-th group of laser data in the focal length coordinate system, represents the z-axis coordinate of the i-th group of laser data in the camera coordinate system, and represents the translation factor of the focal length coordinate system, and represents the scaling factor of the focal length. 6.The method for detecting small road targets based on automatic driving time according to claim 1, wherein, The reducing the driving speed of the intelligent vehicle at the current time further comprises: The actual distance of the intelligent vehicle from the first target detection frame is calculated at the current time to obtain a first target distance; The size relationship between the first target distance and a pre-calculated target distance threshold is determined, wherein the calculation method of the target distance threshold comprises: The driving speed of the intelligent vehicle at the current time is obtained, and a braking distance is calculated based on the driving speed; The braking distance is multiplied by a predetermined distance weight to obtain the target distance threshold, wherein the distance weight is greater than or equal to 1; If the first target distance is less than the target distance threshold, the intelligent vehicle is directly stopped; If the first target distance is greater than or equal to the target distance threshold, the driving speed of the intelligent vehicle at the current time is reduced. 7.The method for detecting small road targets based on automatic driving time according to claim 6, wherein, The first detection frame coordinates of the first target detection frame and the laser detection frame coordinates of the laser target detection frame are respectively: wherein, represents first detection frame coordinates corresponding to the first target detection frame, represents laser detection frame coordinates corresponding to the laser target detection frame, and represents the upper left corner coordinates of the first target detection frame, and represents the right lower corner coordinates of the first target detection frame, and represents the upper left corner coordinates of the laser target detection frame, and represents the right lower corner coordinates of the laser target detection frame.

8. A device for detecting a small target on a road based on autonomous driving, characterized by, The device comprises: A module starting module is configured to start a camera module and a laser module of the intelligent vehicle when the intelligent vehicle starts automatic driving, wherein the vehicle head position of the intelligent vehicle comprises at least one set of camera module and one set of laser module, the original laser data of the intelligent vehicle during driving is obtained by using the laser module, the shooting focal length of the camera module is determined, and the original shooting picture of the intelligent vehicle during driving is obtained based on the shooting focal length, wherein the laser module and the original laser data belong to the laser coordinate system, the camera module belongs to the camera coordinate system, and the original shooting picture belongs to the focal length coordinate system; A target detection frame determination module is configured to perform first target detection on the original shooting picture to obtain a first target detection frame, if the frame area of the first target detection frame is greater than a preset frame threshold, it is determined that the target in the first target detection frame is a large-volume target object, if the frame area of the first target detection frame is less than or equal to the frame threshold, it is determined that the target in the first target detection frame is a small-volume target object, and each pixel point in the first target detection frame is sequentially obtained to obtain a target point set to be corrected; A coordinate system conversion module is configured to convert the original laser data belonging to the laser coordinate system to the focal length coordinate system by using the camera coordinate system to obtain focal length laser data. The target detection frame replacement module is configured to clean the target to-be-corrected point set by using the focal length laser data, to obtain a target cleaned point set, wherein the target cleaned point set corresponds to a corrected detection frame, and to replace the first target detection frame with the corrected detection frame, to obtain a target detection result during automatic driving of the intelligent vehicle. The target detection frame replacement module is configured to determine first detection frame coordinates of the first target detection frame from the target to-be-corrected point set, to determine a laser target detection frame of the target object by using the focal length laser data, wherein the laser target detection frame includes laser detection frame coordinates, to determine an overlap degree of the first target detection frame and the laser target detection frame according to the first detection frame coordinates and the laser detection frame coordinates, to reduce a driving speed of the intelligent vehicle at a current time if the overlap degree of the first target detection frame and the laser target detection frame is less than or equal to a preset overlap threshold, to continue collecting original laser data and original photographed pictures at a next time of driving of the intelligent vehicle, and to return to the first target detection step, and to modify the first target detection frame by using the laser target detection frame to obtain a corrected detection frame if the overlap degree of the first target detection frame and the laser target detection frame is greater than the overlap threshold, wherein the corrected detection frame includes the target cleaned point set. The target detection frame replacement module is configured to calculate an overlapping area of the first target detection frame and the laser target detection frame according to the first detection frame coordinates and the laser detection frame coordinates, to obtain a target overlapping detection frame, to obtain a non-overlapping area of the target overlapping detection frame and the first target detection frame, and to calculate an area ratio of the non-overlapping area to an area of the first target detection frame, to obtain a non-overlapping area ratio. The target detection frame replacement module is configured to determine that the target overlapping detection frame is the corrected detection frame if the non-overlapping area ratio is less than or equal to a preset non-overlapping threshold. The target detection frame replacement module is configured to construct a plurality of straight lines from the target overlapping detection frame to the first target detection frame in a direction, wherein each straight line includes a plurality of groups of coordinates, and each straight line is located in the non-overlapping area. The target detection frame replacement module is configured to calculate pixel gradient values of the plurality of groups of coordinates in each straight line, to determine a coordinate with a maximum pixel gradient value, and to sequentially connect the coordinates with the maximum pixel gradient values of each straight line, to obtain the corrected detection frame.

Images