Obstacle Detection and Interference Removal Method, Device and Computer Equipment in Intelligent Driving

Abstract

The present application relates to a method, device and computer equipment for obstacle detection and interference removal in intelligent driving. The method includes: obtaining lidar data, and obtaining a point cloud image to be processed according to the lidar data; performing line detection on the point cloud image to be processed to obtain a line corresponding to the side wall of the target object in a preset coordinate system, and obtaining the coordinates of stable corner points in the preset coordinate system; obtaining the coordinates of the corner points of the target object according to the line corresponding to the side wall of the target object, the preset length of the target object and the coordinates of the stable corner points; marking the target object area on a preset image according to the coordinates of the corner points of the target object, and marking the obstacle area on the preset image according to the point cloud image to be processed to obtain a marked obstacle image; traversing the obstacle areas in the marked obstacle image, and filtering out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object area. Using this method can achieve good intelligent driving control.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and particularly to a method, device, and computer device for obstacle detection and interference removal in intelligent driving. Background Art

[0002] With the development of intelligent driving technology, autonomous heavy trucks have emerged. An autonomous heavy truck consists of a complex system, mainly including many modules such as perception, planning, decision-making, control, and navigation.

[0003] In traditional technology, the perception module of an autonomous heavy truck is often used as the "eyes", and the lidar is an indispensable sensor in the perception module for accurate ranging. The lidar data is used for planning and decision-making to achieve autonomous driving. The lidar is usually installed on both sides of the front of the autonomous heavy truck. The rotating lidar has a 270° scanning range, and 90° is blocked by the vehicle body.

[0004] However, during the turning process of a vehicle (especially a heavy truck), the front of the vehicle will form a certain angle with the vehicle body or an object on the vehicle body (such as a hanging box installed on the vehicle), which may cause the vehicle body or an object on the vehicle body to be misdetected as an obstacle. It can be seen that without an obstacle detection and interference removal scheme, it will seriously affect the planning and decision-making of autonomous driving vehicles and cannot achieve intelligent driving control well. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a method, device, computer device, and storage medium for obstacle detection and interference removal in intelligent driving.

[0006] A method for obstacle detection and interference removal in intelligent driving, the method includes:

[0007] Obtain lidar data, and obtain a point cloud image to be processed according to the lidar data;

[0008] Perform line detection on the point cloud image to be processed, obtain a line corresponding to the side wall of the target object in a preset coordinate system, and obtain the coordinates of stable corner points in the preset coordinate system. The stable corner points are the fixed corner points of the target object during the vehicle driving process, and the target object is set on the vehicle;

[0009] According to the line corresponding to the side wall of the target object, the preset target object length, and the coordinates of the stable corner points, obtain the coordinates of the corner points of the target object;

[0010] Mark the target object area on a preset image according to the coordinates of the corner points of the target object, and mark the obstacle area on the preset image according to the point cloud image to be processed, to obtain a marked obstacle image;

[0011] Traverse the obstacle regions in the labeled obstacle images, and filter out the misdetected obstacle regions according to the positional relationship between the obstacle regions and the target object regions.

[0012] In one embodiment, obtaining the point cloud image to be processed from lidar data includes:

[0013] Perform data fusion according to the timestamps carried in the lidar data to obtain point cloud data in a preset coordinate system;

[0014] Perform ground segmentation processing on the point cloud data to obtain non-ground point cloud data;

[0015] Filter the non-ground point cloud data according to the preset target object existence region to obtain target point cloud data;

[0016] Project the target point cloud data to obtain the point cloud image to be processed.

[0017] In one embodiment, performing ground segmentation processing on the point cloud data to obtain non-ground point cloud data includes:

[0018] Perform projection on the point cloud data to obtain a point cloud projection map;

[0019] Perform grid division on the point cloud projection map to obtain a grid point cloud map corresponding to the point cloud projection map;

[0020] Calculate the height difference before projection of each grid in the grid point cloud map, and filter out non-ground grids from each grid according to the height difference before projection and a preset height difference threshold to obtain non-ground point cloud data.

[0021] In one embodiment, performing straight line detection on the point cloud image to be processed to obtain a straight line corresponding to the side wall of the target object in a preset coordinate system includes:

[0022] Perform straight line detection on the point cloud image to be processed through a feature detection algorithm to detect all straight lines in the point cloud image to be processed;

[0023] Sort all the straight lines in the point cloud image to be processed according to the straight line length according to a preset sorting range to obtain a straight line corresponding to the side wall of the target object in a preset coordinate system.

[0024] In one embodiment, obtaining the corner point coordinates of the target object according to the straight line corresponding to the side wall of the target object, a preset target object length, and stable corner point coordinates includes:

[0025] Obtain the straight line endpoint coordinates and the straight line length of the straight line corresponding to the side wall of the target object in a preset coordinate system;

[0026] Obtain the first corner point coordinates of the target object based on the stable corner point coordinates, the line endpoint coordinates, the line length, and the preset target object length;

[0027] Obtain the first slope of the target object based on the stable corner point coordinates and the first corner point coordinates of the target object;

[0028] Obtain the set of intercepts corresponding to the first slope of the target object based on the first slope of the target object;

[0029] Obtain the second slope of the target object and the set of intercepts corresponding to the second slope of the target object based on the first slope of the target object, the set of intercepts corresponding to the first slope of the target object, and the preset target object width;

[0030] Obtain the second corner point coordinates and the third corner point coordinates of the target object based on the first slope of the target object, the set of intercepts corresponding to the first slope of the target object, the second slope of the target object, and the set of intercepts corresponding to the second slope of the target object.

[0031] In one embodiment, mark the target object area on the preset image according to the target object corner point coordinates, and mark the obstacle area on the preset image according to the point cloud image to be processed, and the obtained marked obstacle image includes:

[0032] Mark the target object boundary line on the preset image according to the target object corner point coordinates;

[0033] Obtain the target object area according to the target object boundary line, and fill the pixels in the target object area with a preset pixel value to obtain a preset image of the marked target object area;

[0034] Mark the obstacle area on the preset image of the marked target object area according to the point cloud image to be processed to obtain the marked obstacle image.

[0035] In one embodiment, traverse the obstacle areas in the marked obstacle image, and filter out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object areas, including:

[0036] Mark the to-be-detected obstacle image in the marked obstacle image, and perform pixel value detection on the to-be-detected obstacle image;

[0037] When the pixel value of the to-be-detected obstacle image is detected to be the preset pixel value, regard the to-be-detected obstacle image as the misdetected target object obstacle image, and filter the misdetected target object obstacle image from the marked obstacle image.

[0038] In one embodiment, before marking the to-be-detected obstacle image in the marked obstacle image, it further includes:

[0039] Crop the labeled obstacle image according to the preset target object range parameter to obtain a preliminary cropped obstacle image;

[0040] Detect the preliminary cropped obstacle image according to the preset obstacle parameter to be filtered, and filter out the obstacle images that meet the preset obstacle parameter to be filtered from the preliminary cropped obstacle image to obtain the latest labeled obstacle image.

[0041] An obstacle detection and interference removal device in intelligent driving, the device includes:

[0042] An acquisition module, configured to acquire lidar data and obtain a point cloud image to be processed according to the lidar data;

[0043] A line detection module, configured to perform line detection on the point cloud image to be processed, obtain a line corresponding to the side wall of the target object in a preset coordinate system, and obtain the coordinates of stable corner points in the preset coordinate system, where the stable corner points are fixed corner points of the target object during the vehicle driving process, and the target object is set on the vehicle;

[0044] A processing module, configured to obtain the coordinates of the corner points of the target object according to the line corresponding to the side wall of the target object, the preset length of the target object, and the coordinates of the stable corner points;

[0045] A labeling module, configured to label the target object area on a preset image according to the coordinates of the corner points of the target object, and label the obstacle area on the preset image according to the point cloud image to be processed to obtain a labeled obstacle image;

[0046] A detection module, configured to traverse the obstacle areas in the labeled obstacle image and filter out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object areas.

[0047] A computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0048] Acquire lidar data and obtain a point cloud image to be processed according to the lidar data;

[0049] Perform line detection on the point cloud image to be processed, obtain a line corresponding to the side wall of the target object in a preset coordinate system, and obtain the coordinates of stable corner points in the preset coordinate system, where the stable corner points are fixed corner points of the target object during the vehicle driving process, and the target object is set on the vehicle;

[0050] Obtain the coordinates of the corner points of the target object according to the line corresponding to the side wall of the target object, the preset length of the target object, and the coordinates of the stable corner points;

[0051] Mark the target object area on the preset image according to the corner point coordinates of the target object, and mark the obstacle area on the preset image according to the point cloud image to be processed, so as to obtain the marked obstacle image;

[0052] Traverse the obstacle areas in the marked obstacle image, and filter out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object area.

[0053] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0054] Obtain lidar data, and obtain the point cloud image to be processed according to the lidar data;

[0055] Perform line detection on the point cloud image to be processed to obtain a line corresponding to the side wall of the target object in the preset coordinate system, and obtain the stable corner point coordinates in the preset coordinate system. The stable corner point is the fixed corner point of the target object during vehicle driving, and the target object is set on the vehicle;

[0056] According to the line corresponding to the side wall of the target object, the preset target object length, and the stable corner point coordinates, obtain the corner point coordinates of the target object;

[0057] Mark the target object area on the preset image according to the corner point coordinates of the target object, and mark the obstacle area on the preset image according to the point cloud image to be processed, so as to obtain the marked obstacle image;

[0058] Traverse the obstacle areas in the marked obstacle image, and filter out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object area.

[0059] The above-mentioned method, device, computer equipment and storage medium for removing interference in obstacle detection in intelligent driving analyze lidar data to obtain a point cloud image to be processed, perform line detection on the point cloud image to be processed to obtain a line corresponding to the side wall of the target object in a preset coordinate system, and obtain the coordinates of stable corner points in the preset coordinate system. Furthermore, the coordinates of the corner points of the target object can be obtained based on the line corresponding to the side wall of the target object, the preset length of the target object, and the coordinates of the stable corner points. The target object area is marked on the preset image according to the coordinates of the corner points of the target object, and the obstacle area is marked on the preset image according to the point cloud image to be processed to obtain a marked obstacle image. The obstacle areas in the marked obstacle image are traversed, and the misdetected obstacle areas are filtered out according to the positional relationship between the obstacle areas and the target object area. Throughout the process, the determination of the target object area on the preset image can be achieved through the analysis of lidar data. Furthermore, the marked obstacle image is traversed based on the target object area, and the misdetected obstacle areas are filtered out according to the positional relationship between the obstacle areas and the target object area, so that the target object will no longer be misdetected as an obstacle and will not interfere with the decision-making and planning of the autonomous driving heavy truck, thereby achieving good intelligent driving control. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 FIG. 6 is a schematic flowchart of a method for removing interference in obstacle detection in intelligent driving in one embodiment;

[0061] Figure 2 FIG. 7 is a schematic diagram of a method for removing interference in obstacle detection in intelligent driving in one embodiment;

[0062] Figure 3 FIG. 8 is a schematic diagram of a method for removing interference in obstacle detection in intelligent driving in another embodiment;

[0063] Figure 4 FIG. 9 is a schematic diagram of a method for removing interference in obstacle detection in intelligent driving in yet another embodiment;

[0064] Figure 5 FIG. 10 is a schematic diagram of a method for removing interference in obstacle detection in intelligent driving in still another embodiment;

[0065] Figure 6 FIG. 11 is a schematic diagram of a method for removing interference in obstacle detection in intelligent driving in another embodiment;

[0066] Figure 7 FIG. 12 is a schematic flowchart of a method for removing interference in obstacle detection in intelligent driving in another embodiment;

[0067] Figure 8 FIG. 13 is a structural block diagram of a device for removing interference in obstacle detection in intelligent driving in one embodiment;

[0068] Figure 9 It is the internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0069] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] In one embodiment, as Figure 1 shown, a method for removing interference in obstacle detection in intelligent driving is provided. In this embodiment, the example that this method is applied to a server is used for illustration. It can be understood that this method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0071] Step 102: Obtain lidar data, and obtain a point cloud image to be processed according to the lidar data.

[0072] Among them, the lidar data refers to the data sensed by the lidar. The lidar is an indispensable sensor in the perception module of the vehicle and is mainly used for accurate ranging. For example, the lidar can sense obstacles, give the distance between the obstacle and the vehicle body, as well as the specifications such as the length, width and height of the obstacle. The point cloud image to be processed refers to the point cloud image obtained after processing the lidar data and used for analyzing the obstacle situation. For example, the point cloud image to be processed can specifically be a projected point cloud binary image obtained after processing the lidar data through data fusion, segmentation, cropping, projection, etc.

[0073] Specifically, the lidar data is generated through the perception of the lidar, so that the server can obtain the lidar data, convert the lidar data into point cloud data, then segment the point cloud data, screen out the non-ground point cloud data from the point cloud data, and then screen the non-ground point cloud data according to the preset target object existence area, select the target point cloud data to be processed from it, and project the target point cloud data to obtain the point cloud image to be processed. For example, the server can specifically refer to an industrial control computer.

[0074] Step 104: Perform line detection on the point cloud image to be processed to obtain a line corresponding to the side wall of the target object in the preset coordinate system, and obtain the coordinates of the stable corner points in the preset coordinate system. The stable corner points are the fixed corner points of the target object during the driving of the vehicle, and the target object is set on the vehicle.

[0075] Among them, performing straight-line detection on the point cloud image to be processed means detecting all the straight lines in the point cloud image to be processed. The preset coordinate system refers to a pre-set coordinate system. For example, the preset coordinate system can specifically refer to the vehicle body coordinate system. The vehicle body coordinate system takes the midpoint of the front wheel axle of the tractor as the origin and satisfies the right-hand coordinate system. The X direction represents the orientation of the tractor head, the Y direction is perpendicular to the X axis and points to the left side of the vehicle head, and the Z direction is perpendicular to the plane where the XY direction is located and points vertically upward. The target object refers to an object that interferes with obstacle detection. For example, the target object can specifically refer to the hanging box installed on the intelligent driving heavy truck. The straight line corresponding to the side wall of the target object can specifically refer to the straight line representing the length of the side wall of the target object. The corner point refers to the intersection point of the target object in two directions. For example, the corner point can specifically refer to the intersection point of the target object in the vertical direction and the horizontal direction. The vertical direction can be determined by the width of the target object, and the horizontal direction can be determined by the length of the target object. The stable corner point refers to a point that is relatively fixed in position relative to the tractor head of the vehicle when the target object has the vehicle driving direction as the front during vehicle driving. For example, as Figure 2 shown, when the vehicle is driving to the right, the stable corner point of the target object is a point that is relatively fixed in position relative to the tractor head of the vehicle with the right side as the front, that is, point A.

[0076] Specifically, the server can use a feature detection algorithm (such as the Hough transform algorithm, etc.) to perform straight-line detection on the point cloud image to be processed, screen out the straight lines corresponding to the side walls of the target object in the preset coordinate system from the point cloud image to be processed, and obtain the coordinates of the stable corner points in the preset coordinate system. Among them, the method for obtaining the coordinates of the stable corner points in the preset coordinate system can be: pre-obtain at least two sets of vehicle turning scene images with the same turning direction (either the same left or the same right) as the vehicle driving direction, and by comparing the vehicle turning scene images, find the relatively fixed inner corner point coordinates of the target object as the stable corner point coordinates.

[0077] Step 106, obtain the corner point coordinates of the target object according to the straight line corresponding to the side wall of the target object, the preset target object length, and the stable corner point coordinates.

[0078] Among them, the corner point coordinates of the target object refer to the coordinates used to represent the corner points of the target object in the preset coordinate system. The corner point coordinates of the target object can be used to draw the area where the target object is located. Through the coordinates of the three corner points and the stable corner point coordinates, the boundary line of the target object can be depicted in the preset coordinate system. For example, as Figure 2As shown, point A is the coordinate of the stable corner point, and points B, C, and D are the coordinates of the three corner points of the target object. Through these four points, the boundary line of the target object can be depicted in the preset coordinate system. Here, the preset coordinate system has been two-dimensionally projected, that is, the Z coordinate is ignored. The preset target object length refers to the actual length of the target object, which can be obtained from the specification parameters of the target object.

[0079] Specifically, the server will first obtain two endpoints of the straight line corresponding to the side wall of the target object in the preset coordinate system according to the straight line corresponding to the side wall of the target object. These two endpoints can be used to represent the target object. Then, according to the two endpoints, the preset target object length, and the stable corner point coordinates, the slopes and intercepts of the four straight lines corresponding to the side walls of the target object are solved. According to the solved slopes and intercepts of the four straight lines corresponding to the side walls of the target object, the corner point coordinates of the target object are calculated.

[0080] Step 108: Mark the target object area on the preset image according to the corner point coordinates of the target object, and mark the obstacle area on the preset image according to the point cloud image to be processed, to obtain the marked obstacle image.

[0081] Among them, marking the target object area means drawing the target object area on the preset image and filling the pixels in the target object area with a preset pixel value, so that the target object area can be distinguished from other areas on the preset image. Marking the obstacle area on the preset image according to the point cloud image to be processed means performing Euclidean clustering on the point cloud image to be processed according to the preset clustering distance, marking the obstacle image in the point cloud image to be processed, and marking the area corresponding to the obstacle image on the preset image. Here, all obstacle areas need to be marked, including the obstacle areas corresponding to the target object that will interfere with obstacle detection.

[0082] Specifically, the server will mark the boundary line of the target object on the preset image according to the corner point coordinates of the target object, determine the target object area according to the marked boundary line of the target object, and fill the pixels in the target object area with a preset pixel value to obtain the preset image with the marked target object area. Then, mark the obstacle area on the preset image with the marked target object area according to the point cloud image to be processed to obtain the marked obstacle image.

[0083] Step 110: Traverse the obstacle areas in the marked obstacle image, and filter out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object area.

[0084] Specifically, the server first crops the labeled obstacle image according to the preset target object range parameters used to characterize the existence range of the target object to obtain a preliminary cropped obstacle image, and then detects the preliminary cropped obstacle image according to the preset obstacle parameters to be filtered used to characterize small obstacles and strip-shaped obstacles with too large aspect ratios that interfere with detection, filters out the obstacle images that meet the preset obstacle parameters to be filtered from the preliminary cropped obstacle image to obtain the latest labeled obstacle image, and finally labels the obstacle images to be detected in the latest labeled obstacle image, performs pixel value detection on the obstacle images to be detected, detects the misdetected target object obstacle images from the obstacle images to be detected according to the pixel values, and filters out the misdetected target object obstacle images from the latest labeled obstacle image.

[0085] The above method for removing interference in obstacle detection in intelligent driving analyzes the lidar data to obtain a point cloud image to be processed, performs line detection on the point cloud image to be processed to obtain a line corresponding to the side wall of the target object in a preset coordinate system, and obtains the coordinates of stable corner points in the preset coordinate system. Furthermore, according to the line corresponding to the side wall of the target object, the preset target object length, and the coordinates of the stable corner points, the coordinates of the corner points of the target object can be obtained. The target object area is marked on the preset image according to the coordinates of the corner points of the target object, and the obstacle area is marked on the preset image according to the point cloud image to be processed to obtain a labeled obstacle image. The obstacle areas in the labeled obstacle image are traversed, and the misdetected obstacle areas are filtered out according to the positional relationship between the obstacle areas and the target object area. In the whole process, the determination of the target object area on the preset image can be realized through the analysis of the lidar data, and then the labeled obstacle image is traversed according to the target object area, and the misdetected obstacle areas are filtered out according to the positional relationship between the obstacle areas and the target object area, so that the target object will no longer be misdetected as an obstacle and will not interfere with the decision-making and planning of the autonomous driving heavy truck, thus realizing good intelligent driving control.

[0086] In one embodiment, obtaining the point cloud image to be processed according to the lidar data includes:

[0087] Performing data fusion according to the timestamps carried by the lidar data to obtain point cloud data in a preset coordinate system;

[0088] Performing ground segmentation processing on the point cloud data to obtain non-ground point cloud data;

[0089] Filtering the non-ground point cloud data according to the preset target object existence area to obtain target point cloud data;

[0090] Projecting the target point cloud data to obtain the point cloud image to be processed.

[0091] Among them, the timestamp is an attribute in the lidar data header file, which means that each frame of lidar data corresponds to a unique time. Data fusion refers to screening the data of the left and right lidars with adjacent timestamps for the fusion of the left and right lidar data. Ground segmentation processing refers to segmenting the ground point cloud data and the non-ground point cloud data. For example, when performing ground segmentation processing, the grid height difference method can be used, that is, the grids with a height difference exceeding a certain threshold within the same grid are determined as non-ground point data grids, and the non-ground point data within these grids can be combined. The preset target object existence area refers to the area where the target object may exist preset under the preset coordinate system, which is determined according to the specifications of the target object and the vehicle. This preset target existence area includes the x direction and the y direction. For example, when the preset coordinate system is the vehicle body coordinate system and the target object is a hanging box, the preset target object existence area is delimited based on the vehicle body coordinate system. Even when the hanging box turns, it cannot exceed ±Nm in the y direction, and the longest in the x direction is the length of the hanging box + half the length of the vehicle head when driving straight. The value of N can be set according to needs.

[0092] Among them, the target point cloud data refers to the non-ground point cloud data within the preset target object existence area after screening the non-ground point cloud data. The screening method can be to crop the image of the non-ground point cloud data according to the preset target object existence area. For example, the cropping can specifically be ROI (region of interest) cropping. Projection refers to performing a two-dimensional projection, that is, ignoring the elevation z coordinate to form a binary projection point cloud image.

[0093] Specifically, the server will first perform data conversion on the lidar data, uniformly calibrate the lidar data to the preset coordinate system, and then use the timestamp carried by the lidar data for data fusion to form point cloud data based on the preset coordinate system. After obtaining the point cloud data, the server will perform ground segmentation processing on the point cloud data to obtain non-ground point cloud data from the point cloud data. The grid height difference method can be used when performing ground segmentation. After obtaining the non-ground point cloud data, the server will set the ROI area according to the preset target object existence area, perform ROI cropping on the non-ground point cloud data according to the ROI area to obtain the target point cloud data, and perform a two-dimensional projection on the target point cloud data, ignoring the elevation z coordinate, to obtain the binary projection point cloud image, that is, the point cloud image to be processed.

[0094] For example, as Figure 3 shown, it can be seen that in the point cloud image to be processed, the pixels at the positions where there are point clouds are 255 (white), and the pixels at the positions without point clouds are 0 (black). In this point cloud image to be processed, the ground point clouds have been filtered out during the ground segmentation processing to prevent obstacle noise caused by the ground point clouds.

[0095] In this embodiment, by performing data fusion based on the timestamps carried in the lidar data, point cloud data in a preset coordinate system is obtained. The point cloud data is subjected to ground segmentation processing to obtain non-ground point cloud data. The non-ground point cloud data is filtered according to the preset area where the target object exists to obtain target point cloud data. The target point cloud data is projected to obtain a point cloud image to be processed, enabling the acquisition of the point cloud image to be processed.

[0096] In one embodiment, performing ground segmentation processing on the point cloud data to obtain non-ground point cloud data includes:

[0097] Projecting the point cloud data to obtain a point cloud projection image;

[0098] Performing grid division on the point cloud projection image to obtain a grid point cloud image corresponding to the point cloud projection image;

[0099] Calculating the height difference before projection of each grid in the grid point cloud image, and filtering out non-ground grids from each grid according to the height difference before projection and a preset height difference threshold to obtain non-ground point cloud data.

[0100] Among them, the point cloud projection image refers to the image obtained after binary projection of the point cloud data. A grid refers to a segmentation grid preset in a preset coordinate system for dividing the point cloud projection image. The height difference before projection refers to the maximum height difference in the z direction of each point cloud data in the grid before projection, which can be obtained by comparing the coordinates of each point cloud data in the z direction before projection.

[0101] Specifically, the server performs binary projection on the point cloud data, ignoring the elevation z coordinate, to obtain a point cloud projection image. The point cloud projection image is subjected to grid division and cut according to a preset size to obtain a grid point cloud image corresponding to the point cloud projection image. By comparing the maximum height difference in the z direction of each point cloud data in each grid before projection, the height difference before projection of each grid in the grid point cloud image is calculated. According to the height difference before projection and the preset height difference threshold, non-ground grids are filtered out from each grid to obtain non-ground point cloud data. Among them, when the height difference before projection is greater than the preset height difference threshold, it indicates that the distance of the point cloud data in the grid in the z direction is far, and this grid is a non-ground grid. Clustering the point cloud data in this grid can obtain non-ground point cloud data. Among them, the preset size can be set by yourself according to needs.

[0102] In this embodiment, by projecting the point cloud data to obtain a point cloud projection image, performing grid division on the point cloud projection image to obtain a grid point cloud image corresponding to the point cloud projection image, calculating the height difference before projection of each grid in the grid point cloud image, and filtering out non-ground grids from each grid according to the height difference before projection and a preset height difference threshold to obtain non-ground point cloud data, the acquisition of non-ground point cloud data can be realized.

[0103] In one embodiment, performing line detection on the point cloud image to be processed to obtain the lines corresponding to the side wall of the target object in the preset coordinate system includes:

[0104] Performing line detection on the point cloud image to be processed through a feature detection algorithm to detect all the lines in the point cloud image to be processed;

[0105] Sorting all the lines in the point cloud image to be processed according to the line length according to the preset sorting range to obtain the lines corresponding to the side wall of the target object in the preset coordinate system.

[0106] Among them, the feature detection algorithm is used to perform feature detection on the image. In this embodiment, that is, performing line detection on the image to detect the lines in the point cloud image to be processed. For example, the feature detection algorithm can specifically refer to the Hough transform algorithm, which is widely used in image analysis, computer vision, and digital image processing. The Hough transform is used to identify and find the features in an object, such as lines. Line detection utilizes the duality between points and lines.

[0107] Specifically, the server will first perform line detection on the point cloud image to be processed through a feature detection algorithm to detect all the lines in the point cloud image to be processed, and then sort all the lines in the point cloud image to be processed according to the line length according to the preset sorting range, and screen out the line with the longest length as the line corresponding to the side wall of the target object in the preset coordinate system. Among them, taking the Hough detection algorithm to perform line detection on the point cloud image to be processed as an example, the method of performing line detection can be: projecting each point cloud data in the point cloud image to be processed into the Hough space to become corresponding curves, and then using the statistical information in the Hough space to find the slope and intercept of the line, so as to complete the line detection in the point cloud image to be processed. The preset sorting range refers to the line screening range preset in the preset coordinate system, which can be determined according to the specifications of the target object and the offset distance. The offset distance refers to the distance that the target object will deviate from the vehicle when the vehicle turns. For example, when the preset coordinate system is the vehicle body coordinate system and the target object is a hanging box, if the width of the hanging box is 2x and the offset distance is m, then the range in the Y direction is between (-x, x). When turning, the box will deviate outward, so the interval of y direction (- (x + m), (x + m)) is selected for line screening.

[0108] In this embodiment, performing line detection on the point cloud image to be processed through a feature detection algorithm to detect all the lines in the point cloud image to be processed, sorting all the lines in the point cloud image to be processed according to the line length, and obtaining the lines corresponding to the side wall of the target object in the preset coordinate system can realize the acquisition of the lines corresponding to the side wall of the target object.

[0109] In one embodiment, obtaining the corner point coordinates of the target object based on the straight line corresponding to the side wall of the target object, the preset target object length, and the stable corner point coordinates includes:

[0110] Obtaining the straight line endpoint coordinates and the straight line length of the straight line corresponding to the side wall of the target object in the preset coordinate system;

[0111] Obtaining the first corner point coordinates of the target object based on the stable corner point coordinates, the straight line endpoint coordinates, the straight line length, and the preset target object length;

[0112] Obtaining the first slope of the target object based on the stable corner point coordinates and the first corner point coordinates of the target object;

[0113] Obtaining the set of intercepts corresponding to the first slope of the target object according to the first slope of the target object;

[0114] Obtaining the second slope of the target object and the set of intercepts corresponding to the second slope of the target object based on the first slope of the target object, the set of intercepts corresponding to the first slope of the target object, and the preset target object width;

[0115] Obtaining the second corner point coordinates and the third corner point coordinates of the target object based on the first slope of the target object, the set of intercepts corresponding to the first slope of the target object, the second slope of the target object, and the set of intercepts corresponding to the second slope of the target object.

[0116] Wherein, the preset target object width refers to the actual width of the target object and can be obtained through the specification parameters of the target object. The stable corner point coordinates, the first corner point coordinates of the target object, the second corner point coordinates of the target object, and the third corner point coordinates of the target object are used to mark the target object on the preset coordinate system. The first slope of the target object refers to the slope of the first straight line formed by connecting the stable corner point coordinates and the first corner point coordinates of the target object, and the slope of the second straight line with the same slope as the first straight line in the preset coordinate system. The set of intercepts corresponding to the first slope of the target object refers to the intercept of the first straight line and the intercept of the second straight line. The second slope of the target object refers to the slope of the third straight line, and the slope of the fourth straight line with the same slope as the third straight line in the preset coordinate system. The third straight line refers to the straight line passing through the first corner point coordinates of the target object and intersecting with the first straight line, and the fourth straight line refers to the straight line passing through the stable corner point coordinates and intersecting with the first straight line. The set of intercepts corresponding to the second slope of the target object refers to the intercept of the third straight line and the intercept of the fourth straight line.

[0117] For example, as Figure 2 shown, in Figure 2Among them, A is the coordinate of the stable corner point, D is the coordinate of the first corner point of the target object, and C and B are the coordinates of the second corner point of the target object and the coordinates of the third corner point of the target object, respectively. k is the first slope of the target object, and b1 and b3 are the intercept sets corresponding to the first slope of the target object, where b1 is the intercept of the first straight line and b3 is the intercept of the second straight line. -1 / k is the second slope of the target object, and b2 and b4 are the intercept sets corresponding to the second slope of the target object, where b2 is the intercept of the third straight line and b4 is the intercept of the fourth straight line.

[0118] Specifically, the server will first obtain the straight-line endpoint coordinates and the straight-line length of the straight line corresponding to the side wall of the target object in the preset coordinate system, obtain the coordinate of the first corner point of the target object according to the stable corner point coordinate, the straight-line endpoint coordinate, the straight-line length, and the preset target object length, then obtain the first slope of the target object according to the stable corner point coordinate and the coordinate of the first corner point of the target object, obtain the intercept set corresponding to the first slope of the target object according to the first slope of the target object, and then obtain the second slope of the target object and the intercept set corresponding to the second slope of the target object according to the first slope of the target object, the intercept set corresponding to the first slope of the target object, and the preset target object width. Solve the intersection points between the straight lines according to the first slope of the target object, the intercept set corresponding to the first slope of the target object, the second slope of the target object, and the intercept set corresponding to the second slope of the target object to obtain the coordinate of the second corner point of the target object and the coordinate of the third corner point of the target object. Among them, the coordinate of the second corner point of the target object can be obtained by solving the intersection point between the second straight line and the third straight line, and the coordinate of the third corner point of the target object can be obtained by solving the intersection point between the second straight line and the fourth straight line.

[0119] Among them, the formula for solving the coordinate of the first corner point of the target object can be: Among them, H len represents the preset target object length, L represents the straight-line length, (x′ l , y′ l ) and (x l , y l ) are the straight-line endpoint coordinates, and (x b , y b ) is the coordinate of the first corner point of the target object. The formula for solving the first slope of the target object can be: The formula for solving the intercept b1 of the first straight line in the intercept set corresponding to the first slope of the target object can be: b1 = y b - k * x b , similarly, the formulas for solving the intercepts of the other straight lines can be deduced as follows: Intercept of the third straight line: Intercept of the second straight line: Intercept of the fourth straight line: Where Wid represents the preset width of the target object.

[0120] In this embodiment, through the straight line corresponding to the side wall of the target object, the preset length of the target object, and the coordinates of the stable corner points, the coordinates of three corner points of the target object can be obtained.

[0121] In one embodiment, the target object area is marked on the preset image according to the coordinates of the target object corner points, and the obstacle area is marked on the preset image according to the point cloud image to be processed, and the obtained marked obstacle image includes:

[0122] The boundary line of the target object is marked on the preset image according to the coordinates of the target object corner points;

[0123] The target object area is obtained according to the boundary line of the target object, and the pixels in the target object area are filled with a preset pixel value to obtain a preset image with the marked target object area;

[0124] The obstacle area is marked on the preset image with the marked target object area according to the point cloud image to be processed to obtain the marked obstacle image.

[0125] Among them, the boundary line of the target object can be obtained by connecting the coordinates of the target object corner points. The target object area is used to represent the position of the target object on the preset image. The preset pixel value can be set according to needs, as long as it is different from the pixel value of the preset image. For example, when the pixel value of the preset image is 255, the preset pixel value can be set to 0. The obstacle area is used to represent the position of the obstacle on the preset image. For example, as Figure 4 shown, the left figure is the target object area on the preset image when the vehicle is driving straight to the right, and the right figure is the target object area on the preset image when the vehicle is turning.

[0126] Specifically, the server will first mark the boundary line of the target object on the preset image according to the coordinates of the target object corner points, then obtain the target object area according to the boundary line of the target object, and fill the pixels in the target object area with a preset pixel value to obtain a preset image with the marked target object area, and finally mark the obstacle area on the preset image with the marked target object area according to the point cloud image to be processed to obtain the marked obstacle image.

[0127] In this embodiment, by marking the boundary line of the target object on the preset image according to the coordinates of the target object corner points, obtaining the target object area according to the boundary line of the target object, filling the pixels in the target object area with a preset pixel value to obtain a preset image with the marked target object area, and marking the obstacle area on the preset image with the marked target object area according to the point cloud image to be processed to obtain the marked obstacle image, the acquisition of the marked obstacle image can be realized.

[0128] In one embodiment, traversing the obstacle regions in the labeled obstacle image and filtering out the misdetected obstacle regions according to the positional relationship between the obstacle regions and the target object regions includes:

[0129] Labeling the obstacle images to be detected in the labeled obstacle image and performing pixel value detection on the obstacle images to be detected;

[0130] When the pixel value of the obstacle image to be detected is detected as the preset pixel value, regarding the obstacle image to be detected as the misdetected target object obstacle image and filtering out the misdetected target object obstacle image from the labeled obstacle image.

[0131] Among them, the obstacle image to be detected refers to an image of a target object obstacle image that is suspected of being misdetected in the labeled obstacle image and needs to be judged through further detection. Pixel value detection refers to traversing the pixel values of the obstacle image to be detected to detect whether the pixel value of the obstacle image to be detected is the preset pixel value. The preset pixel value refers to the pixel value filled for the pixels in the target object region when labeling the target object region in the preset image, which is different from the pixel values of the pixel points in the non-target object region.

[0132] Specifically, the server will label the obstacle images to be detected in the labeled obstacle image, perform pixel value detection on the obstacle images to be detected. When the pixel value of the obstacle image to be detected is detected as the preset pixel value, it is considered that the position of this obstacle intersects with the real position of the target object and it is a misdetected target obstacle, which needs to be filtered. Regarding the obstacle image to be detected as the misdetected target object obstacle image and filtering out the misdetected target object obstacle image from the labeled obstacle image. Among them, as Figure 6 shown, the obstacle images to be detected can be labeled in the labeled obstacle image in the form of drawing a rectangular border.

[0133] In this embodiment, by detecting the labeled obstacle image according to the target object region, finding out the misdetected target object obstacle image, and filtering out the misdetected target object obstacle image from the labeled obstacle image, it is possible to filter out the misdetected target obstacle image.

[0134] In one embodiment, before labeling the obstacle images to be detected in the labeled obstacle image, it further includes:

[0135] Cropping the labeled obstacle image according to the preset target object range parameter to obtain a preliminary cropped obstacle image;

[0136] Detect the preliminarily cropped obstacle image according to the preset obstacle parameters to be filtered, and filter out the obstacle images that meet the preset obstacle parameters to be filtered from the preliminarily cropped obstacle image to obtain the latest labeled obstacle image.

[0137] Among them, the preset target object range parameter refers to the range where the target object may exist in the preset image under the preset coordinate system, which can be delimited by the target object specifications. The preset obstacle parameters to be filtered refer to the parameters of obstacles with an area and width smaller than a certain threshold. For example, when a vehicle turns, due to the limitation of the fixed radius set by the clustering algorithm, the target object will be detected as more than one obstacle. As shown by the circle in Figure 5 a point cloud aggregation will occur at one end of the front of the target object, and the projected area of the point cloud here is very small and needs to be filtered in advance. At the same time, during slight obstacle avoidance, the point cloud on the side wall of the target object is scarce and no straight line is detected by the Hough transform algorithm. However, the scarce point cloud on the side wall of the target object is detected as a slender obstacle with a very small width, and this obstacle also needs to be filtered out first. Slight obstacle avoidance means that during the driving of the vehicle, small obstacles such as cones and water horses appear on the roadside ahead, and the heavy truck does not need to make a complete lane change, but only needs to occupy half of the adjacent road.

[0138] Specifically, the server will first crop the labeled obstacle image according to the preset target object range parameter to obtain the preliminarily cropped obstacle image, and then detect the preliminarily cropped obstacle image according to the preset obstacle parameters to be filtered, and filter out the obstacle images that meet the preset obstacle parameters to be filtered from the preliminarily cropped obstacle image to obtain the latest labeled obstacle image. In this embodiment, by cropping the labeled obstacle image according to the preset target object range parameter to obtain the cropped obstacle image, the detection range can be reduced, thereby achieving efficient and accurate detection. In one embodiment, as shown in Figure 7As shown, taking the hanging box as the target object, an example of the method for removing interference in obstacle detection in the intelligent driving of the present application is given. The method for removing interference in obstacle detection in the intelligent driving includes the following steps: 1) Perform data fusion according to the timestamps carried by the lidar data to obtain point cloud data in a preset coordinate system. Project the point cloud data to obtain a point cloud projection map. Divide the point cloud projection map into grids to obtain a grid point cloud map corresponding to the point cloud projection map. Calculate the height difference before projection of each grid in the grid point cloud map. According to the height difference before projection and a preset height difference threshold, filter out non-ground grids from each grid to obtain non-ground point cloud data. Filter the non-ground point cloud data according to the preset hanging box existence area to obtain target point cloud data (ROI cropping). Project the target point cloud data (two-dimensional projection) to obtain a point cloud image to be processed (binary image). Perform line detection on the point cloud image to be processed through the Hough transform algorithm to detect all the lines in the point cloud image to be processed. Sort all the lines in the point cloud image to be processed according to the line length to obtain the lines corresponding to the side walls of the hanging box in the preset coordinate system; 2) Obtain the coordinates of the stable corner points in the preset coordinate system (the initial values of the front corner coordinates of the hanging box). The hanging box is set on the vehicle. Obtain the line endpoint coordinates and the line length of the line corresponding to the side wall of the hanging box in the preset coordinate system. According to the coordinates of the stable corner points, the line endpoint coordinates, the line length, and the length of the hanging box, obtain the coordinates of the first corner point of the hanging box. According to the coordinates of the stable corner points and the coordinates of the first corner point of the hanging box, obtain the first slope of the hanging box. According to the first slope of the hanging box, obtain the set of intercepts corresponding to the first slope of the hanging box. According to the first slope of the hanging box, the set of intercepts corresponding to the first slope of the hanging box, and the width of the hanging box, obtain the second slope of the hanging box and the set of intercepts corresponding to the second slope of the hanging box. According to the first slope of the hanging box, the set of intercepts corresponding to the first slope of the hanging box, the second slope of the hanging box, and the set of intercepts corresponding to the second slope of the hanging box, obtain the coordinates of the second corner point and the coordinates of the third corner point of the hanging box. Mark the boundary line of the hanging box on the preset image according to the coordinates of the corner points of the hanging box. Obtain the area of the hanging box according to the boundary line of the hanging box and fill the pixels in the area of the hanging box with a preset pixel value to obtain a preset image with the marked area of the hanging box. Mark the obstacle area on the preset image with the marked area of the hanging box according to the point cloud image to be processed to obtain a marked obstacle image;3) Crop the labeled obstacle image according to the preset target object range parameter to obtain a preliminary cropped obstacle image. Detect the preliminary cropped obstacle image according to the preset obstacle to be filtered parameter, and filter out the obstacle images that meet the preset obstacle to be filtered parameter from the preliminary cropped obstacle image to obtain the latest labeled obstacle image. Label the obstacle image to be detected in the latest labeled obstacle image, and perform pixel value detection on the obstacle image to be detected. When the pixel value of the obstacle image to be detected is detected to be the preset pixel value (set to 0 here), regard the obstacle image to be detected as a misdetected hanging box obstacle image, and filter out the misdetected hanging box obstacle image from the latest labeled obstacle image.

[0139] It should be understood that although Figure 1 the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figure 1 at least a part of the steps in

[0140] In one embodiment, as Figure 8 shown, an obstacle detection interference removal device in intelligent driving is provided, including: an acquisition module 802, a line detection module 804, a processing module 806, a labeling module 808, and a detection module 810, where:

[0141] The acquisition module 802 is configured to acquire lidar data and obtain a point cloud image to be processed according to the lidar data;

[0142] The line detection module 804 is configured to perform line detection on the point cloud image to be processed, obtain a line corresponding to the side wall of the target object in the preset coordinate system, and obtain the stable corner point coordinates in the preset coordinate system. The stable corner point is a fixed corner point of the target object during the vehicle driving process, and the target object is arranged on the vehicle;

[0143] The processing module 806 is configured to obtain the target object corner point coordinates according to the line corresponding to the side wall of the target object, the preset target object length, and the stable corner point coordinates;

[0144] The annotation module 808 is used to annotate the target object area on the preset image according to the corner point coordinates of the target object, and annotate the obstacle area on the preset image according to the point cloud image to be processed, so as to obtain the annotated obstacle image;

[0145] The detection module 810 is used to traverse the obstacle areas in the annotated obstacle image and filter out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object area.

[0146] The above-mentioned obstacle detection and anti-interference device in intelligent driving analyzes the lidar data to obtain the point cloud image to be processed, performs line detection on the point cloud image to be processed to obtain the lines corresponding to the side walls of the target object in the preset coordinate system, and obtains the stable corner point coordinates in the preset coordinate system. Furthermore, according to the lines corresponding to the side walls of the target object, the preset target object length, and the stable corner point coordinates, the corner point coordinates of the target object can be obtained. The target object area is annotated on the preset image according to the corner point coordinates of the target object, and the obstacle area is annotated on the preset image according to the point cloud image to be processed to obtain the annotated obstacle image. The obstacle areas in the annotated obstacle image are traversed, and the misdetected obstacle areas are filtered out according to the positional relationship between the obstacle areas and the target object area. In the whole process, the determination of the target object area on the preset image can be realized through the analysis of the lidar data. Furthermore, the annotated obstacle image is traversed according to the target object area, and the misdetected obstacle areas are filtered out according to the positional relationship between the obstacle areas and the target object area, so that the target object will no longer be misdetected as an obstacle and will not interfere with the decision-making and planning of the autonomous driving heavy truck, thus realizing good intelligent driving control.

[0147] In one embodiment, the acquisition module is further configured to perform data fusion according to the timestamp carried by the lidar data to obtain the point cloud data in the preset coordinate system, perform ground segmentation processing on the point cloud data to obtain the non-ground point cloud data, screen the non-ground point cloud data according to the preset target object existence area to obtain the target point cloud data, and project the target point cloud data to obtain the point cloud image to be processed.

[0148] In one embodiment, the acquisition module is further configured to project the point cloud data to obtain a point cloud projection map, perform grid division on the point cloud projection map to obtain a grid point cloud map corresponding to the point cloud projection map, calculate the height difference before projection of each grid in the grid point cloud map, and screen out the non-ground grids from each grid according to the height difference before projection and the preset height difference threshold to obtain the non-ground point cloud data.

[0149] In one embodiment, the straight line detection module is further configured to perform straight line detection on the point cloud image to be processed through a feature detection algorithm, detect all the straight lines in the point cloud image to be processed, sort all the straight lines in the point cloud image to be processed according to the straight line length according to a preset sorting range, and obtain the straight lines corresponding to the side wall of the target object in the preset coordinate system.

[0150] In one embodiment, the processing module is further configured to obtain the straight line endpoint coordinates and the straight line length of the straight line corresponding to the side wall of the target object in the preset coordinate system, obtain the first corner point coordinates of the target object according to the stable corner point coordinates, the straight line endpoint coordinates, the straight line length, and the preset target object length, obtain the first slope of the target object according to the stable corner point coordinates and the first corner point coordinates of the target object, obtain the intercept set corresponding to the first slope of the target object according to the first slope of the target object, obtain the second slope of the target object and the intercept set corresponding to the second slope of the target object according to the first slope of the target object, the intercept set corresponding to the first slope of the target object, and the preset target object width, and obtain the second corner point coordinates and the third corner point coordinates of the target object according to the first slope of the target object, the intercept set corresponding to the first slope of the target object, the second slope of the target object, and the intercept set corresponding to the second slope of the target object.

[0151] In one embodiment, the annotation module is further configured to mark the boundary line of the target object on the preset image according to the corner point coordinates of the target object, obtain the target object area according to the target object boundary line, fill the pixels in the target object area with a preset pixel value to obtain a preset image with the target object area marked, and mark the obstacle area on the preset image with the target object area marked according to the point cloud image to be processed to obtain a marked obstacle image.

[0152] In one embodiment, the detection module is further configured to mark the obstacle image to be detected in the marked obstacle image, perform pixel value detection on the obstacle image to be detected, and when it is detected that the pixel value of the obstacle image to be detected is the preset pixel value, use the obstacle image to be detected as the misdetected target object obstacle image and filter the misdetected target object obstacle image from the marked obstacle image.

[0153] In one embodiment, the detection module is further configured to crop the marked obstacle image according to the preset target object range parameter to obtain a preliminary cropped obstacle image, detect the preliminary cropped obstacle image according to the preset obstacle parameter to be filtered, and filter out the obstacle images that meet the preset obstacle parameter to be filtered from the preliminary cropped obstacle image to obtain the latest marked obstacle image.

[0154] For the specific limitations of the obstacle detection and interference removal device in intelligent driving, reference can be made to the limitations of the obstacle detection and interference removal method in intelligent driving in the above text, which will not be elaborated here. Each module in the above obstacle detection and interference removal device in intelligent driving can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to the above respective modules.

[0155] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 9 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store lidar data and point cloud data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an obstacle detection and interference removal method in intelligent driving.

[0156] Those skilled in the art can understand that Figure 9 the structure shown in

[0157] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0158] Obtain lidar data, and obtain a point cloud image to be processed according to the lidar data;

[0159] Perform line detection on the point cloud image to be processed, obtain a line corresponding to the side wall of the target object in a preset coordinate system, and obtain the coordinates of stable corner points in the preset coordinate system. The stable corner points are the fixed corner points of the target object during the vehicle's driving, and the target object is set on the vehicle;

[0160] According to the line corresponding to the side wall of the target object, the preset target object length, and the coordinates of the stable corner points, obtain the coordinates of the corner points of the target object;

[0161] Label the target object area on the preset image according to the corner point coordinates of the target object, and label the obstacle area on the preset image according to the point cloud image to be processed, to obtain the labeled obstacle image;

[0162] Traverse the obstacle areas in the labeled obstacle image, and filter out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object area.

[0163] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0164] Perform data fusion according to the timestamps carried in the lidar data to obtain the point cloud data in the preset coordinate system;

[0165] Perform ground segmentation processing on the point cloud data to obtain non-ground point cloud data;

[0166] Filter the non-ground point cloud data according to the preset target object existence area to obtain the target point cloud data;

[0167] Project the target point cloud data to obtain the point cloud image to be processed.

[0168] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0169] Perform projection according to the point cloud data to obtain the point cloud projection map;

[0170] Perform grid division on the point cloud projection map to obtain the grid point cloud map corresponding to the point cloud projection map;

[0171] Calculate the height difference before projection of each grid in the grid point cloud map, and filter out the non-ground grids from each grid according to the height difference before projection and the preset height difference threshold to obtain the non-ground point cloud data.

[0172] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0173] Perform line detection on the point cloud image to be processed through a feature detection algorithm, and detect all the lines in the point cloud image to be processed;

[0174] Sort all the lines in the point cloud image to be processed according to the line length according to the preset sorting range, to obtain the lines corresponding to the side walls of the target object in the preset coordinate system.

[0175] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0176] Obtain the line endpoint coordinates and the line length of the line corresponding to the side wall of the target object in the preset coordinate system;

[0177] Based on the coordinates of the stable corner points, the coordinates of the endpoints of the straight line, the length of the straight line, and the preset length of the target object, obtain the coordinates of the first corner point of the target object;

[0178] Based on the coordinates of the stable corner points and the coordinates of the first corner point of the target object, obtain the first slope of the target object;

[0179] Based on the first slope of the target object, obtain the set of intercepts corresponding to the first slope of the target object;

[0180] Based on the first slope of the target object, the set of intercepts corresponding to the first slope of the target object, and the preset width of the target object, obtain the second slope of the target object and the set of intercepts corresponding to the second slope of the target object;

[0181] Based on the first slope of the target object, the set of intercepts corresponding to the first slope of the target object, the second slope of the target object, and the set of intercepts corresponding to the second slope of the target object, obtain the coordinates of the second corner point and the coordinates of the third corner point of the target object.

[0182] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0183] Mark the boundary line of the target object on the preset image according to the coordinates of the corner points of the target object;

[0184] Based on the boundary line of the target object, obtain the area of the target object, and fill the pixels in the area of the target object with the preset pixel value to obtain the preset image with the marked area of the target object;

[0185] Mark the obstacle area on the preset image with the marked area of the target object according to the point cloud image to be processed, and obtain the marked obstacle image.

[0186] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0187] Mark the image of the obstacle to be detected in the marked obstacle image, and perform pixel value detection on the image of the obstacle to be detected;

[0188] When it is detected that the pixel value of the image of the obstacle to be detected is the preset pixel value, regard the image of the obstacle to be detected as the misdetected target object obstacle image, and filter the misdetected target object obstacle image from the marked obstacle image.

[0189] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0190] Crop the marked obstacle image according to the preset target object range parameter to obtain the preliminary cropped obstacle image;

[0191] Detect the preliminarily cropped obstacle image according to the preset obstacle parameters to be filtered, and filter out the obstacle images that meet the preset obstacle parameters to be filtered from the preliminarily cropped obstacle image, so as to obtain the latest labeled obstacle image.

[0192] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0193] Obtain lidar data, and obtain the point cloud image to be processed according to the lidar data;

[0194] Perform line detection on the point cloud image to be processed to obtain a line corresponding to the side wall of the target object in the preset coordinate system, and obtain the coordinates of the stable corner points in the preset coordinate system. The stable corner points are the fixed corner points of the target object during the vehicle driving process, and the target object is arranged on the vehicle;

[0195] According to the line corresponding to the side wall of the target object, the preset target object length, and the coordinates of the stable corner points, obtain the coordinates of the corner points of the target object;

[0196] Mark the target object area on the preset image according to the coordinates of the corner points of the target object, and mark the obstacle area on the preset image according to the point cloud image to be processed, so as to obtain the labeled obstacle image;

[0197] Traverse the obstacle areas in the labeled obstacle image, and filter out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object areas.

[0198] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented:

[0199] Perform data fusion according to the timestamps carried by the lidar data to obtain the point cloud data in the preset coordinate system;

[0200] Perform ground segmentation processing on the point cloud data to obtain non-ground point cloud data;

[0201] Filter the non-ground point cloud data according to the preset target object existence area to obtain the target point cloud data;

[0202] Project the target point cloud data to obtain the point cloud image to be processed.

[0203] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented:

[0204] Perform projection according to the point cloud data to obtain a point cloud projection map;

[0205] Perform grid division on the point cloud projection map to obtain a grid point cloud map corresponding to the point cloud projection map;

[0206] Calculate the height difference before projection for each grid in the calculated grid point cloud map, and screen out non-ground grids from each grid according to the height difference before projection and a preset height difference threshold to obtain non-ground point cloud data.

[0207] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0208] Perform line detection on the point cloud image to be processed through a feature detection algorithm to detect all the lines in the point cloud image to be processed;

[0209] Sort all the lines in the point cloud image to be processed according to the line length according to a preset sorting range to obtain the lines corresponding to the side wall of the target object in a preset coordinate system.

[0210] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0211] Obtain the line endpoint coordinates and line length of the line corresponding to the side wall of the target object in a preset coordinate system;

[0212] According to the stable corner point coordinates, line endpoint coordinates, line length, and a preset target object length, obtain the first corner point coordinates of the target object;

[0213] According to the stable corner point coordinates and the first corner point coordinates of the target object, obtain the first slope of the target object;

[0214] According to the first slope of the target object, obtain the intercept set corresponding to the first slope of the target object;

[0215] According to the first slope of the target object, the intercept set corresponding to the first slope of the target object, and a preset target object width, obtain the second slope of the target object and the intercept set corresponding to the second slope of the target object;

[0216] According to the first slope of the target object, the intercept set corresponding to the first slope of the target object, the second slope of the target object, and the intercept set corresponding to the second slope of the target object, obtain the second corner point coordinates and the third corner point coordinates of the target object.

[0217] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0218] Mark the boundary line of the target object on a preset image according to the corner point coordinates of the target object;

[0219] Obtain the target object area according to the target object boundary line, and fill the pixels in the target object area with a preset pixel value to obtain a preset image with the target object area marked;

[0220] Mark the obstacle area on the preset image in the area of the target object marked in the point cloud image to be processed, and obtain the marked obstacle image.

[0221] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0222] Mark the obstacle image to be detected in the marked obstacle image, and perform pixel value detection on the obstacle image to be detected;

[0223] When it is detected that the pixel value of the obstacle image to be detected is the preset pixel value, regard the obstacle image to be detected as the misdetected target object obstacle image, and filter the misdetected target object obstacle image from the marked obstacle image.

[0224] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0225] Crop the marked obstacle image according to the preset target object range parameter to obtain a preliminary cropped obstacle image;

[0226] Detect the preliminary cropped obstacle image according to the preset obstacle parameter to be filtered, and filter out the obstacle image that meets the preset obstacle parameter to be filtered from the preliminary cropped obstacle image to obtain the latest marked obstacle image.

[0227] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to the memory, storage, database or other media used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. The volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0228] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0229] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An obstacle detection and interference removal method in intelligent driving, characterized in that, The method includes: Obtaining lidar data and obtaining a point cloud image to be processed based on the lidar data; Performing line detection on the point cloud image to be processed, obtaining a line corresponding to the side wall of the target object in a preset coordinate system, and obtaining the coordinates of stable corner points in the preset coordinate system, where the stable corner points are fixed corner points of the target object during the driving of the vehicle, and the target object is disposed on the vehicle; According to the line corresponding to the side wall of the target object, obtaining two line end points of the line corresponding to the side wall of the target object in the preset coordinate system, and solving for the slopes and intercepts of the four lines corresponding to the side wall of the target object according to the two line end points, the preset target object length, and the coordinates of the stable corner points, and obtaining the coordinates of the corner points of the target object according to the slopes and intercepts of the four lines corresponding to the side wall of the target object; Marking the target object area on a preset image according to the coordinates of the corner points of the target object, and marking the obstacle area on the preset image according to the point cloud image to be processed, obtaining a marked obstacle image; Traversing the obstacle areas in the marked obstacle image, and filtering out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object area.

2. The method according to claim 1, wherein The obtaining the point cloud image to be processed according to the lidar data includes: Performing data fusion according to the timestamp carried by the lidar data to obtain point cloud data in the preset coordinate system; Performing ground segmentation processing on the point cloud data to obtain non-ground point cloud data; Filtering the non-ground point cloud data according to a preset target object existence area to obtain target point cloud data; Performing projection on the target point cloud data to obtain a point cloud image to be processed.

3. The method according to claim 1, wherein The performing line detection on the point cloud image to be processed and obtaining a line corresponding to the side wall of the target object in a preset coordinate system includes: Performing line detection on the point cloud image to be processed through a feature detection algorithm, and detecting all lines in the point cloud image to be processed; Sorting all the lines in the point cloud image to be processed according to the line length according to a preset sorting range, and obtaining a line corresponding to the side wall of the target object in the preset coordinate system.

4. The method according to claim 1, wherein The obtaining two line end points of the line corresponding to the side wall of the target object in the preset coordinate system according to the line corresponding to the side wall of the target object, and solving for the slopes and intercepts of the four lines corresponding to the side wall of the target object according to the two line end points, the preset target object length, and the coordinates of the stable corner points, and obtaining the coordinates of the corner points of the target object according to the slopes and intercepts of the four lines corresponding to the side wall of the target object includes: Obtaining the line end point coordinates and the line length of the line corresponding to the side wall of the target object in the preset coordinate system; Obtaining the coordinates of the first corner point of the target object according to the coordinates of the stable corner points, the line end point coordinates, the line length, and the preset target object length; Obtaining the first slope of the target object according to the coordinates of the stable corner points and the coordinates of the first corner point of the target object; Obtain an intercept set corresponding to the first slope of the target object according to the first slope of the target object; Obtain a second slope of the target object and an intercept set corresponding to the second slope of the target object according to the first slope of the target object, the intercept set corresponding to the first slope of the target object, and a preset target object width; Obtain the second corner point coordinates and the third corner point coordinates of the target object according to the first slope of the target object, the intercept set corresponding to the first slope of the target object, the second slope of the target object, and the intercept set corresponding to the second slope of the target object.

5. The method according to claim 1, wherein The step of marking the target object area on the preset image according to the corner point coordinates of the target object and marking the obstacle area on the preset image according to the to-be-processed point cloud image to obtain the marked obstacle image includes: Mark the boundary line of the target object on the preset image according to the corner point coordinates of the target object; Obtain the target object area according to the boundary line of the target object, and fill the pixels in the target object area with a preset pixel value to obtain a preset image with the marked target object area; Mark the obstacle area on the preset image with the marked target object area according to the to-be-processed point cloud image to obtain the marked obstacle image.

6. The method according to claim 1, characterized in that The step of traversing the obstacle areas in the marked obstacle image and filtering out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object area includes: Mark the to-be-detected obstacle image in the marked obstacle image, and perform pixel value detection on the to-be-detected obstacle image; When it is detected that the pixel value of the to-be-detected obstacle image is the preset pixel value, use the to-be-detected obstacle image as the misdetected target object obstacle image, and filter the misdetected target object obstacle image from the marked obstacle image.

7. The method according to claim 6, characterized in that Before marking the to-be-detected obstacle image in the marked obstacle image, it further includes: Crop the marked obstacle image according to the preset target object range parameter to obtain a preliminary cropped obstacle image; Detect the preliminary cropped obstacle image according to the preset to-be-filtered obstacle parameter, and filter out the obstacle image that meets the preset to-be-filtered obstacle parameter from the preliminary cropped obstacle image to obtain the latest marked obstacle image.

8. An obstacle detection and interference removal device in intelligent driving, characterized in that, The device includes: An acquisition module, configured to acquire lidar data and obtain a to-be-processed point cloud image according to the lidar data; A line detection module, configured to perform line detection on the to-be-processed point cloud image, obtain a line corresponding to the side wall of the target object in a preset coordinate system, and acquire stable corner point coordinates in the preset coordinate system, where the stable corner point is a fixed corner point of the target object during the driving process of the vehicle, and the target object is disposed on the vehicle; A processing module, configured to obtain two endpoints of the straight line corresponding to the side wall of the target object in the preset coordinate system according to the straight line corresponding to the side wall of the target object, and solve for the slopes and intercepts of the four straight lines corresponding to the side wall of the target object according to the two endpoints, the preset length of the target object, and the coordinates of the stable corner points, and obtain the coordinates of the corner points of the target object according to the slopes and intercepts of the four straight lines corresponding to the side wall of the target object; A labeling module, configured to label the target object area on the preset image according to the coordinates of the corner points of the target object, and label the obstacle area on the preset image according to the point cloud image to be processed, so as to obtain a labeled obstacle image; A detection module, configured to traverse the obstacle areas in the labeled obstacle image, and filter out the misdetected obstacle areas according to the positional relationship between the obstacle areas and the target object area.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.

Images