Spilled object detection method and system based on traditional algorithm and deep learning

Abstract

The present invention discloses a method and system for detecting spilled objects based on traditional algorithms and deep learning. The method comprises: performing background filtering, category labeling, data enhancement, rasterization operation and rasterization partitioning, ground filtering, and point cloud compensation on the original point cloud set in sequence to obtain a post-processed point cloud set; performing model training on the post-processed point cloud set as a training sample, and introducing a height-from-ground factor to obtain a target point cloud segmentation network model; performing background filtering, ground filtering, and point cloud compensation on the point cloud to be detected in sequence to obtain a post-processed point cloud to be detected; inputting the post-processed point cloud to be detected into the target point cloud segmentation network model for prediction to obtain the category of each point; extracting points classified as spilled objects and calculating their external bounding boxes to obtain the center position and length, width, and height size information of the spilled objects. The method of the present invention can accurately detect small target spilled objects, and improves the detection accuracy while ensuring the efficiency of spilled object detection.

Description

Technical Field

[0001] The present invention relates to the technical field of spilled object detection, and in particular to a spilled object detection method and system based on traditional algorithms and deep learning. Background Art

[0002] Due to natural disasters such as continuous rainfall or sudden rainstorms, there are foreign objects such as fallen rocks, trees, and soil at the entrance of highway tunnels that can break through multiple protective measures. At the same time, there is the possibility of common road spills such as dropped tires and boxes. Both of these spills will have a serious safety impact on high-speed vehicles. Therefore, an efficient and accurate detection method is needed to identify the spilled objects at the entrance of highway tunnels to ensure traffic safety.

[0003] The detection methods in the prior art generally have the following problems:

[0004] 1. Only deep learning methods are used to detect spilled objects:

[0005] In the high-speed and complex scene of highway tunnel entrances, the amount of point cloud data is huge, and it is usually necessary to scan hundreds of thousands or even more ground point cloud data. If a segmentation algorithm based on deep learning is used, the video memory may not be large enough and the inference speed will be very slow. This is very fatal for such application scenarios with high real-time requirements;

[0006] 2. Only using traditional point cloud algorithms to detect spilled objects:

[0007] Its advantage is fast operation speed, but low precision. Algorithms such as ground extraction cannot completely filter out ground points. Moreover, if the size of the scattered objects is too small, there is a possibility that they cannot be clustered into one target.

[0008] 3. Combining traditional point cloud algorithms with deep learning to detect spilled objects:

[0009] Typically, the existing method is to first filter the ground point cloud through a traditional point cloud algorithm (such as the RANSAC algorithm), and then use the filtered point cloud as training data for model training based on a deep learning network; the same is true for deployment, first use the RANSAC algorithm to filter out the ground point cloud, and then use the deep learning-based network for prediction; however, the ground segmentation by RANSAC is not good, and there may be a small area in some areas where the ground points are not segmented cleanly. In this case, directly inputting the network training model cannot effectively learn the surrounding local features, and the accuracy of the trained model will also decrease; in addition, since most of the scattered objects are small-sized targets and close to the ground, it is difficult to train the segmentation network, and the accuracy of the trained network model cannot be guaranteed. Summary of the invention

[0010] The embodiments of the present invention provide a method and system for detecting spilled objects based on traditional algorithms and deep learning to solve the problem in the prior art of how to improve detection accuracy while ensuring spilled object detection efficiency.

[0011] To achieve the above-mentioned purpose, on the one hand, the present invention provides a method for detecting spilled objects based on traditional algorithms and deep learning, the method comprising: S1, obtaining an original point cloud set, performing background filtering on the original point cloud set to obtain a filtered background point cloud set; labeling each point cloud in the filtered background point cloud set with a category, and performing data enhancement to obtain a data enhanced point cloud set; S2, rasterizing each point cloud in the data enhanced point cloud set, and rasterizing and partitioning all points in each point cloud to obtain a rasterized point set container corresponding to each point cloud; adding a ground height label value of the grid to each point in each point cloud in the data enhanced point cloud set; S3, performing ground filtering on each point cloud in the data enhanced point cloud set through a traditional point cloud algorithm to obtain a filtered ground point cloud set; S4, performing point cloud compensation on each point cloud in the filtered ground point cloud set to obtain a post-processed point cloud set; performing point cloud compensation on the current point cloud in the filtered ground point cloud set includes: performing a neighborhood operation on each point in the current point cloud Search to obtain all the neighborhood points corresponding to each point; perform point cloud compensation on the current point cloud according to all the neighborhood points corresponding to each point and its corresponding rasterized point set container; S5, increase the ground height factor to the cross entropy loss function in the original point cloud segmentation network model to obtain a modified point cloud segmentation network model; perform multiple rounds of training on the post-processed point cloud set by changing the point cloud segmentation network model to obtain the cross entropy loss value of each round; reversely update the modified point cloud segmentation network model according to the cross entropy loss value of the current round; take the modified point cloud segmentation network model corresponding to the training round with the smallest cross entropy loss value as the target point cloud segmentation network model; S6, obtain the point cloud to be detected, perform background filtering, ground filtering, and point cloud compensation on the point cloud to be detected in turn to obtain the post-processed point cloud to be detected; input the post-processed point cloud to be detected into the target point cloud segmentation network model for prediction to obtain the category of each point in the post-processed point cloud to be detected; extract the points classified as scattered objects and calculate their bounding boxes to obtain the center position and length, width and height size information of the scattered objects.

[0012] Optionally, each point cloud in the filtered background point cloud set is labeled with a category, and data enhancement is performed to obtain a data enhanced point cloud set, including: labeling each point in each point cloud in the filtered background point cloud set with a category to obtain a category label for each point; saving each spilled object target in each point cloud separately to obtain a spilled object point cloud set; taking each point cloud in the filtered background point cloud set as the current filtered background point cloud in turn, randomly selecting a preset number of spilled object point clouds from the spilled object point cloud set, and adding them to the current filtered background point cloud to obtain the current data enhanced point cloud.

[0013] Optionally, S2 includes: taking each point cloud in the data enhanced point cloud set as the current data enhanced point cloud in turn, and performing a rasterization operation on the current data enhanced point cloud to obtain multiple grids; rasterizing and partitioning all points in the current data enhanced point cloud to divide each point into a corresponding grid to obtain a current rasterized point set container; averaging the Z values ​​of all points classified as ground in each grid in the current data enhanced point cloud to obtain a ground height value for each grid; adding a label value to each point in the current data enhanced point cloud according to the ground height value of the grid in which it is located, to obtain a ground height label value for each point.

[0014] Optionally, the point cloud compensation for the current point cloud according to all the neighborhood points corresponding to each point and its corresponding rasterized point set container includes: taking each point as the current point in turn, if the number of neighborhood points of the current point is less than a first preset neighborhood point threshold, discarding the current point; if the number of neighborhood points of the current point is greater than a second preset neighborhood point threshold, retaining the current point; if the number of neighborhood points of the current point is greater than or equal to the first preset neighborhood point threshold and less than or equal to the second preset neighborhood point threshold, then according to the rasterized point set container corresponding to the current point, all other points in the grid where the current point is located are added to the currently filtered ground point cloud.

[0015] Optionally, the post-processing point cloud set is subjected to multiple rounds of training by changing the point cloud segmentation network model to obtain the cross entropy loss value of each round, including: performing the current round of training on each post-processing point cloud in the post-processing point cloud set by changing the point cloud segmentation network model to obtain the cross entropy loss value of each post-processing point cloud; and summing and averaging the cross entropy loss values ​​of all post-processing point clouds to obtain the cross entropy loss value of the current round.

[0016] Optionally, the cross entropy loss value of each post-processed point cloud is calculated according to the following formula:

[0017] ;

[0018] Among them, L is the cross entropy loss value of the current post-processing point cloud, N is the total number of points in the current post-processing point cloud, The first points, M is the number of categories in the current post-processing point cloud, and c is the number of points in the current post-processing point cloud. The point is predicted as the cth category by changing the point cloud segmentation network model. The category label of the point is equal to c, then Take 1, otherwise Take 0, The first The predicted probability of the point cloud segmentation network model is changed to the c-th category. The first The height factor of the point above the ground, The first The z coordinate value of the point, The first The ground height label value of each point.

[0019] On the other hand, the present invention provides a spilled object detection system based on traditional algorithms and deep learning, the system comprising: a data processing unit, used to obtain an original point cloud set, perform background filtering on the original point cloud set, and obtain a filtered background point cloud set; perform category labeling on each point cloud in the filtered background point cloud set, and perform data enhancement to obtain a data enhanced point cloud set; a point cloud rasterization unit, used to perform rasterization operations on each point cloud in the data enhanced point cloud set, and perform rasterization partitioning on all points in each point cloud to obtain a rasterized point set container corresponding to each point cloud; add a ground height label value of the grid to each point in each point cloud in the data enhanced point cloud set; a ground filtering unit, used to perform ground filtering on each point cloud in the data enhanced point cloud set through a traditional point cloud algorithm to obtain a filtered ground point cloud set; a point cloud compensation unit, used to perform point cloud compensation on each point cloud in the filtered ground point cloud set to obtain a post-processed point cloud set; performing point cloud compensation on the current point cloud in the filtered ground point cloud set includes: performing a point cloud filter on each point in the current point cloud A neighborhood search is performed to obtain all neighborhood points corresponding to each point; point cloud compensation is performed on the current point cloud according to all neighborhood points corresponding to each point and its corresponding rasterized point set container; a model training unit is used to increase the ground height factor to the cross entropy loss function in the original point cloud segmentation network model to obtain a modified point cloud segmentation network model; the post-processed point cloud set is trained for multiple rounds by the modified point cloud segmentation network model to obtain the cross entropy loss value of each round; the modified point cloud segmentation network model is reversely updated according to the cross entropy loss value of the current round; the modified point cloud segmentation network model corresponding to the training round with the smallest cross entropy loss value is taken as the target point cloud segmentation network model; a detection unit is used to obtain a point cloud to be detected, and the point cloud to be detected is subjected to background filtering, ground filtering, and point cloud compensation in turn to obtain a post-processed point cloud to be detected; the post-processed point cloud to be detected is input into the target point cloud segmentation network model for prediction to obtain the category of each point in the post-processed point cloud to be detected; points classified as scattered objects are extracted and their bounding boxes are calculated to obtain the center position and length, width, and height size information of the scattered objects.

[0020] Optionally, the data processing unit includes: a category labeling subunit, used to label each point in each point cloud in the filtered background point cloud set to obtain a category label for each point; a separation unit, used to separately save each spilled object target in each point cloud to obtain a spilled object point cloud set; a data enhancement subunit, used to take each point cloud in the filtered background point cloud set as the current filtered background point cloud, randomly select a preset number of spilled object point clouds from the spilled object point cloud set, and add them to the current filtered background point cloud to obtain the current data enhanced point cloud.

[0021] Optionally, the point cloud rasterization unit includes: a rasterization subunit, which is used to take each point cloud in the data enhancement point cloud set as the current data enhancement point cloud in turn, and perform rasterization operations on the current data enhancement point cloud to obtain multiple grids; a partitioning subunit, which is used to rasterize and partition all points in the current data enhancement point cloud to divide each point into a corresponding grid to obtain a current rasterized point set container; a ground height labeling subunit, which is used to average the Z values ​​of all points classified as ground in each grid in the current data enhancement point cloud to obtain a ground height value for each grid; and add a label value to each point in the current data enhancement point cloud according to the ground height value of the grid in which it is located to obtain a ground height label value for each point.

[0022] Optionally, the point cloud compensation for the current point cloud according to all the neighborhood points corresponding to each point and its corresponding rasterized point set container includes: taking each point as the current point in turn, if the number of neighborhood points of the current point is less than a first preset neighborhood point threshold, discarding the current point; if the number of neighborhood points of the current point is greater than a second preset neighborhood point threshold, retaining the current point; if the number of neighborhood points of the current point is greater than or equal to the first preset neighborhood point threshold and less than or equal to the second preset neighborhood point threshold, then according to the rasterized point set container corresponding to the current point, all other points in the grid where the current point is located are added to the currently filtered ground point cloud.

[0023] Beneficial effects of the present invention:

[0024] The present invention provides a method and system for detecting spilled objects based on traditional algorithms and deep learning, wherein the method rasterizes the point cloud before the ground is filtered out to obtain a rasterized point set container of the corresponding point cloud, and then performs post-processing such as point cloud compensation on the point cloud after the ground is filtered out, thereby improving the detection accuracy of small-sized spilled objects close to the ground; the method introduces a height factor from the ground and optimizes the cross entropy loss function to a certain extent, so that the entire network training is more inclined to improve the segmentation effect of those spilled object samples (especially small-sized spilled objects that are very close to the ground) and the segmentation effect of the points at the junction of the object and the ground; the method of the present invention improves the detection accuracy while ensuring the efficiency of spilled object detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of a method for detecting spilled objects based on traditional algorithms and deep learning provided by an embodiment of the present invention;

[0026] Figure 2 It is a structural schematic diagram of a spilled object detection system based on traditional algorithms and deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Figure 1 is a flow chart of a method for detecting spilled objects based on traditional algorithms and deep learning provided by an embodiment of the present invention; Figure 1 As shown, the method includes:

[0029] S1. Obtain an original point cloud set, perform background filtering on the original point cloud set to obtain a filtered background point cloud set; perform category labeling on each point cloud in the filtered background point cloud set, and perform data enhancement to obtain a data enhanced point cloud set;

[0030] In an optional implementation, the original point cloud set is obtained by the following process:

[0031] The laser radar was installed on both sides of the tunnel entrance, and personnel were arranged to randomly place the scattered objects in the detection area. Common scattered objects at the entrance of high-speed tunnels, including large, medium and small cartons, stones, tires, etc., were collected, and the point cloud sequence data of continuous t frames was collected and recorded as the original point cloud set { }.

[0032] Performing background filtering on the original point cloud set to obtain a filtered background point cloud set;

[0033] Since there may be background influences such as mountains or green belts on both sides of the tunnel entrance, these background areas are first filtered out of the original point cloud through straight-through filtering. Specifically, each original point cloud in the original point cloud set is subjected to straight-through filtering, and the final processed point cloud set is recorded as the filtered background point cloud set { }.

[0034] The step of labeling each point cloud in the filtered background point cloud set and performing data enhancement to obtain a data enhanced point cloud set includes:

[0035] Label each point in each point cloud in the filtered background point cloud set to obtain a category label for each point;

[0036] The filtered background point cloud set { }, each point in each filtered background point cloud is labeled with a category; in an optional embodiment, we divide the labeled categories into three categories, namely ground, spilled objects, and cars. After the labeling is completed, the category of each point in each filtered background point cloud can be obtained. It should be noted that the present invention does not limit the specific labeling category.

[0037] Since there are relatively few targets for the scattered objects, it is very necessary to use data enhancement. The specific data enhancement processing method is as follows:

[0038] Each spilled object target in each point cloud is saved separately to obtain a spilled object point cloud set;

[0039] The following is an illustration using a filtered background point cloud (current filtered background point cloud):

[0040] Through the above-mentioned category labeling information, all the scattered object targets in the current filtered background point cloud are saved separately to obtain the scattered object point cloud corresponding to each scattered object target;

[0041] All filtered background point clouds are operated according to the above method to obtain the scattered object point cloud set.

[0042] Each point cloud in the filtered background point cloud set is taken as the current filtered background point cloud in turn, and a preset number of scattered object point clouds are randomly selected from the scattered object point cloud set and added to the current filtered background point cloud to obtain the current data enhanced point cloud.

[0043] For the current filtered background point cloud, a preset number of spill point clouds (e.g., S spill point clouds) are randomly selected from the spill point cloud set; each spill point cloud (S spill point clouds) is randomly translated and rotated, and all the spill point clouds (S spill point clouds) after random translation and rotation are added to the current filtered background point cloud to obtain the current data enhanced point cloud;

[0044] All filtered background point clouds are operated according to the above method to obtain the data enhanced point cloud set { }.

[0045] S2, performing a rasterization operation on each point cloud in the data enhanced point cloud set, and performing rasterization partitioning on all points in each point cloud to obtain a rasterized point set container corresponding to each point cloud; adding a ground height label value of the grid where each point in each point cloud in the data enhanced point cloud set is located;

[0046] In an optional embodiment, the S2 includes:

[0047] S21, taking each point cloud in the data enhancement point cloud set as the current data enhancement point cloud in turn, and performing a rasterization operation on the current data enhancement point cloud to obtain a plurality of raster grids;

[0048] Data augmentation point cloud set { Each data augmented point cloud in} is rasterized as follows.

[0049] Assuming that the maximum value of the current data augmented point cloud in the y direction is maxY, the minimum value is minY, the maximum value in the x direction is maxX, the minimum value is minX, the x direction is divided into dx equal intervals, and the y direction is divided into dy equal intervals, then a total of K grids are obtained.

[0050] ;

[0051] S22, rasterizing and partitioning all points in the current data enhanced point cloud to divide each point into a corresponding grid, thereby obtaining a current rasterized point set container;

[0052] Take a point (current point) in the current data augmented point cloud as an example:

[0053] Assume that the coordinates of the current point in the current data augmentation point cloud are q( ), then divide the current point into x-direction indexes , the y-direction index is in the grid.

[0054] The same method is used to divide all points in the current data enhancement point cloud into the corresponding grids, so as to obtain a container, namely the current gridded point set container; the current gridded point set container stores the current data enhancement point cloud. The point set divided into each grid is recorded as .

[0055] S23. Average the Z values ​​of all points classified as ground in each grid in the current data enhanced point cloud to obtain the ground height value of each grid; add a label value to each point in the current data enhanced point cloud according to the ground height value of the grid in which it is located to obtain the ground height label value of each point.

[0056] Take a grid (current grid) in the current data augmentation point cloud as an example:

[0057] Select all points classified as ground from the points divided into the current grid, and average the z values ​​of these points to get , This is the ground height value of the current grid.

[0058] The same method is used to count the ground height values ​​of all grids in the current data augmented point cloud. Then we add a label value to each point in the current data augmented point cloud, and the label value is the ground height value of the grid where the point is located. , recorded as the ground height label value. Finally, the data enhancement point cloud set { The above operation is performed on each data augmented point cloud of}.

[0059] In this way, each data-enhanced point cloud in the data-enhanced point cloud set contains two labels: 1. The category label of each point; 2. The ground height label value of the grid where each point is located.

[0060] At the same time, the rasterized point set container corresponding to each data enhanced point cloud in the data enhanced point cloud set can be obtained, which is recorded as the rasterized point set container set { }.

[0061] S3, performing ground filtering on each point cloud in the data enhanced point cloud set by using a traditional point cloud algorithm to obtain a filtered ground point cloud set;

[0062] The traditional point cloud algorithm is used to perform ground segmentation on each data enhanced point cloud in the data enhanced point cloud set to obtain a non-ground target point cloud after removing the ground points. The traditional point cloud algorithm here includes: ransac (random sampling consistency) algorithm, and the data enhanced point cloud set is ground filtered by a traditional point cloud algorithm (such as ransac algorithm) to obtain a filtered ground point cloud set { }. The ground segmentation using traditional point cloud algorithms (such as the RANSAC algorithm) will not completely segment the ground. Usually, objects such as scattered objects will not be regarded as ground points, that is, it will not fall into the situation of over-segmentation, but will fall into the situation of under-segmentation, that is, after ground segmentation, the ground points are not segmented, but the ground points are identified as non-ground points and retained. Therefore, the following three situations will appear in the filtered ground point cloud: 1. A few scattered ground points are retained in some areas. 2. A large area of ​​ground points is retained in some areas. 3. A small area of ​​ground points is retained in some areas, and the number of retained points is greater than that in case 1 and less than that in case 2.

[0063] S4, performing point cloud compensation on each point cloud in the filtered ground point cloud set to obtain a post-processed point cloud set; performing point cloud compensation on the current point cloud in the filtered ground point cloud set includes: performing a neighborhood search on each point in the current point cloud to obtain all neighboring points corresponding to each point; performing point cloud compensation on the current point cloud according to all neighboring points corresponding to each point and its corresponding gridded point set container;

[0064] S3 mentions the shortcomings of ground extraction by traditional methods, as well as three situations that may occur after ground filtering by traditional methods. In the first situation, if the number of points is too small, it is impossible to form a valid target, so these points can be deleted. For the second situation, if a large area of ​​ground points are retained, since the number of points is large enough, after being input into the neural network, the features can be effectively learned. The most difficult is the third situation, which does not meet the number of points that can be deleted directly, but if these points are directly sent to the neural network to learn features, it is difficult to learn the surrounding local features, and the segmentation effect of the trained model is bound to be poor. The present invention solves this problem by adopting a point cloud compensation method.

[0065] Perform point cloud compensation on each point cloud in the filtered ground point cloud set to obtain a post-processed point cloud set; Take the point cloud compensation process as an example: the current ground point cloud is filtered out. For each point in the , each point is taken as the current point in turn, and a neighborhood search is performed. The present invention adopts a fixed distance search, and the search method adopts a spherical neighborhood, and the distance threshold is set to δd; all points with a distance less than δd are regarded as spherical neighborhood points of the current point, and the number of neighborhood points of the current point is counted. If the number of neighboring points of the current point Less than the first preset neighborhood point count threshold (i.e., the set minimum neighborhood point count threshold) ), then discard the current point; if the number of neighboring points of the current point Greater than the second preset neighborhood point threshold (i.e. the maximum neighborhood point threshold set) ), then keep the current point; if the number of neighboring points of the current point Greater than or equal to the first preset neighborhood point threshold (that is, the set minimum neighborhood point threshold ) and is less than or equal to the second preset neighborhood point threshold (that is, the set maximum neighborhood point threshold ), then according to the grid point set container corresponding to the current point (i.e., the gridded point set container obtained in S2), all other points in the grid where the current point is located are added to the currently filtered ground point cloud The present invention calls this process point cloud compensation.

[0066] Use the same method to filter out the ground point cloud Perform the above operations on all points in to get the current post-processed point cloud .

[0067] To filter out the ground point cloud set { The above operation is performed on each filtered ground point cloud in} to obtain the post-processed point cloud set { }; Each post-processed point cloud set contains two labels: 1. The category label of each point; 2. The ground height label value of the grid where each point is located.

[0068] S5. Add a ground height factor to the cross entropy loss function in the original point cloud segmentation network model to obtain a modified point cloud segmentation network model; perform multiple rounds of training on the post-processed point cloud set through the modified point cloud segmentation network model to obtain a cross entropy loss value for each round; reversely update the modified point cloud segmentation network model according to the cross entropy loss value of the current round; take the modified point cloud segmentation network model corresponding to the training round with the smallest cross entropy loss value as the target point cloud segmentation network model;

[0069] The original point cloud segmentation network model is a RandLA-Net network model; the network achieves a good balance between algorithm efficiency and effect through random sampling and local feature aggregation.

[0070] The cross entropy loss function is generally used in point cloud segmentation tasks. The present invention optimizes the cross entropy loss function to a certain extent and introduces a height factor from the ground, so that the entire network training tends to improve the segmentation effect of those scattered object samples (especially small-sized scattered objects that are very close to the ground) and the segmentation effect of the points at the junction of objects and the ground.

[0071] After the cross entropy loss function in the original point cloud segmentation network model introduces the ground height factor, the modified point cloud segmentation network model is obtained;

[0072] The post-processed point cloud set is trained for multiple rounds by changing the point cloud segmentation network model, and the cross entropy loss value of each round includes:

[0073] Each post-processed point cloud in the post-processed point cloud set is trained in the current round by changing the point cloud segmentation network model to obtain the cross entropy loss value of each post-processed point cloud;

[0074] The cross entropy loss value of each post-processed point cloud is calculated according to the following formula:

[0075] ;

[0076] Among them, L is the cross entropy loss value of the current post-processing point cloud, N is the total number of points in the current post-processing point cloud, The first points, M is the number of categories in the current post-processing point cloud, and c is the number of points in the current post-processing point cloud. The point is predicted as the cth category by changing the point cloud segmentation network model. The category label of the point is equal to c, then Take 1, otherwise Take 0, The first The predicted probability of the point cloud segmentation network model is changed to the c-th category. The first The height factor of the point above the ground, The first The z coordinate value of the point, The first The ground height label value of each point, and The smaller the difference between these two values, the The greater the height factor of the point above the ground.

[0077] The cross entropy loss values ​​of all post-processed point clouds are summed and averaged to obtain the cross entropy loss value of the current round.

[0078] The following is a specific example:

[0079] The processed point cloud set is trained for the first round by changing the point cloud segmentation network model to obtain the cross entropy loss value of the first round, and the changed point cloud segmentation network model is updated by back propagation according to the cross entropy loss value of the first round; the processed point cloud set is trained for the second round by changing the point cloud segmentation network model (i.e., the updated changed point cloud segmentation network model) to obtain the cross entropy loss value of the second round, and the changed point cloud segmentation network model is updated again by back propagation according to the cross entropy loss value of the second round, and the above process is repeated. It is assumed that the maximum number of training rounds is K. The training is stopped after K rounds of training. Finally, the changed point cloud segmentation network model corresponding to the training round with the smallest cross entropy loss value is taken as our optimal network model, i.e., the target point cloud segmentation network model, called Mseg.

[0080] S6. Obtain the point cloud to be detected, and perform background filtering, ground filtering, and point cloud compensation on the point cloud to be detected in turn to obtain the post-processed point cloud to be detected; input the post-processed point cloud to be detected into the target point cloud segmentation network model for prediction to obtain the category of each point in the post-processed point cloud to be detected; extract the points classified as spilled objects and calculate their bounding boxes to obtain the center position and length, width, and height size information of the spilled objects.

[0081] In an optional embodiment, a laser radar is installed on both sides of the tunnel entrance to obtain a point cloud to be detected, and the point cloud to be detected is subjected to a straight-through filtering to remove interference from the background area to obtain a filtered background point cloud to be detected. Then, the filtered background point cloud to be detected is subjected to a point cloud rasterization operation and a rasterization partitioning operation according to S2 to obtain a rasterized point set container corresponding to the filtered background point cloud to be detected. Then, according to S3, the ground segmentation based on the traditional point cloud algorithm is used to perform ground filtering on the filtered background point cloud to be detected to obtain a filtered ground point cloud to be detected; then, the filtered ground point cloud to be detected is subjected to point cloud compensation according to S4 to obtain a post-processed point cloud to be detected; finally, the post-processed point cloud to be detected is input into the optimal network model trained in S5, namely, the target point cloud segmentation network model Mseg, to perform point cloud segmentation to obtain the category of each point in the post-processed point cloud to be detected, and the points classified as scattered objects are extracted, and their external bounding boxes are calculated to obtain the center position and length, width and height size information of the scattered objects.

[0082] Figure 2 A spilled object detection system based on traditional algorithms and deep learning is provided in an embodiment of the present invention. Figure 2 As shown, the system includes:

[0083] The data processing unit 201 is used to obtain an original point cloud set, perform background filtering on the original point cloud set to obtain a filtered background point cloud set; perform category labeling on each point cloud in the filtered background point cloud set, and perform data enhancement to obtain a data enhanced point cloud set;

[0084] The data processing unit 201 includes:

[0085] A category labeling subunit is used to label each point in each point cloud in the filtered background point cloud set to obtain a category label for each point;

[0086] A separation unit is used to save each scattered object target in each point cloud separately to obtain a scattered object point cloud set;

[0087] The data enhancement subunit is used to take each point cloud in the filtered background point cloud set as the current filtered background point cloud in turn, randomly select a preset number of scattered object point clouds from the scattered object point cloud set and add them to the current filtered background point cloud to obtain the current data enhanced point cloud.

[0088] The point cloud rasterization unit 202 is used to perform a rasterization operation on each point cloud in the data enhanced point cloud set, and perform rasterization partitioning on all points in each point cloud to obtain a rasterized point set container corresponding to each point cloud; and to add a ground height label value of the grid where each point in each point cloud in the data enhanced point cloud set is located;

[0089] The point cloud rasterization unit 202 includes:

[0090] A rasterization subunit is used to sequentially use each point cloud in the data enhancement point cloud set as the current data enhancement point cloud, perform a rasterization operation on the current data enhancement point cloud, and obtain multiple grids;

[0091] A partitioning subunit is used to perform rasterization partitioning on all points in the current data augmented point cloud, so as to divide each point into a corresponding grid and obtain a current rasterized point set container;

[0092] The ground height labeling subunit is used to average the Z values ​​of all points classified as ground in each grid in the current data enhanced point cloud to obtain the ground height value of each grid; and to add a label value to each point in the current data enhanced point cloud according to the ground height value of the grid in which it is located to obtain the ground height label value of each point.

[0093] A ground filtering unit 203, configured to perform ground filtering on each point cloud in the data enhanced point cloud set by using a traditional point cloud algorithm to obtain a filtered ground point cloud set;

[0094] The point cloud compensation unit 204 is used to perform point cloud compensation on each point cloud in the filtered ground point cloud set to obtain a post-processed point cloud set; performing point cloud compensation on the current point cloud in the filtered ground point cloud set includes: performing a neighborhood search on each point in the current point cloud to obtain all neighboring points corresponding to each point; performing point cloud compensation on the current point cloud according to all neighboring points corresponding to each point and its corresponding gridded point set container;

[0095] The point cloud compensation for the current point cloud according to all the neighborhood points corresponding to each point and the corresponding gridded point set container includes:

[0096] Take each point as the current point in turn. If the number of neighborhood points of the current point is less than the first preset neighborhood point threshold, discard the current point; if the number of neighborhood points of the current point is greater than the second preset neighborhood point threshold, retain the current point; if the number of neighborhood points of the current point is greater than or equal to the first preset neighborhood point threshold and less than or equal to the second preset neighborhood point threshold, then according to the rasterized point set container corresponding to the current point, all other points in the grid where the current point is located are added to the currently filtered ground point cloud.

[0097] The model training unit 205 is used to increase the ground height factor to the cross entropy loss function in the original point cloud segmentation network model to obtain a modified point cloud segmentation network model; the post-processed point cloud set is trained for multiple rounds through the modified point cloud segmentation network model to obtain a cross entropy loss value for each round; the modified point cloud segmentation network model is reversely updated according to the cross entropy loss value of the current round; and the modified point cloud segmentation network model corresponding to the training round with the smallest cross entropy loss value is taken as the target point cloud segmentation network model;

[0098] The detection unit 206 is used to obtain the point cloud to be detected, and perform background filtering, ground filtering, and point cloud compensation on the point cloud to be detected in turn to obtain the post-processed point cloud to be detected; input the post-processed point cloud to be detected into the target point cloud segmentation network model for prediction to obtain the category of each point in the post-processed point cloud to be detected; extract the points classified as scattered objects and calculate their external bounding boxes to obtain the center position and length, width and height size information of the scattered objects.

[0099] The system of the present invention corresponds to the above method, and the specific implementation of the system will not be repeated here.

[0100] Beneficial effects of the present invention:

[0101] The present invention reduces the redundancy of point cloud data through multi-stage data preprocessing, point cloud enhancement, rasterization, and point cloud compensation, so that the subsequent deep learning model can run efficiently with lower computing resources, and point cloud enhancement and point cloud compensation significantly improve the detection accuracy of small-sized, close-to-the-ground spills. The present invention introduces a height-from-the-ground factor and optimizes the traditional cross-entropy loss function, so that the segmentation effect of small-sized spills during training is significantly improved.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting spilled objects based on traditional algorithms and deep learning, characterized in that: include: S1, obtaining an original point cloud set, and performing background filtering on the original point cloud set to obtain a filtered background point cloud set; Label each point cloud in the filtered background point cloud set by category, and perform data enhancement to obtain a data enhanced point cloud set; S2, performing a rasterization operation on each point cloud in the data enhanced point cloud set, and performing rasterization partitioning on all points in each point cloud to obtain a rasterized point set container corresponding to each point cloud; Adding a ground height label value of the grid where each point in each point cloud in the data enhancement point cloud set is located; S3, performing ground filtering on each point cloud in the data enhanced point cloud set by using a traditional point cloud algorithm to obtain a filtered ground point cloud set; S4, performing point cloud compensation on each point cloud in the filtered ground point cloud set to obtain a post-processed point cloud set; Point cloud compensation for the current point cloud after filtering out the ground point cloud set includes: performing neighborhood search on each point in the current point cloud to obtain all neighboring points corresponding to each point; performing point cloud compensation on the current point cloud according to all neighboring points corresponding to each point and its corresponding gridded point set container; S5. Add a ground height factor to the cross entropy loss function in the original point cloud segmentation network model to obtain a modified point cloud segmentation network model; perform multiple rounds of training on the post-processed point cloud set through the modified point cloud segmentation network model to obtain a cross entropy loss value for each round; reversely update the modified point cloud segmentation network model according to the cross entropy loss value of the current round; take the modified point cloud segmentation network model corresponding to the training round with the smallest cross entropy loss value as the target point cloud segmentation network model; S6. Obtain the point cloud to be detected, perform background filtering, ground filtering, and point cloud compensation on the point cloud to be detected in turn, and obtain the post-processed point cloud to be detected; input the post-processed point cloud to be detected into the target point cloud segmentation network model for prediction, and obtain the category of each point in the post-processed point cloud to be detected; extract the points classified as scattered objects and calculate their external bounding boxes to obtain the center position and length, width, and height size information of the scattered objects; The post-processed point cloud set is trained for multiple rounds by changing the point cloud segmentation network model, and the cross entropy loss value of each round includes: Each post-processed point cloud in the post-processed point cloud set is trained in the current round by changing the point cloud segmentation network model to obtain the cross entropy loss value of each post-processed point cloud; Sum and average the cross entropy loss values ​​of all post-processed point clouds to get the cross entropy loss value of the current round; The cross entropy loss value of each post-processed point cloud is calculated according to the following formula: Where L is the cross entropy loss value of the current post-processed point cloud, N is the total number of points in the current post-processed point cloud, i is the i-th point in the current post-processed point cloud, M is the number of categories in the current post-processed point cloud, c is the i-th point in the current post-processed point cloud predicted as the c-th category by changing the point cloud segmentation network model, if the category label of the i-th point is equal to c, then y ic Take 1, otherwise y ic Take 0, p ic The predicted probability of the cth category predicted by changing the point cloud segmentation network model for the i-th point in the current post-processed point cloud, is the height factor of the i-th point in the current post-processing point cloud, z i is the z coordinate value of the i-th point in the current post-processing point cloud, h i It is the ground height label value of the i-th point in the current post-processed point cloud.

2. The method according to claim 1, characterized in that The step of labeling each point cloud in the filtered background point cloud set and performing data enhancement to obtain a data enhanced point cloud set includes: Label each point in each point cloud in the filtered background point cloud set to obtain a category label for each point; Each spilled object target in each point cloud is saved separately to obtain a spilled object point cloud set; Each point cloud in the filtered background point cloud set is taken as the current filtered background point cloud in turn, and a preset number of scattered object point clouds are randomly selected from the scattered object point cloud set and added to the current filtered background point cloud to obtain the current data enhanced point cloud.

3. The method according to claim 1, characterized in that The S2 includes: Each point cloud in the data enhancement point cloud set is used as the current data enhancement point cloud in turn, and the current data enhancement point cloud is rasterized to obtain multiple grids; Rasterize and partition all points in the current data-enhanced point cloud to divide each point into a corresponding grid and obtain the current rasterized point set container; The Z values ​​of all points classified as ground in each grid in the current data-enhanced point cloud are averaged to obtain the ground height value of each grid; a label value is added to each point in the current data-enhanced point cloud according to the ground height value of the grid in which it is located to obtain the ground height label value of each point.

4. The method according to claim 3, characterized in that The point cloud compensation for the current point cloud according to all the neighborhood points corresponding to each point and the corresponding gridded point set container includes: Take each point as the current point in turn. If the number of neighborhood points of the current point is less than the first preset neighborhood point threshold, discard the current point; if the number of neighborhood points of the current point is greater than the second preset neighborhood point threshold, retain the current point; if the number of neighborhood points of the current point is greater than or equal to the first preset neighborhood point threshold and less than or equal to the second preset neighborhood point threshold, then according to the rasterized point set container corresponding to the current point, all other points in the grid where the current point is located are added to the current filtered ground point cloud.

5. A spilled object detection system based on traditional algorithms and deep learning, characterized in that: include: A data processing unit, used for acquiring an original point cloud set, performing background filtering on the original point cloud set, and obtaining a filtered background point cloud set; Label each point cloud in the filtered background point cloud set by category, and perform data enhancement to obtain a data enhanced point cloud set; A point cloud rasterization unit, used for performing a rasterization operation on each point cloud in the data enhanced point cloud set, and performing rasterization partitioning on all points in each point cloud to obtain a rasterized point set container corresponding to each point cloud; Adding a ground height label value of the grid where each point in each point cloud in the data enhancement point cloud set is located; A ground filtering unit, used for performing ground filtering on each point cloud in the data enhanced point cloud set by using a traditional point cloud algorithm to obtain a filtered ground point cloud set; A point cloud compensation unit is used to perform point cloud compensation on each point cloud in the filtered ground point cloud set to obtain a post-processed point cloud set; Point cloud compensation for the current point cloud after filtering out the ground point cloud set includes: performing neighborhood search on each point in the current point cloud to obtain all neighboring points corresponding to each point; performing point cloud compensation on the current point cloud according to all neighboring points corresponding to each point and its corresponding gridded point set container; The model training unit is used to increase the ground height factor to the cross entropy loss function in the original point cloud segmentation network model to obtain a modified point cloud segmentation network model; the post-processed point cloud set is trained for multiple rounds through the modified point cloud segmentation network model to obtain a cross entropy loss value for each round; the modified point cloud segmentation network model is reversely updated according to the cross entropy loss value of the current round; and the modified point cloud segmentation network model corresponding to the training round with the smallest cross entropy loss value is taken as the target point cloud segmentation network model; The detection unit is used to obtain the point cloud to be detected, and perform background filtering, ground filtering, and point cloud compensation on the point cloud to be detected in turn to obtain the post-processed point cloud to be detected; the post-processed point cloud to be detected is input into the target point cloud segmentation network model for prediction to obtain the category of each point in the post-processed point cloud to be detected; the points classified as scattered objects are extracted and their external bounding boxes are calculated to obtain the center position and length, width, and height size information of the scattered objects; The post-processed point cloud set is trained for multiple rounds by changing the point cloud segmentation network model, and the cross entropy loss value of each round includes: Each post-processed point cloud in the post-processed point cloud set is trained in the current round by changing the point cloud segmentation network model to obtain the cross entropy loss value of each post-processed point cloud; Sum and average the cross entropy loss values ​​of all post-processed point clouds to get the cross entropy loss value of the current round; The cross entropy loss value of each post-processed point cloud is calculated according to the following formula: Where L is the cross entropy loss value of the current post-processed point cloud, N is the total number of points in the current post-processed point cloud, i is the i-th point in the current post-processed point cloud, M is the number of categories in the current post-processed point cloud, c is the i-th point in the current post-processed point cloud predicted as the c-th category by changing the point cloud segmentation network model, if the category label of the i-th point is equal to c, then y ic Take 1, otherwise y ic Take 0, p ic The predicted probability of the cth category predicted by changing the point cloud segmentation network model for the i-th point in the current post-processed point cloud, is the height factor of the i-th point in the current post-processing point cloud, z i is the z coordinate value of the i-th point in the current post-processing point cloud, h i It is the ground height label value of the i-th point in the current post-processed point cloud.

6. The system according to claim 5, characterized in that The data processing unit comprises: A category labeling subunit is used to label each point in each point cloud in the filtered background point cloud set to obtain a category label for each point; A separation unit is used to save each scattered object target in each point cloud separately to obtain a scattered object point cloud set; The data enhancement subunit is used to take each point cloud in the filtered background point cloud set as the current filtered background point cloud in turn, randomly select a preset number of scattered object point clouds from the scattered object point cloud set and add them to the current filtered background point cloud to obtain the current data enhanced point cloud.

7. The system according to claim 5, characterized in that The point cloud rasterization unit comprises: A rasterization subunit is used to sequentially use each point cloud in the data enhancement point cloud set as the current data enhancement point cloud, and perform a rasterization operation on the current data enhancement point cloud to obtain multiple grids; A partitioning subunit is used to perform rasterization partitioning on all points in the current data augmented point cloud, so as to divide each point into a corresponding grid and obtain a current rasterized point set container; The ground height labeling subunit is used to average the Z values ​​of all points classified as ground in each grid in the current data enhanced point cloud to obtain the ground height value of each grid; and to add a label value to each point in the current data enhanced point cloud according to the ground height value of the grid in which it is located to obtain the ground height label value of each point.

8. The system according to claim 7, characterized in that The point cloud compensation for the current point cloud according to all the neighborhood points corresponding to each point and the corresponding gridded point set container includes: Take each point as the current point in turn. If the number of neighborhood points of the current point is less than the first preset neighborhood point threshold, discard the current point; if the number of neighborhood points of the current point is greater than the second preset neighborhood point threshold, retain the current point; if the number of neighborhood points of the current point is greater than or equal to the first preset neighborhood point threshold and less than or equal to the second preset neighborhood point threshold, then according to the rasterized point set container corresponding to the current point, all other points in the grid where the current point is located are added to the current filtered ground point cloud.

Images