A multi-feature fusion method for dust removal in mining areas

Abstract

A multi-feature fusion dust filtering method for mining areas, comprising: S1: acquiring point cloud data through a sensor in an echo detection module, the same beam of laser light generates multiple echo signals, and the penetration characteristics of obstacles are judged by analyzing the point cloud data of different echoes corresponding to each beam of laser light, and output point cloud data is obtained by analyzing multiple echoes; S2: inputting the output point cloud data into a point cloud semantic detection network in a point cloud semantic detection module, learning features, constructing and outputting point cloud data; S3: inputting the point cloud data output by the point cloud semantic detection module into an intensity detection module, performing intensity analysis to obtain analysis results; S4: inputting the analysis results after intensity analysis into a perception module, and the perception module outputs the perception results after processing, and inputting the perception results into a post-processing module as the final target point cloud, and the post-processing module performs curvature analysis and outputs decision-level results.

Description

Technical Field

[0001] The present invention relates to an automated system for dust filtration in a mining environment, and more specifically to a method for dust filtration in a mining environment utilizing multi-feature fusion. Background Art

[0002] Since the 21st century, it has been an important development topic to achieve high automation of mining work with the support of vehicle-mounted laser radar unmanned driving technology. However, high concentrations of dust and harsh environments pose higher challenges to the laser radar sensors of unmanned mining vehicles. It is very important to use emerging technologies to solve the dust problem in mining areas and realize automated work in mining areas.

[0003] Existing dust analysis methods include using sensors to detect and return information, which can be compared with data samples to identify dust. However, the complex and diverse mining areas and harsh working environments limit the stable operation of multi-sensors, and when some sensors have problems, the reliability of the entire system cannot be guaranteed; the dust density in mining areas is high, and the use of existing analysis methods still cannot achieve the desired filtering effect; when the vehicle arrives in a new environment, the existing methods that rely on data samples will weaken their ability to handle dust.

[0004] Based on this, the art needs a new fusion method that can adapt to new environments and can stably and accurately perform automatic dust filtration in high-concentration dust and harsh environments. Summary of the invention

[0005] In order to solve the above-mentioned problems in the prior art and overcome the shortcomings of the prior art, an embodiment of the present invention provides a multi-feature fusion method for dust filtration in mining areas to improve the working efficiency of mine cars, especially unmanned mine cars, in special scenarios.

[0006] A multi-feature fusion mining dust filtration method provided according to an embodiment of the present invention comprises the following steps:

[0007] S1: The echo detection module obtains point cloud data through the sensor. The same laser beam generates multiple echo signals. The penetration characteristics of the obstacle are determined by analyzing the point cloud data of different echoes corresponding to each laser beam. The output point cloud data is obtained by analyzing multiple echoes.

[0008] S2: inputting the output point cloud data into a point cloud semantic detection network in a point cloud semantic detection module, learning features, constructing and outputting point cloud data;

[0009] S3: inputting the point cloud data output by the point cloud semantic detection module into the strength detection module to perform strength analysis to obtain analysis results;

[0010] S4: The analysis result after the intensity analysis is input into the perception module, and the perception module outputs the perception result after processing. The perception result is input into the post-processing module as the final target point cloud, and the post-processing module performs curvature analysis and outputs the decision-level result.

[0011] In an optional embodiment, the step S1 may further include the following steps:

[0012] S1-1: Use a laser radar sensor to obtain echo data, analyze the two echo data obtained by the laser radar sensor, and use the time of flight ranging method TOF (Time Of Flight) to measure the distance, where the distance d is:

[0013] d=(v*t) / 2,

[0014] Where v represents the speed of light, t represents the flight time, and d represents the distance of the obstacle.

[0015] For the double echoes at the same time, the measured distances are d1 and d2 respectively, and the distance difference S is:

[0016] s=|d2-d1|

[0017] The distance difference s is compared with the threshold. If s>threshold, the laser data at a farther distance is retained. Otherwise, if s<threshold, the data of the strongest echo signal is retained, and the retained echo signal data is the output point cloud data.

[0018] In an optional embodiment, the step S2 may further include the following steps:

[0019] S2-1: Collect point cloud data of dust from working vehicles in the mining area in different scenes through a laser radar sensor, and use the output point cloud data obtained by the echo detection module to create a database as training data;

[0020] S2-2: Perform spherical mapping on the point cloud data collected by the sensor, and map the point cloud data output by the echo detection module into a pseudo image, including

[0021] Assume that the coordinates of a point in space are p = (x, y, z), and the vertical receptive field of the sensor is f down and f up It means that f down and f up are the maximum and minimum values ​​of the vertical receptive field respectively. The coordinates (u, v) of the pixel coordinate system corresponding to point p are expressed as:

[0022]

[0023] Among them, r is the distance from p to the origin, f is the vertical sensing range of the sensor and f=|f down |+|f up |, f down and f up are the maximum and minimum values ​​of the vertical receptive field, respectively. w and h represent the width and height of the pseudo image, respectively. Finally, the mapping relationship between the three-dimensional coordinates and the pixel coordinate system is obtained, that is, the correspondence between any point p in space and the coordinates (u, v) in the pixel coordinate system. Through this relationship, x, y, z, intensity, r, are stored in the corresponding position of the pixel coordinate system to form a pseudo image of w*h*5, where intensity represents the reflection intensity.

[0024] S2-3: Processing the w*h*5 pseudo image through a context feature extraction layer to obtain more comprehensive feature information;

[0025] S2-4: The encoder uses dilated convolution to downsample the acquired more comprehensive feature information and inputs it into the inverse encoder. The inverse encoder performs an inverse convolution operation to upsample the feature information to obtain the inverse encoder result. The output layer performs a 1*1 convolution on the inverse encoder result, modifies the number of channels to match the actual category, and outputs a point cloud with point cloud category information.

[0026] In an optional embodiment, the step S3 may further include the following steps:

[0027] S3-1: Input the point cloud data output by the point cloud semantic detection module into the strength detection module, and the strength detection module divides the point cloud data into effective areas.

[0028] -15≤x≤15, and

[0029] -15≤y≤15,

[0030] Among them, x and y are the coordinates representing the point cloud. In this way, the point cloud within the effective range is traversed. If the reflection intensity of the point cloud is within the reflection intensity threshold range of the dust, the point cloud is discarded, otherwise the point cloud is retained, thereby obtaining the analysis result of the intensity analysis.

[0031] In an optional embodiment, the step S4 may further include the following steps:

[0032] S4-1: Obtain the analysis result, and the post-processing module performs further processing, obtains n-1 (n>3) nearest points for any point p in the analysis result, decentralizes the n discrete points, and solves the covariance matrix.

[0033] Assume the coordinate data of n discrete points is

[0034]

[0035] Find the mean of the coordinates in each direction

[0036]

[0037] The result after decentralization is

[0038]

[0039] Constructing the covariance matrix

[0040]

[0041] Among them, x, y, z are the coordinates of the three-dimensional space points in three directions, n is the number of points, D is the intermediate variable for constructing the covariance matrix C, and C is the covariance matrix.

[0042] S4-2: Perform singular value decomposition on the covariance matrix C to obtain eigenvalues ​​λ0, λ1, and λ2, where λ0 is the smallest eigenvalue, λ1 is the second largest eigenvalue, and λ2 is the largest eigenvalue. Use surface variation to obtain the approximate curvature δ of each point.

[0043] δ=λ0 / (λ0+λ1+λ2),

[0044] Assume that the final perception result point cloud is S = S i (i=1,2…,n), where S i is the target point cloud of the perception result, and S i Solve the mean of the point cloud curvature, filter out the results with larger mean values, thereby judging obstacles, filtering out dust, and achieving decision-level dust filtering. The multi-feature fusion mining dust filtering method provided by the embodiment of the present invention has at least the following beneficial effects compared with the prior art.

[0045] 1. By analyzing the optical characteristics of laser radar and the physical characteristics of dust, the present invention proposes a method of using double echo dust filtering for the working environment of the mining area. Using a multi-echo dust filtering method, such as a double echo dust filtering method, dust with penetrating properties can be directly filtered out, which can cope with scenes with less dust such as living areas in mining areas, which also improves the computing speed of the semantic detection module.

[0046] 2. Dust has irregular features and requires more semantic understanding of the overall context. In the present invention, residual dilated convolution is applied to extract context features of the point cloud semantic detection module to fuse features of different receptive fields, so that the model can learn overall and local information. At the same time, dilated convolution can increase features without reducing the resolution of feature maps, making point cloud semantic segmentation more accurate. In the representation method of point clouds, spherical mapping is used to map point clouds into 2D pseudo images, which improves the computing speed.

[0047] 3. Curvature is a decision-level filter for the perception results, so that the final obstacle output is guaranteed again. The curvature method is to perform feature analysis on the clustered point cloud, with the aim of reflecting the clutter characteristics of dust. Using the PCA method, singular value decomposition is performed on the point cloud data, and the curvature of the point cloud is estimated using the variational idea, ultimately achieving the purpose of describing the clutter degree of the point cloud. This is a new form of discrete degree representation, which has a better recognition effect on high-concentration dust.

[0048] 4. Aiming at the problem that the high performance of LiDAR cannot be realized in the harsh mining environment, the present invention proposes a multi-feature fusion method. The method combining point cloud semantic segmentation with multiple features increases the robustness of the algorithm to cope with the changing physical environment of the mining area. When arriving at a new environment, it can still cope with the complex environment on site in terms of echo, curvature, reflection intensity, etc.

[0049] By reference to the accompanying drawings and the following description, other devices, equipment, systems, methods, features and advantages of the present invention will be apparent. All other such systems, methods, features and advantages included in this description are within the scope of the present invention and are protected by the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention may be better understood by reference to the accompanying drawings, wherein the components in the figures are not to be considered to scale, emphasis instead being placed upon illustrating the principles of the invention.

[0051] Figure 1 A schematic diagram showing the effect of dust on double echoes in a mine dust removal method using multi-feature fusion provided in accordance with an embodiment of the present invention is shown.

[0052] Figure 2 A schematic diagram of PCA hyperplane fitting in a mine dust removal method with multi-feature fusion provided in accordance with an embodiment of the present invention is shown.

[0053] Figure 3 A curvature comparison diagram of point cloud data of cars, mining trucks, and dust in a multi-feature fusion mining dust filtration method provided in an embodiment of the present invention is shown.

[0054] Figure 4 A schematic diagram of the processing flow of a pre-processing module in a multi-feature fusion mining dust filtration method provided in accordance with an embodiment of the present invention is shown.

[0055] Figure 5 A schematic diagram of the structure and flow of a point cloud semantic detection module in a system of a method for dust removal in a mining area implementing multi-feature fusion provided in accordance with an embodiment of the present invention is shown.

[0056] Figure 6 A schematic flow chart of post-processing curvature analysis performed by a post-processing module in a multi-feature fusion mining dust filtration method provided in accordance with an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0057] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the technical terms or scientific terms used in the present invention should be the common meanings understood by those skilled in the art to which the present invention belongs.

[0058] Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

[0059] Figure 1 A schematic diagram showing the effect of dust on double echoes in a mine dust removal method using multi-feature fusion provided in accordance with an embodiment of the present invention is shown. Figure 2 A schematic diagram of PCA hyperplane fitting in a mine dust removal method with multi-feature fusion provided in accordance with an embodiment of the present invention is shown. Figure 3 The curvature comparison diagram of the point cloud data of the car, the mining truck and the dust is shown. As shown in the figure, considering the high frequency and high precision characteristics of the laser radar, in the multi-feature fusion mining dust filtering method provided according to an embodiment of the present invention, the laser radar sensor is used for dust detection.

[0060] Regarding the dual echo technology, the ranging technology of LiDAR includes triangulation ranging, time of flight ranging (Time OfFlight) and other methods. The working principle of time of flight ranging is: the laser transmitter emits laser, the laser is reflected by the obstacle, and then received by the laser receiver to obtain the obstacle position. Since the reflection of light is related to the material of the medium, distance and other factors, the laser receiver can receive multiple laser echoes, such as Figure 1 Dust has the characteristic of penetrating. By analyzing the characteristics of the echo, we can determine whether the object is dust or a solid obstacle.

[0061] In an embodiment of the present invention, deep learning semantic segmentation technology is used to achieve semantic-level understanding of point clouds. The present invention forms a 2D pseudo image as a representation of the point cloud through spherical mapping of the point cloud. Compared with the voxelization method, the point cloud is mapped to a two-dimensional image, which is faster in the network. Dust has various shapes, and the selection of the receptive field is very important for comprehensive and holistic contextual information. A simple way to expand the receptive field is to expand the width of the convolution kernel, but this also brings computational overhead. The network in the embodiment of the present invention uses residual hole convolution to realize feature extraction of the context, and then through the encoder and decoder, as well as the output layer, finally realizes the semantic-level segmentation of the point cloud.

[0062] Curvature describes the degree of curvature at a point on the surface. Due to the suspension characteristics of dust and the influence of dust penetration, the laser point cloud of dust will show a messy and uneven characteristic. Using the idea of ​​PCA, the point cloud samples are decentralized, and then the covariance matrix of the decentralized data is decomposed by singular value to solve the eigenvalue. The eigenvector corresponding to the minimum eigenvalue is the normal vector of the fitting plane of the point cloud data, such as Figure 2 The curvature of the point cloud data at a certain point can be approximated as the surface variation at that point, so as to perform approximate curvature analysis on the obstacle point cloud of the perception result and separate the dust point cloud. The results of the curvature comparison of different objects are shown in Figure 3 As shown in the figure, the x-axis represents the sample number and the y-axis represents the average value of the curvature of the perceived result. It can be seen that the curvature of the dust object is relatively large, which is consistent with the messy nature of reality.

[0063] In addition to xyz information, point cloud data also includes reflection intensity information. Reflection intensity is generally related to the wavelength of the laser pulse, the brightness and texture of the medium surface, transmission distance, etc. The brighter the reflective medium, the higher the reflectivity. Although reflection intensity is related to many factors, when in the same area, it can be approximately considered that reflection intensity is only related to the medium surface. By establishing a corresponding relationship between the material and its reflection intensity, the dust filtration relationship can be achieved.

[0064] Figure 4 A schematic diagram of the processing flow of a pre-processing module in a multi-feature fusion mining dust filtration method provided in accordance with an embodiment of the present invention is shown. Figure 5 A schematic diagram of the structure and flow of a point cloud semantic detection module in a system of a method for dust removal in a mining area implementing multi-feature fusion provided in accordance with an embodiment of the present invention is shown. Figure 6 The following is a schematic diagram showing the processing flow of curvature analysis performed by the post-processing module of the multi-feature fusion mining dust filtration method according to an embodiment of the present invention. Figure 4-6 ,in Figure 4The following figure shows the echo detection module, point cloud semantic segmentation module, intensity detection module process in steps S1, S2, and S3. Figure 5 The model structure diagram of the point cloud semantic segmentation module in step S2 below is shown. Figure 6 The structure of the post-processing module in step S4. According to an embodiment of the present invention, a multi-feature fusion method for filtering dust in a mining area is provided, which includes the following steps.

[0065] S1: In the echo detection module, point cloud data is obtained through sensors. The same laser beam generates multiple echo signals. The penetration characteristics of obstacles are determined by analyzing the different echo point cloud data corresponding to each laser beam. The output point cloud data is obtained by analyzing multiple echoes.

[0066] S2: Input the output point cloud data into the point cloud semantic detection network in the point cloud semantic detection module, learn features, construct and output point cloud data. Learning features is part of semantic segmentation, and after semantic segmentation, each point cloud point carries semantic information.

[0067] S3: Input the point cloud data output by the point cloud semantic detection module into the strength detection module to perform strength analysis to obtain analysis results.

[0068] S4: The analysis result after the intensity analysis is input into the perception module, and the perception module outputs the perception result after processing. The perception result is input into the post-processing module as the final target point cloud, and the post-processing module performs curvature analysis and outputs the decision-level result.

[0069] In an optional implementation, the step S1 may further include the following steps.

[0070] S1-1: Analyze the two echo data collected by the sensor. In this embodiment, the sensor uses a laser radar sensor. It should be understood that in other embodiments, other suitable sensors can be used as needed.

[0071] Table 1-1

[0072] Echo First echo Second echo distance <![CDATA[d1]]> <![CDATA[d2]]>

[0073] In this embodiment, the distance measurement method can use the time of flight distance measurement method TOF (Time Of Flight) for measurement, and the distance d = (v*t) / 2, where v represents the speed of light, t represents the flight time, and d represents the distance of the obstacle. For the double echoes at the same time, the measured distances are d1 and d2 respectively, as shown in the above Table 1-1, and the distance difference is:

[0074] s=|d2-d1|.

[0075] The distance difference s is compared with the preset threshold. If s>threshold, it indicates that penetration has occurred this time, and the laser data at a farther distance (i.e., the echo signal data at a farther distance) is retained. Otherwise, if s<threshold, it indicates that the distances of the two echoes are close, and the data of the strongest echo signal is retained. The retained echo signal data is the output point cloud data. The size of the threshold is related to the actual height of the vehicle and can reflect the penetration characteristics of the obstacle.

[0076] Using double-echo or multiple-echo dust filtering methods, penetrating dust can be directly filtered out, which can cope with scenes with less dust such as mining areas and living areas. This is also conducive to improving the computing speed and processing efficiency of the semantic detection module.

[0077] refer to Figure 5 In an optional implementation, the step S2 may further include the following steps.

[0078] S2-1: Collect point cloud data of dust from working vehicles in the mining area in different scenes through sensors, and generate a database of the output point cloud data obtained through the output of the echo detection module as training data. In this embodiment, the sensor uses a laser radar sensor. It should be understood that in other embodiments, other suitable sensors can be used as needed.

[0079] S2-2: Input the point cloud data collected by the sensor into the point cloud semantic detection module, perform spherical mapping on the point cloud data in the point cloud semantic detection module, and map the point cloud data output by the echo detection module into a pseudo image, including

[0080] Assume that the coordinates of a point in space are p = (x, y, z), and the vertical receptive field of the sensor is f down and f up It means that f down and f up are the maximum and minimum values ​​of the vertical receptive field respectively. The coordinates (u, v) of the pixel coordinate system corresponding to point p are expressed as:

[0081]

[0082] in, r is the distance from p to the origin, f is the vertical sensing range of the sensor and f=|f down |+|f up|, w and h represent the width and height of the pseudo image respectively, and finally obtain the mapping relationship between the 3D coordinates and the pixel coordinate system, that is, the correspondence between any point p in space and the coordinates (u, v) in the pixel coordinate system. Through this relationship, x, y, z, intensity, r, are stored in the corresponding position of the pixel coordinate system to form a pseudo image of w*h*5. Among them, intensity represents the reflection intensity, which is an attribute of the point cloud point. The reflection intensity of different materials is different.

[0083] S2-3: The w*h*5 pseudo image formed in S2-2 obtains more comprehensive feature information through the context feature extraction layer of the point cloud semantic detection network. Specifically, residual dilated convolution can be used in the context feature extraction layer to extract context semantic features. Optionally, the residual dilated convolution can use 1*1 and 5*5 convolution kernels to perform dilated convolution operations, and then the convolution results are residually summed, so that the pseudo image finally learns global and local information.

[0084] S2-4: The encoder uses a dilated convolution to achieve downsampling of the acquired more comprehensive feature information and inputs it into the inverse encoder. The inverse encoder achieves upsampling of the feature information through a deconvolution operation to obtain the inverse encoder result. The output layer performs a 1*1 convolution on the inverse encoder result, modifies the number of channels to match the actual category, and outputs a point cloud with point cloud category information (which can be represented by a label). The output point cloud with point cloud category information is the output point cloud data. In this step, the output point cloud category includes dust point clouds and non-dust point clouds, and then the category probability of the point cloud can be obtained through the softmax (activation function) layer, and the probability of each point being dust is determined and filtered out. Among them, after the feature information data is input into the point cloud semantic detection network, the output results of each point cloud semantic detection network are high-dimensional features.

[0085] In this step, the point cloud is assigned category information, and the point cloud points identified as obstacles (i.e., non-dust) are subjected to intensity analysis to further filter out dust points. Furthermore, dust has irregular characteristics, which requires more semantic understanding of the overall context. In this embodiment of the present invention, residual hole convolution is applied during context feature extraction of the point cloud semantic detection module to fuse features of different receptive fields, so that the model can learn overall and local information. At the same time, hole convolution can achieve the increase of features without reducing the resolution of feature maps, making point cloud semantic segmentation more accurate. In terms of point cloud representation, spherical mapping is used to map the point cloud into a 2D pseudo-image, which effectively improves the computing speed.

[0086] In an optional implementation, the step S3 may further include the following steps.

[0087] S3-1: Input the point cloud data output by the point cloud semantic detection module into the intensity detection module. The intensity detection module divides the laser point cloud (i.e., the point cloud data) into effective areas, requiring -15≤x≤15 and -15≤y≤15, where x and y are the coordinates representing the point cloud. In this way, the point cloud within the effective range of the point cloud data is traversed. If the reflection intensity of the point is within the reflection intensity threshold interval of the dust, the point is discarded, otherwise the point is retained, thereby obtaining the analysis result of the intensity analysis. The purpose of this step is that the reflection intensity of the same reflective medium is related to the change in distance, and the reflection intensity characteristics of the dust are stable within the effective area. In this embodiment, the above-mentioned -15≤x≤15, -15≤y≤15 are preferred range values. In other embodiments, other suitable range values ​​can be selected as needed.

[0088] In an optional implementation, the step S4 may further include the following steps.

[0089] Since the point cloud is composed of isolated points, PCA (Principal Component Analysis) is used to estimate the curvature of each point in the point cloud. For each point, n-1 (n>3) adjacent points are found and the curvature is estimated using the PCA+surface variation method. The purpose of n>3 here is that at least three points are required to determine a plane.

[0090] S4-1: Obtain the analysis results, and the post-processing module performs further processing to obtain n-1 (n>3) nearest points for any point p in the analysis results, decentralized the n discrete points (i.e., point p and its n-1 nearest points), and solve its covariance matrix.

[0091] Assume the coordinate data of n discrete points is

[0092]

[0093] Find the mean of the coordinates in each direction

[0094]

[0095] The result after decentralization is

[0096]

[0097] Constructing the covariance matrix

[0098]

[0099] Among them, x, y, and z are the coordinates of the three-dimensional space points in three directions, n is the number of points, D is the intermediate variable for constructing the covariance matrix C, and C is the covariance matrix.

[0100] S4-2: Perform singular value decomposition on the covariance matrix C to obtain eigenvalues ​​λ0, λ1, and λ2. λ0 is the smallest eigenvalue, λ1 is the second largest eigenvalue, and λ2 is the largest eigenvalue. Use surface variation: δ = λ0 / (λ0+λ1+λ2) to obtain the approximate curvature δ of each point. Suppose the final perception result point cloud is S = S i (i=1,2…,n), where S i is the target point cloud of the perception result, and S i Solve the mean of the point cloud curvature and filter out the results with larger mean values, so as to judge the obstacles and filter out the dust, thus achieving decision-level dust removal. A larger mean value indicates a high degree of dispersion, which is a dust point cloud.

[0101] In step S4, target obstacles are obtained by clustering (one obstacle corresponds to one point cloud cluster), and it is analyzed whether the obstacle is dust, and the dust is filtered out.

[0102] It should be understood that the foregoing only describes some embodiments according to the present invention, and changes, modifications, additions and / or variations may be made without departing from the scope and essence of the disclosed embodiments, which are illustrative rather than restrictive. In addition, the described embodiments relate to the most practical and preferred embodiments currently considered, and it should be understood that the embodiments should not be limited to the disclosed embodiments, but rather, are intended to cover different modifications and equivalent arrangements included in the essence and scope of the embodiments. In addition, the various embodiments described above may be used in conjunction with other embodiments, such as aspects of one embodiment may be combined with aspects of another embodiment to achieve yet another embodiment. In addition, each independent feature or component of any given component may constitute another embodiment.

[0103] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some or all of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of the claims and specification of the present invention.

Claims

1. A multi-feature fusion method for dust removal in mining areas, characterized in that The following steps are involved: S1: The echo detection module obtains point cloud data through the sensor. The same laser beam generates multiple echo signals. The penetration characteristics of the obstacle are determined by analyzing the point cloud data of different echoes corresponding to each laser beam. The output point cloud data is obtained by analyzing multiple echoes. S2: inputting the output point cloud data into a point cloud semantic detection network in a point cloud semantic detection module, learning features, constructing and outputting point cloud data; S3: inputting the point cloud data output by the point cloud semantic detection module into the strength detection module to perform strength analysis to obtain analysis results; S4: Input the analysis result after the intensity analysis into the perception module, and the perception module outputs the perception result after processing, and inputs the perception result as the final target point cloud into the post-processing module, and the post-processing module performs curvature analysis and outputs the decision-level result; Wherein, the step S1 further comprises the following steps: S1-1: Use a laser radar sensor to obtain echo data, analyze the two echo data obtained by the laser radar sensor, and use the time of flight ranging method TOF (Time Of Flight) to measure the distance, where the distance d is: d=(v*t) / 2, Where v represents the speed of light, t represents the flight time, and d represents the distance of the obstacle. For the double echoes at the same time, the measured distances are d1 and d2 respectively, and the distance difference s is: s=|d2-d1| Compare the distance difference s with the threshold. If s>threshold, retain the laser data farther away. Otherwise, if s<threshold, retain the data of the strongest echo signal. The retained echo signal data is the output point cloud data. Wherein, the step S2 further comprises the following steps: S2-1: Collect point cloud data of dust from working vehicles in the mining area in different scenes through a laser radar sensor, and use the output point cloud data obtained by the echo detection module to create a database as training data; S2-2: Perform spherical mapping on the point cloud data collected by the sensor, and map the point cloud data output by the echo detection module into a pseudo image, including Assume that the coordinates of a point in space are p = (x, y, z), and the vertical receptive field of the sensor is f down and f up It means that f down and f up are the maximum and minimum values ​​of the vertical receptive field respectively. The coordinates (u, v) of the pixel coordinate system corresponding to point p are expressed as: Among them, r is the distance from p to the origin, f is the vertical sensing range of the sensor and f=|f down |+|f up |, f down and f up are the maximum and minimum values ​​of the vertical receptive field, respectively. w and h represent the width and height of the pseudo image, respectively. Finally, the mapping relationship between the three-dimensional coordinates and the pixel coordinate system is obtained, that is, the correspondence between any point p in space and the coordinates (u, v) in the pixel coordinate system. Through this relationship, x, y, z, intensity, r, are stored in the corresponding position of the pixel coordinate system to form a pseudo image of w*h*5, where intensity represents the reflection intensity. S2-3: Processing the w*h*5 pseudo image through a context feature extraction layer to obtain more comprehensive feature information; S2-4: The encoder uses dilated convolution to downsample the acquired more comprehensive feature information and inputs it into the inverse encoder. The inverse encoder performs an inverse convolution operation to upsample the feature information to obtain the inverse encoder result. The output layer performs a 1*1 convolution on the inverse encoder result, modifies the number of channels to match the actual category, and outputs a point cloud with point cloud category information.

2. A multi-feature fusion mining dust removal method as claimed in claim 1, characterized in that The step S3 further comprises the following steps: S3-1: Input the point cloud data output by the point cloud semantic detection module into the strength detection module, and the strength detection module divides the point cloud data into effective areas. -15≤x≤15, and -15≤y≤15， Among them, x and y are the coordinates representing the point cloud. In this way, the point cloud within the effective range is traversed. If the reflection intensity of the point cloud is within the reflection intensity threshold range of the dust, the point cloud is discarded, otherwise the point cloud is retained, thereby obtaining the analysis result of the intensity analysis.

3. A multi-feature fusion mining dust removal method as claimed in claim 1, characterized in that Described step S4 further comprises the following steps: S4-1: Obtain the analysis result, and the post-processing module performs further processing, obtains n-1 (n>3) nearest points for any point p in the analysis result, decentralizes the n discrete points, and solves the covariance matrix. Assume the coordinate data of n discrete points is Find the mean of the coordinates in each direction The result after decentralization is Constructing the covariance matrix Among them, x, y, z are the coordinates of the three-dimensional space points in three directions, n is the number of points, D is the intermediate variable for constructing the covariance matrix C, and C is the covariance matrix. S4-2: Perform singular value decomposition on the covariance matrix C to obtain eigenvalues ​​λ0, λ1, and λ2, where λ0 is the smallest eigenvalue, λ1 is the second largest eigenvalue, and λ2 is the largest eigenvalue. Use surface variation to obtain the approximate curvature δ of each point. δ=λ0 / (λ0+λ1+λ2), Assume that the final perception result point cloud is S = S i (i=1,2…,n), where S i is the target point cloud of the perception result, and S i Solve the mean of the point cloud curvature and filter out the results with larger mean values, so as to judge the obstacles, filter out the dust, and achieve decision-level dust removal.

Images