Material pile edge detection method and device, medium and product

The 3D point cloud data of the material stack is obtained and processed through 4D mmWave radar, combined with the fuzzy twin support vector machine and differential evolution algorithm, the problem of difficulty in identifying the edge of the material stack is solved, and high precision is achieved in complex environments Test to meet the needs of port automation control.

CN119068170APending Publication Date: 2024-12-03YANSHAN UNIV +2
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411049525.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing 3D mmWave radar is difficult to effectively identify the edge of the material pile in the automation construction of the port dry bulk terminal. In addition, traditional visual and lidar methods are greatly affected by the environment such as weather, lighting and sand and dust, and cannot meet the needs of semi-automation and automation. Control requirements.

Method used

4D mmWave radar is used to obtain the three-dimensional point cloud data at the edge of the material pile, and convert it to the dam foundation yard coordinate system through rotation and translation, and perform direct or statistical filtering to remove the point clouds in the noise and non-material stack area, using fuzzy twin support vector machines and differentials Evolutionary algorithms optimize model parameters to improve the accuracy of material pile edge detection.

Benefits of technology

Steadily obtain three-dimensional spatial information in complex environments, improve the accuracy and classification accuracy of material pile edge detection, and meet the needs of semi-automated and automated control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068170A_ABST
    Figure CN119068170A_ABST
Patent Text Reader

Abstract

The invention discloses a material pile edge detection method and device, a medium and a product, and relates to the technical field of 4D millimeter wave radar point cloud processing. The method comprises the following steps: converting to-be-measured three-dimensional point cloud data from a millimeter-wave radar coordinate system to a dam foundation storage yard coordinate system to obtain three-dimensional point cloud data under the dam foundation storage yard coordinate system; performing straight-through filtering processing or statistical filtering processing on the three-dimensional point cloud data under the millimeter-wave radar coordinate system and the three-dimensional point cloud data under the dam foundation storage yard coordinate system to obtain target three-dimensional point cloud data; and dividing the target three-dimensional point cloud data according to a frame sequence to obtain sample point clouds, extracting features of the sample point clouds, and inputting the features of the sample point clouds into the material pile edge detection model to obtain a material pile edge detection result. According to the invention, parameter selection of the material pile edge detection model is optimized by using the differential evolution algorithm, and the precision of material pile edge detection can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of 4D millimeter wave radar point cloud processing, and in particular to a pile edge detection method, device, medium and product. Background Art

[0002] With the continuous strengthening of the trend of economic globalization, the volume of cargo transportation between countries and regions is growing. As a sea and land transportation hub for material transportation, coastal ports play a role in the transshipment and distribution of bulk cargo, and play a vital role in global trade exchanges. As the main operating machinery currently used for material loading and unloading in dry bulk cargo yards, the realization of automatic and semi-automatic operation functions of reclaimers is the key to improving port operation efficiency. In the port yard, dry bulk materials are naturally dropped and stacked by the stacker, and the original material pile shape is conical. In the process of the reclaimer constantly rotating and retrieving materials, the shape of the material pile is constantly changing, especially for the recognition and judgment of edge samples. In order to obtain the edge morphological characteristics and classification of the material pile in real time, 4D millimeter wave radar and related point cloud improved processing algorithms are used to complete the judgment of the edge morphology of the material pile.

[0003] In the traditional sense, millimeter wave radar generally refers to 3D millimeter wave radar, and the acquired data mainly includes the distance, speed and azimuth angle information of the target object. 4D millimeter wave radar can obtain the radial distance, pitch angle, azimuth angle, speed, etc. of the target object. On the basis of the original advantages of 3D millimeter wave radar, it can additionally obtain pitch angle information for the target height, and the resolution of radar point cloud and the number of single-frame point clouds are further improved, which can present the spatial outline of the target object to a certain extent. By extracting the spatial features of several frames of point clouds, the real three-dimensional spatial form of the target can be reflected, providing more discriminative spatial information for point cloud morphology classification. At present, the bulk cargo picking operation in the port yard mainly relies on manual control, and the classification and judgment of the bulk cargo pile morphology also mainly relies on the operator's visual, visual images, laser radar and other methods. Based on visual image processing, KamariM et al. used deep learning to perform semantic segmentation on the scanning results, and segmented the target from the environment after identifying it, but the visual method lacks spatial three-dimensional information and has large measurement errors. In the construction of dry bulk terminal automation, the port uses 3D laser scanning technology to scan and integrate the piles in real time, convert them into intuitive 3D scanning images, and extract the control data required by other modules. The twin overlapping sensitive margin classifier proposed by Zhang Yu (Research and Application of Perception System in Unmanned Operation of Reclaimer) uses 3D millimeter wave radar to obtain the edge data of the pile for training and identify the edge of the pile, but the 3D millimeter wave radar can only return a two-dimensional plane point cloud, which lacks local spatial information and needs to be combined with the external bucket wheel power characteristics, and does not effectively solve the problem of parameter optimization selection during the training process. At present, relying on the visual judgment of operators does not meet the requirements of semi-automation and automation control of reclaimers in industrial upgrading, and the judgment of operators is greatly affected by on-site conditions such as obstacle angle obstruction and lighting. Although optical devices such as surveillance cameras and lidar can obtain a good monitoring angle by adjusting the installation position, to ensure their normal operation, they must have good weather, working environment, lighting and other conditions. However, in actual operations, weather conditions such as rain, snow and fog are inevitable, and good lighting conditions cannot be fully met at night. In addition, in a high dust working environment, dust covering the optical sensor will cause it to completely lose its perception ability. Summary of the invention

[0004] The object of the present invention is to provide a method, device, medium and product for detecting the edge of a pile of materials, which can improve the accuracy of detecting the edge of a pile of materials.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for detecting a pile edge, comprising:

[0007] Acquire the three-dimensional point cloud data to be measured of the cutting surface of the current stockpile; the three-dimensional point cloud data to be measured is obtained by a 4D millimeter wave radar installed on the side of the reclaimer;

[0008] The three-dimensional point cloud data to be measured is converted from the millimeter wave radar coordinate system to the dam foundation yard coordinate system to obtain the three-dimensional point cloud data in the dam foundation yard coordinate system;

[0009] Performing through filtering or statistical filtering on the three-dimensional point cloud data in the millimeter-wave radar coordinate system and the three-dimensional point cloud data in the dam foundation stockpile coordinate system to obtain target three-dimensional point cloud data; the target three-dimensional point cloud data is the point cloud from which noise points, outliers and non-stockpile areas have been removed in the millimeter-wave radar coordinate system;

[0010] The target three-dimensional point cloud data is divided into sample point clouds according to frame sequence, the features of the sample point clouds are extracted, and the features of the sample point clouds are input into a pile edge detection model to obtain a pile edge detection result; the pile edge detection model is a data set made based on multiple historical three-dimensional point cloud data of the pile cutting surface, trained using a fuzzy twin support vector machine, and obtained by optimizing model parameter selection using a differential evolution algorithm.

[0011] Optionally, the millimeter wave radar coordinate system takes the midpoint of the 4D millimeter wave radar as the origin and the center axis direction of the reclaimer arm as the X L Axis; the vertical direction of the central axis of the reclaimer arm is Y L Axis; perpendicular to X L O L Y L The direction of the plane is Z L axis.

[0012] Optionally, the three-dimensional point cloud data to be measured is converted from the millimeter wave radar coordinate system to the dam foundation yard coordinate system to obtain the three-dimensional point cloud data in the dam foundation yard coordinate system, including:

[0013] Determine any point in the three-dimensional point cloud data to be measured as the current point;

[0014] Determine the coordinates of the current point in the reclaimer rotation center coordinate system according to the millimeter wave radar coordinate system-reclaimer rotation center coordinate system rotation matrix and the millimeter wave radar coordinate system-reclaimer rotation center coordinate system translation vector, as well as the coordinates of the current point in the millimeter wave radar coordinate system;

[0015] Determine the ordinate of the current point in the reclaimer rotation center coordinate system as the ordinate of the current point in the dam foundation yard coordinate system;

[0016] Determine the sum of the distance the reclaimer travels along the dam foundation and the horizontal coordinate of the current point in the reclaimer rotation center coordinate system as the horizontal coordinate of the current point in the dam foundation yard coordinate system;

[0017] Traverse the three-dimensional point cloud data to be measured and obtain all the three-dimensional point cloud data in the dam foundation yard coordinate system.

[0018] Optionally, before obtaining the three-dimensional point cloud data to be measured of the cutting surface of the current stockpile, the following steps are also included:

[0019] Acquire multiple historical 3D point cloud data of the cutting surface of the stockpile;

[0020] The historical three-dimensional point cloud data is converted from the millimeter wave radar coordinate system to the dam foundation yard coordinate system to obtain the historical three-dimensional point cloud data in the dam foundation yard coordinate system;

[0021] Performing through filtering or statistical filtering on the historical three-dimensional point cloud data in the millimeter-wave radar coordinate system and the historical three-dimensional point cloud data in the dam foundation yard coordinate system to obtain historical filtered three-dimensional point cloud data in the millimeter-wave radar coordinate system;

[0022] The historical filtered three-dimensional point cloud data is divided into a plurality of point cloud samples with a preset number of frames as a step length, the features of the point cloud samples are extracted as data set samples, and the sample categories of the data set samples are used as labels to obtain a data set; the sample categories are edge samples and in-pile samples; the edge samples in the data set are positive samples, and the in-pile samples are negative samples;

[0023] With data set samples as input and labels as output, the differential evolution algorithm is used to optimize the parameters of the fuzzy twin support vector machine to obtain the stockpile edge detection model.

[0024] A computer device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned material pile edge detection method.

[0025] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned method for detecting the edge of a pile of materials.

[0026] A computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned method for detecting the edge of a pile of materials.

[0027] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0028] The present invention provides a method, device, medium and product for detecting the edge of a pile of materials. The method uses a 4D millimeter-wave radar to obtain point cloud data of the edge area and non-edge area of ​​the pile of materials; obtains the coordinates of the point cloud in different spatial coordinate systems by rotation and translation; then performs direct and statistical filtering on the point cloud in the corresponding coordinate system to remove noise points, outliers and point clouds in non-pile areas; manually selects features, creates a data set, and uses a custom fuzzy membership function to calculate the membership value of each sample point; iteratively completes the selection and optimization of fuzzy twin support vector machine (FTSVM) parameters through a differential evolution algorithm (DE) to obtain a highly reliable model; finally, inputs the target sample data into a support vector machine classifier after parameter optimization to complete the pile of materials edge detection and discrimination. In addition, during the reclaimer's rotation and reclaiming process, the three-dimensional morphology at the edge of the pile has obvious morphological differences from that at the non-edge, and the 4D millimeter-wave radar sensor can stably and normally obtain the three-dimensional spatial information of the target object in complex on-site operating environments and severe weather conditions. Therefore, the 4D millimeter-wave radar sensor is used to collect the morphological point clouds of the edge and non-edge areas of the pile, and the improved algorithm is used to improve the classification accuracy, thereby achieving the pile edge detection task. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 A schematic flow chart of a method for detecting a material pile edge provided in Example 1 of the present invention;

[0031] Figure 2 It is a first principle diagram of a pile edge detection method according to the present invention;

[0032] Figure 3 It is a second principle diagram of a pile edge detection method according to the present invention;

[0033] Figure 4 It is a side view schematic diagram of the installation position of the millimeter wave radar, the millimeter wave radar coordinate system, and the rotation center coordinate system in the method of the present invention;

[0034] Figure 5 It is a top view schematic diagram of the millimeter wave radar coordinate system, the rotation center coordinate system, and the dam foundation yard coordinate system in the method of the present invention;

[0035] Figure 6 It is a schematic diagram for determining the demarcation angle;

[0036] Figure 7This is a fitness value change diagram of optimizing the parameters of the classification model using the differential evolution algorithm on data 1 according to an embodiment of the present invention;

[0037] Figure 8 This is a fitness value change diagram of optimizing classification model parameters using a differential evolution algorithm on data 2 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The object of the present invention is to provide a method, device, medium and product for detecting the edge of a pile of materials, which can improve the accuracy of detecting the edge of a pile of materials.

[0040] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] Example 1

[0042] like Figure 1 As shown, a pile edge detection method in this embodiment includes:

[0043] Step 101: Obtain the three-dimensional point cloud data to be measured of the cutting surface of the current stockpile. The three-dimensional point cloud data to be measured is obtained by a 4D millimeter wave radar installed on the side of the reclaimer.

[0044] Step 102: convert the three-dimensional point cloud data to be measured from the millimeter wave radar coordinate system to the dam foundation yard coordinate system to obtain the three-dimensional point cloud data in the dam foundation yard coordinate system.

[0045] Step 103: Perform through-filtering or statistical filtering on the three-dimensional point cloud data in the millimeter-wave radar coordinate system and the three-dimensional point cloud data in the dam foundation stockyard coordinate system to obtain the target three-dimensional point cloud data. The target three-dimensional point cloud data is the point cloud in the millimeter-wave radar coordinate system that has been freed of noise points, outliers, and non-stockpile areas. The filtered point clouds are point clouds of the same batch. Steps 101 and 102 respectively obtain the coordinates in the millimeter-wave radar coordinate system and the dam foundation stockyard coordinate system. The millimeter-wave radar coordinate system uses through-filtering and statistical filtering methods to remove points that do not meet the requirements. The remaining points meet the requirements in the radar coordinate system. When converted to the stockyard coordinate system, some points that do not meet the requirements may appear again. For example, the stockyard coordinate system uses through-filtering to remove points in the non-stockpile area. After the final filtering, the final available point cloud is obtained. The three-dimensional coordinates used by the available point cloud are in the millimeter-wave radar coordinate system. Therefore, the temporary conversion to the stockyard coordinate system is to filter out points that do not meet the requirements in the stockyard. These points cannot be seen in the millimeter-wave radar coordinate system.

[0046] Step 104: Divide the target 3D point cloud data into sample point clouds according to the frame sequence, extract the features of the sample point clouds, and input the features of the sample point clouds into the stockpile edge detection model to obtain the stockpile edge detection result. The stockpile edge detection model is a data set made from multiple historical 3D point cloud data of the stockpile cutting surface, trained using a fuzzy twin support vector machine, and optimized using a differential evolution algorithm to select model parameters.

[0047] The millimeter wave radar coordinate system takes the midpoint of the 4D millimeter wave radar as the origin and the central axis direction of the reclaimer arm as the X L The vertical direction of the central axis of the reclaimer arm is Y L Axis. Perpendicular to X L O L Y L The direction of the plane is Z L axis.

[0048] Step 102 includes:

[0049] Step 102 - 1 : Determine any point in the three-dimensional point cloud data to be measured as the current point.

[0050] Step 102-2: Determine the coordinates of the current point in the reclaimer rotation center coordinate system based on the millimeter wave radar coordinate system-reclaimer rotation center coordinate system rotation matrix and the millimeter wave radar coordinate system-reclaimer rotation center coordinate system translation vector, as well as the coordinates of the current point in the millimeter wave radar coordinate system.

[0051] Step 102-3: Determine the ordinate of the current point in the reclaimer rotation center coordinate system as the ordinate of the current point in the dam foundation yard coordinate system.

[0052] Step 102-4: Determine the sum of the distance traveled by the reclaimer along the dam foundation and the horizontal coordinate of the current point in the reclaimer rotation center coordinate system as the horizontal coordinate of the current point in the dam foundation yard coordinate system. Here, the horizontal and vertical coordinates refer to the coordinates on the X-axis and Y-axis in the corresponding coordinate system.

[0053] Step 102-5: traverse the three-dimensional point cloud data to be measured to obtain all three-dimensional point cloud data in the dam foundation yard coordinate system.

[0054] Before step 101, the method further includes:

[0055] Step 105: Acquire a plurality of historical three-dimensional point cloud data of the cutting surface of the stockpile.

[0056] Step 106: convert the historical three-dimensional point cloud data from the millimeter wave radar coordinate system to the dam foundation yard coordinate system to obtain the historical three-dimensional point cloud data in the dam foundation yard coordinate system.

[0057] Step 107: performing direct filtering or statistical filtering on the historical three-dimensional point cloud data in the millimeter wave radar coordinate system and the historical three-dimensional point cloud data in the dam foundation yard coordinate system to obtain the historical filtered three-dimensional point cloud data in the millimeter wave radar coordinate system.

[0058] Step 108: Divide the historical filtered 3D point cloud data into multiple point cloud samples with a preset number of frames as the step size, extract the features of the point cloud samples to make data set samples for training, and mark the positive and negative sample categories. The sample categories are edge samples and heap samples. In the data set, edge samples are positive samples, and heap samples are negative samples.

[0059] Step 109: Using the data set samples as input and the labels as output, a differential evolution algorithm is used to optimize the parameter selection of the fuzzy twin support vector machine to obtain a stockpile edge detection model.

[0060] A method for detecting the edge of a pile of materials in this embodiment is described in detail below. Figure 2 and Figure 3 , this embodiment includes:

[0061] Step 1: During the reclaimer's rotation and reclaiming process, use the 4D millimeter-wave radar installed on the left side of the reclaimer's bucket wheel to obtain the three-dimensional point cloud data of the current material pile cutting surface.

[0062] In this step, the basic coordinate system is established as follows Figure 4 As shown in the figure, the 4D millimeter wave radar is installed on the left side of the boom platform of the reclaimer. The coordinate system X L The axis is parallel to the center axis of the reclaimer arm and points to the far end of the pile. L The axis is perpendicular to the central axis of the reclaimer arm and points to the left. The origin of the millimeter wave radar coordinate system is defined as O LThe reclaimer rotation center coordinate system takes the initial rotation center of the reclaimer as its origin and is defined as e , X e The axis is parallel to the dam foundation track and points to the far end. e The axis direction is always perpendicular to the ground and points upward. The dam foundation yard coordinate system takes the reclaimer's rotation center at the starting point of the dam foundation track as the origin. The X-axis is parallel to the dam foundation track direction and points to the far end. The Z-axis direction is always perpendicular to the ground and points upward. During the rotation process, the millimeter wave radar scanning area passes through the edge of the pile at least once and includes the non-pile area. At the same time, the encoder is used to obtain the current pitch angle and rotation angle of the reclaimer.

[0063] Step 2: Perform coordinate system conversion on the three-dimensional space point cloud data in the millimeter wave radar coordinate system obtained in step 1, and convert it to the dam foundation yard coordinate system O-XYZ.

[0064] In order to obtain the point cloud in the effective yard area between the adjacent parallel dam foundation tracks, the point cloud based on the radar coordinate system obtained by the 4D millimeter wave radar is converted to the dam foundation yard coordinate system O-XYZ. The relevant schematic coordinate system in this step is as follows Figure 5 shown.

[0065] The pitch angle of the reclaimer arm is obtained through the encoder. When the arm is raised, the pitch angle is positive, and when the arm is lowered, the pitch angle is negative. L The axis rotates, and its rotation angle is the pitch angle of the reclaimer arm, which finally realizes the X-axis of the millimeter wave radar coordinate system. L The axis remains parallel to the horizontal ground. The coordinate system rotation formula is expressed as:

[0066]

[0067] Where θ is the pitch angle of the reclaimer arm obtained by the encoder, [x r y r z r ] Τ With [x L y L z L ] Τ They are the three-dimensional spatial coordinates of the point cloud of the millimeter-wave radar coordinate system before and after pitch angle correction.

[0068] When the reclaimer arm is parallel to the ground, measure the translation distance from the origin of the millimeter-wave radar coordinate system to the origin of the reclaimer rotation center coordinate system. Due to the existence of the arm pitch angle, the actual translation vector needs to be calculated. The reclaimer arm rotation angle is obtained through the encoder, and the point cloud in the millimeter-wave radar coordinate system after pitch angle correction is rotated and translated to the reclaimer rotation center coordinate system to obtain the point cloud in the coordinate system O e -Xe Y e Z e The translation and rotation calculation process of the coordinate system of the three-dimensional point cloud data can be expressed as:

[0069]

[0070]

[0071] Where α is the rotation angle of the reclaimer arm. a ,y a 、z a They represent the translation distances in each coordinate axis direction from the origin of the millimeter wave radar coordinate system to the origin of the reclaimer rotation center coordinate system under the condition that the reclaimer arm is parallel to the ground, corresponding to x d ,y d 、z d It is the translation distance in the coordinate axis direction from the origin of the millimeter wave radar coordinate system to the origin of the reclaimer rotation center coordinate system when the pitch angle of the reclaimer arm is not zero. e y e z e ] Τ is the coordinate position of the point cloud in the rotation center coordinate system.

[0072] The encoder is used to obtain the travel distance of the reclaimer on the dam foundation, and the millimeter-wave radar point cloud is converted into the dam foundation yard coordinate system.

[0073]

[0074] where x b The distance the reclaimer travels along the dam foundation obtained by the encoder.

[0075] Step 3: In the radar point cloud frames corresponding to the sample points, use straight-through filtering and statistical filtering to remove noise points, outliers, and point clouds in non-stockpile areas in the 3D point cloud data. Extract features to divide the point cloud into positive and negative samples, and create sample training and test data sets.

[0076] For the point cloud in the radar coordinate system obtained in step 1, according to the X L , Y L , Z LThe distance coordinates of the axis, radar reflection cross-section RCS, and false alarm probability are selected in an appropriate numerical range for direct filtering. In the radar coordinate system, a certain number of point cloud frames are selected, and the average distance between points in the point cloud of these frames is calculated. For a specific three-dimensional point, if the average distance from other points exceeds the average distance between any points, the point is removed, and the point cloud statistical filtering is completed. For the point cloud obtained in step 2 that has been converted from the radar coordinate system to the yard coordinate system, the point cloud in the non-yard area is filtered out according to the limiting conditions on the spatial distance. After filtering, artificial point cloud feature extraction is performed in the millimeter-wave radar coordinate system. The basic features of the point cloud are shown in Table 1.

[0077] Table 1 Point cloud features and their meanings

[0078]

[0079]

[0080] Each sample point uses 20 frames of point cloud data as the sample point cloud to extract features, and according to the frame sequence, the ring buffer data is updated every 5 frames to obtain a new sample point.

[0081] In order to eliminate the adverse effects caused by the large scale differences between the dimensions of the features, Z-Score normalization is used to normalize each dimension of the sample features to a certain range. The normalization formula is:

[0082]

[0083] In the formula, x mean , σ X are the mean and standard deviation of the sample set respectively.

[0084] For the classification of positive and negative samples, the sample categories are marked with the specific rotation angle of the reclaimer arm as the dividing line. Figure 6 As shown in FIG. 1 , the distance d1 from the 4D millimeter-wave radar along the direction of the boom to the front edge of the bucket wheel is expressed as: d1=r+d0, where r is the radius of the bucket wheel and d0 is the distance from the 4D millimeter-wave radar along the direction parallel to the boom to the rotation center of the bucket wheel. Since the horizontal field of view angle range of the 4D millimeter-wave radar used in this embodiment is -50° to +50°, and in the millimeter-wave radar coordinate system, the horizontal field of view plane is parallel to the X L O L Y L Plane parallel, valid point cloud is taken from Y L axis positive half axis, so when the longitudinal distance is d1, the millimeter wave radar is at Y L The maximum detection distance on the positive half axis is d′2=d1tan(50°). LIn the direction of the Y axis, the distance between the bucket wheel and the farthest detection point is d2 = d′2 + d′2′, where d′2′ is the distance between the bucket wheel and the farthest detection point. L The distance from the origin of the millimeter-wave radar in the axial direction. d2 is much smaller than the rotation radius. To simplify the calculation, d2 can be approximately equal to the rotation circumference of the edge area. At this time, the rotation angle of the reclaimer in the edge area is calculated to be β = (d2 / 2πr e )×360°,r e is the rotation radius of the reclaimer arm. At this point, the cutoff rotation angle β for data set classification and labeling is determined. Within the range of the rotation cutoff angle neighboring β, all samples are marked as edge samples, and the rest are samples in the pile. Finally, the training and test data sets are divided according to the ratio.

[0085] Step 4: Use the custom fuzzy membership function to calculate the fuzzy membership value of each sample point in the positive and negative samples.

[0086] In order to reduce the influence of factors such as sample distribution, imbalance of positive and negative samples on classification accuracy, membership values ​​are assigned to sample points through corresponding membership functions to further distinguish the importance of different samples. However, most traditional membership function constructions are designed based on the spatial distance between centers of the same category, with high membership values ​​assigned to those close to the category center and low membership values ​​assigned to those far from the category center. When there are irregular data with discrete block-like density distribution, this membership design method often cannot reflect the actual data distribution well. To this end, the present invention proposes a new fuzzy membership function.

[0087] First, the Gaussian probability density function is used to describe the degree of outlier of a sample in the training set relative to the positive and negative samples. For example, any sample point in the positive sample can be described as a Gaussian probability distribution, the mean of which is the eigenvector of the sample point, and its covariance matrix is ​​obtained from all positive samples. Take a sample x as an example m , within the range of k nearest neighbors, the degree of outliers relative to positive and negative samples can be expressed as:

[0088]

[0089]

[0090] where d m+ ,d m- are the measure of the density of positive and negative samples within the k nearest neighbors of a sample point of a specific class, N(x m |x m ,Σ + ) is the eigenvector x m is the Gaussian distribution with mean vector at x m The probability density function value at N(x m|x i ,Σ + ) is the eigenvector x i is the Gaussian distribution with mean vector at x m The probability density function value at Σ + ,Σ - is the covariance matrix of positive samples and negative samples. Specifically, the number of neighbor points k=20.

[0091] For a sample x m , the fuzzy membership function is defined as:

[0092]

[0093] Where D + , D - are the positive sample set and the negative sample set respectively, and ε is a small constant to prevent the denominator from being 0.

[0094] Step 5: Train the classification model and use the differential evolution algorithm (DE) to optimize the parameter selection of the fuzzy twin support vector machine (FTSVM).

[0095] FTSVM is trained using the training data set obtained in the previous steps to obtain a support vector machine model for point cloud edge morphology classification. FTSVM uses two non-parallel hyperplanes to separate data in high-dimensional space. In the case of nonlinear classification, the quadratic programming problem can be expressed as follows:

[0096]

[0097] In the formula, ω + ,ω - are the normal vectors of the positive and negative splitting hyperplanes, respectively. + , b - are the biases of the positive and negative segmentation hyperplanes respectively. K(·) is the kernel function. C1=C3, C2=C4 are the coefficient parameters to be set. + 、s - are the fuzzy membership values ​​of positive and negative samples respectively. + 、e - is a unit vector. + , - is the slack variable. + , X - , X are positive samples, negative samples, and the union of positive and negative samples respectively.

[0098] Solve the quadratic programming problem to obtain the decision function:

[0099]

[0100] In the present invention, the point cloud morphology classification problem is a nonlinear problem, and it is necessary to map the original feature vector to a high-dimensional space through a nonlinear transformation, and perform linear classification in the high-dimensional space. Specifically, in this embodiment, the support vector machine kernel function K(·) uses a radial basis kernel function to complete feature mapping, and its expression is:

[0101]

[0102] In the formula, x i 、x j are two eigenvectors, σ k is the kernel parameter.

[0103] In view of the fact that the FTSVM model parameter selection is random and the actual operation performance is greatly affected by the parameter selection, the present invention uses the differential evolution algorithm to optimize the support vector machine parameters to obtain good model parameters, and finally obtains better point cloud classification performance and realizes the detection of the edge of the pile. In the support vector machine, C1, C2, C3, and C4 are the coefficient parameters to be set, which can control the complexity of the model and balance the generalization ability and fitting accuracy of the model. k As a kernel parameter, it determines the distribution of data after being mapped to the new feature space, which in turn affects the selection of support vectors.

[0104] Apply differential evolution algorithm to obtain the optimal parameter σ k , C1, C2 steps:

[0105] Step 1: Set the population size, individual vector dimension, upper and lower bounds of each dimension, and maximum number of iterations of the differential evolution algorithm. The initialization process of the initial population individuals is:

[0106]

[0107] p i,j represents the i-th individual in the population, j represents the j-th component in the parameter vector, Indicates the lower bound of the jth parameter component. Specifically, set the population size v_size = 50. The maximum number of iterations iter_num = 100. Set the solution dimension v_dim = 3 according to the number of optimal parameters. Set the initial upper bound of the parameter ub = [2 1 ,2 4 ,2 4 ], the initial lower bound lb=[2 -1 ,2 -4 ,2 -4 ].

[0108] Step 2: Perform mutation operation on the individuals in the population to obtain the mutant individuals. Suppose the i-th mutant individual of the g-th generation is:

[0109]

[0110] In the formula, p rand1 、p rand2 、p rand3 are different individuals randomly selected from the population, F is the variation factor, and Each dimension should satisfy its upper and lower bound conditions.

[0111] Step 3: Use each individual in the population and its offspring mutant individuals to perform a crossover operation to ensure that at least one dimension of the new generation of individuals after the crossover operation is provided by the mutant individual. For the j component of individual i, the crossover formula is expressed as:

[0112]

[0113] In the formula, j rand is a randomly selected dimension.

[0114] Step 4: Each individual in the population corresponds to a set of support vector machine parameters, and the data is trained under this parameter set. In the differential evolution algorithm, the fitness function value is taken from the mean of the geometric mean (G-mean) obtained by N-fold cross validation of the model on the training set. The next generation of individuals is obtained through selection operations, and its expression is:

[0115]

[0116] In the formula, f i (·) is the fitness function value of the ith individual. In this embodiment, considering that there may be different degrees of imbalance between the number of positive and negative samples in the actual data set, and the total number of samples in the data set is small, an N-fold cross-validation evaluation model is used, and the mean of the geometric mean (G-mean) is used as the fitness function. The classification accuracy of positive and negative samples is comprehensively considered, and the final individual fitness function expression is as follows:

[0117]

[0118] In the formula, TP represents the number of samples that are actually positive and predicted as positive. FN represents the number of samples that are actually positive and predicted as positive. TN represents the number of samples that are actually negative and predicted as negative. FP represents the number of samples that are actually negative and predicted as positive. N is the number of cross-validation folds, specifically, N=5.

[0119] Step5: Return to Step2 and iterate until the maximum number of iterations is reached.

[0120] Figure 7 , Figure 8 To show the changing trend of fitness values ​​in the process of optimizing model parameters using differential evolution algorithm on two sets of data.

[0121] Step 6: Input the target sample data into the support vector machine classifier after parameter optimization to complete the pile edge detection.

[0122] Specifically, in this embodiment, the target sample data to be classified and identified is obtained from a test set.

[0123] All experimental procedures in this embodiment are completed in Matlab2022b. The training set, test set, improved method, implementation process, etc. are completed according to the above steps. This embodiment uses 2 sets of rotation sample data, which are labeled 1 and 2 respectively. In the embodiment, four classification models are compared, among which SVM and FTSVM cen The model uses default parameters. cen -DE and FTSVM pdf -DE parameters are optimized using DE algorithm. FTSVM cen and FTSVM pdf The models respectively use the fuzzy membership function based on the category center and the fuzzy membership function based on the probability density function proposed by the present invention. Among them, data 1 has 690 training samples and 174 test samples. Data 2 has 605 training samples and 153 test samples. The classification accuracy and G-mean results are all taken from the test set. It can be seen from the experimental result comparison table that the support vector machine using the fuzzy membership function proposed by the present invention, under the optimization of the DE algorithm, compares different model parameter setting methods and fuzzy membership functions. On data 1, compared with the first three classification models, the classification accuracy is increased by 7.47%, 14.94%, and 1.72%, respectively, and G-mean is increased by 0.1274, 0.2416, and 0.012, respectively. On data 2, compared with the first three classification models, the classification accuracy rates are increased by 5.23%, 18.96%, and 1.31%, respectively, and the G-mean is increased by 0.0681, 0.2789, and 0.0166, respectively, which verifies the effectiveness of the method of the present invention. The comparison results are shown in Table 2.

[0124] Table 2 Comparison of results of the method of the present invention

[0125]

[0126] Example 2

[0127] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a pile edge detection method in Example 1.

[0128] Example 3

[0129] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a pile edge detection method in Example 1.

[0130] Example 4

[0131] A computer program product includes a computer program, and when the computer program is executed by a processor, the method for detecting a pile edge in embodiment 1 is implemented.

[0132] Example 5

[0133] A computer device, which may be a database. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pending transactions. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a pile edge detection method in Example 1 is implemented.

[0134] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0135] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present invention may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present invention may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0136] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for detecting the edge of a pile, characterized in that: include: Obtain the three-dimensional point cloud data to be measured on the cutting surface of the current stockpile; The three-dimensional point cloud data to be measured is obtained by a 4D millimeter wave radar installed on the side of the reclaimer; The three-dimensional point cloud data to be measured is converted from the millimeter wave radar coordinate system to the dam foundation yard coordinate system to obtain the three-dimensional point cloud data in the dam foundation yard coordinate system; Performing through filtering or statistical filtering on the three-dimensional point cloud data in the millimeter wave radar coordinate system and the three-dimensional point cloud data in the dam foundation yard coordinate system to obtain target three-dimensional point cloud data; The target three-dimensional point cloud data is a point cloud from which noise points, outliers and non-stockpile areas have been removed in a millimeter-wave radar coordinate system; The target three-dimensional point cloud data is divided into sample point clouds according to frame sequence, the features of the sample point clouds are extracted, and the features of the sample point clouds are input into a pile edge detection model to obtain a pile edge detection result; the pile edge detection model is a data set made based on multiple historical three-dimensional point cloud data of the pile cutting surface, trained using a fuzzy twin support vector machine, and obtained by optimizing model parameter selection using a differential evolution algorithm.

2. A method for detecting the edge of a pile of materials according to claim 1, characterized in that: The millimeter wave radar coordinate system takes the midpoint of the 4D millimeter wave radar as the origin and the central axis direction of the reclaimer arm as the X L Axis; the vertical direction of the central axis of the reclaimer arm is Y L Axis; perpendicular to X L O L Y L The direction of the plane is Z L axis.

3. A method for detecting the edge of a pile of materials according to claim 1, characterized in that: The three-dimensional point cloud data to be measured is converted from the millimeter wave radar coordinate system to the dam foundation yard coordinate system to obtain the three-dimensional point cloud data in the dam foundation yard coordinate system, including: Determine any point in the three-dimensional point cloud data to be measured as the current point; Determine the coordinates of the current point in the reclaimer rotation center coordinate system according to the millimeter wave radar coordinate system-reclaimer rotation center coordinate system rotation matrix and the millimeter wave radar coordinate system-reclaimer rotation center coordinate system translation vector, as well as the coordinates of the current point in the millimeter wave radar coordinate system; Determine the ordinate of the current point in the reclaimer rotation center coordinate system as the ordinate of the current point in the dam foundation yard coordinate system; Determine the sum of the distance the reclaimer travels along the dam foundation and the horizontal coordinate of the current point in the reclaimer rotation center coordinate system as the horizontal coordinate of the current point in the dam foundation yard coordinate system; Traverse the three-dimensional point cloud data to be measured and obtain all the three-dimensional point cloud data in the dam foundation yard coordinate system.

4. A method for detecting the edge of a pile of materials according to claim 1, characterized in that: Before obtaining the three-dimensional point cloud data to be measured on the cutting surface of the current stockpile, it also includes: Acquire multiple historical 3D point cloud data of the cutting surface of the stockpile; The historical three-dimensional point cloud data is converted from the millimeter wave radar coordinate system to the dam foundation yard coordinate system to obtain the historical three-dimensional point cloud data in the dam foundation yard coordinate system; Performing through filtering or statistical filtering on the historical three-dimensional point cloud data in the millimeter-wave radar coordinate system and the historical three-dimensional point cloud data in the dam foundation yard coordinate system to obtain historical filtered three-dimensional point cloud data in the millimeter-wave radar coordinate system; The historical filtered three-dimensional point cloud data is divided into a plurality of point cloud samples with a preset number of frames as a step length, the features of the point cloud samples are extracted as data set samples, and the sample categories of the data set samples are used as labels to obtain a data set; the sample categories are edge samples and in-pile samples; the edge samples in the data set are positive samples, and the in-pile samples are negative samples; With data set samples as input and labels as output, the differential evolution algorithm is used to optimize the parameters of the fuzzy twin support vector machine to obtain the stockpile edge detection model.

5. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a pile edge detection method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a pile edge detection method according to any one of claims 1 to 4 is implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, a pile edge detection method according to any one of claims 1 to 4 is implemented.

Citation Information

Cited By

  • 4D millimeter wave radar intelligent inventory stock bin management method based on electric scanning principle

    CN121353347A

  • Stacking method of bucket-wheel stacker-reclaimer for optimizing storage yard utilization rate

    CN122415616A