A method for three-dimensional reconstruction of a material surface of a mining shovel by introducing a SIREN periodic activation function neural network

Abstract

This invention provides a method for 3D reconstruction of the material surface in electric shovels by introducing a SIREN periodic activation function neural network, belonging to the field of intelligent operation of mining machinery and industrial 3D vision technology. This invention simultaneously collects multimodal data of the material pile, which is preprocessed to form an initial 3D point cloud. Subsequently, using a SIREN network with a periodic activation function, the sparse point cloud is used as input to learn a continuous, differentiable implicit symbolic distance field, faithfully fitting a complex geometric surface of the material surface containing rich high-frequency details. This invention can quickly and faithfully reconstruct a 3D model of the material surface with rich details from sparse and noisy point cloud data collected on-site by electric shovels, directly serving the key problem of optimal excavation trajectory planning, realizing a closed-loop process from perception to planning for electric shovels, and significantly improving the intelligence level and operational efficiency of electric shovel operations.

Description

Technical Field

[0001] This invention belongs to the field of intelligent operation of mining machinery and industrial three-dimensional vision technology, and relates to a method for three-dimensional reconstruction of the material surface of mining electric shovels by introducing a SIREN periodic activation function neural network. Background Technology

[0002] As a core piece of equipment in open-pit mining, the level of intelligence in electric shovels directly affects the mine's production efficiency and safety. In an automated electric shovel system, rapid and high-precision 3D reconstruction of the material pile (i.e., the material surface) is a crucial prerequisite for trajectory planning, bucket capacity control, and energy consumption optimization. However, the harsh operating environment of electric shovels (high dust levels and strong vibrations) often results in noisy, unevenly dense, or even severely missing point cloud data of the material surface collected by sensors such as lidar. This poses a significant challenge to traditional material surface modeling methods.

[0003] SIREN (Sinusoidal Representation Networks) is an implicit neural representation technique that uses a sine function as the activation function. It can represent complex signals (such as 3D shapes and images) as continuous implicit functions, thereby achieving high-fidelity modeling of complex geometric details. Compared with traditional neural networks, SIREN performs better in fitting high-frequency details and accurately calculating spatial derivatives, providing a continuous and high-precision 3D geometric representation foundation for electric shovel surface reconstruction.

[0004] SIREN, through its unique sinusoidal periodic activation function, transforms sparse, noisy LiDAR point cloud data collected by electric shovels into a continuous implicit function (such as a signed distance field). Compared to traditional methods such as polynomial response surface (PRS) or radial basis functions, SIREN can accurately capture high-frequency details such as steep edges and local concavities in the material pile, and is more robust to data gaps and noise. This high-precision reconstruction provides an accurate geometric basis for the trajectory planning of the electric shovel. Combined with its continuous function representation, it facilitates the calculation of dynamic excavation volume and energy consumption optimization through integrated dynamic models. Ultimately, under constraints such as bucket capacity, it achieves more precise and energy-efficient autonomous excavation operations.

[0005] In industrial settings such as metallurgy and mining, 3D reconstruction of material surfaces (e.g., ore piles, furnace charge surfaces) is a crucial step in optimizing operational efficiency and achieving automated control. However, existing material surface reconstruction methods (such as polynomial response surface models, radial basis function interpolation, or traditional triangular meshing techniques) have significant limitations when dealing with sparse, noisy point cloud data collected in industrial settings: First, traditional methods are insufficient in capturing high-frequency geometric features (e.g., steep edges, local concavities and convexities), resulting in overly smooth reconstructed surfaces and severe loss of detail; second, they are highly dependent on data quality and integrity, easily exhibiting distortions such as holes and breaks under point cloud defects or noise interference; third, it is difficult to balance computational efficiency and reconstruction accuracy, making it hard to meet real-time control requirements. Especially for automated operations of heavy equipment such as electric shovels, real-time and accurate perception of the material surface morphology is required to plan the excavation trajectory, but existing technologies cannot achieve high-fidelity reconstruction under harsh working conditions (point cloud defects caused by dust and vibration).

[0006] Mine surface reconstruction has been extensively studied. Liu Yu et al. proposed a method based on semantic segmentation networks to determine semantic masks from information data and to perform cross-modal interaction from images to point clouds (Chinese Invention Patent CN121353264A). However, the core drawback of this method lies in the limitations of feature extraction and alignment processes, and it places extremely high demands on sensor accuracy, computational efficiency, environmental robustness, and architectural flexibility. For irregular excavation faces, it is difficult to achieve accurate 3D reconstruction using this method. This invention overcomes the bottlenecks of traditional methods in detail preservation, noise resistance, and real-time performance by introducing a SIREN periodic activation function neural network to a 3D reconstruction method for mine electric shovels, thus solving the problem of high-precision and high-efficiency surface reconstruction under sparse noisy point clouds. Summary of the Invention

[0007] To address the problems of existing technologies, this invention provides a method for 3D reconstruction of the material surface in mining electric shovels by introducing a SIREN periodic activation function neural network. Specifically, it is a real-time, high-precision reconstruction method for the material surface in electric shovel operations based on a SIREN neural network. This method simultaneously acquires multimodal data of the material pile using a binocular camera and a lidar mounted on the shovel boom. After point cloud denoising and coordinate normalization preprocessing, an initial 3D point cloud is formed. Subsequently, using a SIREN network with a periodic activation function, the sparse point cloud is used as input to learn a continuous, differentiable implicit symbolic distance field. This allows for high-fidelity fitting of the complex geometric surface of the material surface, containing rich high-frequency details. This method can quickly and accurately reconstruct a richly detailed 3D model of the material surface from sparse, noisy point cloud data collected on-site by the electric shovel, directly serving the key problem of optimal excavation trajectory planning. This achieves a closed-loop process from perception to planning for the electric shovel, significantly improving the intelligence level and operational efficiency of electric shovel operations.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: This paper describes a method for reconstructing the 3D shape of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network. Specifically, it is a method for achieving a high-fidelity parameterized representation of the geometric shape of the material surface based on the periodic sinusoidal activation function unique to the SIREN neural network. The method includes the following steps: The first step is environmental perception, which involves collecting geographic information as raw point cloud data. Using binocular cameras and LiDAR, comprehensive geographic information of the excavation site is collected, including topographic and geomorphological environmental data, as raw data point cloud.

[0009] The second step is to preprocess the raw point cloud data collected by the binocular camera and LiDAR. The preprocessing includes point cloud denoising based on statistical filtering and radius filtering methods, resulting in relatively clean point cloud data with significantly reduced noise, preserved key geometric features, and optimized data volume.

[0010] Step 2.1, for each point in the raw point cloud data collected by the binocular camera and LiDAR... Calculate its distance to the nearest The average distance of each neighboring point , The nearest neighboring point is the target point. Each point, regardless of how far away they actually are. For The value of is first determined from empirical values. Start with a value of 50, observe the filtering effect, fine-tune around this value, and determine the optimal value by combining visualization or quantitative indicators.

[0011] (1) in, This means The nth "nearest neighbor" point There are 10 points, of which 10 are points. , traverse the neighboring points.

[0012] Calculate the mean μ and standard deviation σ of the average distance between all points in the raw point cloud data collected by the binocular camera and lidar. (2) Where N represents the total number of data points involved in the "average distance" calculation, and i is the "index variable" of the summation process, used to iterate through each of these N data points.

[0013] Calculate each point in the original data point cloud Average distance to its K nearest neighbors Then seek all The mean μ and standard deviation σ are then satisfied. Points are identified as outliers and removed. Here, μ reflects the average local density of the point cloud, and σ measures the dispersion of the local density. A positive scaling factor is used to adjust the stringency of noise determination.

[0014] This denoising method can effectively filter out random "flying points" in the original data point cloud, but its effectiveness depends on the density of neighboring points. It may not be effective for point clouds with non-uniform distribution, so further denoising is required.

[0015] Step 2.2, after removing outlier noise points in step 2.1, performs local density analysis on each point in the original data point cloud. Count the number of neighborhood points within a sphere centered at r. .like Less than the set threshold Then determine Noise points are removed; if Not less than the set threshold If a point is found to be a neighboring point, then that point is retained as a neighboring point. The set of all retained neighboring points constitutes a point cloud. .

[0016] Calculate the local mean of the number of secondary neighbors and standard deviation As shown in formula (3): (3) The threshold The determination method is as follows, as shown in formula (4): (4) in, This is the local mean of the number of secondary neighbors within the neighborhood; The standard deviation of the number of secondary neighbors within the neighborhood. It is the adjustment coefficient (let it be here) =2). Among them, This is the local mean of the number of secondary neighbors within the neighborhood. The standard deviation of the number of secondary neighbors within the neighborhood. This is an index variable. It represents the index at point... Within the neighborhood of a sphere centered at r = 0.5m, the first... Neighboring points, For the point The total number of points contained within a sphere centered at a radius of r = 0.5m is given by... Each specific point within the neighborhood (i.e., the first) After taking the neighborhood points as the center, draw a sphere with a radius of r = 0.5m. This represents the number of points inside the sphere.

[0017] Step 2.3: Process the point cloud obtained in Step 2.2 after preserving the neighborhood points. Coordinate normalization is performed to map the data to a normalized space Ω, generating clean point cloud data G. This clean point cloud data provides stable and efficient input for the SIREN neural network. The specific steps are as follows: First, the point cloud obtained in step 2.2 after retaining neighborhood points is transformed using translation centering. The center of the network is moved to the origin to eliminate the influence of the absolute position on network training, as shown in formulas (5) and (6): (5) in, To preserve the point cloud of neighboring points The coordinates of the centroid; for The Middle The original coordinates of each point; The coordinates are those of the centroid. for Points.

[0018] Centralized coordinates obtained by translating a point cloud for: (6) in, The point cloud obtained in step 2.2 after preserving neighborhood points No. The coordinates of each point after centering; These are the centered coordinate components.

[0019] Then, the point cloud obtained in step 2.2 after retaining the neighborhood points is calculated using formula (7). The maximum distance from all points to the origin is the maximum norm. The point cloud obtained in step 2.2 after preserving neighborhood points is then processed. Divide all point coordinates by this maximum value to obtain the point cloud after retaining neighborhood points obtained in step 2.2. The entire scale is adjusted to the range [-1, 1], as shown in formula (8): (7) (8) in, Point cloud with maximum norm and neighborhood points preserved. The maximum distance from all points in the set to the origin; for Coordinate components; For the first The normalized coordinates of each point. These are the normalized coordinate components, ranging from [-1, 1].

[0020] In step 2.3, the point cloud obtained in step 2.2 after retaining the neighborhood points... All are scaled to [-1, 1], therefore the canonical space is defined as Ω = Ω represents the gauge space. All coordinate vectors obtained after coordinate normalization... The set is the pure point cloud data G, which not only conforms to the periodicity of the SIREN activation function, but also avoids training difficulties caused by input values ​​that are too large or too small.

[0021] The third step involves training the SIREN network to transform the preprocessed, clean point cloud data from the second step into a continuous, differentiable, and high-fidelity geometric model of the material surface. This invention employs the signed distance function (SDF) as an implicit representation of the scene and leverages the superior characteristics of the SIREN network to learn and parameterize this SDF field. Specifically: Step 3.1: Use the SIREN network model as the activation function to replace the traditional ReLU or TanH to accurately model the high-frequency details of the material surface.

[0022] Step 3.1.1: Forward propagation process (information flows from the input layer to the output layer); For an L-layer SIREN network, the l-th layer is transformed as shown in Equation (9): (9) in, This represents the output vector of the l-th layer. This represents the output vector of the (l-1)th layer. and Let these represent the weight matrix and bias vector of the l-th layer, respectively; It is the frequency scaling factor. For the first layer (input coordinate p), =p. Final output layer That is, prediction .

[0023] Weight of each layer All elements are randomly initialized according to a uniform distribution. This provides a good and stable starting point for subsequent gradient descent optimization, as shown in equation (10): (10) Where n represents the dimension, This is the initial variance. It scales the weights by dividing by the square root of the input dimension n, thus keeping the output variance around 1.

[0024] Step 3.1.2: Initialize the parameters in formula (9) Achieve periodic activation and high-frequency detail fitting effects.

[0025] parameter The frequency of the activation function is controlled. For the first layer (input coordinates), it is usually set to 30. This allows the network to sense high-frequency changes during initialization, laying the foundation for capturing material surface details.

[0026] Step 3.2: Construct a pseudo-SDF-based loss function to supervise the SDF values ​​predicted in step 3.1.

[0027] Since accurate ground truth SDF values ​​cannot be obtained in advance at the excavation site, this embodiment adopts a viewpoint-dependent pseudo-SDF supervision strategy. Using the real-time pose of the electric shovel bucket as a spatial reference frame, and combining it with point cloud normal information, a self-supervised loss function is constructed to guide the SIREN network in learning a physically intuitive signed distance field.

[0028] Step 3.2.1, Spatial Sampling and Nearest Neighbor Distance Calculation Within the canonical space Ω obtained in step 2.3, a set of free query points Q is randomly sampled using a stratified sampling strategy. For each free query point q∈Q, the nearest Euclidean distance d(q,G) to the pure point cloud G is calculated, as shown in formula (11): (11) Where q is any three-dimensional free query point sampled in the canonical space Ω defined in step 2.3, G is the set of clean point clouds output in step 2.3, and p is any three-dimensional point in the point cloud set G. It is the shortest straight-line distance from the free point q to the point cloud G.

[0029] Step 3.2.2, Normal vector estimation based on local covariance matrix The excavator's operating status is introduced as prior knowledge. An RTK-GPS positioning terminal is deployed on the bucket to directly measure the bucket's absolute three-dimensional coordinates in the physical world coordinate system. .

[0030] The real-time acquired position of the electric shovel bucket The three-dimensional coordinates in physical space are used to obtain the point cloud after preserving neighboring points in step 2.2. By applying the same normalization parameters for translation and scaling, we obtain the normalized electric shovel bucket position vector located in the normal space Ω. As shown in formula (12).

[0031] (12) in, This represents the real-time three-dimensional coordinates of the electric shovel bucket in a standard spatial coordinate system. This represents the line-of-sight vector pointing from the free point q towards the bucket.

[0032] Find the nearest point of the 3D free query point q in the point cloud G. and with Using a circle as the center, search for its 20 nearest neighbors to form a local neighborhood set. .

[0033] Calculate the local neighborhood point set center of mass And construct a 3×3 covariance matrix C, as shown in formulas (13) and (14): (13) (14) Where C is a symmetric positive semi-definite matrix describing the degree of dispersion of the point cloud distribution in the local neighborhood. For local neighborhood point set The center of mass.

[0034] The covariance matrix C is decomposed into eigenvalues, and the eigenvector corresponding to the smallest eigenvalue is taken as the original normal vector and normalized, as shown in formula (15): (15) in, It is the eigenvector corresponding to the smallest eigenvalue of matrix C. It is the original unit normal vector without direction correction. Define from the nearest point Pointing to the bucket position The vector is denoted as As shown in formula (16).

[0035] (16) like This indicates that the normal vector points inwards from the material pile. By reversing it so that it always points outwards from the material pile (towards the air), we obtain the oriented unit normal vector. As shown in formula (17).

[0036] (17) in, It is the final output unit normal vector with a consistent outward orientation.

[0037] 3.2.3 Symbol determination and pseudo-truth value construction based on bucket viewpoint Using the normal after orientation With line of sight vector The sign attribute of the free point q is determined by the angle relationship between the two points, as shown in formula (18): (18) Here, sign(q) represents the sign property of the free point q.

[0038] By combining distance and sign, pseudo-true values ​​are generated for supervised training. As shown in formula (19): (19) in It is the pseudo-SDF ground truth constructed from the free point q, which serves as the target output of the SIREN network.

[0039] 3.3 Construction of Composite Loss Function 3.3.1 Constructing the loss of data items Constructing data item loss Measuring network predictions With pseudo-truth value The difference is shown in formula (20): (20) in, In step 3.2.1, the SIREN network is located at the coordinate point. Predicted SDF value; Coordinates The corresponding actual SDF value; It is the set of spatial points sampled during training; This is the total number of sampling points. Minimizing this means making the network's output value as close as possible to the true SDF value, which is fundamental to reconstructing the accurate surface shape.

[0040] Step 3.3.2, apply the Eikonal constraint term. Physical constraints are introduced to ensure that the network learns a truly signed distance function, as shown in equation (21): (twenty one) in, The SIREN network at coordinate points Predicted SDF value; It is the gradient operator, representing the function Regarding spatial coordinates The vector of partial derivatives.

[0041] Step 3.3.3, the loss function adopted. It is the weighted sum of steps 3.3.1 and 3.3.2, as shown in formula (22): (twenty two) In the parameter initialization settings, =1, =0.1; Optimizer uses Adam, learning rate... The training run consists of 1000 epochs. The goal is to achieve a trained network that can generalize from sparse, noisy point clouds to a complete, continuous, and physically reliable high-precision surface model.

[0042] Step 3.3: Use formulas (23) and (24) to calculate the shortest distance from the spatial point cloud to the material surface point cloud using the signed distance function SDF, and define the outer side of the material surface at the excavation site as positive and the inner side as negative.

[0043] (twenty three) (twenty four) in, Represents the point The signed distance value at the location; Represents the point The closest distance to point cloud S; Represents the three-dimensional coordinates of any point in space; This represents the set of point clouds on the material surface obtained after preprocessing. Point The negative of the nearest distance to point cloud S; A signed distance function that can accurately fit the geometry of the material surface A parameterized neural network is a trained SIREN model.

[0044] The fourth step is to convert the implicit field into an explicit mesh; Step 4.1, SDF field prediction: The region occupied by all the clean point cloud data obtained from coordinate normalization in step 2.3 is a normalized space (e.g., [-1, 1]). 3 The process takes place within a cube. This specification space encompasses the entire effective area containing the surface of the material pile. A 256-dimensional space is created within the specification space. 3For each grid point, the SDF value of the clean point cloud data generated in the normalized space after coordinate normalization in step 2.3 is obtained by querying the SIREN network model trained in step 3.3. For each coordinate point (x, y, z), the SIREN network model performs a forward propagation calculation as shown in step 3.1.1 and outputs a scalar value, which serves as the predicted SDF value for that coordinate point. This process is highly parallelized on the GPU. The predicted SDF values ​​obtained from all clean point cloud data are collected and arranged according to their positions in the normalized space, resulting in a discrete, regular three-dimensional scalar field. Each predicted SDF value represents the shortest distance from its corresponding input clean point cloud data to the surface of the material pile. The sign of the predicted SDF value is used to distinguish whether it is inside or outside the surface of the material pile.

[0045] Step 4.2, extract isosurfaces with predicted SDF values ​​of 0: Using the Marching Cubes algorithm, isosurfaces with SDF = 0 are extracted from the SDF field predicted in step 4.1. These isosurfaces are output as a standard triangular mesh model to complete the three-dimensional reconstruction and obtain the triangular mesh model of the material pile surface.

[0046] The beneficial effects of this invention are as follows: The core advantage of this invention in the technical field lies in its fundamental improvement of the accuracy, physical rationality, and engineering practicality of 3D reconstruction of material surfaces in mining electric shovels by introducing a SIREN neural network based on a sinusoidal periodic activation function. Specifically, the infinitely differentiable nature of the SIREN network allows it to accurately fit high-frequency variations in the signal, thus perfectly reproducing the steep edges and local unevenness of the material pile surface during reconstruction, achieving a fidelity far exceeding that of traditional ReLU or TanH activation function networks. Simultaneously, since the derivative of a sine function is still a sine function, this network can not only accurately fit the signed distance function (SDF) field itself but also naturally and accurately fit its derivatives of all orders. This characteristic makes the introduction of physical priors such as Eikonal constraints direct and stable, ensuring that the learned SDF field is geometrically correct, thereby generating a physically reliable material surface model. Furthermore, SIREN learns a continuous implicit function representation, which is equivalent to obtaining an infinite-resolution parameterized expression of the material surface geometry, possessing powerful generalization capabilities and allowing for model querying and reconstruction from any resolution. At the engineering level, this method exhibits excellent robustness to noise and outliers in the original point cloud. Combined with preprocessing filtering strategies, it can robustly generate complete and smooth surfaces from sparse, noisy measured data. The entire process is highly automated, and the forward inference of the SIREN network can be highly parallelized on GPUs, meeting the stringent requirements of processing speed and reliability in mining operations. This lays a solid technical foundation for subsequent precise mining planning, volume measurement, and intelligent operations. Attached Figure Description

[0047] Figure 1 To implement the flowchart; Figure 2 This section compares the SIREN activation function with traditional ReLU and Tanh activation functions. Detailed Implementation

[0048] The present invention will be further described below with reference to specific implementation examples.

[0049] This embodiment provides a method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network. Figure 1 As shown, this is specifically a high-fidelity parameterized representation scheme for the geometry of a material surface based on the periodic sinusoidal activation function unique to the SIREN neural network. The scheme includes the following steps: To achieve the above objectives, the technical solution adopted by the present invention is as follows: This paper describes a method for reconstructing the 3D shape of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network. Specifically, it is a method for achieving a high-fidelity parameterized representation of the geometric shape of the material surface based on the periodic sinusoidal activation function unique to the SIREN neural network. The method includes the following steps: The first step is environmental perception, which involves collecting geographic information as raw point cloud data. Using binocular cameras and LiDAR, comprehensive geographic information of the excavation site is collected, including topographic and geomorphological environmental data, as raw data point cloud.

[0050] The second step is to preprocess the raw point cloud data collected by the binocular camera and LiDAR. The preprocessing includes point cloud denoising based on statistical filtering and radius filtering methods, resulting in relatively clean point cloud data with significantly reduced noise, preserved key geometric features, and optimized data volume.

[0051] Step 2.1, for each point in the raw point cloud data collected by the binocular camera and LiDAR... Calculate its distance to the nearest The average distance of each neighboring point , The nearest neighboring point is the target point. Each point, regardless of how far away they actually are. For The value of is first determined from empirical values. Start with a value of 50, observe the filtering effect, fine-tune around this value, and determine the optimal value by combining visualization or quantitative indicators.

[0052] (1) in, This means The nth "nearest neighbor" point There are 10 points, of which 10 are points. , traverse the neighboring points.

[0053] Calculate the mean μ and standard deviation σ of the average distance between all points in the raw point cloud data collected by the binocular camera and lidar. (2) Where N represents the total number of data points involved in the "average distance" calculation, and i is the "index variable" of the summation process, used to iterate through each of these N data points.

[0054] Calculate each point in the original data point cloud Average distance to its K nearest neighbors Then seek all The mean μ and standard deviation σ are then satisfied. Points are identified as outliers and removed. Here, μ reflects the average local density of the point cloud, and σ measures the dispersion of the local density. A positive scaling factor is used to adjust the stringency of noise determination.

[0055] This denoising method can effectively filter out random "flying points" in the original data point cloud, but its effectiveness depends on the density of neighboring points. It may not be effective for point clouds with non-uniform distribution, so further denoising is required.

[0056] Step 2.2, after removing outlier noise points in step 2.1, performs local density analysis on each point in the original data point cloud. Count the number of neighborhood points within a sphere centered at r. .like Less than the set threshold Then determine Noise points are removed; if Not less than the set threshold If a point is found to be a neighboring point, then that point is retained as a neighboring point. The set of all retained neighboring points constitutes a point cloud. .

[0057] Calculate the local mean of the number of secondary neighbors and standard deviation As shown in formula (3): (3) The threshold The determination method is as follows, as shown in formula (4): (4) in, This is the local mean of the number of secondary neighbors within the neighborhood; The standard deviation of the number of secondary neighbors within the neighborhood. It is the adjustment coefficient (let it be here) =2). Among them, This is the local mean of the number of secondary neighbors within the neighborhood. The standard deviation of the number of secondary neighbors within the neighborhood. This is an index variable. It represents the index at point... Within the neighborhood of a sphere centered at r = 0.5m, the first... Neighboring points, For the point The total number of points contained within a sphere centered at a radius of r = 0.5m is given by... Each specific point within the neighborhood (i.e., the first) After taking the neighborhood points as the center, draw a sphere with a radius of r = 0.5m. This represents the number of points inside the sphere.

[0058] Step 2.3: Process the point cloud obtained in Step 2.2 after preserving the neighborhood points. Coordinate normalization is performed to map the data to a normalized space Ω, generating clean point cloud data G. This clean point cloud data provides stable and efficient input for the SIREN neural network. The specific steps are as follows: First, the point cloud obtained in step 2.2 after retaining neighborhood points is transformed using translation centering. The center of the network is moved to the origin to eliminate the influence of the absolute position on network training, as shown in formulas (5) and (6): (5) in, To preserve the point cloud of neighboring points The coordinates of the centroid; for The Middle The original coordinates of each point; The coordinates are those of the centroid. for Points.

[0059] Centralized coordinates obtained by translating a point cloud for: (6) in, The point cloud obtained in step 2.2 after preserving neighborhood points No. The coordinates of each point after centering; These are the centered coordinate components.

[0060] Then, the point cloud obtained in step 2.2 after retaining the neighborhood points is calculated using formula (7). The maximum distance from all points to the origin is the maximum norm. The point cloud obtained in step 2.2 after preserving neighborhood points is then processed. Divide all point coordinates by this maximum value to obtain the point cloud after retaining neighborhood points obtained in step 2.2. The entire scale is adjusted to the range [-1, 1], as shown in formula (8): (7) (8) in, Point cloud with maximum norm and neighborhood points preserved. The maximum distance from all points in the set to the origin; for Coordinate components; For the first The normalized coordinates of each point. These are the normalized coordinate components, ranging from [-1, 1].

[0061] In step 2.3, the point cloud obtained in step 2.2 after retaining the neighborhood points... All are scaled to [-1, 1], therefore the canonical space is defined as Ω = Ω represents the gauge space. All coordinate vectors obtained after coordinate normalization... The set is the pure point cloud data G, which not only conforms to the periodicity of the SIREN activation function, but also avoids training difficulties caused by input values ​​that are too large or too small.

[0062] The third step involves training the SIREN network to transform the preprocessed, clean point cloud data from the second step into a continuous, differentiable, and high-fidelity geometric model of the material surface. This invention employs the signed distance function (SDF) as an implicit representation of the scene and leverages the superior characteristics of the SIREN network to learn and parameterize this SDF field. Specifically: Step 3.1: Use the SIREN network model as the activation function to replace the traditional ReLU or TanH to accurately model the high-frequency details of the material surface.

[0063] Step 3.1.1: Forward propagation process (information flows from the input layer to the output layer); For an L-layer SIREN network, the l-th layer is transformed as shown in Equation (9): (9) in, This represents the output vector of the l-th layer. This represents the output vector of the (l-1)th layer. and Let these represent the weight matrix and bias vector of the l-th layer, respectively; It is the frequency scaling factor. For the first layer (input coordinate p), =p. Final output layer That is, prediction .

[0064] Weight of each layer All elements are randomly initialized according to a uniform distribution. This provides a good and stable starting point for subsequent gradient descent optimization, as shown in equation (10): (10) Where n represents the dimension, This is the initial variance. It scales the weights by dividing by the square root of the input dimension n, thus keeping the output variance around 1.

[0065] Step 3.1.2: Initialize the parameters in formula (9) Achieve periodic activation and high-frequency detail fitting effects.

[0066] parameter The frequency of the activation function is controlled. For the first layer (input coordinates), it is usually set to 30. This allows the network to sense high-frequency changes during initialization, laying the foundation for capturing material surface details.

[0067] Step 3.2: Construct a pseudo-SDF-based loss function to supervise the SDF values ​​predicted in step 3.1.

[0068] Since accurate ground truth SDF values ​​cannot be obtained in advance at the excavation site, this embodiment adopts a viewpoint-dependent pseudo-SDF supervision strategy. Using the real-time pose of the electric shovel bucket as a spatial reference frame, and combining it with point cloud normal information, a self-supervised loss function is constructed to guide the SIREN network in learning a physically intuitive signed distance field.

[0069] Step 3.2.1, Spatial Sampling and Nearest Neighbor Distance Calculation Within the canonical space Ω obtained in step 2.3, a set of free query points Q is randomly sampled using a stratified sampling strategy. For each free query point q∈Q, the nearest Euclidean distance d(q,G) to the pure point cloud G is calculated, as shown in formula (11): (11) Where q is any three-dimensional free query point sampled in the canonical space Ω defined in step 2.3, G is the set of clean point clouds output in step 2.3, and p is any three-dimensional point in the point cloud set G. It is the shortest straight-line distance from the free point q to the point cloud G.

[0070] Step 3.2.2, Normal vector estimation based on local covariance matrix The excavator's operating status is introduced as prior knowledge. An RTK-GPS positioning terminal is deployed on the bucket to directly measure the bucket's absolute three-dimensional coordinates in the physical world coordinate system. .

[0071] The real-time acquired position of the electric shovel bucket The three-dimensional coordinates in physical space are used to obtain the point cloud after preserving neighboring points in step 2.2. By applying the same normalization parameters for translation and scaling, we obtain the normalized electric shovel bucket position vector located in the normal space Ω. As shown in formula (12).

[0072] (12) in, This represents the real-time three-dimensional coordinates of the electric shovel bucket in a standard spatial coordinate system. This represents the line-of-sight vector pointing from the free point q towards the bucket.

[0073] Find the nearest point of the 3D free query point q in the point cloud G. and with Using a circle as the center, search for its 20 nearest neighbors to form a local neighborhood set. .

[0074] Calculate the local neighborhood point set center of mass And construct a 3×3 covariance matrix C, as shown in formulas (13) and (14): (13) (14) Where C is a symmetric positive semi-definite matrix describing the degree of dispersion of the point cloud distribution in the local neighborhood. For local neighborhood point set The center of mass.

[0075] The covariance matrix C is decomposed into eigenvalues, and the eigenvector corresponding to the smallest eigenvalue is taken as the original normal vector and normalized, as shown in formula (15): (15) in It is the eigenvector corresponding to the smallest eigenvalue of matrix C. It is the original unit normal vector without direction correction. Define from the nearest point Pointing to the bucket position The vector is denoted as As shown in formula (16).

[0076] (16) like This indicates that the normal vector points inwards from the material pile. By reversing it so that it always points outwards from the material pile (towards the air), we obtain the oriented unit normal vector. As shown in formula (17).

[0077] (17) in, It is the final output unit normal vector with a consistent outward orientation.

[0078] 3.2.3 Symbol determination and pseudo-truth value construction based on bucket viewpoint Using the normal after orientation With line of sight vector The sign attribute of the free point q is determined by the angle relationship between the two points, as shown in formula (18): (18) Here, sign(q) represents the sign property of the free point q.

[0079] By combining distance and sign, pseudo-true values ​​are generated for supervised training. As shown in formula (19): (19) in It is the pseudo-SDF ground truth constructed from the free point q, which serves as the target output of the SIREN network.

[0080] 3.3 Construction of Composite Loss Function 3.3.1 Constructing the loss of data items Constructing data item loss Measuring network predictions With pseudo-truth value The difference is shown in formula (20): (20) in, In step 3.2.1, the SIREN network is located at the coordinate point. Predicted SDF value; Coordinates The corresponding actual SDF value; It is the set of spatial points sampled during training; This is the total number of sampling points. Minimizing this means making the network's output value as close as possible to the true SDF value, which is fundamental to reconstructing the accurate surface shape.

[0081] Step 3.3.2, apply the Eikonal constraint term. Physical constraints are introduced to ensure that the network learns a truly signed distance function, as shown in equation (21): (twenty one) in, The SIREN network at coordinate points Predicted SDF value; It is the gradient operator, representing the function Regarding spatial coordinates The vector of partial derivatives.

[0082] Step 3.3.3, the loss function adopted. It is the weighted sum of steps 3.3.1 and 3.3.2, as shown in formula (22): (twenty two) In the parameter initialization settings, =1, =0.1; Optimizer uses Adam, learning rate... The training run consists of 1000 epochs. The goal is to achieve a trained network that can generalize from sparse, noisy point clouds to a complete, continuous, and physically reliable high-precision surface model.

[0083] Step 3.3: Use formulas (23) and (24) to calculate the shortest distance from the spatial point cloud to the material surface point cloud using the signed distance function SDF, and define the outer side of the material surface at the excavation site as positive and the inner side as negative.

[0084] (twenty three) (twenty four) in, Represents the point The signed distance value at the location; Represents the point The closest distance to point cloud S; The three-dimensional coordinates of any point in space; This represents the set of point clouds on the material surface obtained after preprocessing. Point The negative of the nearest distance to point cloud S; A signed distance function that can accurately fit the geometry of the material surface A parameterized neural network is a trained SIREN model.

[0085] The fourth step is to convert the implicit field into an explicit mesh; Step 4.1, SDF field prediction: The region occupied by all the clean point cloud data obtained from coordinate normalization in step 2.3 is a normalized space (e.g., [-1, 1]). 3 The process takes place within a cube. This specification space encompasses the entire effective area containing the surface of the material pile. A 256-dimensional space is created within the specification space. 3For each grid point, the SDF value of the clean point cloud data generated in the normalized space after coordinate normalization in step 2.3 is obtained by querying the SIREN network model trained in step 3.3. For each coordinate point (x, y, z), the SIREN network model performs a forward propagation calculation as shown in step 3.1.1 and outputs a scalar value, which serves as the predicted SDF value for that coordinate point. This process is highly parallelized on the GPU. The predicted SDF values ​​obtained from all clean point cloud data are collected and arranged according to their positions in the normalized space, resulting in a discrete, regular three-dimensional scalar field. Each predicted SDF value represents the shortest distance from its corresponding input clean point cloud data to the surface of the material pile. The sign of the predicted SDF value is used to distinguish whether it is inside or outside the surface of the material pile.

[0086] Step 4.2, extract isosurfaces with predicted SDF values ​​of 0: Using the Marching Cubes algorithm, isosurfaces with SDF = 0 are extracted from the SDF field predicted in step 4.1. These isosurfaces are output as a standard triangular mesh model to complete the three-dimensional reconstruction and obtain the triangular mesh model of the material pile surface.

[0087] The embodiments described above are merely illustrative of the implementation of the invention, but should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements are all within the scope of protection of the present invention.

Claims

1. A method for three-dimensional reconstruction of the material surface in mining electric shovels by introducing a SIREN periodic activation function neural network, characterized in that... The material surface three-dimensional reconstruction method includes the following steps: The first step is environmental perception, which involves collecting geographic information as raw point cloud data. Using binocular cameras and LiDAR, comprehensive geographic information of the excavation site is collected, including topographic and geomorphological environmental data, as raw data point cloud. The second step is to preprocess the raw point cloud data collected by the binocular camera and LiDAR. The preprocessing includes point cloud denoising based on statistical filtering and radius filtering methods to obtain clean point cloud data. The third step involves training the SIREN network to transform the preprocessed, clean point cloud data from the second step into a continuous, differentiable, and high-fidelity geometric model of the material surface. This invention employs the signed distance function (SDF) as an implicit representation of the scene and leverages the superior characteristics of the SIREN network to learn and parameterize this SDF field. Specifically: Step 3.1: Use the SIREN network model as the activation function to replace the traditional ReLU or TanH to accurately model the high-frequency details of the material surface; Step 3.2: Construct a pseudo-SDF-based loss function to supervise the SDF values ​​predicted in step 3.1; Using the real-time pose of the electric shovel bucket as a spatial reference frame, and combining it with point cloud normal information, a self-supervised loss function is constructed to guide the SIREN network to learn a signed distance field that conforms to physical intuition. Step 3.3 Constructing the composite loss function; Step 3.4: Calculate the shortest distance from the spatial point cloud to the material surface point cloud using the signed distance function SDF, and define the outer side of the material surface at the excavation site as positive and the inner side as negative. The fourth step is to convert the implicit field into an explicit mesh; Step 4.1, SDF field prediction: Step 4.2: Extract isosurfaces with predicted SDF values ​​of 0.

2. The method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network according to claim 1, characterized in that, The second step is as follows: Step 2.1, for each point in the raw point cloud data collected by the binocular camera and LiDAR... Calculate its distance to the nearest The average distance of each neighboring point , The nearest neighboring point is the target point. One point; Calculate the mean μ and standard deviation σ of the average distance between all points in the raw point cloud data collected by the binocular camera and lidar. Calculate each point in the original data point cloud Average distance to its K nearest neighbors Then seek all The mean μ and standard deviation σ will satisfy... Points are identified as outliers and removed; where μ reflects the average local density of the point cloud, and σ measures the dispersion of the local density. A positive scaling factor; Step 2.2, after removing outlier noise points in step 2.1, performs local density analysis on each point in the original data point cloud. Count the number of neighborhood points within a sphere centered at r. ;like Less than the set threshold Then determine Noise points are removed; if Not less than the set threshold If a point is selected, it is retained as a neighboring point; the set of all retained neighboring points constitutes a point cloud. ; Step 2.3: Process the point cloud obtained in Step 2.2 after preserving the neighborhood points. The coordinates are normalized and mapped to a normalized space Ω to generate clean point cloud data G; the specific steps are as follows: First, the point cloud obtained in step 2.2 after retaining neighborhood points is transformed using translation centering. The center of the network is moved to the origin to eliminate the influence of the absolute position on network training, as shown in formulas (5) and (6): （5） in, To preserve the point cloud of neighboring points The coordinates of the centroid; for The Middle The original coordinates of each point; The coordinates are those of the centroid. for points; Centralized coordinates obtained by translating a point cloud for: （6） in, The point cloud obtained in step 2.2 after preserving neighborhood points No. The coordinates of each point after centering; These are the centered coordinate components; Then, the point cloud obtained in step 2.2 after retaining the neighborhood points is calculated using formula (7). The maximum distance from all points to the origin is the maximum norm; the point cloud obtained in step 2.2 after retaining neighborhood points is... Divide all point coordinates by this maximum value, and then use the point cloud obtained in step 2.2 after retaining the neighborhood points. The entire range is scaled to [-1, 1], as shown in formula (8): （7） （8） in, Point cloud with maximum norm and neighborhood points preserved. The maximum distance from all points in the set to the origin; for Coordinate components; For the first Normalized coordinates of each point; These are normalized coordinate components, ranging from [-1, 1]. The gauge space is defined as Ω= Ω represents the normalized space; all coordinate vectors obtained after coordinate normalization. The set is the pure point cloud data G.

3. The method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network according to claim 2, characterized in that, In step 2.1: The average distance The formulas for calculating the mean μ and standard deviation σ are as follows: （1） in, This means The nth "nearest neighbor" point There are 10 points, of which 10 are points. traverse the neighboring points; （2） Where N represents the total number of data points participating in the "average distance" calculation, and i is the "index variable" of the summation process, used to iterate through each of these N data points.

4. The method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network according to claim 3, characterized in that, In step 2.2: Calculate the local mean of the number of secondary neighbors and standard deviation As shown in formula (3): （3） The threshold The determination method is as follows, as shown in formula (4): （4） in, This is the local mean of the number of secondary neighbors within the neighborhood; The standard deviation of the number of secondary neighbors within the neighborhood. It is the adjustment coefficient.

5. The method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network according to claim 4, characterized in that, Step 3.1 specifically involves: Step 3.1.1: Forward propagation process, information flows from the input layer to the output layer; For an L-layer SIREN network, the l-th layer is transformed as shown in Equation (9): （9） in, This represents the output vector of the l-th layer. This represents the output vector of the (l-1)th layer. and Let these represent the weight matrix and bias vector of the l-th layer, respectively; It is the frequency scaling factor; for the first layer, i.e., the input coordinate p, =p; final output layer That is, prediction ; Weight of each layer All are randomly initialized according to a uniform distribution; as shown in formula (10): （10） Where n represents the dimension, It initializes the variance; it scales the weights by dividing by the square root of the input dimension n, thus keeping the output variance around 1. Step 3.1.2: Initialize the parameters in formula (9) Achieve periodic activation and high-frequency detail fitting effects.

6. The method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network according to claim 5, characterized in that, Step 3.2 specifically involves: Step 3.2.1, Spatial Sampling and Nearest Neighbor Distance Calculation Within the canonical space Ω obtained in step 2.3, a set of free query points Q is randomly sampled using a stratified sampling strategy; for each free query point q∈Q, the nearest Euclidean distance d(q,G) to the pure point cloud G is calculated, as shown in formula (11): （11） Where q is any three-dimensional free query point sampled in the canonical space Ω defined in step 2.3, G is the set of clean point clouds output in step 2.3, and p is any three-dimensional point in the point cloud set G; It is the shortest straight-line distance from the free point q to the point cloud G; Step 3.2.2, estimation of the normal vector based on the local covariance matrix; The excavator's operating status is introduced as prior knowledge; an RTK-GPS positioning terminal is deployed on the bucket to directly measure the bucket's absolute three-dimensional coordinates in the physical world coordinate system. ; The real-time acquired position of the electric shovel bucket The three-dimensional coordinates in physical space are used to obtain the point cloud after preserving neighboring points in step 2.

2. By applying the same normalization parameters for translation and scaling, we obtain the normalized electric shovel bucket position vector located in the normal space Ω. As shown in formula (12); （12） in, This represents the real-time three-dimensional coordinates of the electric shovel bucket in a standard spatial coordinate system. This represents the line-of-sight vector from the free point q towards the bucket; Find the nearest point of the 3D free query point q in the point cloud G. and with Using a circle as the center, search for its 20 nearest neighbors to form a local neighborhood set. ; Calculate the local neighborhood point set center of mass And construct a 3×3 covariance matrix C, as shown in formulas (13) and (14): （13） （14） Where C is a symmetric positive semi-definite matrix describing the degree of dispersion of the point cloud distribution in the local neighborhood. For local neighborhood point set The center of mass; The covariance matrix C is decomposed into eigenvalues, and the eigenvector corresponding to the smallest eigenvalue is taken as the original normal vector and normalized, as shown in formula (15): （15） in, It is the eigenvector corresponding to the smallest eigenvalue of matrix C; It is the original unit normal vector without direction correction. Define from the nearest point Pointing to the bucket position The vector is denoted as As shown in formula (16); （16） like This indicates that the normal vector points inwards from the material pile. By reversing it so that it always points outwards from the material pile (towards the air), we obtain the oriented unit normal vector. As shown in formula (17); （17） in, It is the final output unit normal vector with a consistent outward orientation; Step 3.2.3 Symbol determination and pseudo-truth value construction based on bucket viewpoint; Using the normal after orientation With line of sight vector The sign attribute of the free point q is determined by the angle relationship between the two points, as shown in formula (18): （18） Where sign(q) represents the sign property of the free point q; By combining distance and sign, pseudo-true values ​​are generated for supervised training. As shown in formula (19): （19） in It is the pseudo-SDF ground truth constructed from the free point q, which serves as the target output of the SIREN network.

7. The method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network according to claim 6, characterized in that, Step 3.3 specifically involves: Step 3.3.1 Construct the data item loss ; Constructing data item loss Measuring network predictions With pseudo-truth value The difference is shown in formula (20): （20） in, In step 3.2.1, the SIREN network is located at the coordinate point. Predicted SDF value; Coordinates The corresponding actual SDF value; It is the set of spatial points sampled during training; This is the total number of sampling points; Step 3.3.2, apply the Eikonal constraint term. Physical constraints are introduced to ensure that the network learns a truly signed distance function, as shown in equation (21): （21） in, The SIREN network at coordinate points Predicted SDF value; It is the gradient operator, representing the function Regarding spatial coordinates The vector of partial derivatives; Step 3.3.3, the loss function adopted. It is the weighted sum of steps 3.3.1 and 3.3.2, as shown in formula (22): (twenty two).

8. The method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network according to claim 7, characterized in that, Step 3.4 specifically involves: Formulas (23) and (24) are used to calculate the shortest distance from the spatial point cloud to the material surface point cloud using the signed distance function SDF, and it is agreed that the outer side of the material surface at the excavation site is positive and the inner side is negative, as shown below: （23） （24） in, Represents the point The signed distance value at the location; Represents the point The closest distance to point cloud S; The three-dimensional coordinates of any point in space; This represents the set of point clouds on the material surface obtained after preprocessing. Point The negative of the nearest distance to point cloud S; A signed distance function that can accurately fit the geometry of the material surface A parameterized neural network is a trained SIREN model.

9. The method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network according to claim 8, characterized in that, Step 4.1 specifically involves: The area occupied by all the clean point cloud data obtained from coordinate normalization in step 2.3 is a normalized space. Multiple grid points are created in the normalized space. The SDF value of each clean point cloud data generated in the normalized space after coordinate normalization in step 2.3 is obtained by querying the SIREN network model trained in step 3.

3. For each coordinate point (x, y, z), the SIREN network model performs the forward propagation calculation shown in step 3.1.1 once and outputs a scalar value, which is used as the predicted SDF value of the coordinate point. The predicted SDF values ​​obtained from all clean point cloud data are collected and arranged according to their positions in the normalized space, resulting in a discrete, regular three-dimensional scalar field. Each predicted SDF value represents the shortest distance from the corresponding input clean point cloud data to the surface of the material pile. The sign of the predicted SDF value is used to distinguish whether it is inside or outside the surface of the material pile.

10. The method for three-dimensional reconstruction of the material surface in a mining electric shovel by introducing a SIREN periodic activation function neural network according to claim 9, characterized in that, Step 4.2 specifically involves: Using the Marching Cubes algorithm, isosurfaces with SDF = 0 are extracted from the SDF field predicted in step 4.

1. These isosurfaces are output as a standard triangular mesh model to complete the three-dimensional reconstruction and obtain the triangular mesh model of the material pile surface.

Images