A photovoltaic pile driver terrain recognition method, system, terminal and storage medium

Abstract

The application relates to a photovoltaic pile driver terrain identification method and system, a terminal and a storage medium, and relates to the technical field of intelligent equipment. The application comprises the following steps: analyzing camera pixel data, laser radar data, a camera internal parameter matrix, a camera radar external parameter matrix and a radar satellite external parameter matrix to determine pixel camera data, radar world data and camera world data; fusing the radar world data and the camera world data to determine simultaneous space fusion data and coordinate point gray values; analyzing the coordinate point gray values and the simultaneous space fusion data to determine spatial terrain features and pile driving terrain classification; and analyzing the radar world data, the camera world data, the spatial terrain features and the pile driving terrain classification to determine real-time terrain modeling. The application has the effects of improving the photovoltaic pile driving terrain identification accuracy and terrain updating efficiency.

Description

Technical Field

[0001] This application relates to the technical field of smart devices, and in particular to a method, system, terminal and storage medium for terrain recognition of photovoltaic piling machines. Background Technology

[0002] Terrain recognition for photovoltaic piling machines refers to the process of identifying complex terrains such as deserts, mountains, and hills in real time, modeling the complex terrain where the piling area is located, and providing terrain data support for photovoltaic piling machines.

[0003] In related technologies, when modeling complex terrain, terrain recognition technology using UAV-borne LiDAR is usually adopted. By generating hundreds of thousands of laser pulses per second, high-density point cloud data is obtained. Finally, the geometric constraints of ground control points are combined with the point cloud data to perform real-time global updates on the terrain model of the piling area.

[0004] Regarding the aforementioned technologies, when identifying the terrain of the piling area using lidar, the timing and spatial angle deviations in the location data from BeiDou positioning, the attitude data from inertial navigation, and the texture data from visual images lead to low accuracy in multi-source data fusion. Furthermore, when performing global updates on complex terrain, the piling operation only causes local terrain changes, and remodeling the global model results in low terrain update efficiency, leaving room for improvement. Summary of the Invention

[0005] To improve the accuracy and efficiency of terrain recognition for photovoltaic piling machines, this application provides a method, system, terminal, and storage medium for terrain recognition of photovoltaic piling machines.

[0006] Firstly, this application provides a method for terrain recognition of photovoltaic piling machines, employing the following technical solution: A method for terrain recognition of photovoltaic piling machines includes: Acquire camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, and radar satellite extrinsic parameter matrix; Analyze camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, and radar satellite extrinsic parameter matrix to determine pixel camera data, radar world data, and camera world data. The radar world data and camera world data are fused to determine the spatiotemporal fused data and the grayscale values ​​of coordinate points. The grayscale values ​​of coordinate points and spatiotemporal fusion data are analyzed to determine spatial terrain features and pile-driven terrain classification. Analyze radar world data, camera world data, spatial terrain features, and pile-driven terrain classification to determine real-time terrain modeling.

[0007] Optionally, the steps of analyzing camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera-radar extrinsic parameter matrix, and radar-satellite extrinsic parameter matrix to determine pixel camera data, radar world data, and camera world data include: The lidar data is converted into radar camera data based on the camera radar extrinsic parameter matrix. The radar camera data is converted into radar pixel data based on the camera intrinsic parameter matrix. Data is extracted from radar pixel data based on camera pixel data to determine pixel depth data; Based on the camera intrinsic parameter matrix, the pixel depth data and camera pixel data are converted into pixel camera data; The pixel camera data is converted into camera radar data based on the camera radar extrinsic parameter matrix. Based on the radar satellite extrinsic matrix, camera radar data and lidar data are converted into camera world data and radar world data.

[0008] Optionally, the steps of fusing radar world data and camera world data to determine the simultaneous spatiotemporal fused data and coordinate point grayscale values ​​include: Obtain the real-time coordinates and world coordinates of the pile driver; Calculate the Euclidean distance between the world coordinate point and the real-time position of the pile driver to determine the coordinate point distance; Input the coordinate point distance into the preset distance weight model to determine the coordinate point distance weight; Data is extracted from the camera's world data to determine the grayscale value of the coordinate point; Input the grayscale value of the coordinate point into the preset grayscale weight model to determine the grayscale weight of the coordinate point; Based on a preset number of neighboring points, data is extracted from the world coordinate points and their grayscale values ​​to determine the neighboring elevation data and the neighboring grayscale data. Input the number of neighborhood points, neighborhood elevation data, and neighborhood grayscale data into the preset texture calculation model to determine the texture balance coefficient; Radar world data and camera world data are fused based on texture balance coefficient, coordinate point grayscale weight, and coordinate point distance weight to determine the spatiotemporal fused data.

[0009] Optionally, the steps of analyzing the grayscale values ​​of coordinate points and spatiotemporally fused data to determine spatial terrain features and piling-up terrain classification include: Simultaneous spatiotemporal fusion data is sampled according to a preset global sampling model to determine global core point data; Global core point data and spatiotemporal fusion data are input into a preset local dynamic sampling model to determine the core point neighborhood data; Calculate the coordinate difference between the neighborhood data of the core point and the global core point data to determine the relative coordinates of the neighborhood; The relative coordinates of the neighborhood and the neighborhood data of the core point are input into the preset local coding model to determine the local coding features; Local encoded features and preset training weight vectors are input into a preset attention aggregation model to determine spatial terrain features; The spatial terrain features, global core point data, core point neighborhood data, and coordinate point gray values ​​are analyzed to determine the terrain classification for piling.

[0010] Optionally, the steps for analyzing spatial terrain features, global core point data, core point neighborhood data, and coordinate point grayscale values ​​to determine the pile driving terrain classification include: Integrate global core point data and core point neighborhood data to determine local point cloud data; Calculate the standard deviation of the grayscale values ​​at coordinate points to determine the texture value of those points. The texture value and gray value of the coordinate point are integrated to determine the global image features; Based on a preset temporal sliding window, data extraction is performed on spatial terrain features, local point cloud data, and global image features to determine temporal terrain feature groups, temporal local feature groups, and temporal image feature groups; The temporal terrain feature set, temporal local feature set, and temporal image feature set are input into a preset temporal extraction model to determine the temporal terrain features; Obtain decision learning datasets; A pre-defined feature decision model is trained based on a decision learning dataset and a pre-defined focus loss function to determine the terrain classification model; Temporal and spatial terrain features are input into the terrain classification model to determine the terrain classification for piling.

[0011] Optionally, the steps for analyzing radar world data, camera world data, spatial terrain features, and staking terrain classification to determine real-time terrain modeling include: The covariance of radar world data is calculated based on preset spatial neighborhood points to determine the coordinate point covariance. Input the coordinate point covariance into the preset decomposed curvature model to determine the principal curvature of the coordinate point; Data analysis is performed on the principal curvatures of the coordinate points to determine the maximum coordinate curvature. The principal curvature of the coordinate points is normalized based on the maximum coordinate curvature to determine the terrain curvature weights. The terrain curvature weights, the preset robust error function, radar world data, and camera world data are input into the preset curvature-weighted ICP model to determine the optimal rotation matrix and the optimal translation vector. The lidar data and camera world data are fused based on the optimal rotation matrix and optimal translation vector to determine the fused point cloud data. We analyze the fused point cloud data, spatial terrain features, and pile-driven terrain classification to determine the real-time terrain modeling.

[0012] Optionally, the steps for real-time terrain modeling include analyzing the fused point cloud data, spatial terrain features, and piling terrain classifications: Data extraction is performed on the fused point cloud data according to the preset judgment time window to determine the judgment point cloud data; The point cloud data is divided according to the preset incremental update grid to determine the grid point cloud data; Calculate the density of the grid point cloud data in the incrementally updated grid to determine the grid point cloud density; Input the grid point cloud density into the preset density change rate model to determine the point cloud density change rate; Data extraction is performed on the grid point cloud data to determine the grid point cloud elevation; Calculate the elevation difference between the grid point cloud elevations to determine the maximum elevation change; The point cloud density change rate and maximum elevation change corresponding to the grid point cloud data are filtered according to the preset update limit, preset density change rate threshold and preset elevation change threshold to determine the grid data to be updated. The fused point cloud data, the grid data to be updated, and the spatial terrain features are input into a preset octree incremental update model to determine the terrain model to be rendered. The pile driving terrain classification and the terrain model to be rendered are input into the preset parallel rendering model to determine the real-time terrain modeling.

[0013] Secondly, this application provides a terrain recognition system for photovoltaic piling machines, which adopts the following technical solution: A terrain recognition system for photovoltaic piling machines, comprising: The acquisition module is used to acquire camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, and radar satellite extrinsic parameter matrix; A memory for storing a program for a photovoltaic piling machine terrain recognition method as described in any of the preceding claims; The processor and the program in the memory can be loaded and executed by the processor to implement a photovoltaic piling machine terrain recognition method as described in any of the above.

[0014] Thirdly, this application provides a smart terminal, which adopts the following technical solution: A smart terminal includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any of the preceding claims, a method for terrain recognition of a photovoltaic piling machine.

[0015] Fourthly, this application provides a computer storage medium capable of storing corresponding programs, which facilitates improving the accuracy of terrain recognition and the efficiency of terrain updating for photovoltaic piling machines, and adopts the following technical solution: A computer-readable storage medium storing a computer program capable of being loaded by a processor and executed by any of the above-described photovoltaic piling machine terrain recognition methods.

[0016] In summary, this application includes at least one of the following beneficial technical effects: 1. By analyzing camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, and radar satellite extrinsic parameter matrix, the camera pixel data and LiDAR data are transformed into radar world data and camera world data in world coordinates determined by satellite coordinates. Then, the radar world data and camera world data are fused to determine the spatiotemporal fused data and coordinate point gray values. Based on the spatiotemporal fused data, soil echo intensity, and coordinate point gray values, spatial terrain features and pile-driven terrain classification are determined. Finally, real-time terrain modeling is determined based on spatial terrain features and pile-driven terrain classification, thereby improving the spatiotemporal synchronization of multi-source data. 2. By determining the coordinate point distance weight and coordinate point grayscale weight respectively based on the coordinate point distance and coordinate point grayscale value, and then combining the coordinate point grayscale value, satellite data elevation, radar data elevation and camera world elevation to determine the texture balance coefficient, and finally fusing radar world data and camera world data according to the texture balance coefficient, coordinate point grayscale weight and coordinate point distance weight to determine the spatiotemporal fused data, thereby fusing multi-source data according to adaptive weight and reducing the fusion error of multi-source data; 3. Data extraction is performed on the fused point cloud data through a judgment time window to determine the judgment point cloud data. Then, the judgment point cloud data is divided according to the incremental update grid to determine the grid point cloud data. Based on the grid point cloud data, the point cloud density change rate and maximum elevation change are determined. Then, the point cloud density change rate and maximum elevation change corresponding to the grid point cloud density are filtered according to the density change threshold and elevation change threshold. The grid point cloud density with the number of data exceeding the threshold is determined to be greater than the lower limit of the update limit data, thereby determining the grid to be updated. The data in the grid to be updated is incrementally updated for global modeling, thereby avoiding terrain update errors caused by camera shake and other factors, improving the accuracy of terrain recognition for photovoltaic piling machines, and improving the efficiency of terrain modeling. Attached Figure Description

[0017] Figure 1 This is a flowchart of a photovoltaic piling machine terrain recognition method in an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating the analysis of camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, and radar satellite extrinsic parameter matrix in this embodiment of the application to determine pixel camera data, radar world data, and camera world data.

[0019] Figure 3 This is a flowchart in this application embodiment of fusing radar world data and camera world data to determine the simultaneous spatiotemporal fused data and coordinate point grayscale values.

[0020] Figure 4 This is a flowchart in this application embodiment of analyzing the gray values ​​of coordinate points and spatiotemporal fusion data to determine spatial terrain features and pile-driving terrain classification.

[0021] Figure 5 This is a flowchart in this application embodiment that analyzes spatial terrain features, global core point data, core point neighborhood data, and coordinate point grayscale values ​​to determine the pile driving terrain classification.

[0022] Figure 6 This application embodiment analyzes radar world data, camera world data, spatial terrain features, and pile-driven terrain classification to determine the flowchart for real-time terrain modeling.

[0023] Figure 7 This is a flowchart in this application embodiment that analyzes fused point cloud data, spatial terrain features, and pile-driven terrain classification to determine real-time terrain modeling. Detailed Implementation

[0024] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figures 1 to 7 The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0025] This application discloses a method, system, terminal, and storage medium for terrain recognition of photovoltaic piling machines. Specifically, it discloses a processing terminal that acquires camera pixel data, LiDAR data, camera intrinsic parameter data, camera radar extrinsic parameter data, and radar satellite extrinsic parameter data. The processing terminal analyzes these data, converting them into radar world data and camera world data in world coordinates determined by satellite coordinates. It then fuses the radar world data and camera world data to determine simultaneous spatiotemporal fused data and coordinate point grayscale values. Based on the simultaneous spatiotemporal fused data, soil echo intensity, and coordinate point grayscale values, it determines spatial terrain features and piling terrain classification. Finally, it determines real-time terrain modeling based on spatial terrain features and piling terrain classification, thereby improving the spatiotemporal synchronization of multi-source data.

[0026] Reference Figure 1 This application discloses a method for terrain recognition of photovoltaic piling machines, including the following steps: Step S100: Obtain camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, and radar satellite extrinsic parameter matrix.

[0027] Among them, camera pixel data refers to the pixel data generated by the camera. Its corresponding coordinate system is a two-dimensional pixel coordinate system, which is calibrated by the corner points of the checkerboard. It includes pixel grayscale data and pixel two-dimensional coordinate data, which are determined by the processing terminal by directly acquiring the two-dimensional image data collected by the camera.

[0028] LiDAR data refers to the three-dimensional point cloud coordinate data obtained by LiDAR point cloud acquisition. The corresponding coordinate system is the LiDAR coordinate system, which is calibrated with the center of the radar equipment as the origin and combined with its own mechanical structure. It includes the laser point cloud coordinates and the soil echo intensity and soil moisture content corresponding to each point cloud coordinate. The processing terminal first retrieves the point cloud data acquired by the LiDAR to determine the coordinates of each point cloud in the LiDAR coordinate system. Then, it retrieves the intensity of the reflected signal received after the LiDAR emits a laser pulse to the area to be piled. This data is positively correlated with the size of soil particles and reflects the surface roughness. Then, it retrieves the data from the high-precision humidity sensor deployed on the pile tip of the pile driver to determine the soil moisture content data corresponding to each point cloud coordinate. After determining the point cloud coordinates, soil echo intensity, and soil moisture content, the three are integrated and determined.

[0029] The camera intrinsic parameter matrix refers to the intrinsic parameter transformation matrix between the two-dimensional data captured by the camera and the three-dimensional data in the camera coordinate system. It is determined by the processing terminal by first retrieving a chessboard image formed by multiple sets of image data captured by the camera, then extracting the world coordinates and pixel coordinates of the chessboard corner points to construct the perspective projection equation, and iteratively solving the intrinsic parameters based on the Zhang calibration method and the maximum likelihood estimation method.

[0030] The camera radar extrinsic parameter matrix refers to the extrinsic parameter transformation matrix between the coordinate points of the camera's 3D coordinate data and the coordinate points of the lidar's 3D coordinate data. The processing terminal constructs a rotation matrix and a translation vector, establishes the relationship between the camera data coordinate points and the lidar data coordinate points based on the lidar coordinates and camera coordinates at the corner points of the checkerboard grid, substitutes the pixel coordinates after camera intrinsic parameter transformation, constructs an overdetermined system of equations, solves it using the least squares method, and obtains the optimal extrinsic parameters through SVD decomposition.

[0031] The radar satellite extrinsic parameter matrix refers to the extrinsic parameter transformation matrix that maps the coordinate points of the lidar three-dimensional coordinate data to the world coordinate system calibrated by the BeiDou satellite coordinate system. The processing terminal constructs a rotation matrix and a translation vector, establishes the relationship between the lidar data coordinate points and the BeiDou satellite data coordinate points based on the lidar data coordinates of the corresponding ground control points of the BeiDou satellite and the coordinates of the BeiDou satellite data points, substitutes the lidar data coordinate points into the equations, constructs an overdetermined system of equations, solves it using the least squares method, and obtains the optimal extrinsic parameters through SVD decomposition.

[0032] Step S101: Analyze the camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera-radar extrinsic parameter matrix, and radar-satellite extrinsic parameter matrix to determine the pixel camera data, radar world data, and camera world data.

[0033] Among them, pixel camera data refers to camera pixel data transformed into the camera's three-dimensional coordinate system after transformation by the camera's intrinsic parameter matrix. The corresponding coordinate system is the three-dimensional camera coordinate system, calibrated based on the camera's own optical structure. Radar world data refers to lidar data transformed into the satellite world coordinate system after transformation by the radar satellite extrinsic parameter matrix. Camera world data refers to camera pixel data transformed into the satellite world coordinate system after transformation by the camera's intrinsic parameter matrix, camera-radar extrinsic parameter matrix, and radar-satellite extrinsic parameter matrix. All three are determined by the processing terminal through analysis of the camera pixel data, lidar data, camera intrinsic parameter matrix, camera-radar extrinsic parameter matrix, and radar-satellite extrinsic parameter matrix. Specific analysis steps are described in [reference needed]. Figure 2 The steps in the process.

[0034] Step S102: Fuse radar world data and camera world data to determine the simultaneous spatiotemporal fused data and coordinate point grayscale values.

[0035] Among them, spatiotemporal fused data refers to spatiotemporally aligned fused data obtained by fusing radar world data and camera world data in the satellite world coordinate system. The coordinate point grayscale value refers to the grayscale value corresponding to each data point in the camera world data. Both are determined by the processing terminal through analysis of the radar world data and camera world data; the specific analysis steps are described in [reference needed]. Figure 3 The steps in the process.

[0036] Step S103: Analyze the gray values ​​of coordinate points and the spatiotemporal fusion data to determine the spatial terrain features and the classification of the pile-driving terrain.

[0037] Spatial terrain features refer to the global terrain features of the area to be piled. Pile driving terrain classification refers to the probability of the terrain type in the area to be piled, such as the probability that area A in the area to be piled is sandy (a%), grassland (b%), and mountainous (c%). Both are determined by the processing terminal through analysis of the coordinate point grayscale values ​​and spatiotemporal fusion data. Specific analysis steps are detailed in [reference needed]. Figure 4 The steps in the process.

[0038] Step S104: Analyze radar world data, camera world data, spatial terrain features, and pile-driven terrain classification to determine real-time terrain modeling.

[0039] Real-time terrain modeling refers to the real-time terrain modeling of the area to be driven for piling. This is determined by the processing terminal through analysis of LiDAR data, pixel camera data, spatial terrain features, and piling terrain classification. Specific analysis steps are detailed below. Figure 6 The steps in the process.

[0040] Reference Figure 2 The steps for analyzing camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera-radar extrinsic parameter matrix, and radar-satellite extrinsic parameter matrix to determine pixel camera data, radar world data, and camera world data include: Step S200: Convert the lidar data into radar camera data based on the camera radar extrinsic parameter matrix.

[0041] Among them, the radar camera data refers to the lidar data transformed into the camera's three-dimensional coordinate system by the camera radar extrinsic parameter matrix. The processing terminal first multiplies the coordinate points corresponding to the lidar data by the rotation vector in the camera radar extrinsic parameter matrix, and then adds the translation vector in the camera radar extrinsic parameter matrix.

[0042] Step S201: Convert radar camera data into radar pixel data based on the camera intrinsic parameter matrix.

[0043] Among them, radar pixel data refers to radar camera data that has been transformed into a two-dimensional pixel coordinate system by the camera intrinsic parameter matrix. The processing terminal determines the coordinate projection perspective transformation of the radar camera data into the pixel two-dimensional coordinate system by the camera intrinsic parameter matrix.

[0044] Step S202: Extract radar pixel data from camera pixel data to determine pixel depth data.

[0045] Among them, pixel depth data refers to the height data corresponding to each pixel point. It is determined by the processing terminal by extracting the height data of the corresponding coordinate point in the meta-radar coordinate system of the radar pixel data based on the coordinate point corresponding to the camera pixel data.

[0046] Step S203: Convert the pixel depth data and camera pixel data into pixel camera data based on the camera intrinsic parameter matrix.

[0047] The pixel camera data is consistent with the pixel camera data in step S101. The processing terminal first constructs the coordinates of the two-dimensional pixel data into homogeneous coordinates, then multiplies the homogeneous coordinates by the inverse of the camera intrinsic parameter matrix, and calculates the matrix transformation result by multiplying it by the pixel depth data.

[0048] Step S204: Convert pixel camera data into camera radar data based on the camera radar extrinsic parameter matrix.

[0049] Among them, camera radar data refers to pixel camera data that has been transformed into the lidar coordinate system through the camera radar extrinsic parameter matrix. The processing terminal then projects the coordinates of the pixel camera data into the lidar coordinate system based on the camera radar extrinsic parameter matrix.

[0050] Step S205: Convert camera radar data and lidar data into camera world data and radar world data based on the radar satellite extrinsic matrix.

[0051] The camera world data is consistent with the camera world data in step S101. It is determined by the processing terminal by first converting the coordinates of the camera radar data into homogeneous coordinates, and then multiplying the homogeneous coordinates by the radar satellite extrinsic parameter matrix.

[0052] The radar world data is consistent with the radar world data in step S101. The processing terminal first converts the coordinates of the lidar data into homogeneous coordinates, and then multiplies the homogeneous coordinates by the radar satellite extrinsic parameter matrix.

[0053] Reference Figure 3 The steps for fusing radar world data and camera world data to determine the simultaneous spatiotemporal fused data and coordinate point grayscale values ​​include: Step S300: Obtain the real-time coordinates and world coordinates of the piling machine.

[0054] Among them, the real-time coordinates of the pile driver refer to the position coordinates of the center point of the pile driver in the world coordinate system, which is determined by the processing terminal by directly retrieving the coordinates of the center point of the pile driver determined by Beidou satellite positioning.

[0055] World coordinates refer to the aggregated coordinates of radar world data and camera world data. They are determined by the processing terminal by retrieving all coordinates from both radar and camera world data and then summarizing the coordinate data.

[0056] Step S301: Calculate the Euclidean distance between the world coordinate point and the real-time position of the pile driver to determine the coordinate point distance.

[0057] The coordinate point distance refers to the distance between the world coordinate point and the real-time position of the pile driver, which is determined by the processing terminal by calculating the Euclidean distance between the world coordinate point and the real-time position of the pile driver.

[0058] Step S302: Input the coordinate point distance into the preset distance weight model to determine the coordinate point distance weight.

[0059] The distance-weighted model is a weight determination model based on the squared decay characteristic, ensuring that the weight of a coordinate point decreases rapidly with increasing distance. This allows the weight of a coordinate point to be determined based on its distance from the pile driver's coordinates, improving the accuracy of multi-source data fusion. The specific model formula is as follows: .

[0060] In the formula, Distance weights for coordinate points This represents the distance between coordinate points.

[0061] The coordinate point distance weight refers to the distance fusion weight of each world coordinate point determined based on the coordinate point distance, which is calculated by the processing terminal by inputting the coordinate point distance into the distance weight model.

[0062] Step S303: Extract data from the camera's world data to determine the grayscale value of the coordinate point.

[0063] The grayscale value of the coordinate point is consistent with the grayscale value of the coordinate point in step S102, and is determined by the processing terminal through data extraction of grayscale data in the camera world data.

[0064] Step S304: Input the grayscale value of the coordinate point into the preset grayscale weight model to determine the grayscale weight of the coordinate point.

[0065] The gray-level weighting model refers to a normalization model that transforms image gray-level information into quantifiable coordinate point gray-level weights by normalizing the gray-level values ​​of coordinate points. The specific model formula is as follows: .

[0066] In the formula, For the grayscale weight of coordinate points, This represents the grayscale value of the coordinate point.

[0067] The grayscale weight of a coordinate point refers to the grayscale fusion weight of each world coordinate point determined based on its grayscale value. It is calculated and determined by the processing terminal by inputting the grayscale value of the coordinate point into the grayscale weight model.

[0068] Step S305: Extract data from the world coordinate point and the gray value of the coordinate point according to the preset number of neighboring points to determine the neighborhood elevation data and neighborhood gray value data.

[0069] The number of neighborhood points refers to the number of neighborhood data points for each coordinate point. The operator determines the initial number of neighborhood points based on the terrain recognition accuracy requirements of the photovoltaic piling machine, and then fine-tunes the initial number of neighborhood points based on the actual terrain modeling situation.

[0070] Neighborhood elevation data refers to the elevation data of neighboring points corresponding to each world coordinate point. The processing terminal first determines the nearest neighbor coordinate point based on the number of neighboring points, and then extracts and determines the elevation data of the neighboring coordinate point.

[0071] Neighborhood grayscale data refers to the grayscale value of the neighboring points corresponding to each data coordinate point. The processing terminal first determines the nearest neighboring coordinate point to the world coordinate point based on the number of neighboring points, and then extracts and determines the grayscale data of the neighboring coordinate points.

[0072] Step S306: Input the number of neighboring points, neighborhood elevation data, and neighborhood grayscale data into the preset texture calculation model to determine the texture balance coefficient.

[0073] Among them, the texture calculation model refers to an algorithm model that dynamically adjusts the texture features and combined feature weights based on the texture and geometric complexity of the terrain to be piled up. The specific model formula is as follows: , , .

[0074] In the formula, This is the texture balance coefficient. The average value of the neighborhood grayscale data. For neighborhood grayscale data, Let i be the neighborhood of point i. This is the average value of the elevation data in the neighborhood. For neighborhood elevation data, The number of neighboring points. The standard deviation of the neighborhood grayscale data. is the standard deviation of the neighborhood elevation data.

[0075] The texture balance coefficient is a dynamic balance coefficient used to balance the texture features and geometric features of the terrain. In areas with rich texture, the grayscale weight of the terrain is increased, and in areas with significant geometric features, the distance weight of the terrain is increased. It is calculated and determined by the processing terminal by inputting the number of neighboring points, neighboring elevation data, and neighboring grayscale data into the texture calculation model.

[0076] Step S307: Fuse radar world data and camera world data according to texture balance coefficient, coordinate point grayscale weight and coordinate point distance weight to determine the spatiotemporal fused data.

[0077] The spatiotemporal fusion data is consistent with the spatiotemporal fusion data in step S102. The processing terminal sets an objective function based on the weighted least squares principle to fuse the multi-source data. The specific fusion formula is as follows: .

[0078] In the formula, For simultaneous spatiotemporal data fusion, This is the texture balance coefficient. Distance weights for coordinate points For the grayscale weight of coordinate points, For radar world data, For camera world data.

[0079] Reference Figure 4 The steps for analyzing the grayscale values ​​of coordinate points and spatiotemporally fused data to determine spatial terrain features and piling terrain classification include: Step S400: Sample the spatiotemporal fusion data according to the preset global sampling model to determine the global core point data.

[0080] Among them, the global sampling model refers to a sampling model that performs global sampling of spatiotemporally fused data based on the principle of farthest point sampling, ensuring that the sampling points cover the entire sampling area to the maximum extent. The specific model formula is as follows: .

[0081] In the formula, The coordinates of the global core point data. This is the set of data from the first j-1 core points that have been sampled. These are the coordinates of the candidate spatiotemporal fusion point data.

[0082] Global core point data refers to the core point data obtained by global sampling of simultaneous spatiotemporal fusion data. The processing terminal first samples the simultaneous spatiotemporal fusion data according to the global sampling model to determine the coordinates of the core points. Then, it extracts the core point data from the simultaneous spatiotemporal fusion data based on the coordinates and integrates the core point data with the core point coordinates to determine the core point data.

[0083] Step S401: Input the global core point data and the spatiotemporal fusion data into the preset local dynamic sampling model to determine the core point neighborhood data.

[0084] The local dynamic sampling model refers to a sampling model that, after determining the neighborhood range based on the dynamic sphere query radius, performs nearest-point sampling on the spatiotemporally fused data with a fixed preset number of neighborhood sampling points, centered on the global core point data coordinates, and using the dynamic sphere query radius as the radius. The specific formula for calculating the dynamic sphere query radius is as follows: , .

[0085] In the formula, For the dynamic sphere query radius, The standard deviation of the elevation of data within the neighborhood of the global core point data. This represents the average elevation of the global core point data within its neighborhood. The elevation of the data within the neighborhood of the global core point data. The number of preset neighborhood sampling points is fixed, and the specific method for determining it is the same as the method for determining the number of neighborhood points in step S305.

[0086] Step S402: Calculate the coordinate difference between the neighborhood data of the core point and the global core point data to determine the relative coordinates of the neighborhood.

[0087] Among them, the relative coordinates of the neighborhood refer to the relative coordinates of the neighborhood data of the core point determined based on the coordinates of the global core point data. The processing terminal determines the relative coordinates of the neighborhood data of the core point and the coordinates of the global core point data by calculating the difference between the coordinates of the neighborhood data of the core point and the coordinates of the global core point data.

[0088] Step S403: Input the relative coordinates of the neighborhood and the neighborhood data of the core point into the preset local coding model to determine the local coding features.

[0089] Among them, the local coding model refers to the algorithm model based on the shared multilayer perceptron to enhance the features of the relative coordinates of the neighborhood and the neighborhood data of the core points corresponding to the relative coordinates. The multilayer perceptron is trained on point cloud data with manually labeled terrain. The model realizes the progressive mapping from the original terrain features to the higher-order terrain features through two hidden layers. The first hidden layer performs linear transformation and nonlinear excitation on the neighborhood data of the core points corresponding to the original relative coordinates and the global core point data to extract simple geometric association features and attribute association features of terrain features. The second layer captures complex local terrain features through fully connected layers and nonlinear activation functions to form higher-order features.

[0090] Local coding features refer to terrain feature data after feature extraction, which are determined by the processing terminal by inputting the relative coordinates of the neighborhood, the neighborhood data of the core point, and the global core point data into the local coding model.

[0091] Step S404: Input the local encoded features and the preset training weight vector into the preset attention aggregation model to determine the spatial terrain features.

[0092] The training weight vector refers to the weight vector used in the attention aggregation model to focus on key local features. Its dimension is consistent with the local encoded features. The operator first obtains the terrain classification result based on the labeled point cloud data, then measures the error between the terrain classification result and the true label based on the terrain classification result and the cross-entropy loss function, then calculates the gradient of the loss function with respect to the weight vector using the chain rule, and finally uses the optimizer to update the weight vector according to the gradient.

[0093] The attention aggregation model refers to a model that assigns importance weights to each local feature in the neighborhood based on an attention mechanism, aggregates the neighborhood features according to the importance weights, and performs max pooling on the aggregated features to finally obtain the global macro-terrain features. The specific model formula is as follows: , .

[0094] In the formula, For spatial topographic features, As feature importance weights, For local coding features, For each neighborhood corresponding to the core point's neighborhood data, This is for training weight vectors.

[0095] The spatial terrain features are consistent with those in step S103, and are determined by the processing terminal by inputting local encoded features and training weight vectors into the attention aggregation model.

[0096] Step S405: Analyze the spatial terrain features, global core point data, core point neighborhood data, and coordinate point gray values ​​to determine the pile driving terrain classification.

[0097] The pile driving terrain classification is consistent with that in step S103, and is determined by the processing terminal through analysis of spatial terrain features, global core point data, core point neighborhood data, and coordinate point grayscale values. Specific analysis steps are detailed below. Figure 5 The steps in the process.

[0098] Reference Figure 5 The steps for determining the pile driving terrain classification include analyzing spatial terrain features, global core point data, core point neighborhood data, and coordinate point gray values. Step S500: Integrate the global core point data and the core point neighborhood data to determine the local point cloud data.

[0099] Among them, local point cloud data refers to the core sampled point cloud data of the terrain to be piled, which is determined by the processing terminal by integrating the global core point data and the core point neighborhood data.

[0100] Step S501: Calculate the standard deviation of the grayscale values ​​at the coordinate points to determine the texture values ​​at the coordinate points.

[0101] Among them, the texture value of the coordinate point refers to the standard deviation of the gray value of the coordinate point, which reflects the texture complexity and contrast of the local terrain. It is determined by the processing terminal by calculating the standard deviation of the gray value of the coordinate point.

[0102] Step S502: Integrate the texture value and gray value of the coordinate point to determine the global image features.

[0103] Among them, global image features refer to the global terrain image features within the area to be piled, which are determined by the processing terminal by integrating the texture value and gray value of the coordinate points.

[0104] Step S503: Extract spatial terrain features, local point cloud data and global image features according to the preset time-series sliding window to determine the time-series terrain feature group, time-series local feature group and time-series image feature group.

[0105] The time-series sliding window refers to a fixed-length sliding window used to extract time-series data. The end of the sliding window is aligned with the current terrain identification time. Terrain data is extracted backward from the current terrain identification time. The time-series extraction window is determined by the operator based on the data requirements of the time-series extraction model, the data processing capabilities, and the accuracy of the pile driver's terrain identification.

[0106] The temporal terrain feature group refers to the spatial terrain features extracted through a temporal sliding window. The processing terminal first integrates the spatial terrain features stored in the system in a temporal order, and then extracts and determines the spatial terrain features within the temporal range of the sliding window frame by frame in a temporal order.

[0107] Temporal local feature groups refer to spatial terrain features extracted through a temporal sliding window. The processing terminal first integrates the local point cloud data stored in the system in a temporal order, and then extracts the local point cloud data within the temporal range of the sliding window frame by frame in a temporal order.

[0108] Temporal image features refer to global image features extracted through a temporal sliding window. The processing terminal first integrates the global image features stored in the system in a temporal order, and then extracts and determines the global image features within the temporal range of the sliding window frame by frame in a temporal order through the temporal sliding window.

[0109] Step S504: Input the temporal terrain feature group, temporal local feature group, and temporal image feature group into the preset temporal extraction model to determine the temporal terrain features.

[0110] Among them, the temporal extraction model refers to the algorithm model that analyzes temporal features based on long short-term memory network and finally obtains temporal terrain features. This model is trained by temporal terrain feature group, temporal local feature group and temporal image data labeled with terrain features.

[0111] Temporal terrain features refer to the characteristics of temporal terrain data. For example, when the elevation of a certain area is continuously and slowly decreasing in multiple consecutive frames, the temporal terrain features of that area can be identified as dynamic loose sand. This is determined by the processing terminal through inputting the temporal terrain feature group, the temporal local feature group, and the temporal image feature group into the temporal extraction model for analysis.

[0112] Step S505: Obtain the decision learning dataset.

[0113] Among them, the decision learning dataset refers to the dataset used to train the feature decision model, which includes temporal features and spatial features with terrain classification labels. The processing terminal accesses the terrain database, retrieves point cloud data with terrain labels and camera data, and processes the point cloud data and camera data based on the current terrain data processing flow to obtain the temporal data features and spatial data features.

[0114] Step S506: Train a preset feature decision model based on the decision learning dataset and a preset focus loss function to determine the terrain classification model.

[0115] The focus loss function is a training function designed to reduce the imbalance error in the classification model caused by the uneven number of terrain samples in the construction scenario. It is based on the principle of balancing class weights and focusing on difficult-to-classify samples. The specific function formula is as follows: , .

[0116] In the formula, The focus loss value measures the error between the model's prediction and the data labels. This is a data balancing factor used to balance the training effect of different amounts of terrain category data on the model. The total number of samples in the decision learning dataset. Let be the total number of data in the i-th class of the decision learning dataset. The total number of terrain categories in the decision learning dataset. This represents the probability that the decision training model will predict the terrain within the decision learning set during the training process. The focusing parameters for the decision learning dataset are used to reduce the contribution of easily classified samples to the total loss. The optimal focusing parameters for the decision learning dataset are determined by the operator through baseline comparison, grid search, and metric selection.

[0117] Step S507: Input the temporal terrain features and spatial terrain features into the terrain classification model to determine the pile driving terrain classification.

[0118] The pile driving terrain classification is consistent with the pile driving terrain classification in step S405, and is determined by the processing terminal by inputting temporal terrain features and spatial terrain features into the terrain classification model.

[0119] Reference Figure 6 The analysis of radar world data, camera world data, spatial terrain features, and pile-driven terrain classification determines the steps involved in real-time terrain modeling, including: Step S600: Calculate the covariance of radar world data based on preset spatial neighborhood points to determine the coordinate point covariance.

[0120] The number of spatial neighbor points is consistent with the number of neighbor points in step S305, and is determined by the operator by first setting the initial number of neighbor points and then fine-tuning it based on the terrain recognition results.

[0121] Coordinate point covariance refers to the covariance data of spatial terrain features. It is determined by the processing terminal based on the number of neighboring points marked by the spatial neighboring points. The radar world data coordinate points are extracted sequentially according to the distance between the radar world data coordinate points, and then the covariance of the radar world data coordinate points in the neighborhood is calculated.

[0122] Step S601: Input the coordinate point covariance into the preset decomposed curvature model to determine the principal curvature of the coordinate point.

[0123] The decomposition curvature model refers to first decomposing the coordinate point covariance data through SVD to obtain the maximum variance direction, the second largest variance direction, and the minimum variance direction of the corresponding neighborhood points for each radar world data. Then, by calculating the ratio of the eigenvalue corresponding to the minimum variance direction to the sum of eigenvalues, it reflects the curvature of the coordinate point corresponding to each lidar data in the region, thereby ensuring the registration and focusing of key terrain features between the radar world data and the camera world data.

[0124] The principal curvature of a coordinate point refers to the degree of curvature of the data within the neighborhood of the radar world data corresponding to the coordinate point covariance data. It is determined by the processing terminal by inputting the coordinate point covariance into the decomposed curvature model.

[0125] Step S602: Perform data analysis on the principal curvature of the coordinate points to determine the maximum coordinate curvature.

[0126] The maximum coordinate curvature refers to the maximum value of the principal curvature of the coordinate point. The maximum value of the principal curvature of the coordinate point is determined by the processing terminal through data analysis of the principal curvature of the coordinate point.

[0127] Step S603: Normalize the principal curvature of the coordinate points based on the maximum coordinate curvature to determine the terrain curvature weight.

[0128] Among them, the terrain curvature weight refers to the terrain feature weight determined by the principal curvature of the coordinate point. It is determined by the processing terminal by normalizing the principal curvature of the coordinate point according to the maximum coordinate curvature and calculating the quotient of the principal curvature of each coordinate point to the maximum coordinate curvature.

[0129] Step S604: Input the terrain curvature weights, the preset robust error function, radar world data, and camera world data into the preset curvature-weighted ICP model to determine the optimal rotation matrix and the optimal translation vector.

[0130] The robust error function refers to the error function constructed based on the Huber kernel function. It is the objective function for iterative optimization in the ICP algorithm, used to replace the traditional ICP error function, thereby improving the robustness of the objective function, reducing interference from noise, and improving the accuracy of terrain recognition. The specific error function is as follows: , .

[0131] In the formula, For robust error function, This is the optimal rotation matrix obtained through iterative solving of the curvature-weighted ICP model. The optimal translation vector is obtained through iterative solving of the curvature-weighted ICP model. For Huber kernel function, This refers to the error quantity in the Huber kernel function, i.e. , This refers to camera world data, specifically the target point cloud data used in data fusion. This refers to radar world data, specifically the source point cloud data used in data fusion. The upper limit of lidar accuracy is determined by the operator based on the lidar's accuracy. This represents the terrain curvature weight.

[0132] The curvature-weighted ICP model refers to an improved ICP iterative optimization algorithm. This algorithm replaces the centroid and covariance matrix in the traditional ICP iterative optimization algorithm with a curvature-weighted centroid and a curvature-weighted covariance matrix, and replaces the error function in the traditional ICP iterative optimization algorithm with a robust error function. This improves the robustness of the ICP algorithm and its ability to focus on key terrain features. After determining the curvature-weighted covariance matrix, the model performs SVD decomposition on the covariance matrix to obtain the rotation matrix and translation vector. The rotation matrix and translation vector are substituted into the robust error function to determine the robust error. Iteration stops when the absolute value of the error change between two consecutive iterations is less than a set value and the Frobenius norm of the rotation matrix error is less than a set value. The rotation matrix and translation vector corresponding to the iteration results are then determined as the optimal rotation matrix and optimal translation vector. The specific model formula is as follows: , , .

[0133] In the formula, For the weighted centroid of the camera world data, As the weighted centroid of radar world data, For weighted covariance, As the terrain curvature weight, For camera world data, Data from Radar World.

[0134] The optimal rotation matrix is ​​the rotation matrix that minimizes the fusion error when fusing radar world data and camera world data. It is determined by the processing terminal by inputting terrain curvature weights, robust error functions, radar world data, and camera world data into a curvature-weighted ICP model.

[0135] The optimal translation vector is the translation vector that minimizes the fusion error when fusing radar world data and camera world data. It is determined by the processing terminal by inputting terrain curvature weights, robust error functions, radar world data, and camera world data into a curvature-weighted ICP model.

[0136] Step S605: Fuse the lidar data and camera world data according to the optimal rotation matrix and optimal translation vector to determine the fused point cloud data.

[0137] Among them, fused point cloud data refers to the fusion of radar world data and camera world data, which is determined by the processing terminal by registering the radar world data and camera world data according to the optimal rotation matrix and the optimal translation vector.

[0138] Step S606: Analyze the fused point cloud data, spatial terrain features, and pile-driven terrain classification to determine the real-time terrain modeling.

[0139] The real-time terrain modeling step is consistent with the real-time terrain modeling in step S104. It is determined by the processing terminal through analysis of fused point cloud data, spatial terrain features, and piled terrain classification. The specific analysis steps are as follows: Figure 7 The steps in the process.

[0140] Reference Figure 7 The analysis of fused point cloud data, spatial terrain features, and pile-driven terrain classification determines the steps involved in real-time terrain modeling, including: Step S700: Extract data from the fused point cloud data according to the preset judgment time window to determine the judgment point cloud data.

[0141] Among them, the judgment time window refers to the sliding time window used to judge the terrain update needs. This time window takes the current terrain recognition time as the time end point and extracts a fixed length of fused point cloud data backward. It is determined by the operator based on the confidence requirements of terrain point cloud changes and the accuracy requirements of terrain recognition.

[0142] Judgment point cloud data refers to the fused point cloud data extracted through the judgment time window, used to determine whether the terrain change is a real terrain change. The processing terminal first integrates the fused point cloud data stored in the system in time sequence, and then extracts the fused point cloud data frame by frame in time sequence within the time range of the sliding window through the judgment time window.

[0143] Step S701: Divide the point cloud data according to the preset incremental update grid to determine the grid point cloud data.

[0144] The incremental update grid refers to the smallest terrain grid unit used for incremental terrain updating, which is divided into areas to be piling up. It is determined by the operator based on the point cloud density of the lidar data and the accuracy of terrain recognition.

[0145] Grid point cloud data refers to the judgment point cloud data obtained after incrementally updating the grid. The judgment point cloud data is divided and determined by the processing terminal according to the incrementally updated grid.

[0146] Step S702: Calculate the density of the grid point cloud data in the incremental update grid to determine the grid point cloud density.

[0147] Among them, the grid point cloud density refers to the density of grid point cloud data in each incremental update grid. It is determined by the processing terminal by determining the area of ​​the incremental grid based on the size of the incremental update grid, then counting the number of grid point cloud data in each incremental update grid, and finally calculating the quotient of the number of grid point cloud data and the volume of the incremental grid.

[0148] Step S703: Input the grid point cloud density into the preset density change rate model to determine the point cloud density change rate.

[0149] The density change rate model refers to the formula model used to calculate the density change rate of point clouds within adjacent computational grids. The specific model formula is as follows: .

[0150] In the formula, The rate of change of point cloud density. The density of the grid point cloud within the current grid to be calculated. This represents the density of grid point clouds within adjacent grids.

[0151] The point cloud density change rate refers to the rate of change of point cloud density between adjacent grids, which is calculated and determined by the processing terminal by inputting the grid point cloud density into the density change rate model.

[0152] Step S704: Extract data from the grid point cloud data to determine the grid point cloud elevation.

[0153] Among them, grid point cloud elevation refers to the elevation of each data point in the grid point cloud data, which is determined by the processing terminal through data extraction of the grid point cloud data.

[0154] Step S705: Calculate the elevation difference of the grid point cloud to determine the maximum elevation change.

[0155] The maximum elevation change refers to the maximum elevation change between adjacent grids, which is determined by the processing terminal by first calculating the elevation difference between the grid point cloud elevations of adjacent grids, and then performing data analysis on the elevation difference.

[0156] Step S706: Based on the preset lower limit of the update limit, the preset density change rate threshold, and the preset elevation change threshold, the point cloud density change rate and the maximum elevation change corresponding to the grid point cloud data are filtered to determine the grid data to be updated.

[0157] The lower limit for updating out-of-limit data refers to the lower limit of the number of points cloud density change rate and maximum elevation change exceeding the corresponding density change rate threshold and elevation change threshold. When the number of points cloud density change rate and maximum elevation change exceeding the corresponding density change rate threshold and elevation change threshold exceeds the lower limit for updating out-of-limit data, the grid is identified as a grid to be updated. This is determined by the operator based on the terrain update accuracy requirements and confidence requirements, thereby avoiding erroneous updates due to camera shake and other factors, and improving terrain recognition accuracy.

[0158] The lower limit of density change rate refers to the lower threshold of the density change rate of grid point cloud data when the terrain changes. It is determined by the operator through actual measurement of terrain change data.

[0159] The elevation change threshold refers to the lower limit of the maximum elevation change corresponding to the grid point cloud data when the terrain changes. It is determined by the operator through actual measurement of the terrain change data.

[0160] The grid data to be updated refers to the grid point cloud data waiting to be updated. The processing terminal first filters the point cloud density change rate and maximum elevation change corresponding to the grid point cloud data based on the lower limit of density change rate and the elevation change threshold. Then, it determines the total number of grid point cloud data in multiple frames within the judgment time window whose point cloud density change rate and maximum elevation change exceed the lower limit of density change rate and the elevation change threshold. When the total number of data exceeding the threshold is greater than the lower limit of the update over-limit threshold, the grid point cloud data is determined as grid data to be updated.

[0161] Step S707: Input the fused point cloud data, the grid data to be updated, and the spatial terrain features into the preset octree incremental update model to determine the terrain model to be rendered.

[0162] Among them, the octree incremental update model refers to an algorithm model based on the octree spatial partitioning algorithm to update the grid data to be updated. This model takes the spatial terrain features as the root node and recursively divides the space into eight leaf nodes until the leaf side length reaches a set value. Then, based on the incremental update grid corresponding to the grid data to be updated, the leaf nodes to be updated are determined. After the nodes to be updated are determined, the spatial terrain features are updated sequentially according to the fused point cloud data corresponding to the nodes to be updated.

[0163] Terrain modeling to be rendered refers to terrain modeling that has undergone incremental updates and is waiting to be rendered. It is determined by the processing terminal by analyzing and inputting fused point cloud data, grid data to be updated, and spatial terrain features into an octree incremental update model.

[0164] Step S708: Input the pile driving terrain classification and the terrain model to be rendered into the preset parallel rendering model to determine the real-time terrain model.

[0165] Among them, the parallel rendering model refers to the rendering model that implements 1024 cores of parallel rendering based on the CUDA framework. This model first performs perspective projection transformation on the terrain model to be rendered, transforming the terrain model to be rendered into screen coordinates. Then, it updates the depth buffer data through atomic operations, retaining the nearest point to avoid occlusion. Finally, it assigns colors to the corresponding sand, mountains and grassland according to the terrain classification results and writes them into the color buffer.

[0166] Real-time terrain modeling is consistent with real-time terrain modeling in step S104, and is determined by the processing terminal by inputting the pile terrain classification and the terrain model to be rendered into the parallel rendering model for analysis.

[0167] Based on the same inventive concept, embodiments of this application provide a photovoltaic piling machine terrain recognition system, including: The acquisition module is used to acquire camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, radar satellite extrinsic parameter matrix, real-time coordinates of the pile driver, world coordinate points, and decision learning dataset; A memory for storing a program for a terrain recognition method for photovoltaic piling machines; The processor and memory can load and execute programs to implement a terrain recognition method for photovoltaic piling machines.

[0168] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0169] This application provides a computer-readable storage medium storing a computer program that can be loaded and executed by a processor to perform a photovoltaic piling machine terrain recognition method.

[0170] Computer storage media include, for example, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code.

[0171] Based on the same inventive concept, this application provides a smart terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded and executed by the processor to perform a photovoltaic piling machine terrain recognition method.

[0172] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0173] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A method for terrain recognition of photovoltaic piling machines, characterized in that, include: Acquire camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, and radar satellite extrinsic parameter matrix; Analyze camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, and radar satellite extrinsic parameter matrix to determine pixel camera data, radar world data, and camera world data. The radar world data and camera world data are fused to determine the spatiotemporal fused data and the grayscale values ​​of coordinate points. The grayscale values ​​of coordinate points and spatiotemporal fusion data are analyzed to determine spatial terrain features and pile-driven terrain classification. Analyze radar world data, camera world data, spatial terrain features, and pile-driven terrain classification to determine real-time terrain modeling.

2. The method for terrain recognition of photovoltaic piling machines according to claim 1, characterized in that, The steps for analyzing camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera-radar extrinsic parameter matrix, and radar-satellite extrinsic parameter matrix to determine pixel camera data, radar world data, and camera world data include: The lidar data is converted into radar camera data based on the camera radar extrinsic parameter matrix. The radar camera data is converted into radar pixel data based on the camera intrinsic parameter matrix. Data is extracted from radar pixel data based on camera pixel data to determine pixel depth data; Based on the camera intrinsic parameter matrix, the pixel depth data and camera pixel data are converted into pixel camera data; The pixel camera data is converted into camera radar data based on the camera radar extrinsic parameter matrix. Based on the radar satellite extrinsic matrix, camera radar data and lidar data are converted into camera world data and radar world data.

3. The method for terrain recognition of photovoltaic piling machines according to claim 1, characterized in that, The steps for fusing radar world data and camera world data to determine the simultaneous spatiotemporal fused data and coordinate point grayscale values ​​include: Obtain the real-time coordinates and world coordinates of the pile driver; Calculate the Euclidean distance between the world coordinate point and the real-time position of the pile driver to determine the coordinate point distance; Input the coordinate point distance into the preset distance weight model to determine the coordinate point distance weight; Data is extracted from the camera's world data to determine the grayscale value of the coordinate point; Input the grayscale value of the coordinate point into the preset grayscale weight model to determine the grayscale weight of the coordinate point; Based on a preset number of neighboring points, data is extracted from the world coordinate points and their grayscale values ​​to determine the neighboring elevation data and the neighboring grayscale data. Input the number of neighborhood points, neighborhood elevation data, and neighborhood grayscale data into the preset texture calculation model to determine the texture balance coefficient; Radar world data and camera world data are fused based on texture balance coefficient, coordinate point grayscale weight, and coordinate point distance weight to determine the spatiotemporal fused data.

4. The method for terrain recognition of photovoltaic piling machines according to claim 1, characterized in that, The steps for analyzing coordinate point grayscale values ​​and spatiotemporally fused data to determine spatial terrain features and piling-up terrain classification include: Simultaneous spatiotemporal fusion data is sampled according to a preset global sampling model to determine global core point data; Global core point data and spatiotemporal fusion data are input into a preset local dynamic sampling model to determine the core point neighborhood data; Calculate the coordinate difference between the neighborhood data of the core point and the global core point data to determine the relative coordinates of the neighborhood; The relative coordinates of the neighborhood and the neighborhood data of the core point are input into the preset local coding model to determine the local coding features; Local encoded features and preset training weight vectors are input into a preset attention aggregation model to determine spatial terrain features; The spatial terrain features, global core point data, core point neighborhood data, and coordinate point gray values ​​are analyzed to determine the terrain classification for piling.

5. The method for terrain recognition of photovoltaic piling machines according to claim 4, characterized in that, The steps for determining the pile driving terrain classification include analyzing spatial terrain features, global core point data, core point neighborhood data, and coordinate point grayscale values: Integrate global core point data and core point neighborhood data to determine local point cloud data; Calculate the standard deviation of the grayscale values ​​at coordinate points to determine the texture value of those points. The texture value and gray value of the coordinate point are integrated to determine the global image features; Based on a preset temporal sliding window, data extraction is performed on spatial terrain features, local point cloud data, and global image features to determine temporal terrain feature groups, temporal local feature groups, and temporal image feature groups; The temporal terrain feature set, temporal local feature set, and temporal image feature set are input into a preset temporal extraction model to determine the temporal terrain features; Obtain decision learning datasets; A pre-defined feature decision model is trained based on a decision learning dataset and a pre-defined focus loss function to determine the terrain classification model; Temporal and spatial terrain features are input into the terrain classification model to determine the terrain classification for piling.

6. The method for terrain recognition of photovoltaic piling machines according to claim 1, characterized in that, Analyzing radar world data, camera world data, spatial terrain features, and staking terrain classification to determine the steps involved in real-time terrain modeling includes: The covariance of radar world data is calculated based on preset spatial neighborhood points to determine the coordinate point covariance. Input the coordinate point covariance into the preset decomposed curvature model to determine the principal curvature of the coordinate point; Data analysis is performed on the principal curvatures of the coordinate points to determine the maximum coordinate curvature. The principal curvature of the coordinate points is normalized based on the maximum coordinate curvature to determine the terrain curvature weights. The terrain curvature weights, the preset robust error function, radar world data, and camera world data are input into the preset curvature-weighted ICP model to determine the optimal rotation matrix and the optimal translation vector. The lidar data and camera world data are fused based on the optimal rotation matrix and optimal translation vector to determine the fused point cloud data. We analyze the fused point cloud data, spatial terrain features, and pile-driven terrain classification to determine the real-time terrain modeling.

7. The method for terrain recognition of photovoltaic piling machines according to claim 6, characterized in that, The analysis of fused point cloud data, spatial terrain features, and pile-driven terrain classification determines the steps involved in real-time terrain modeling: Data extraction is performed on the fused point cloud data according to the preset judgment time window to determine the judgment point cloud data; The point cloud data is divided according to the preset incremental update grid to determine the grid point cloud data; Calculate the density of the grid point cloud data in the incrementally updated grid to determine the grid point cloud density; Input the grid point cloud density into the preset density change rate model to determine the point cloud density change rate; Data extraction is performed on the grid point cloud data to determine the grid point cloud elevation; Calculate the elevation difference between the grid point cloud elevations to determine the maximum elevation change; The point cloud density change rate and maximum elevation change corresponding to the grid point cloud data are filtered according to the preset update limit, preset density change rate threshold and preset elevation change threshold to determine the grid data to be updated. The fused point cloud data, the grid data to be updated, and the spatial terrain features are input into a preset octree incremental update model to determine the terrain model to be rendered. The pile driving terrain classification and the terrain model to be rendered are input into the preset parallel rendering model to determine the real-time terrain modeling.

8. A terrain recognition system for photovoltaic piling machines, characterized in that, include: The acquisition module is used to acquire camera pixel data, LiDAR data, camera intrinsic parameter matrix, camera radar extrinsic parameter matrix, and radar satellite extrinsic parameter matrix; A memory for storing a program for a photovoltaic piling machine terrain recognition method as described in any one of claims 1 to 7; The processor and the program in the memory can be loaded and executed by the processor to implement the photovoltaic piling machine terrain recognition method as described in any one of claims 1 to 7.

9. A smart terminal, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 7 for a photovoltaic piling machine terrain recognition method.

10. A computer-readable storage medium, characterized in that, The computer program is stored and can be loaded by a processor and executed as described in any one of claims 1 to 7, which is a method for identifying the terrain of a photovoltaic piling machine.

Images