Mine vehicle trajectory prediction method based on deep learning and related equipment

Abstract

The application relates to the technical field of automatic driving of vehicles, and discloses a mine vehicle trajectory prediction method based on deep learning and related equipment. The method comprises the following steps: collecting multi-source perception data and site capacity states in a target mine operation environment, and performing coordinate system conversion and noise filtering to generate a standard perception data set; constructing a mine vehicle interaction scene feature vector through mine operation scene feature coding and multi-source information fusion, and extracting a time-space relationship to obtain a deep time-space feature representation; selecting a point rotation target for a mine vehicle based on the deep time-space feature representation and the site capacity state, determining a target operation point of each mine vehicle, and obtaining a point allocation decision result; performing Freinet coordinate conversion and safe driving area construction on the point allocation decision result, and performing trajectory planning on the safe driving area to generate a final prediction result. The application realizes intelligent point rotation decision of mine multi-vehicle cooperation, and improves trajectory prediction accuracy and operation safety.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method and related equipment for predicting the trajectory of mining vehicles based on deep learning. Background Technology

[0002] In the field of autonomous driving technology in mining, trajectory prediction is a crucial component of vehicle path planning and decision-making control. Accurate trajectory prediction for mining trucks in complex operating environments is the most critical and challenging stage in autonomous driving systems. Accurately predicting the future trajectory of mining trucks is essential for mining companies, autonomous driving research institutions, and mine safety management departments, directly impacting mining efficiency, safety assessments, and overall operating costs. Therefore, an effective trajectory prediction method for handling complex mining scenarios is crucial for ensuring the reliability and safety of autonomous driving for mining trucks.

[0003] Currently, autonomous driving in mines primarily employs traditional rule-based algorithms for path planning and decision control, handling various traffic scenarios and operational tasks through pre-defined logical rules. Some research attempts to apply deep learning techniques to trajectory prediction to better model vehicle motion patterns and environmental interactions. However, these methods still face challenges in handling multi-vehicle interactions, adapting to dynamic operating environments, and responding to unexpected scenarios. Furthermore, these prediction methods often overlook unique characteristics of mine operations, such as work site rotation, site capacity limitations, dust interference, and complex terrain, factors that can significantly impact trajectory prediction and path decision-making. In other words, existing mine vehicle trajectory prediction methods neglect the unique characteristics and multi-element interactions of mining operation scenarios, resulting in low accuracy and adaptability of the final trajectory prediction results. Summary of the Invention

[0004] The main objective of this invention is to address the problem that existing methods for predicting the trajectory of mining vehicles neglect the unique characteristics and multi-element interactions of mining operation scenarios, resulting in low accuracy and adaptability of the final trajectory prediction results.

[0005] The first aspect of this invention provides a deep learning-based method for predicting the trajectory of mining vehicles. The method includes: collecting raw multi-source sensing data and mine site capacity status from the target mining environment; performing coordinate system transformation and mine environment noise filtering on the raw multi-source sensing data to obtain a standard sensing dataset; performing feature encoding and multi-source information fusion on the standard sensing dataset to obtain a mining vehicle interaction scene feature vector; extracting the spatiotemporal relationship between the mining vehicle and obstacles from the mining vehicle interaction scene feature vector to obtain a deep spatiotemporal feature representation; based on the deep spatiotemporal feature representation and the mine site capacity status, selecting target locations for mining vehicles in the target mining environment to determine the target operating locations for each mining vehicle, obtaining a location allocation decision result; performing Frenet coordinate transformation and constructing a safe driving area for the mining vehicles based on the location allocation decision result to obtain a safe driving area; and planning the driving trajectory of the mining vehicles within the safe driving area to generate a trajectory prediction result.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the step of collecting original multi-source sensing data and mine site capacity status in the target mine operation environment, and performing coordinate system transformation and mine environment noise filtering on the original multi-source sensing data to obtain a standard sensing dataset includes: collecting various environmental sensing information in the target mine operation environment to obtain original multi-source sensing data, and collecting mine site operation status information of the target mine to obtain mine site capacity status, wherein the various environmental sensing information includes vehicle historical trajectory data, work point information, obstacle distribution data, road reference line information, and current mine vehicle status information; performing a unified transformation of the original multi-sensor coordinate system on the original multi-source sensing data to obtain unified coordinate data, and performing various mine environment noise filtering on the unified coordinate data to obtain a standard sensing dataset.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of performing feature encoding and multi-source information fusion on the standard perception dataset to obtain a mining truck interaction scene feature vector includes: extracting temporal features from the historical trajectory data of vehicles in the standard perception dataset to obtain a trajectory feature vector; extracting the spatial coordinates and functional attribute information of the operation point information in the standard perception dataset to generate a point feature vector; extracting the geometric features of obstacles from the obstacle distribution data in the standard perception dataset to obtain an obstacle feature vector; extracting the road topology features from the road reference line information in the standard perception dataset to obtain a road feature vector; and performing weighted fusion calculation on the trajectory feature vector, the point feature vector, the obstacle feature vector, and the road feature vector based on a preset interaction feature weighting value to generate a mining truck interaction scene feature vector.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of extracting the spatiotemporal relationship between the mine truck and obstacles from the feature vector of the mine truck interaction scene to obtain a deep spatiotemporal feature representation includes: encoding the spatial location of various mining environmental elements into the feature vector of the mine truck interaction scene to obtain mining operation space features; and representing the mine truck movement time sequence into the feature vector of the mine truck interaction scene to obtain mine truck movement time sequence features; combining the mining operation space features and the mine truck movement time sequence features in a spatiotemporal manner to obtain a mining spatiotemporal fusion feature vector; and using a preset multilayer transformer network to calculate the association weight between the mine truck and obstacles in the mining spatiotemporal fusion feature vector to obtain mining operation attention weights; and performing deep feature modeling on the mining spatiotemporal fusion feature vector based on the mining operation attention weights to obtain a deep spatiotemporal feature representation.

[0009] Optionally, in the fourth implementation of the first aspect of the present invention, the step of selecting target mining locations for mining trucks in the target mining operation environment based on the depth spatiotemporal feature representation and the mine site capacity status, determining the target operation location for each mining truck, and obtaining the location allocation decision result includes: determining the real-time capacity information and queuing status of each initial operation location in the target mine based on the mine site capacity status, and obtaining location status information; performing joint decision calculation on the initial operation locations of multiple mining trucks in the target mining operation environment based on the depth spatiotemporal feature representation, the location status information, and the current mining truck status information, and obtaining a multi-vehicle location allocation probability distribution; and determining the target operation location with the highest probability for each mining truck based on the multi-vehicle location allocation probability distribution, and obtaining the location allocation decision result.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of performing joint decision-making calculations on the initial working points of multiple mining trucks in the target mining operation environment based on the depth spatiotemporal feature representation, the point status information, and the current mining truck status information to obtain a multi-vehicle point allocation probability distribution includes: generating a multi-vehicle point decision query vector based on the depth spatiotemporal feature representation and the current mining truck status information, and generating a point capacity key-value vector based on the point status information; calculating the attention weight between the multi-vehicle point decision query vector and the point capacity key-value vector to obtain a point matching attention distribution; and calculating the allocation probability of each mining truck to each initial working point in the target mining operation environment based on the point matching attention distribution and a preset point allocation optimization target to obtain a multi-vehicle point allocation probability distribution.

[0011] Optionally, in the sixth implementation of the first aspect of the present invention, the step of performing Frenet coordinate transformation and constructing a safe driving area for the mine trucks based on the point allocation decision results to obtain a safe driving area includes: extracting the coordinates of the target work points corresponding to each mine truck in the point allocation decision results to obtain target driving point information; constructing a Frenet coordinate system with the target mine road reference centerline as the reference based on the target driving point information and the road reference line information to obtain coordinate system parameters; performing Frenet coordinate transformation on the target driving point information and the mine truck position information in the current mine truck status information based on the coordinate system parameters to obtain a Frenet coordinate path; and constructing the drivable area boundary around the recommended trajectory path based on the Frenet coordinate path and preset mine truck safe operation parameters to obtain a safe driving area.

[0012] A second aspect of this invention provides a deep learning-based mine vehicle trajectory prediction device, comprising: a data preprocessing module for collecting raw multi-source sensing data and mine site capacity status in the target mine operation environment, and performing coordinate system transformation and mine environment noise filtering on the raw multi-source sensing data to obtain a standard sensing dataset; a feature encoding module for performing feature encoding and multi-source information fusion on the standard sensing dataset to obtain a mine vehicle interaction scene feature vector, and extracting the spatiotemporal relationship between the mine vehicle and obstacles from the mine vehicle interaction scene feature vector to obtain a deep spatiotemporal feature representation; a prediction decision module for selecting mine location rotation targets for mine vehicles in the target mine operation environment based on the deep spatiotemporal feature representation and the mine site capacity status, determining the target operation location for each mine vehicle, and obtaining a location allocation decision result; and a trajectory planning module for performing Frenet coordinate transformation and constructing a safe driving area for the mine vehicle based on the location allocation decision result to obtain a safe driving area, and planning the mine vehicle driving trajectory within the safe driving area to generate a trajectory prediction result.

[0013] A third aspect of the present invention provides a deep learning-based mining vehicle trajectory prediction device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the deep learning-based mining vehicle trajectory prediction device to perform the various steps of the deep learning-based mining vehicle trajectory prediction method described above.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the aforementioned deep learning-based mining vehicle trajectory prediction method.

[0015] The above-mentioned deep learning-based method and related equipment for predicting the trajectory of mining vehicles. In this embodiment of the invention, multi-source sensing data and site capacity status in the target mining operation environment are collected, and coordinate system unification and noise filtering are performed to obtain a standard sensing dataset. Then, based on the feature encoding of the mining vehicle interaction scene and the fusion of multi-source information, spatiotemporal relationships are extracted to obtain a deep spatiotemporal feature representation and construct an association model between the mining vehicle and obstacles, generating high-dimensional scene representation data. Next, based on the deep spatiotemporal feature representation and the mine site capacity status, intelligent point rotation decision is performed to determine the target operation point of each mining vehicle and obtain the point allocation decision result. Finally, through Frenet coordinate transformation and safe area construction, a constrained trajectory space is obtained, and trajectory planning optimization is performed on the safe driving area to generate the mining vehicle trajectory prediction result. By employing a hierarchical feature fusion and decision-priority collaborative planning strategy, the accuracy of mine vehicle trajectory prediction is addressed. Particularly in areas such as multi-source information encoding, spatiotemporal relationship modeling, and point rotation decision-making, the unique characteristics and complexity of mine operations are fully considered, improving trajectory prediction accuracy. Furthermore, an end-to-end optimization architecture integrating deep learning and traditional planning is adopted, achieving an organic combination of intelligent decision-making and trajectory planning while enhancing adaptability to complex mine environments. In addition, through multi-layer transformer networks and attention mechanisms, mine vehicle behavior patterns and multi-vehicle collaborative relationships are effectively identified, thereby achieving intelligent and accurate prediction of mine vehicle operations as a whole.

[0016] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the first embodiment of the deep learning-based mining vehicle trajectory prediction method in this invention.

[0019] Figure 2 This is a schematic diagram of an embodiment of the deep learning-based mining vehicle trajectory prediction device of the present invention;

[0020] Figure 3 This is a schematic diagram of one embodiment of a deep learning-based mining vehicle trajectory prediction device according to the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0023] To facilitate understanding of this embodiment, the specific process of this embodiment is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the deep learning-based mining vehicle trajectory prediction method in this invention includes:

[0024] 101. Collect raw multi-source sensing data and mine site capacity status in the target mine operation environment, and perform coordinate system transformation and mine environment noise filtering on the raw multi-source sensing data to obtain a standard sensing dataset.

[0025] In this embodiment, various environmental perception information in the target mine's operating environment is collected to obtain raw multi-source perception data, and mine site operation status information is collected to obtain mine site capacity status. The various environmental perception information includes vehicle historical trajectory data, work point information, obstacle distribution data, road reference line information, and current mine vehicle status information. The raw multi-source perception data is subjected to a unified transformation of the multi-sensor coordinate system to obtain unified coordinate data, and various mine environment noise filters are applied to the unified coordinate data to obtain a standard perception dataset.

[0026] In practical applications, within the target mine operating environment, five types of environmental perception information are collected using sensor devices such as LiDAR, cameras, GPS positioning equipment, and IMU inertial measurement units. Vehicle historical trajectory data is continuously recorded by GPS positioning equipment at 0.1-second intervals, with each trajectory point containing latitude and longitude coordinates, altitude, and a corresponding timestamp, forming a chronologically ordered set of trajectory points. Operation point information is obtained through a mine digital map system and on-site surveying equipment, including: basic information such as coordinates, capacity, and equipment type of loading and unloading points obtained through the operation management system; attributes such as the location, passage width, and traffic restrictions of the operation area entrances and exits obtained through on-site surveying; parameters such as the center coordinates, available area, and turning radius of U-turn points determined through road surveying; and information such as the coordinate range, number of parking spaces, and waiting restrictions of temporary stopping points determined through operation process analysis. For example, a mine may have 20 different operational points with various functions, including 3 loading points, 2 unloading points, 4 entrances and exits, 6 U-turn points, and 5 temporary stopping points. Comprehensive acquisition of work site information provides mining trucks with end-to-end location selection support from entering to leaving the work area, enabling complete intelligent planning of mining operations. Obstacle distribution data is obtained by LiDAR sensors scanning at a frequency of 10 times per second, including static obstacles such as rock piles, the spatial location and geometric dimensions of working equipment, and dynamic obstacles such as the real-time location and movement status of other operating vehicles. Road reference line information is extracted from the mine's digital map database, including the coordinate sequence of road centerlines, road width, radius of curvature, and longitudinal slope, etc. Road reference line information refers to digital data describing the geometry and spatial orientation of mine roads. Current mining truck status information is collected in real time through onboard GPS and IMU sensors, including the truck's own position coordinates, speed, and heading angle, as well as similar status parameters of other mining trucks in the vicinity obtained through workshop communication equipment. Mine site capacity status is collected in real time through the mine dispatch management system, including the number of vehicles currently parked at each work site, the number of vehicles waiting in queue, and the estimated completion time of operations, etc. Site capacity status refers to the real-time occupancy and remaining passage capacity of each work site within the mine's operating area.

[0027] The raw multi-source sensing data is transformed from its respective sensor coordinate system to a unified geodetic coordinate system, eliminating spatial reference differences between different sensors. The coordinate system transformation process consists of four steps: First, based on the sensor's installation position on the vehicle, the data in the LiDAR coordinate system is converted to the vehicle coordinate system, adding the sensor's positional offset relative to the vehicle's center. Second, using the vehicle's pitch, roll, and yaw angles measured by the IMU, the vehicle coordinate system data is converted to a horizontal ground coordinate system. Third, combining the geographic location information provided by GPS, the ground coordinate system data is converted to a unified geodetic coordinate system. Fourth, the same coordinate transformation process is performed on road reference line information and work point information. The transformation calculation uses coordinate transformation formulas, handling positional offsets through addition and angular rotations through trigonometric functions. For example, if a lidar is installed 1.5 meters forward and 2.0 meters upward from the center of the vehicle roof, and it detects an obstacle 30 meters ahead, the system first adds a 1.5-meter offset to the measured distance, resulting in an actual distance of 31.5 meters relative to the front of the vehicle. Then, combining the vehicle's orientation angle and GPS position, the absolute position of the obstacle in the geodetic coordinate system is calculated. GPS data is converted from latitude and longitude format to planar coordinate format for easier distance and angle calculations. This coordinate system conversion ensures that all sensor data is processed under the same spatial reference, eliminating positional deviations caused by inconsistencies in coordinate systems.

[0028] Multi-level filtering algorithms are used to process the unique environmental noise interference in the mine for the coordinate unified data. For ranging noise generated by the lidar in dusty environments, an outlier detection method comparing adjacent data points is employed. Specifically, the distance difference between the current measurement point and the previous measurement point is calculated. When the difference exceeds a reasonable range determined based on the maximum vehicle speed, the current data is identified as dust interference noise and is removed. The reasonable range is calculated based on the maximum speed of the mine truck, 15 meters per second. For GPS signal positioning jumps caused by terrain obstruction in the mine, a Kalman filter algorithm combined with IMU data is used for position estimation smoothing. The Kalman filter algorithm is a recursive calculation method that predicts the position at the next moment and corrects the prediction result based on the measurement value, effectively suppressing random jumps in GPS signals. For high-frequency vibration noise from the IMU generated by the vehicle traveling on rough roads, a low-pass filter is used to filter out excessively high-frequency vibration signals, retaining low-frequency signal components reflecting normal vehicle movement. The filtering frequency threshold is set to the upper limit of the frequency of normal vehicle steering and acceleration / deceleration. The specific execution flow of the filtering process is as follows: First, the original data is arranged in chronological order to identify data points with abnormal numerical changes; then, an appropriate filtering algorithm is selected based on the data type for processing; finally, the continuity of the filtered data is checked, and missing data is appropriately interpolated and supplemented. Mine environmental noise filtering effectively improves the accuracy and stability of the sensing data, reduces the adverse effects of mine-specific environmental factors such as dust obstruction, terrain interference, and road bumps on data quality, and ensures that subsequent deep learning algorithms can perform accurate feature extraction and trajectory prediction calculations based on reliable standard sensing datasets.

[0029] 102. Perform feature encoding and multi-source information fusion on the standard perception dataset to obtain the feature vector of the mining truck interaction scene, and extract the spatiotemporal relationship between the mining truck and obstacles from the feature vector of the mining truck interaction scene to obtain the deep spatiotemporal feature representation.

[0030] In this embodiment, temporal features are extracted from the historical vehicle trajectory data in the standard perception dataset to obtain trajectory feature vectors; spatial coordinates and functional attribute information of the work point information in the standard perception dataset are extracted to generate point feature vectors; geometric features of obstacles are extracted from the obstacle distribution data in the standard perception dataset to obtain obstacle feature vectors; road topology features are extracted from the road reference line information in the standard perception dataset to obtain road feature vectors; based on preset interactive feature weighting values, the trajectory feature vectors, point feature vectors, obstacle feature vectors, and road feature vectors are weighted and fused to generate mining truck interaction scene feature vectors; spatial location encoding of various mining environmental elements is performed on the mining truck interaction scene feature vectors to obtain mining operation space features; and temporal representation of the mining truck movement time sequence is performed on the mining truck interaction scene feature vectors to obtain mining truck movement time sequence features. The process involves: spatiotemporally combining the spatial features of mining operations and the temporal features of mine car movement to obtain a spatiotemporal fusion feature vector; then using a pre-defined multi-layer transformer network to calculate the association weights between the mine car and obstacles within the spatiotemporal fusion feature vector to obtain the mining operation attention weights (i.e., performing feature decomposition of the spatiotemporal fusion feature vector for various mining operations elements, and transforming the decomposition results into attention calculation vector groups to obtain a mining trajectory attention vector group, which includes a mining trajectory query vector and a mining trajectory key value vector; calculating the original similarity score between the mining trajectory query vector and the mining trajectory key value vector, and then calculating the attention weights of the mine car to different mining operations elements and obstacles based on the original similarity score to obtain the mining operation attention weights); and finally, performing deep feature modeling on the spatiotemporal fusion feature vector to obtain a deep spatiotemporal feature representation.

[0031] In practical applications, temporal features are extracted from the historical vehicle trajectory data in the standard perception dataset to obtain a trajectory feature vector. The historical vehicle trajectory data is a series of position coordinates and motion state information arranged in chronological order. Specifically, firstly, the historical trajectory data of each mining truck is arranged chronologically to form a time series containing position coordinates, speed, and heading angle. Then, a fixed-length time analysis segment is set, with the segment length set to 10 seconds. The 10-second timeframe is chosen because the typical time for a mining truck to complete a turn or obstacle avoidance maneuver is 8-12 seconds. Within this timeframe, the average speed, maximum speed, rate of change of speed, rate of change of heading angle, and distance traveled are calculated as motion statistics. Next, a time-shifting strategy is used, moving the analysis segment forward at 1-second intervals to gradually extract trajectory features from different time periods. Finally, the extracted feature values ​​are arranged in a fixed order to form a trajectory feature vector containing 50 values, corresponding to 20 speed features, 15 direction features, and 15 acceleration features. For example, the historical trajectory of a mining truck over the past 10 seconds shows an average speed of 8 meters per second, a maximum speed of 12 meters per second, and a heading angle change rate of 3 degrees per second. These values ​​are sequentially filled into the corresponding positions of the trajectory feature vector. Temporal feature extraction can quantify the historical motion patterns of mining trucks. Especially in the environment of mine curves and slopes, historical trajectory features can reflect the adaptive driving patterns of mining trucks to specific road conditions, providing a data foundation for predicting future trajectories based on behavioral patterns.

[0032] Spatial coordinates and functional attribute information of work sites are extracted from the standard sensing dataset to generate site feature vectors. The work site information includes the geographical location and functional characteristics of each work site. Spatial coordinates refer to the precise location of the site in the mine coordinate system, and functional attribute information refers to characteristic parameters such as the work type, equipment configuration, and capacity limitations of the site. Specifically, the three-dimensional spatial coordinates of each work point are read, including the X-coordinate, Y-coordinate, and heading (the orientation angle of the point). The coordinate values ​​are directly filled into the first three positions of the feature vector. The orientation angle refers to the direction angle information of the work point, indicating the standard parking direction of the vehicle at that point. Then, the functional attribute information of the work point is extracted, including parameters such as point type, maximum number of vehicles that can be accommodated, type of supporting equipment, and work efficiency. The point type is coded using a numerical method: loading point is coded as 1, unloading point as 2, entrance / exit point as 3, turning point as 4, and temporary parking point as 5. Other attributes are directly represented by numerical values. Next, the real-time status attributes of the work point are calculated, including dynamic information such as the number of vehicles currently occupied, remaining capacity, average waiting time, and number of vehicles in queue. These status values ​​are scaled proportionally to the range of 0 to 1 for easy calculation and processing. Finally, the position coordinates and orientation angle (3 values), functional attribute (12 values), and status attribute (15 values) are combined in sequence to form a 30-dimensional point feature vector. For example, an entrance / exit point is located at coordinates (1200 meters, 800 meters), with an orientation angle of 45 degrees, a point type code of 3, a maximum capacity of 3 vehicles, and currently occupied by 1 vehicle. This information is sequentially filled into the corresponding positions of a 30-dimensional vector. The generation of point feature vectors can digitally describe the location, orientation, and operational status of various mine operation points on a two-dimensional plane. In a mine environment that includes multiple types of points such as loading / unloading points, entrances / exits, turning points, and temporary parking points, comprehensive point features facilitate the algorithm's automated processing of full-process operation point selection and vehicle scheduling.

[0033] Obstacle distribution data in a standard perception dataset is used to extract geometric features of obstacles, resulting in obstacle feature vectors. The obstacle distribution data contains the location and shape information of various static and dynamic obstacles in the mining environment. Geometric feature extraction refers to extracting geometric parameters such as length, width, height, and area from the spatial shape of the obstacles. Specifically, obstacle data points obtained from LiDAR scanning are grouped. Data points that are close together are grouped into the same obstacle. The grouping criterion is that adjacent data points are grouped together if the distance between them is less than 0.5 meters. The grouping distance of 0.5 meters is based on the minimum interval distance of common obstacles in mines, such as vehicles and equipment. Then, the geometric features of each group of obstacle data points are calculated. The length, width, and height of the obstacle are calculated using the maximum and minimum coordinate values ​​of the data points. The center position of the obstacle is calculated using the average coordinates of all data points. The size of the obstacle is estimated using the number of data points. Next, the motion characteristics of the obstacles are analyzed. Obstacles that remain in place are marked as static, while the speed and direction of motion of obstacles that change position are calculated. Finally, the values ​​of geometric features and motion features are combined and encoded in sequence into a 40-dimensional obstacle feature vector, including 3-dimensional position features, 3-dimensional size features, 4-dimensional shape features, and 30-dimensional motion features. For example, a static obstacle might be a pile of ore. Data point analysis reveals a length of 5 meters, a width of 3 meters, a height of 2 meters, center coordinates of (500 meters, 300 meters), and a speed of 0. These parameters are sequentially filled into the obstacle's feature vector. Geometric feature extraction accurately describes the spatial occupancy and movement of various obstacles in the mining environment, providing precise environmental constraints for mine truck obstacle avoidance path planning. Especially in obstacle-dense work areas, detailed geometric features help calculate safe passage zones.

[0034] Road topology features are extracted from road reference line information in the standard perception dataset to obtain road feature vectors. Road reference line information includes the shape, connectivity, and traffic parameters of roads within the mine, while road topology features refer to network structure characteristics such as road connectivity, node distribution, and path length. Specifically, the coordinate point sequence of the road reference line is converted into road geometric parameters. The curvature of the road is calculated by the arc formed by three adjacent coordinate points, and the curvature is represented by the radius of curvature. The smaller the radius of curvature, the sharper the curve. The slope angle of the road is calculated by the height difference between adjacent coordinate points, and the road width is calculated by the road boundary points. Then, the connection relationship of the road is analyzed to identify structural elements such as road forks, merging points, straight sections, and curved sections. A fork point is the location where a road splits into multiple roads, and a merging point is the location where multiple roads merge into one road. These structural types are identified by the connection pattern of the coordinate points. Next, the road's traffic attributes are extracted, including parameters such as road grade, speed limit, traffic direction, and pavement material. These attributes are converted into numerical codes. The road grade is divided into three categories: main roads, secondary roads, and maintenance roads, coded as 3, 2, and 1, respectively. Finally, the 15 values ​​of geometric parameters, 10 values ​​of structural information, and 10 values ​​of traffic attributes are combined according to category to form a 35-dimensional road feature vector. For example, a road segment is a left-turn lane. The radius of curvature is calculated to be 30 meters using three points, the slope is calculated to be 5 degrees using the height difference, the road width is 6 meters, and the speed limit is 25 kilometers per hour. These parameters are then sequentially filled into the corresponding positions of the road feature vector. Road structure feature extraction can digitally describe the geometry and traffic rules of the mine road network, providing accurate road constraint information for mine truck path selection algorithms. Especially in complex mine road environments, quantified road features facilitate the algorithm's automatic selection of the most suitable driving path.

[0035] Based on the preset interactive feature weighting values ​​(numerical weights set according to the differences in the importance of different feature vectors to trajectory prediction in the actual corresponding type of mine), the trajectory feature vector, point feature vector, obstacle feature vector and road feature vector are weighted and fused to generate the mine truck interaction scene feature vector. Specifically, four importance weights are assigned to the feature vectors: trajectory features (0.3), location features (0.25), obstacle features (0.25), and road features (0.2). This weighting is determined based on the influence of various information types in the mine truck's driving decision-making process, with the trajectory feature having the highest weight because historical behavior patterns are most important for predicting future trajectories. The four feature vectors are then adjusted to the same length by adding zeros to the shorter vectors to expand them to 155 dimensions, ensuring uniformity for subsequent calculations. Next, a weighted merging calculation is performed: the trajectory feature vector is multiplied by 0.3, the location feature vector by 0.25, the obstacle feature vector by 0.25, and the road feature vector by 0.2. The four weighted results are then summed at corresponding positions. Finally, the merged feature vector is standardized, bringing all values ​​to the range of 0 to 1. Standardization is achieved by dividing by the largest value in the vector. For example, at a certain moment, the weighted merging of the four feature vectors yields a 155-dimensional feature vector representing the mine truck's interaction scenario, comprehensively reflecting the current overall environmental state. Weighted fusion computing can integrate different types of environmental information into a unified digital representation, enabling collaborative description of multi-source information. Especially in complex scenarios of multi-element interaction in mines, the fused feature vector can comprehensively reflect the overall environmental status of the mining truck.

[0036] The spatial location encoding of the feature vector of the mining truck interaction scene is performed on various mining environmental factors to obtain the spatial features of the mining operation. The temporal representation of the mining truck movement sequence is also performed on the feature vector of the mining truck interaction scene to obtain the temporal features of the mining truck movement. Here, mining environmental factors refer to various spatial factors affecting the operation of the mining truck, spatial location encoding converts geographical location information into numerical location identifiers, and the temporal sequence of the mining truck movement is the movement change pattern of the mining truck in the time dimension. Specifically, spatial location-related feature components are extracted from the 155-dimensional feature vector of the mine truck interaction scene. These include location information such as mine truck coordinates, obstacle coordinates, point coordinates, and road coordinates. These spatial coordinates are processed using a trigonometric function encoding method. Specifically, a sine function is applied to the X-coordinate, and a cosine function is applied to the Y-coordinate. A 64-dimensional spatial encoding vector is generated by combining trigonometric functions of different frequencies, thus obtaining the spatial features of the mine operation. The temporal representation of the mine truck's movement sequence is as follows: Time-varying feature components are extracted from the feature vector of the mine truck interaction scene. These include time information such as trajectory time series, obstacle movement, and point state changes. This time series data is processed using a time-step encoding method. Specifically, the features of the current time step are combined with the features of the previous five time steps to generate a 64-dimensional time-code vector, thus obtaining the temporal features of the mine truck's movement. For example, the current position coordinates of the mine truck (1000 meters, 600 meters) are converted into a 64-dimensional spatial feature vector through sine and cosine function encoding. The change in the movement state between the current moment and the previous five moments is converted into a 64-dimensional temporal feature vector through time-step encoding. Spatiotemporal separation coding can capture the spatial distribution patterns and temporal evolution patterns of environmental elements respectively, providing a structured spatiotemporal information foundation for subsequent attention calculations.

[0037] The spatial features of mine operations and the temporal features of mine car movement are spatiotemporally combined to obtain a mine spatiotemporal fusion feature vector. A pre-defined multi-layer transformer network (a deep learning network structure based on an attention mechanism) is then used to calculate the correlation weights between mine cars and obstacles (i.e., to calculate the numerical values ​​of the mutual influence between different elements) on the mine spatiotemporal fusion feature vector, thus obtaining the mine operation attention weights. Specifically, the 64-dimensional spatial features of mine operations and the 64-dimensional temporal features of mine car movement are concatenated. The concatenation operation is achieved by arranging the values ​​of the two vectors in sequence, forming a 128-dimensional mine spatiotemporal fusion feature vector. The attention weight calculation process is as follows: First, the 128-dimensional mine spatiotemporal fusion feature vector is input into a multi-layer neural network for feature processing. The neural network contains three computational layers. Each layer performs linear transformations on the input features through matrix multiplication and addition operations, converting the 128-dimensional input features into multiple 64-dimensional sub-features. Then, the processed features are processed into a query-key value vector. The process involves transformation, generating a mine trajectory query vector and a mine trajectory key-value vector. The query vector represents the current information needs of the mining truck, while the key-value vector represents the information content of environmental elements. Both types of vectors are 64-dimensional. Next, the similarity between the query vector and the key-value vector is calculated by multiplying the corresponding positional values ​​and then summing the results. The similarity score reflects the degree of association between the mining truck and different environmental elements. Finally, the similarity score is normalized using an exponential function calculation and normalization operation to obtain the mine operation attention weight. The weight value represents the allocation of attention the mining truck gives to different environmental elements. For example, when the mining truck approaches a work point, the attention weight value related to the work point is higher; when the mining truck encounters an obstacle, the attention weight value related to the obstacle is higher. Attention weight calculation can automatically identify important environmental elements in the current scene, enabling the mining truck to intelligently perceive complex environments. Especially in mining operation environments with multiple coexisting elements, the attention mechanism helps the mining truck focus on the most important environmental information.

[0038] Based on the attention weights of mining operations, deep feature modeling is performed on the spatiotemporal fusion feature vector of the mine (i.e., layer-by-layer abstraction and transformation of input features through a multi-layer neural network) to obtain a deep spatiotemporal feature representation. Specifically, the importance weights of the features are weighted by multiplying the corresponding values ​​of the mining operation attention weights and the spatiotemporal fusion feature vector, highlighting the numerical contribution of important features and reducing the numerical influence of unimportant features. Then, the weighted feature vector is input into the deep neural network for multi-layer computation. The neural network contains three fully connected computation layers. Each layer processes the input through matrix multiplication, addition, and nonlinear function transformation. The first layer converts the 128-dimensional input into a 256-dimensional output, the second layer converts the 256-dimensional input into a 128-dimensional output, and the third layer converts the 128-dimensional input into a 64-dimensional output. After each layer's computation, numerical standardization is performed by calculating the mean and variance to adjust the feature values ​​and ensure numerical stability. Finally, a 64-dimensional deep spatiotemporal feature representation is output. This feature vector contains deep correlation information and spatiotemporal interaction relationships between the mine truck and environmental elements. For example, when a mining truck is in a complex intersection scenario, deep spatiotemporal feature representation can comprehensively reflect the complex interaction relationships between the mining truck and multiple obstacles, roads, and points. Deep feature processing can extract high-level abstract features from raw perception data, achieving a deep understanding of complex mining operation scenarios. This provides feature inputs containing rich semantic information for subsequent point-of-origin decision-making and trajectory planning algorithms. Especially in complex interaction scenarios that are difficult to handle by traditional methods, deep feature representation can automatically discover effective feature combination patterns.

[0039] 103. Based on the deep spatiotemporal feature representation and the capacity status of the mine site, the target selection of mine site rotation is carried out for the mine cars in the target mine operation environment, the target operation site of each mine car is determined, and the site allocation decision result is obtained.

[0040] In this embodiment, based on the capacity status of the mine site, the real-time capacity information and queuing status of each initial work point in the target mine are determined to obtain the point status information; based on the depth spatiotemporal feature representation, point status information, and current mine car status information, joint decision calculation is performed on the initial work points of multiple mine cars in the target mine operation environment to obtain the multi-car point allocation probability distribution (i.e., based on the depth spatiotemporal feature representation and current mine car status information, a multi-car point decision query vector is generated, and based on the point status information, a point capacity key value vector is generated; the attention weight between the multi-car point decision query vector and the point capacity key value vector is calculated to obtain the point matching attention distribution; based on the point matching attention distribution and the preset point allocation optimization target, the allocation probability of each mine car to each initial work point in the target mine operation environment is calculated to obtain the multi-car point allocation probability distribution); based on the multi-car point allocation probability distribution, the target work point with the highest probability corresponding to each mine car is determined to obtain the point allocation decision result.

[0041] In practical applications, the operational information of each initial work point is determined through real-time data query and status calculation. The specific implementation process is as follows: First, the basic capacity parameters of each initial work point are read from the mine dispatch management system. The initial work point refers to the starting functional point in the mine car operation process, including different types of work points such as loading points, unloading points, entrance and exit points, turning points, and temporary stopping points. The read content includes fixed information such as the maximum number of vehicles that can be accommodated at each point, standard operation time, and equipment operating status. Then, the number of mine cars currently parked at each point is counted in real time through the GPS vehicle positioning system. The counting method is to calculate the number of vehicles within 30 meters of each point with a speed of less than 1 meter per second, and then subtract the maximum capacity from the current number of vehicles to obtain the remaining capacity value. Next, the queuing status information of each point is obtained through operation monitoring cameras and sensors, including dynamic information such as the number of vehicles waiting for operation, the estimated waiting time, and the progress of operation completion. The queuing status information refers to the current vehicle queuing situation and the estimated time required to complete the operation at each work point. Finally, the real-time capacity information and queuing status are organized into a point status information dataset in the form of a data table. The dataset contains four core status parameters: point number, remaining capacity, number of vehicles in queue, and estimated waiting time. For example, a loading point may have a maximum capacity of 4 vehicles. GPS positioning shows 2 vehicles currently parked, calculating a remaining capacity of 2 vehicles. The monitoring system shows 1 vehicle waiting in queue, with an estimated waiting time of 6 minutes based on the work progress. This information constitutes the complete status information for that loading point. Real-time acquisition of loading point status information accurately reflects the current operational status of various work points in the mine, providing accurate decision-making basis for multi-vehicle coordinated scheduling. Especially during peak mining operations, real-time status information helps avoid the decline in operational efficiency caused by excessive vehicle aggregation at the same location.

[0042] Based on deep spatiotemporal feature representation, location status information, and current mine truck status information, a demand-supply matching method is used to perform joint location allocation calculations for multiple mine trucks. The multi-truck location decision query vector generation process is as follows: A 64-dimensional deep spatiotemporal feature representation is concatenated with 32-dimensional current mine truck status information, including real-time parameters such as the location coordinates, speed, load status, and target task type of each mine truck. The concatenation operation forms a 96-dimensional comprehensive feature vector by linking the two data vectors end-to-end. Then, matrix multiplication is used to convert the 96-dimensional comprehensive feature vector into a 64-dimensional multi-truck location decision query vector. The conversion process uses a pre-trained 96×64-dimensional weight matrix, which represents the response of each mine truck to different... The demand characteristics of initial work sites; the process of generating the site capacity key-value vector is as follows: The parameters such as remaining capacity, number of vehicles in queue, and estimated waiting time in the site status information are numerically standardized. The standardization method is to divide each parameter by the maximum value of that type of parameter across all sites, adjusting the numerical range of all parameters to between 0 and 1. Then, the standardized status parameters are combined with fixed attributes such as the site's geographical coordinates, function type, and orientation angle, and arranged sequentially to form a 32-dimensional site capacity key-value vector. This vector represents the supply capacity characteristics of each initial work site. The calculation process for the point-matching attention distribution is as follows: The similarity between the multi-vehicle point-matching decision query vector and the point-matching capacity key vector is calculated by multiplying the values ​​at corresponding positions of the two vectors and summing them to obtain the matching score. Then, the matching score of each mine car with all points is normalized by dividing each score by the sum of all scores for that mine car to obtain the point-matching attention distribution. This distribution reflects the degree of matching between each mine car and each initial work point. The calculation process for the multi-vehicle point allocation probability distribution is as follows: Probability calculation is performed based on the point-matching attention distribution and the preset point allocation optimization objectives. The point allocation optimization objectives include minimizing total waiting time, maximizing work efficiency, and balancing point load. Multiple optimization objectives are combined and calculated using a weighted summation method. The waiting time weight is set to 0.4, the work efficiency weight is set to 0.4, and the load balancing weight is set to 0.2. The weight settings are determined based on the importance of time efficiency and load balance in mining operations. Then, the comprehensive score when each mine car is allocated to each initial work point is calculated. Finally, the comprehensive score is converted into an allocation probability value using the softmax function. For example, the decision query vector of a mining truck matches the capacity key-value vector of loading point 3 with a score of 0.85. Simultaneously, this point has sufficient remaining capacity and a short waiting time. After comprehensive calculation, the probability of this mining truck being assigned to loading point 3 is 0.72. Joint decision-making enables globally optimized allocation for multiple vehicles, avoiding congestion caused by multiple mining trucks simultaneously selecting the same location. Especially in scenarios where multiple types of mining operations are conducted in parallel, the joint decision-making mechanism can significantly improve overall operational efficiency and resource utilization.

[0043] Based on the probability distribution of multi-vehicle location allocation, the optimal target operation location for each mining truck is determined by the highest probability priority strategy. The specific decision-making process is as follows: First, extract the probability vector for each mining truck from the probability distribution of multi-vehicle allocation points. The length of the probability vector is equal to the total number of initial working points, and each value in the vector represents the probability of the mining truck being allocated to the corresponding point. Then, find the maximum value in the probability vector of each mining truck by traversing all values ​​in the probability vector to find the maximum value and its corresponding initial working point number, and use this point as the initial target working point for the mining truck. Next, perform capacity conflict detection to check if there are conflicts where multiple mining trucks select the same point and the number of selected vehicles exceeds the remaining capacity of the point. Conflict detection is achieved by comparing the number of allocated vehicles at each point with the remaining capacity of the point. Finally, perform conflict resolution. For points with conflicts, sort the mining trucks according to their allocation probability values ​​for that point from high to low, retain the allocation result of the mining truck with the highest probability value until the point capacity limit is reached, and reassign the mining trucks with lower probability values ​​to the second highest probability point in their probability vectors that has remaining capacity. Repeat the detection and allocation process until all conflicts are resolved, forming the final point allocation decision result. For example, if the probability of mine car A being assigned to loading point 3 is 0.8, and the probability of mine car B being assigned to the same point is 0.6, and if the remaining capacity of loading point 3 can only accommodate one car, then mine car A will be prioritized for assignment to loading point 3, while mine car B will be reassigned to loading point 5, which has the second highest probability (0.5) in its probability vector and also has remaining capacity. The point allocation decision result includes complete allocation information for each mine car, such as the target work point number, estimated arrival time, and estimated work time. The highest probability priority strategy ensures that each mine car obtains the most suitable target work point, while conflict detection and resolution mechanisms guarantee the practical feasibility of the allocation result. Especially in complex mining environments, automated point allocation decisions can effectively avoid the limitations of experience and decision delays in traditional manual scheduling, achieving intelligent and optimized scheduling of mine car operations.

[0044] 104. Perform Frenet coordinate transformation on the point allocation decision results and construct the safe driving area of ​​the mine car to obtain the safe driving area. Plan the driving trajectory of the mine car in the safe driving area and generate trajectory prediction results.

[0045] In this embodiment, the coordinates of the target work points corresponding to each mining truck are extracted from the point allocation decision results to obtain the target driving point information. Based on the target driving point information and road reference line information, a Frenet coordinate system with the target mine road reference center line as the reference is constructed to obtain the coordinate system parameters. Based on the coordinate system parameters, Frenet coordinate transformation is performed on the target driving point information and the mining truck position information in the current mining truck status information to obtain the Frenet coordinate path. Based on the Frenet coordinate path and the preset mining truck safety operation parameters, the drivable area boundary around the recommended trajectory path is constructed to obtain the safe driving area. Based on the safe driving area and the target driving points of multiple mining trucks, the starting point, ending point, and various mine driving planning constraints for the trajectory planning of each mining truck are set. Based on the starting point and ending point, a preset path planning strategy is used to search for a path within the safe driving area to obtain the initial planned path. A polynomial curve is used to smooth and optimize the initial planned path to obtain a smooth trajectory curve. Based on various mine driving planning constraints, constraint detection is performed on the smooth trajectory curve to generate the trajectory prediction result. In addition, after generating the mine vehicle trajectory prediction results, the process includes: constructing a basic mine operation scenario for the target mine based on a standard perception dataset to obtain basic scenario data, and adding pre-set special operation scenario data to the basic scenario data to obtain difficult scenario data; using a deep learning-based mine vehicle trajectory prediction method, predicting mine vehicle trajectories using the difficult scenario data, and collecting actual trajectory data of the mine vehicles during actual operation, and comparing the deviation data between the trajectory prediction data and the actual trajectory data to obtain difficult scenario prediction deviation data; based on the difficult scenario prediction deviation data, adjusting and optimizing the parameters of various deep learning networks in the deep learning-based mine vehicle trajectory prediction method to obtain optimized network parameters, and generating a new deep learning-based mine vehicle trajectory prediction method based on the optimized network parameters.

[0046] In practical applications, target path information refers to the specific path data that the mining truck needs to travel, including the starting point, the ending point, and the sequence of coordinate points along the way. The Frenet coordinate system is a coordinate system with the road centerline as a reference, where the vertical coordinate 's' represents the distance along the road centerline, and the horizontal coordinate 'd' represents the offset distance perpendicular to the road centerline. Specifically, the target work point number corresponding to each mining truck is read from the point allocation decision results. The corresponding planar coordinate information, including the X-coordinate, Y-coordinate, and orientation angle, is queried in the mine's electronic map database using this point number to form the target driving point information. This target driving point information refers to the coordinates and direction data of the target location that the mining truck needs to reach. Then, based on the current position of the mining truck and the coordinates of the target work point, a feasible road path connecting the two points is searched in the mine road network. The search method uses a road segment connection approach, starting from the starting point and gradually connecting adjacent road segments until the ending point is reached. Each time, the adjacent road segment closest to the ending point is selected for connection, forming a complete road connection sequence as the recommended path. Then… The road reference line information corresponding to the path is extracted. By reading the geometric attributes such as the road centerline coordinate point sequence, road width value, curve curvature value, and longitudinal slope value stored in the road database, the basic data of the recommended trajectory path is formed. Finally, the Frenet coordinate system is constructed. The Frenet coordinate system is a local coordinate system established with the road reference centerline as the reference. The specific construction method is to select the road starting point as the coordinate origin, establish the longitudinal axis s along the forward direction of the road centerline, and establish the transverse axis l in the direction perpendicular to the road centerline. The specific direction of the coordinate axis is determined by calculating the direction angle of the road centerline at each coordinate point, and the coordinate system parameters are obtained, including the X and Y coordinates of the origin, the angle of the longitudinal axis, and the angle of the transverse axis. For example, given a mining truck's target work point coordinates (1500m, 1200m, 45-degree orientation), a recommended trajectory path with a total length of 800m is obtained by connecting road segments. This path includes five road segments: a main road, a secondary road, and a work road. The road reference line contains 40 coordinate points spaced 20 meters apart. A Freni coordinate system is established based on the starting coordinates (500m, 400m) and a 30-degree angle in the road's direction of travel. The Freni coordinate system transforms the complex two-dimensional path planning problem into two independent calculation problems: longitudinal distance and lateral offset. This simplifies trajectory calculations on winding mine roads, especially in environments with many curves and slopes, where the Freni coordinate system provides a more intuitive description of the vehicle's position relative to the road centerline.

[0047] Based on coordinate system parameters, Frenet coordinate transformation is performed on the target point information and mine car position information to construct a safe driving zone. The coordinate transformation process is as follows: the planar coordinates of the target driving point and the current mine car position are transformed into the Frenet coordinate system. The transformation method is as follows: first, the cumulative distance from each coordinate point to the road starting point along the road centerline is calculated as the longitudinal coordinate s. The calculation method is to measure the length of each segment along the road centerline and sum them up. Then, the shortest vertical distance from each coordinate point to the road centerline is calculated as the lateral coordinate l. The calculation method is to draw a perpendicular line from the coordinate point to the road centerline and measure the length of the perpendicular line to form the Frenet coordinate path. This path is represented by the (s, l) coordinate pair to indicate the vehicle's position in the road coordinate system. The safe driving zone construction process is as follows: based on the Frenet coordinate path and preset mine car safe operation parameters, the boundary range of the drivable area is determined. The mine car safe operation parameters include constraints such as the actual vehicle width, safe buffer distance, and road boundary restrictions. The vehicle width is set to 2.5 meters (actual width of the mining car) plus a safety margin of 0.25 meters on each side, totaling 3 meters. The safety buffer distance is set to a braking buffer zone of 3 meters in both the front and rear directions of the vehicle. These parameters are determined based on the physical dimensions and braking performance of the mining car. Then, using the road reference line as the central axis, a lateral safety boundary is formed by extending 1.5 meters to the left and 1.5 meters to the right. A longitudinal safety boundary is formed by extending 3 meters forward and 3 meters backward along the road, thus forming a rectangular drivable area boundary. Finally, a lidar scan is used to check for static or dynamic obstacles within the boundary area. The scanning method involves the lidar scanning the objects in the area at a frequency of 10 times per second. For areas where obstacles are detected, the boundary range is narrowed inward according to the outer contour of the obstacle to ensure that there is no risk of collision within the safe driving area, thus obtaining the final safe driving area. For example, the current position of a mining truck, converted to Freyner coordinates (s = 50 meters, l = 0.2 meters), indicates that it has traveled 50 meters along the road centerline and deviated 0.2 meters to the right. The target driving point, after conversion, has coordinates (s = 850 meters, l = -0.5 meters). Based on a 3-meter vehicle width and a 3-meter safety buffer distance, the safe driving area has a lateral width of 3 meters and a longitudinal length of 806 meters. The construction of the safe driving area provides clear safety boundary constraints for mining trucks, ensuring that the planned trajectory is physically feasible and meets safety requirements. Especially in the narrow roads and complex terrain environments of mines, the predefined safe area can effectively prevent vehicles from deviating from the road or colliding with obstacles.

[0048] Based on the safe driving area, trajectory planning and optimization are performed to generate the final trajectory prediction result. The trajectory planning process is as follows: Trajectory planning parameters are set for each mining car, including setting the starting point as the Freni coordinates of the current position of the mining car, setting the ending point as the Freni coordinates of the target driving point, and various mining driving planning constraints, including a maximum speed of 25 km / h, a maximum acceleration of 2 m / s², a maximum steering angle of 30 degrees, and a minimum turning radius of 8 meters. The speed limit is determined according to mining safety regulations, the acceleration limit is determined according to the characteristics of the mining car's power system, the steering angle limit prevents the vehicle from overturning on slopes, and the turning radius limit adapts to the geometry of the mining road. Then, a grid search method is used to plan the path within the safe driving area. The search method involves dividing the safe driving area into regular square grids at 1-meter intervals, and gradually expanding the search from the starting grid to the adjacent grids in eight directions: up, down, left, right, upper left, upper right, lower left, and lower right. Evaluation indicators such as the path length to each grid, the angle of change of direction, and the closest distance to obstacles are calculated. The grid with the highest comprehensive evaluation score is selected as the next path point. The search process is repeated until the endpoint grid is reached, forming an initial planned path composed of continuous grid center points. Next, a segmented straight-line connection method is used to smooth and optimize the initial planned path. The optimization method involves selecting key control points spaced 5 meters apart on the path and using cubic spline interpolation to determine the smooth curve passing through these control points. The interpolation calculation ensures that the position, direction, and curvature of adjacent path segments at the connection points are continuous and consistent, generating a continuous and smooth trajectory curve. Finally, constraint checks are performed, checking each checkpoint on the smooth trajectory curve at 1-meter intervals to ensure it meets various mine driving planning constraints. The check method involves calculating the instantaneous velocity, acceleration, steering angle, and turning radius of each checkpoint on the trajectory and comparing them with constraint limits of 25 km / h, 2 m / s², 30 degrees, and 8 meters, respectively. Trajectory segments exceeding the constraints are adjusted locally by reducing the target driving speed or increasing the turning radius. This check and adjustment process is repeated until the entire trajectory fully meets the constraints, generating the final trajectory prediction result. For example, the initial planned path of a mining truck contains 800 path points spaced 1 meter apart. After cubic spline interpolation smoothing optimization, the jagged abrupt changes in the path are eliminated. Constraint detection finds that the turning angle of the 600th path point is 35 degrees, which exceeds the 30-degree limit. By increasing the turning radius at this point from 6 meters to 10 meters, the turning angle is adjusted to 28 degrees. Finally, a smooth trajectory prediction result with a length of 800 meters that meets all constraints is generated.Segmented straight-line connection and spline interpolation optimization can eliminate abrupt changes in direction and discontinuities in the path, generating a smooth trajectory suitable for the actual driving of the mining truck. Constraint detection ensures the safety of the trajectory and the physical executability of the vehicle. Especially in the complex working environment of the mine, the optimized and constrained trajectory can ensure that the mining truck can safely and efficiently complete the autonomous driving task from the current position to the target work point.

[0049] Furthermore, after generating the mining vehicle trajectory prediction results, the deep learning-based mining vehicle trajectory prediction method can be optimized. The basic mining operation scenario refers to the typical environmental conditions during normal daily mining operations; special operation scenario data refers to abnormal or extreme situations that occur during mining operations; difficult example scenario data refers to a comprehensive scenario dataset containing various complex and difficult situations; trajectory prediction data refers to the predicted trajectory results calculated by the algorithm based on the input scenario; actual trajectory data refers to the actual trajectory records of mining vehicles driving in the real environment; difficult example prediction deviation data refers to the difference between the predicted and actual results; deep learning network parameters refer to adjustable parameters such as the weights and biases of each layer in the neural network; and optimized network parameters refer to parameter values ​​obtained through adjustment that can reduce prediction errors. Specifically, data from normal operating periods is selected from the standard perception dataset, and perception data under conditions such as vehicles traveling at the prescribed speed, equipment operating normally at the work site, and good weather conditions are chosen. This data is then organized into basic scenario data according to chronological order and spatial location. Then, data editing software is used to insert data records of various special operating scenarios into the basic scene data. This includes adding data on suddenly appearing vehicle obstacles ahead of the normal driving path, adding extra queuing vehicle data to the vehicle count records at the work site, and modifying visibility values ​​in weather condition parameters to simulate severe weather. For example, in the work site data where there were originally only two vehicles in queue, the number of vehicles in queue is changed to seven to simulate a congestion scenario; in the data of an open area 100 meters ahead of the mining truck, the position and speed information of an incoming vehicle is inserted. The modified scene data is merged and saved with the original basic scene data to form difficult scenario data that includes both normal and abnormal situations. This systematically covers various complex situations that may be encountered in mining operations. The difficult scenario data is then input one by one into the mining vehicle trajectory prediction method. The algorithm calculates and outputs the corresponding future trajectory prediction results based on the input data such as vehicle status, obstacle positions, and work site information for each scenario, including the position coordinates for each second within the next 10 seconds. Simultaneously, using vehicle-mounted GPS devices and dashcams at the mine site, under actual environmental conditions similar to the challenging scenario data, the actual driving trajectory of the mine truck driver was recorded, obtaining the actual position coordinate data every second. Then, the coordinate data of the predicted trajectory and the actual trajectory at the same time were compared one by one. The straight-line distance between the predicted and actual positions was calculated as the position deviation, the difference between the predicted and actual speeds was calculated as the speed deviation, and the difference between the predicted and actual driving direction angles was calculated as the direction deviation. For example, at the 5th second, the predicted position was (X = 150 meters, Y = 200 meters), and the actual position was (X = 153 meters, Y = 197 meters). The position deviation distance was then the square root of 3 squared plus 3 squared, equal to 4.2 meters. The deviation values ​​for all times and all scenarios were recorded and organized to form a challenging scenario prediction deviation data table.This approach allows for precise identification of prediction error patterns and problems in various complex scenarios. Further analysis of the difficult-case prediction deviation data table reveals the average deviation across different scenario types, identifying scenario categories and corresponding prediction tasks where deviations exceed set thresholds. A backpropagation algorithm is then used to adjust network parameters, transmitting the prediction deviation as an error signal back to each network layer. The parameter values ​​of the corresponding layers are adjusted proportionally to the error magnitude. For example, if the average deviation for lateral position prediction in a vehicle merging scenario is found to be 6 meters, exceeding the acceptable range of 2 meters, the network layer responsible for lateral position calculation is located, and the weight parameters handling merging vehicle features in that layer are increased by 10% to 20% from their original values. Multiple rounds of parameter adjustment are automatically executed by a computer program. After each round, the prediction accuracy is retested using difficult-case scenario data. Adjustment stops when the average position deviation is reduced to within 2 meters and the velocity deviation to within 1 meter per second. All the final adjusted network parameter values ​​are saved as an optimized network parameter file, replacing the corresponding parameters in the original prediction method, generating a new deep learning-based mining vehicle trajectory prediction method. Through systematic parameter optimization, the trajectory prediction method can achieve higher prediction accuracy and adaptability when facing complex mining operation scenarios.

[0050] In this embodiment of the invention, the accuracy problem of mine vehicle trajectory prediction is solved by using a hierarchical feature fusion and decision-priority collaborative planning strategy. In particular, in terms of multi-source information encoding, spatiotemporal relationship modeling, and point rotation decision-making, the unique characteristics and complexity of mine operations are fully considered, thus improving the accuracy of trajectory prediction. Furthermore, an end-to-end optimization architecture that integrates deep learning and traditional planning is adopted, which not only achieves the organic combination of intelligent decision-making and trajectory planning, but also enhances the adaptability to complex mine environments. In addition, through a multi-layer transformer network and attention mechanism, the behavior patterns of mine vehicles and the collaborative relationships of multiple vehicles are effectively identified, thereby achieving intelligent and accurate prediction of mine vehicle operations as a whole.

[0051] The above describes the deep learning-based mining vehicle trajectory prediction method in the embodiments of the present invention. The following describes the deep learning-based mining vehicle trajectory prediction device in the embodiments of the present invention. Please refer to [link / reference]. Figure 2 One embodiment of the deep learning-based mining vehicle trajectory prediction device of the present invention includes:

[0052] The data preprocessing module 201 is used to collect raw multi-source sensing data and mine site capacity status in the target mine operation environment, and to perform coordinate system transformation and mine environment noise filtering on the raw multi-source sensing data to obtain a standard sensing dataset.

[0053] The feature encoding module 202 is used to perform feature encoding and multi-source information fusion on the standard perception dataset to obtain a feature vector of the mine truck interaction scene, and to extract the spatiotemporal relationship between the mine truck and obstacles from the feature vector of the mine truck interaction scene to obtain a deep spatiotemporal feature representation.

[0054] The prediction and decision module 203 is used to select target locations for mining trucks in the target mining operation environment based on the depth spatiotemporal feature representation and the mining site capacity status, determine the target operation location of each mining truck, and obtain the location allocation decision result.

[0055] The trajectory planning module 204 is used to perform Frenet coordinate transformation on the point allocation decision results and construct the safe driving area of ​​the mine car to obtain the safe driving area, and to plan the driving trajectory of the mine car in the safe driving area to generate trajectory prediction results.

[0056] In this embodiment of the invention, the accuracy problem of mine vehicle trajectory prediction is solved by using a hierarchical feature fusion and decision-priority collaborative planning strategy. In particular, in terms of multi-source information encoding, spatiotemporal relationship modeling, and point rotation decision-making, the unique characteristics and complexity of mine operations are fully considered, thus improving the accuracy of trajectory prediction. Furthermore, an end-to-end optimization architecture that integrates deep learning and traditional planning is adopted, which not only achieves the organic combination of intelligent decision-making and trajectory planning, but also enhances the adaptability to complex mine environments. In addition, through a multi-layer transformer network and attention mechanism, the behavior patterns of mine vehicles and the collaborative relationships of multiple vehicles are effectively identified, thereby achieving intelligent and accurate prediction of mine vehicle operations as a whole.

[0057] above Figure 2 The deep learning-based mine vehicle trajectory prediction device in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The deep learning-based mine vehicle trajectory prediction device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0058] Figure 3This is a schematic diagram of a deep learning-based mining vehicle trajectory prediction device 300 provided in an embodiment of the present invention. The deep learning-based mining vehicle trajectory prediction device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the deep learning-based mining vehicle trajectory prediction device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the deep learning-based mining vehicle trajectory prediction device 300.

[0059] The deep learning-based mining vehicle trajectory prediction device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The illustrated structure of the deep learning-based mining vehicle trajectory prediction device does not constitute a limitation on deep learning-based mining vehicle trajectory prediction devices. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0060] The present invention also provides a deep learning-based mining vehicle trajectory prediction device. The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs each step of the deep learning-based mining vehicle trajectory prediction method in the above embodiments.

[0061] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the various steps of the deep learning-based mining vehicle trajectory prediction method.

[0062] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0063] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0064] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0065] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based method for predicting the trajectory of mining vehicles, characterized in that, The deep learning-based method for predicting mining vehicle trajectories includes: Collect raw multi-source sensing data and mine site capacity status in the target mine operation environment, and perform coordinate system transformation and mine environment noise filtering on the raw multi-source sensing data to obtain a standard sensing dataset; The standard perception dataset is subjected to feature encoding and multi-source information fusion for mining operation scenarios to obtain a feature vector of mining truck interaction scenarios. The spatiotemporal relationship between mining trucks and obstacles is extracted from the feature vector of mining truck interaction scenarios to obtain a deep spatiotemporal feature representation. Based on the depth spatiotemporal feature representation and the mine site capacity status, the target mine location rotation target selection is performed on the mine trucks in the target mine operation environment to determine the target operation location of each mine truck and obtain the location allocation decision result. The point allocation decision results are subjected to Frenet coordinate transformation and the safe driving area of ​​the mine truck is constructed to obtain the safe driving area. The driving trajectory of the mine truck is planned in the safe driving area to generate trajectory prediction results. The step of selecting target work locations for mining trucks in the target mine operation environment based on the depth spatiotemporal feature representation and the mine site capacity status, and obtaining the work location allocation decision result, includes: determining the real-time capacity information and queuing status of each initial work location in the target mine based on the mine site capacity status, and obtaining the work location status information; performing joint decision calculation on the initial work locations of multiple mining trucks in the target mine operation environment based on the depth spatiotemporal feature representation, the work location status information, and the current mining truck status information, and obtaining the multi-vehicle work location allocation probability distribution; and determining the target work location with the highest probability for each mining truck based on the multi-vehicle work location allocation probability distribution, and obtaining the work location allocation decision result.

2. The deep learning-based mining vehicle trajectory prediction method according to claim 1, characterized in that, The process involves collecting raw multi-source sensing data and mine site capacity status from the target mine's operating environment, and then performing coordinate system transformation and mine environment noise filtering on the raw multi-source sensing data to obtain a standard sensing dataset, including: Multiple environmental perception information in the target mine's operating environment is collected to obtain raw multi-source perception data, and mine site operation status information is collected to obtain mine site capacity status. The multiple environmental perception information includes vehicle historical trajectory data, work point information, obstacle distribution data, road reference line information, and current mine vehicle status information. The original multi-source sensing data is subjected to a unified transformation of the multi-sensor coordinate system to obtain unified coordinate data, and the unified coordinate data is subjected to various mine environment noise filters to obtain a standard sensing dataset.

3. The deep learning-based mining vehicle trajectory prediction method according to claim 2, characterized in that, The step of performing feature encoding and multi-source information fusion on the standard perception dataset to obtain a feature vector of the mining truck interaction scenario includes: Temporal feature extraction is performed on the vehicle historical trajectory data in the standard perception dataset to obtain trajectory feature vectors; Extract the spatial coordinates and functional attribute information of the work site locations from the standard sensing dataset to generate site feature vectors; The geometric features of obstacles are extracted from the obstacle distribution data in the standard perception dataset to obtain obstacle feature vectors; The road topology features of the road reference line information in the standard perception dataset are extracted to obtain the road feature vector; Based on preset interactive feature weighting values, the trajectory feature vector, the point feature vector, the obstacle feature vector, and the road feature vector are weighted and fused to generate a mining truck interactive scene feature vector.

4. The deep learning-based mining vehicle trajectory prediction method according to claim 1, characterized in that, The step of extracting the spatiotemporal relationship between the mining truck and obstacles from the feature vector of the mining truck interaction scene to obtain a deep spatiotemporal feature representation includes: The spatial location of various mining environmental elements is encoded into the feature vector of the mining truck interaction scene to obtain the mining operation space features, and the time representation of the mining truck movement time sequence is performed on the feature vector of the mining truck interaction scene to obtain the mining truck movement time sequence features. The spatial features of the mining operation and the temporal features of the mine car movement are combined in a spatiotemporal manner to obtain a spatiotemporal fusion feature vector of the mine. The association weight between the mine car and obstacles is calculated by using a preset multilayer transformer network to obtain the mine operation attention weight. Based on the attention weights of the mining operations, deep feature modeling is performed on the spatiotemporal fusion feature vector of the mine to obtain a deep spatiotemporal feature representation.

5. The deep learning-based method for predicting the trajectory of mining vehicles according to claim 2, characterized in that, The step of jointly deciding on the initial operating locations of multiple mining trucks in the target mining environment based on the depth spatiotemporal feature representation, the location status information, and the current mining truck status information, to obtain a multi-truck location allocation probability distribution, includes: Based on the depth spatiotemporal feature representation and the current mining truck status information, a multi-vehicle location decision query vector is generated, and based on the location status information, a location capacity key-value vector is generated. The attention weight between the multi-vehicle location decision query vector and the location capacity key vector is calculated to obtain the location matching attention distribution; Based on the point-matching attention distribution and the preset point allocation optimization target, the allocation probability of each mine car to each initial work point in the target mine operation environment is calculated to obtain the multi-car point allocation probability distribution.

6. The deep learning-based mining vehicle trajectory prediction method according to claim 2, characterized in that, The point allocation decision results are subjected to Frenet coordinate transformation and the construction of a safe driving area for mine trucks to obtain a safe driving area, including: The coordinates of each mine truck corresponding to the target work point are extracted from the point allocation decision results to obtain the target driving point information. Based on the target driving point information and the road reference line information, a Frenet coordinate system with the target mine road reference center line as the reference is constructed to obtain the coordinate system parameters. Based on the coordinate system parameters, Freni coordinate transformation is performed on the target driving marker location information and the mine car position information in the current mine car status information to obtain the Freni coordinate path. Based on the Freni coordinate path and the preset mine car safe operation parameters, the drivable area boundary around the recommended trajectory path is constructed to obtain the safe driving area.

7. A deep learning-based mining vehicle trajectory prediction device, characterized in that, The deep learning-based mining vehicle trajectory prediction device includes: The data preprocessing module is used to collect raw multi-source sensing data and mine site capacity status in the target mine operation environment, and to perform coordinate system transformation and mine environment noise filtering on the raw multi-source sensing data to obtain a standard sensing dataset. The feature encoding module is used to encode the features of the mining operation scenario and fuse multi-source information in the standard perception dataset to obtain the feature vector of the mining truck interaction scenario, and to extract the spatiotemporal relationship between the mining truck and obstacles from the feature vector of the mining truck interaction scenario to obtain a deep spatiotemporal feature representation. The prediction and decision module is used to select target mining locations for mining trucks in the target mining operation environment based on the deep spatiotemporal feature representation and the mining site capacity status, determine the target operation location of each mining truck, and obtain the location allocation decision result. The trajectory planning module is used to perform Frenet coordinate transformation on the point allocation decision results and construct the safe driving area of ​​the mine truck to obtain the safe driving area, and to plan the driving trajectory of the mine truck in the safe driving area to generate trajectory prediction results. The step of selecting target work locations for mining trucks in the target mine operation environment based on the depth spatiotemporal feature representation and the mine site capacity status, and obtaining the work location allocation decision result, includes: determining the real-time capacity information and queuing status of each initial work location in the target mine based on the mine site capacity status, and obtaining the work location status information; performing joint decision calculation on the initial work locations of multiple mining trucks in the target mine operation environment based on the depth spatiotemporal feature representation, the work location status information, and the current mining truck status information, and obtaining the multi-vehicle work location allocation probability distribution; and determining the target work location with the highest probability for each mining truck based on the multi-vehicle work location allocation probability distribution, and obtaining the work location allocation decision result.

8. A deep learning-based mining vehicle trajectory prediction device, characterized in that, The deep learning-based mining vehicle trajectory prediction device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the deep learning-based mining vehicle trajectory prediction device to perform the steps of the deep learning-based mining vehicle trajectory prediction method as described in any one of claims 1-6.

9. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the deep learning-based mining vehicle trajectory prediction method as described in any one of claims 1-6.

Images