Multi-vehicle joint trajectory prediction method based on ego vehicle fusion and related equipment

Abstract

The present application relates to the technical field of vehicle automatic driving, and discloses a multi-vehicle joint trajectory prediction method based on self-vehicle fusion and related equipment. The method comprises the following steps: collecting multi-vehicle state data and mine environment data in a target mine, and preprocessing and lane-changing behavior calibration to generate a standard multi-vehicle motion data set; extracting self-vehicle planning intention information through self-vehicle planning trajectory and operation task information, and extracting mine environment features and vehicle dynamic distribution from the standard multi-vehicle motion data set to construct a mine scene context representation; performing attention weight calculation and feature fusion through operation task features in the self-vehicle planning intention information and vehicle interaction features in the mine scene context representation, generating an intention-enhanced scene representation and constructing a multi-modal query result; and generating a multi-vehicle cooperative operation scheme through multi-vehicle cooperative prediction, path conflict detection and cooperative optimization scheduling. The accuracy of the mine multi-vehicle trajectory prediction and the safety of the cooperative operation are improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a multi-vehicle joint trajectory prediction method and related equipment based on autonomous vehicle fusion. Background Technology

[0002] In the field of mining automation technology, multi-vehicle collaborative trajectory prediction is a crucial component of intelligent scheduling and safe operation of mining vehicles. Trajectory prediction and intent recognition based on vehicle interaction are the most critical and challenging stages in multi-vehicle collaboration. Accurately predicting the future trajectories and driving intentions of multiple vehicles in a mining environment is essential for mining operators, equipment manufacturers, and safety regulators, directly impacting operational efficiency assessment, route optimization, and overall operational safety. Therefore, an effective multi-vehicle joint trajectory prediction method is crucial for ensuring the coordination and safety of multi-vehicle operations in mining.

[0003] Currently, deep learning techniques are used to analyze historical vehicle trajectory data and build predictive models to assess future vehicle behavior, or attention mechanisms are applied to vehicle interaction modeling to better capture the inter-vehicle relationships. However, these methods still face challenges in integrating vehicle planning intentions, handling complex interactions, and adapting to the dynamic environment of mining operations. Furthermore, these predictive methods often overlook unique characteristics of mining operations, such as changes in load conditions, loading point queuing, and complex terrain constraints, which can significantly impact trajectory prediction and intention recognition. In other words, existing multi-vehicle trajectory prediction methods neglect vehicle planning intentions and unique characteristics of mining operations, resulting in low accuracy in the final trajectory prediction results. Summary of the Invention

[0004] The main objective of this invention is to address the problem that existing multi-vehicle trajectory prediction methods neglect the planning intentions of individual vehicles and some unique characteristics of mining operations, resulting in low accuracy of the final trajectory prediction results.

[0005] The first aspect of this invention provides a multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion. The method includes: collecting multi-vehicle status data and mine environment data from a target mine; preprocessing and lane-changing behavior calibration of the multi-vehicle status data and mine environment data to obtain a standard multi-vehicle motion dataset; wherein the multi-vehicle status data includes autonomous vehicle driving data and surrounding vehicle status data; extracting intent from the autonomous vehicle's planned trajectory and task information in the standard multi-vehicle motion dataset to obtain autonomous vehicle planning intent information; and further processing the standard multi-vehicle motion data... The system extracts environmental features from the mining site and constructs a dynamic vehicle distribution to obtain a mining site scene context representation. Attention weights are calculated and features are fused between the task features in the autonomous vehicle planning intent information and the vehicle interaction features in the mining site scene context representation to obtain an intent-enhanced scene representation. Trajectory and intent queries are constructed from this intent-enhanced scene representation to obtain multimodal query results. Multi-vehicle collaborative prediction is performed on the multimodal query results to obtain joint prediction results. Multi-vehicle path conflict detection and collaborative optimization scheduling are then performed on the joint prediction results to generate a multi-vehicle collaborative operation scheme.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the step of collecting multi-vehicle status data and mine environment data in the target mine, and preprocessing and lane-change behavior calibration of the multi-vehicle status data and mine environment data to obtain a standard multi-vehicle motion dataset, wherein the multi-vehicle status data includes the vehicle's own driving data and surrounding vehicle status data, including: collecting various types of driving data of the vehicle itself in the target mine to obtain its own driving data, collecting multi-dimensional motion status data of the surrounding vehicles of the vehicle itself to obtain surrounding vehicle status data, and collecting various types of static environmental information of the target mine to obtain mine environment data; performing median filtering and weighted average filtering on the vehicle's own driving data and the surrounding vehicle status data to obtain filtered motion data, and performing linear interpolation compensation on missing values ​​in the filtered motion data to obtain complete motion data; performing lane-change behavior detection on the complete motion data to obtain lane-change annotation data, and performing a mine global coordinate system-one transformation on the lane-change annotation data and the mine environment data to obtain a standard multi-vehicle motion dataset.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the vehicle driving data includes the vehicle's planned trajectory and task information. The step of extracting intent from the vehicle's planned trajectory and task information in the standard multi-vehicle motion dataset to obtain vehicle planning intent information includes: extracting path point sequences and calculating geometric features from the vehicle's planned trajectory in the standard multi-vehicle motion dataset to obtain trajectory geometric feature data; fitting expected speed curves and identifying key path nodes on the trajectory geometric feature data to obtain trajectory motion feature data; performing hierarchical decomposition of the task information in the standard multi-vehicle motion dataset into a mining operation process to obtain task planning data; identifying mining vehicle motion patterns on the task planning data to obtain motion pattern data; labeling the task planning data and the motion pattern data with multiple preset driving intent combinations to obtain intent labeling data; and encoding the intent labeling data using a multi-layer embedded network to obtain vehicle planning intent information.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of extracting mine environment features and constructing vehicle dynamic distribution from the standard multi-vehicle motion dataset to obtain a mine scene context representation includes: extracting road network topology and loading / unloading points from the mine environment data in the standard multi-vehicle motion dataset to obtain static environmental features, and performing spatial location encoding and functional area division on the static environmental features to obtain mine environment features; extracting historical motion states and labeling load states from the standard multi-vehicle motion dataset to obtain vehicle temporal features, and performing motion parameter numerical vectorization on the vehicle temporal features to obtain trajectory representation data; mapping node relationships and constructing an adjacency matrix on the trajectory representation data and the mine environment features to obtain a dynamic interaction graph structure, and extracting mine vehicle interaction features and loading point queuing features from the dynamic interaction graph structure to obtain a mine scene context representation.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of calculating attention weights and fusing features between the task features in the autonomous vehicle planning intent information and the vehicle interaction features in the mine scene context representation to obtain an intent-enhanced scene representation includes: decomposing the task features of the autonomous vehicle planning intent information to obtain a task feature vector; extracting vehicle interaction features from the mine scene context representation to obtain an interaction feature vector; constructing a query matrix from the task feature vector to obtain an intent query representation; constructing a key-value matrix from the interaction feature vector to obtain a scene key-value representation; calculating the correlation between the intent query representation and the scene key-value representation to obtain an attention score matrix; calculating the weight distribution of the attention score matrix to obtain an attention weight matrix; performing weighted summation and feature aggregation on the attention weight matrix and the interaction feature vector to obtain weighted interaction features; and fusing the weighted interaction features with the task feature vector to obtain an intent-enhanced scene representation.

[0010] Optionally, in the fifth implementation of the first aspect of the present invention, the construction of trajectory query and intent query for the intent-enhanced scene representation to obtain multimodal query results includes: performing mine movement pattern recognition and trajectory type classification on the intent-enhanced scene representation to obtain trajectory modality configuration; performing mine operation decision type recognition and driving intent type classification on the intent-enhanced scene representation to obtain intent modality configuration; constructing query features on the trajectory modality configuration to form a trajectory query tensor; and constructing query features on the intent modality configuration to form an intent query tensor; constructing mine-workshop adjacency relationships and calculating operation area weights on the trajectory query tensor and the intent query tensor to obtain mine interaction weight distribution; and aggregating mine vehicle node information and fusing loading point information on the mine interaction weight distribution to obtain enhanced query features; and performing cross-attention calculation and feature matching on the enhanced query features and the intent-enhanced scene representation to obtain multimodal query results.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of performing multi-vehicle collaborative prediction on the multimodal query results to obtain joint prediction results includes: dividing the multimodal query results into multi-branch channels and allocating feature representations to obtain trajectory query data, intent query data, and planning query data, wherein the multi-branch channels include trajectory prediction channels, intent decision channels, and planning coordination channels; extracting trajectory query vectors and calculating position and velocity predictions on the trajectory query data to obtain trajectory prediction outputs, and generating trajectory coordinates and estimating motion states at future time steps on the trajectory prediction outputs to obtain mine vehicle trajectory prediction data; and performing multi-vehicle collaborative prediction on the intent query data... The system extracts intent query vectors and classifies driving behavior to obtain intent classification outputs. It then calculates the probability distribution of various driving intent combinations from these outputs to obtain mining truck intent decision data. The system extracts the vehicle's planned trajectory from the planning query data and compares it with the predicted trajectories of other vehicles to obtain consistency verification results. These results are then used to identify trajectory conflicts and detect logical contradictions to obtain a coordination requirement assessment. Finally, the system matches the mining truck trajectory prediction data with the mining truck intent decision data and verifies their logical relationships to obtain a coordination prediction result. Finally, the system merges the coordination prediction result with the coordination requirement assessment to obtain a joint prediction result.

[0012] A second aspect of the present invention provides a multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion. The device includes: a data preprocessing module for collecting multi-vehicle status data and mine environment data in a target mine, and preprocessing and calibrating the multi-vehicle status data and mine environment data to obtain a standard multi-vehicle motion dataset, wherein the multi-vehicle status data includes autonomous vehicle driving data and surrounding vehicle status data; and a feature extraction module for extracting intent from the autonomous vehicle planning trajectory and task information in the standard multi-vehicle motion dataset to obtain autonomous vehicle planning intent information, and for extracting intent from the standard multi-vehicle motion dataset. The system extracts environmental features from the mining site and constructs a dynamic vehicle distribution to obtain a mining scene context representation. An intent fusion module calculates attention weights and fuses features between the task features in the autonomous vehicle planning intent information and the vehicle interaction features in the mining scene context representation to obtain an intent-enhanced scene representation. It then constructs trajectory queries and intent queries based on this intent-enhanced scene representation to obtain multimodal query results. A collaborative prediction module performs multi-vehicle collaborative prediction on the multimodal query results to obtain joint prediction results. Finally, it performs multi-vehicle path conflict detection and collaborative optimization scheduling on the joint prediction results to generate a multi-vehicle collaborative operation scheme.

[0013] A third aspect of the present invention provides a multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion to perform the various steps of the above-described multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the above-described multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion.

[0015] The above-mentioned multi-vehicle joint trajectory prediction method and related equipment based on autonomous vehicle fusion. In this embodiment of the invention, multi-vehicle status data and mine environment data in the target mine are collected, preprocessed, and lane-changing behavior is calibrated to obtain a standard multi-vehicle motion dataset. Then, based on the autonomous vehicle's planned trajectory and task information, intent is extracted to obtain autonomous vehicle planning intent information and construct a mine scene context representation to generate vehicle dynamic distribution data. Next, attention weights are calculated and features are fused based on the task features in the autonomous vehicle's planning intent information and the vehicle interaction features in the mine scene context representation to obtain an intent-enhanced scene representation. Then, multimodal query results are constructed through trajectory query and intent query. Finally, multi-vehicle collaborative prediction is performed on the multimodal query results to generate joint prediction results. Based on this, multi-mine truck path conflict detection and collaborative optimization scheduling are performed to obtain a multi-vehicle collaborative operation scheme. By employing hierarchical intent fusion and collaborative prediction, the accuracy problem of multi-vehicle trajectory prediction in mining operations has been solved. In particular, in terms of vehicle planning intent fusion, vehicle interaction modeling, and collaborative decision-making, the unique characteristics and dynamics of mining operations are fully considered, thus improving trajectory prediction accuracy. Furthermore, the adoption of multimodal query fusion and collaborative optimization strategies not only realizes the correlation calculation between intent recognition, trajectory prediction, and conflict detection, but also enhances the reliability of collaborative decision-making. In addition, through attention mechanisms and multi-branch prediction, vehicle behavioral intent and operating trajectory are better identified, thereby achieving intelligent and accurate prediction of multi-vehicle collaborative operations in mining operations as a whole.

[0016] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the first embodiment of the multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion in this invention.

[0019] Figure 2 This is a schematic diagram of an embodiment of the multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion in this invention.

[0020] Figure 3 This is a schematic diagram of an embodiment of a multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion in this invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0023] To facilitate understanding of this embodiment, the specific process of this embodiment is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion in this invention includes:

[0024] 101. Collect multi-vehicle status data and mine environment data in the target mine, and preprocess and calibrate lane-changing behavior of multi-vehicle status data and mine environment data to obtain a standard multi-vehicle motion dataset, wherein multi-vehicle status data includes the vehicle's own driving data and the status data of surrounding vehicles.

[0025] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0026] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0027] In this embodiment, various driving data of the mine car itself in the target mine are collected to obtain its own driving data. Multi-dimensional motion state data of the mine cars around it are collected to obtain surrounding vehicle state data. Various static environmental information of the target mine is collected to obtain mine environment data. Median filtering and weighted average filtering are applied to the mine car's driving data and surrounding vehicle state data to obtain filtered motion data. Linear interpolation is used to compensate for missing values ​​in the filtered motion data to obtain complete motion data. Lane change behavior detection is performed on the complete motion data to obtain lane change annotation data. The lane change annotation data and the mine environment data are then transformed into a global coordinate system of the mine to obtain a standard multi-vehicle motion dataset.

[0028] In practical applications, various types of driving data of the mine cars themselves are collected in the target mine. The methods for acquiring the relevant data in the embodiments are diverse and flexible. Data can be acquired through a pre-constructed digital simulation environment of the mine and a mine car operation simulation system, or by deploying sensor devices in the actual mine operation environment to collect relevant data in real time. This application does not specifically limit the data acquisition method. Regardless of the data acquisition method used, the various types of operating data, environmental data, and historical data collected are applicable to the mine car starting slippage detection method of this invention, and can also be used for algorithm verification in a simulation environment and trajectory prediction analysis for multi-vehicle collaborative operations. The above description also applies to other data acquisition processes involved subsequently, including real-time data that can be from the actual mine or real-time data from the simulation environment. To facilitate understanding and explanation, this application's embodiment uses the method of real-time data collection via sensor devices in an actual mining operation environment as an example for detailed illustration. Specifically, the vehicle's GPS positioning system records the latitude and longitude coordinates and altitude of the mining truck every second; the inertial measurement unit measures triaxial acceleration and angular velocity in real time; and the vehicle's CAN bus system acquires vehicle operating status parameters such as engine speed, wheel speed, and steering wheel angle, as well as work task information. These data collectively constitute the vehicle's own driving data. For sensing surrounding mining trucks, a lidar device installed on the vehicle scans the surrounding environment. This device measures the distance and relative position to other mining trucks by emitting laser beams and receiving reflected signals. Simultaneously, it works with millimeter-wave radar to measure the relative speed of other mining trucks, forming surrounding vehicle status data. Thus, the vehicle's own driving data and surrounding vehicle status data constitute multi-dimensional motion state data (this data simultaneously includes a set of multiple motion parameters such as position coordinates, speed, acceleration, and heading angle). The mining environment data is collected by obtaining static information such as road centerlines, road widths, and slopes through pre-mapped high-precision maps, and by acquiring the specific coordinates of loading and unloading areas through fixed monitoring equipment distributed at key locations in the mining area. This data acquisition step provides a comprehensive perception information foundation for multi-vehicle collaborative prediction, ensuring that the system can simultaneously grasp its own status, the surrounding environment, and the movement of other vehicles.

[0029] The collected vehicle driving data and surrounding vehicle status data are preprocessed. First, median filtering is performed, which involves taking a sliding window of consecutive odd-numbered data points in chronological order for each data sequence, sorting the data within the window by value, and taking the median as the filtering result. This method can effectively remove abnormal values ​​caused by momentary sensor malfunctions. Next, weighted average filtering is performed, assigning different weights to data from multiple consecutive time points and calculating the average. Data closer to the current time has a higher weight, thus smoothing data fluctuations and obtaining filtered motion data. When missing values ​​appear in the filtered motion data, linear interpolation is used for compensation. Specifically, the two nearest valid data points before and after the missing data point are found, and the value of the missing point is calculated proportionally based on the time interval. The formula is: Missing value = Previous valid value + (Next valid value - Previous valid value) × (Missing time - Previous time) / (Next time - Previous time). This method completes the data to form complete motion data.

[0030] Lane change behavior detection is performed on complete motion data by continuously monitoring the lateral position changes of mining trucks on the road to identify lane change actions. Specifically, the detection method calculates the cumulative lateral displacement of the mining truck within a fixed time period. When this cumulative displacement exceeds a preset lane width threshold, a lane change is determined to have occurred. This is achieved by continuously sliding the data in short time steps within a fixed time window, calculating the lateral displacement change at each window position. This method ensures accurate capture of the start and end times of lane changes, forming lane change annotation data (structured data recording the time interval, position coordinates, and direction of the lane change). Next, a coordinate system transformation is performed, converting the position information of all vehicles from their respective local coordinate systems to a unified global coordinate system of the mine. The global coordinate system is established with a fixed reference point in the mine as its origin. Coordinate transformation calculations unify position data from different sources to the same spatial reference frame, specifically through translation and rotation transformations, ultimately obtaining a standard multi-vehicle motion dataset. This step provides key driving behavior features for subsequent intent recognition through behavior calibration, and coordinate unification ensures the spatial consistency and comparability of multi-vehicle data.

[0031] 102. Extract the intention of the autonomous vehicle planning trajectory and task information in the standard multi-vehicle motion dataset to obtain the autonomous vehicle planning intention information, and extract the mining environment features and construct the dynamic distribution of vehicles in the standard multi-vehicle motion dataset to obtain the mining scene context representation.

[0032] In this embodiment, path point sequences and geometric features are extracted from the self-planned trajectory in the standard multi-vehicle motion dataset to obtain trajectory geometric feature data. Then, expected speed curve fitting and key path node identification are performed on the trajectory geometric feature data to obtain trajectory motion feature data. The task information in the standard multi-vehicle motion dataset is hierarchically decomposed into mining operation processes to obtain task planning data. Then, mining truck motion pattern recognition is performed on the task planning data to obtain motion pattern data. Finally, various preset driving intention combinations are labeled on the task planning data and motion pattern data to obtain intention labeling data. Finally, intention feature encoding is performed on the intention labeling data using a multi-layer embedding network to obtain the self-planned trajectory. Intent information; road network topology extraction and loading / unloading point extraction are performed on the mine environment data in the standard multi-vehicle motion dataset to obtain static environmental features. Spatial location encoding and functional area division are then performed on the static environmental features to obtain mine environment features. Historical motion state extraction and load state labeling are performed on the standard multi-vehicle motion dataset to obtain vehicle temporal features. Motion parameter numerical vectorization is then performed on the vehicle temporal features to obtain trajectory representation data. Node relationship mapping and adjacency matrix construction are performed on the trajectory representation data and mine environment features to obtain a dynamic interaction graph structure. Mine vehicle interaction features and loading point queuing features are extracted from the dynamic interaction graph structure to obtain a mine scene context representation.

[0033] In practical applications, path point sequences are extracted from the planned trajectories of autonomous vehicles in standard multi-vehicle motion datasets. This involves reading the vehicle's predetermined driving path and sampling the continuous trajectory curve into discrete path points at fixed intervals. Each path point contains coordinates and a timestamp. Subsequently, geometric feature calculations are performed. Geometric parameters such as curvature, steering angle, and path length are calculated for the lines connecting adjacent path points. Curvature is calculated using a three-point circular arc fitting method, and the steering angle is determined by the angle between the direction vectors of adjacent path segments, forming trajectory geometric feature data. Next, the trajectory geometric feature data is fitted with a desired speed curve. Based on the speed planning requirements of the mining truck on different road sections, a polynomial fitting method is used to connect the discrete speed points into a smooth speed curve. Simultaneously, key path nodes in the trajectory are identified, including turning points, gradient change points, loading points, and unloading points. These nodes are automatically identified by analyzing abrupt changes in the trajectory geometric features, forming trajectory motion feature data. The desired speed curve refers to the ideal driving speed variation curve pre-planned based on road conditions and operational requirements. This step transforms complex trajectory information into structured geometric and motion features, enabling a precise understanding of the vehicle's planning intent and expected behavior patterns.

[0034] The task information in the standard multi-vehicle motion dataset is hierarchically decomposed into mining operation processes. The complete mining truck operation task is broken down into multiple sub-task levels. The top level represents the overall operation objective, such as completing a loading and transportation cycle. The middle level represents specific operation stages, such as heading to the loading point, loading, transporting to the unloading point, and unloading. The bottom level represents specific operational actions, such as starting, accelerating, turning, and braking, forming task planning data. Subsequently, mining truck motion pattern recognition is performed on the task planning data. By analyzing the typical motion characteristics of mining trucks in different operation stages, different motion patterns are identified, including loading, transportation, unloading, and waiting modes. Each mode corresponds to a specific speed range, steering behavior, and parking characteristics. For example, in loading mode, the mining truck has a lower speed and frequently performs precise positioning adjustments, while in transportation mode, it maintains a relatively stable driving speed, forming motion pattern data. This step, through structured decomposition of the operation process, accurately understands the behavioral characteristics of mining trucks in different operation stages, improving the accuracy of intent recognition and the targeting of predictions.

[0035] The task planning data and motion pattern data are labeled with various preset combinations of driving intentions (referring to a classification system of various driving behavior intentions predefined according to the characteristics of mining operations). Based on the actual needs of mining operations, basic driving intentions such as straight-ahead intention, left-turn intention, right-turn intention, acceleration intention, deceleration intention, and stopping intention are preset, as well as operation-specific intentions such as loading intention, unloading intention, and yielding intention. These basic intentions are combined to form composite intention labels, such as the combination of "acceleration + straight-ahead + heading to loading point". The mine vehicle behavior in each time period is labeled accordingly, forming intention labeling data. Subsequently, the intention labeling data is encoded using a multi-layer embedding network. The embedding layer of the neural network converts discrete intention labels into continuous numerical vectors. The multi-layer embedding network contains multiple fully connected layers, each processed by a non-linear activation function, transforming high-dimensional sparse intention labels into low-dimensional dense feature vectors, forming the vehicle's planning intention information. This step, through standardized intention classification and feature encoding, can accurately understand and represent the vehicle's driving intentions, providing crucial prior intention information for subsequent multi-vehicle cooperative prediction.

[0036] This paper extracts the road network topology (a graphical representation of the connections between roads) from the mine environment data in the standard multi-vehicle motion dataset. By analyzing the connections between mine roads, topological information such as road centerlines, intersection locations, and road widths are extracted to form a graph structure representation of the road network, where road segments are edges and intersections are nodes. Simultaneously, loading and unloading points are extracted. Precise coordinates, service ranges, and capacity limitations of all loading and unloading points are obtained from a pre-defined mine site map, forming static environmental features. Then, spatial location encoding is performed on these static features, converting the two-dimensional coordinate information into a computationally suitable encoding format. A grid encoding method is used to divide the mine area into a regular grid, with each location corresponding to a unique grid code. Functional area division is also performed, dividing the entire area into different functional zones such as loading areas, transportation channels, unloading areas, and parking / waiting areas based on the mine's operational functions. Each functional zone is assigned a corresponding area identifier code, forming the mine environment features. This step, by structurally representing the mine environment information, enables an accurate understanding of the spatial environment and functional constraints of the mining vehicles, improving the adaptability and accuracy of trajectory prediction in specific environments.

[0037] Historical motion states are extracted from a standard multi-vehicle motion dataset. Motion parameters such as position coordinates, velocity, acceleration, and heading angle for each mining truck over a past period are extracted from the time-series data, forming a historical trajectory sequence for each truck. Simultaneously, load status is labeled. Based on the truck's load sensor data or operational task information, the load status of each truck at each moment is labeled, including different load levels such as empty, half-loaded, and fully loaded, as load status directly affects the truck's acceleration performance and braking distance, forming the vehicle's temporal characteristics. Subsequently, the vehicle's temporal characteristics are vectorized into motion parameter values. The time-series data containing multi-dimensional motion parameters such as position, velocity, and acceleration are converted into numerical vectors in a unified format. Data standardization ensures that parameters of different dimensions have the same numerical range, forming trajectory representation data. This step, by extracting and standardizing historical motion information, enables more accurate behavior prediction based on the vehicle's motion history and load status. In particular, considering load status significantly improves the accuracy of predictions in a mining environment.

[0038] A node relationship mapping is performed on the trajectory representation data and mine environment characteristics. Each mining truck is treated as a dynamic node, and static facilities such as loading and unloading points are treated as fixed nodes. The connection relationships between nodes are determined by calculating the spatial distance and functional correlation between nodes. Subsequently, an adjacency matrix is ​​constructed. The adjacency matrix is ​​a square matrix where rows and columns represent different nodes. The matrix elements indicate whether there is a connection relationship or connection strength between corresponding nodes. For node pairs that are close to each other or have interactive influence, corresponding connection weights are set in the adjacency matrix to form a dynamic interaction graph structure. Next, mining truck interaction features are extracted from the dynamic interaction graph structure. The interaction features such as relative position, relative speed, and movement trend between adjacent mining trucks are analyzed. At the same time, queuing features at loading points are extracted (referring to the data set describing the queuing status and time characteristics of mining trucks waiting for service at loading points), including information such as queuing length, waiting time, and service efficiency at each loading point. These features reflect the dynamic interaction status in the mining operation and form a contextual representation of the mining scene. This step integrates scattered vehicle and environmental information into a unified graph structure representation, enabling a comprehensive understanding of the complex interactions and operational constraints among multiple workshops in the mine, and providing a complete scenario understanding foundation for accurate collaborative prediction.

[0039] 103. Perform attention weight calculation and feature fusion on the task features in the autonomous vehicle planning intent information and the vehicle interaction features in the mine scene context representation to obtain the intent-enhanced scene representation. Then, construct trajectory query and intent query on the intent-enhanced scene representation to obtain multimodal query results.

[0040] In this embodiment, the autonomous vehicle planning intent information is decomposed into task feature vectors, and vehicle interaction features are extracted from the mine scene context representation to obtain interaction feature vectors. A query matrix is ​​constructed from the task feature vectors to obtain an intent query representation, and a key-value matrix is ​​constructed from the interaction feature vectors to obtain a scene key-value representation. The correlation between the intent query representation and the scene key-value representation is calculated to obtain an attention score matrix, and a weight distribution is calculated from the attention score matrix to obtain an attention weight matrix. The attention weight matrix and the interaction feature vectors are weighted and summed, and feature aggregation is performed to obtain weighted interaction features. The weighted interaction features and the task feature vectors are then fused to obtain an intent-enhanced scene representation. The process involves enhancing scene representation to identify mine movement patterns and classify trajectory types, resulting in trajectory modality configuration. It also involves enhancing intent scene representation to identify mine operation decision types and classify driving intent types, resulting in intent modality configuration. Query features are constructed from trajectory modality configuration to form a trajectory query tensor, and from intent modality configuration to form an intent query tensor. Mine-workshop adjacency relationships are constructed from the trajectory query tensor and intent query tensor, and operation area weights are calculated to obtain mine interaction weight distribution. This distribution is then aggregated with mine vehicle node information and fused with loading point information to obtain enhanced query features. Finally, cross-attention calculation and feature matching are performed between the enhanced query features and the intent-enhanced scene representation to obtain multimodal query results.

[0041] In practical applications, the autonomous vehicle planning intent information is decomposed into task features. This involves classifying and extracting the autonomous vehicle planning intent information, which includes task objectives, path planning, and time constraints, according to feature types. For example, from the planning of "going to loading point A to perform a loading task," target location features, task type features, and time features are extracted. These features are then converted into fixed-length numerical vectors through one-hot encoding and numerical processing, forming the task feature vector. Simultaneously, vehicle interaction features are extracted from the mine scene context representation. Interaction parameters such as distance difference, speed difference, and relative position angle between vehicles are calculated from the scene data. These parameters are combined into numerical vectors describing the interaction state between vehicles, forming the interaction feature vector. Next, a query matrix is ​​constructed from the task feature vector through a linear transformation operation. Specifically, the task feature vector is multiplied by a learnable transformation weight matrix. The initial values ​​of this transformation weight matrix are set using a random initialization method, and the matrix dimension is determined based on the length of the task feature vector and the length of the target query vector. During system training, the weight parameters in the matrix are continuously adjusted through a backpropagation algorithm to effectively convert the task features into a matrix form suitable for query operations, forming an intent query representation. Simultaneously, two different linear transformation operations are performed on the interaction feature vector to generate a key matrix and a value matrix, respectively. The key matrix is ​​used for matching calculations with the query matrix, and the value matrix contains the actual feature values, forming a scene key-value representation. This step, through standardized feature decomposition and matrix transformation operations, converts complex intent and scene information into a matrix form that is easy to process, enabling accurate numerical matching between autonomous vehicle planning intent and multi-vehicle interaction scenarios.

[0042] The process involves calculating the relevance between the intent query representation and the scene key-value representation. This is achieved by multiplying the query matrix with the transpose of the key matrix using matrix multiplication, resulting in a numerical matrix reflecting the degree of matching between the query content and the scene content. The value of each element in the matrix represents the strength of the association between the corresponding query and scene elements, forming an attention score matrix. Subsequently, a weight distribution calculation is performed on the attention score matrix. An exponential function is applied to each row of data, and all elements in each row are divided by the sum of the elements in that row, ensuring that the sum of the data in each row equals 1. This transforms the original relevance score into a standardized probability weight distribution, forming an attention weight matrix. For example, when a vehicle plans to go to a loading point, scene elements related to the loading point receive higher weight values, while elements related to other areas receive lower weight values. This step transforms abstract relationships into quantifiable weight allocations through precise numerical calculations, enabling the automatic identification of key influencing elements in the scene based on the vehicle's intent.

[0043] The attention weight matrix and interaction feature vectors are weighted and summed. Each weight value in the attention weight matrix is ​​multiplied by the corresponding element of the interaction feature vector, and then all weighted results are accumulated positionally to obtain an interaction feature representation that incorporates intent guidance information. Next, feature aggregation is performed, concatenating multiple weighted feature vectors according to their feature dimensions to merge the originally scattered feature information into a unified feature vector, forming a weighted interaction feature. Then, the weighted interaction feature and task feature vector are fused. This is done either by concatenating the two feature vectors end-to-end into a longer vector or by summing the values ​​at corresponding positions element-wise, forming a unified feature representation that simultaneously contains vehicle intent and scene interaction information—an intent-enhanced scene representation. This step achieves a deep integration of vehicle planning intent with scene understanding through specific numerical operations, thus fully considering the influence of vehicle intent when analyzing the behavior of other vehicles.

[0044] The intent-enhanced scene representation is used for mine movement pattern recognition. Based on set speed thresholds and position change patterns, the movement features in the scene representation are classified into loading, transportation, and unloading modes. Simultaneously, trajectory types are categorized based on geometric features, including basic types such as straight-line trajectories, left-turn trajectories, right-turn trajectories, and reversing trajectories. The recognition results are encoded into category labels and corresponding feature descriptions, forming a trajectory modality configuration. Simultaneously, the intent-enhanced scene representation is used for mine operation decision type recognition. Based on vehicle location, movement status, and task progress, the current operation stage is determined, identifying decision types such as waiting to load, executing loading, and transportation. Furthermore, based on features such as speed changes and steering behavior, driving intent types such as acceleration intent, deceleration intent, and lane-changing intent are identified, forming an intent modality configuration. Then, corresponding query features are constructed for each trajectory type in the trajectory modality configuration. The trajectory type identifier and feature parameters are converted into a three-dimensional tensor structure. The first dimension of the tensor represents different trajectory types, the second dimension represents the time series, and the third dimension represents the feature vector, forming a trajectory query tensor. The same tensor construction process is performed on the intent modality configuration, forming an intent query tensor. This step, through explicit classification criteria and tensor construction operations, enables the simultaneous handling of trajectory prediction and intent recognition tasks, improving the completeness and practicality of the prediction results.

[0045] Adjacency relationships between mine workshops are constructed using the trajectory query tensor and intent query tensor. The Euclidean distance between workshops is calculated, and a distance less than a set threshold is marked as 1 in the adjacency matrix to indicate an adjacency relationship; otherwise, it is marked as 0, forming an adjacency matrix describing the vehicle network connectivity. Simultaneously, work area weights are calculated, assigning weight coefficients based on vehicle density and interaction frequency within the mining area. Loading areas, due to higher vehicle concentration, are assigned a weight of 0.4, transport channels 0.3, and unloading areas 0.3, forming a mining area interaction weight distribution. The adjacency matrix is ​​then multiplied by the feature vectors of the corresponding vehicle nodes to aggregate mine vehicle node information, and the feature information of adjacent vehicles is weighted and averaged according to their connectivity. Furthermore, the coordinates, service capacity, and current queue length of loading points are encoded into feature vectors, which are then concatenated and fused with the vehicle feature vectors to complete loading point information fusion, forming an enhanced query feature containing both dynamic vehicle information and static facility information. This step, through a clear distance calculation and weight allocation mechanism, constructs a complete vehicle association network and a regional importance evaluation system.

[0046] Cross-attention calculation is performed on enhanced query features and intent-enhanced scene representations. Enhanced query features are used as query input, and intent-enhanced scene representations are used as key-value inputs. The dot product score between the query and the key is calculated through matrix multiplication, followed by scaling and softmax normalization to obtain the attention weight distribution. Feature matching is then performed, multiplying the calculated attention weights by the value matrix. The most relevant scene features are selected and combined according to their weights, prioritizing features with higher attention weights in the final result to form a multimodal query result. For example, when querying "intent to queue at the loading point for mining trucks," the focus is on information related to vehicle distribution, queue length, and service efficiency near the loading point, while information far from the loading point is assigned lower weights. This step, through specific matrix operations and weight matching, achieves a precise correspondence between query requirements and scene information, significantly improving the targeting and accuracy of multi-vehicle collaborative prediction.

[0047] 104. Perform multi-vehicle collaborative prediction on the multimodal query results to obtain joint prediction results, and perform multi-mine truck path conflict detection and collaborative optimization scheduling on the joint prediction results to generate a multi-vehicle collaborative operation plan.

[0048] In this embodiment, the multimodal query results are divided into multi-branch channels and feature representations are assigned to obtain trajectory query data, intent query data, and planning query data. The multi-branch channels include trajectory prediction channels, intent decision channels, and planning coordination channels. Trajectory query vector extraction and position / velocity prediction calculations are performed on the trajectory query data to obtain trajectory prediction output. The trajectory prediction output is then used to generate trajectory coordinates for future time steps and estimate motion states to obtain mine truck trajectory prediction data. Intent query vector extraction and driving behavior classification are performed on the intent query data to obtain intent classification output. The probability distribution of various driving intent combinations is calculated on the intent classification output to obtain mine truck intent decision data. The planning query data is then compared with the predicted trajectories of other vehicles to obtain… The process involves obtaining consistency verification results, identifying trajectory conflicts and detecting logical contradictions to arrive at a coordination requirement assessment. Matching and verifying the logical relationships between mine truck trajectory prediction data and mine truck intention decision data yields a coordination prediction result. This result is then fused with the coordination requirement assessment to obtain a joint prediction result. The joint prediction result is used to construct a global scene map of the target mine, resulting in a multi-vehicle interaction scene map. The vehicle occupancy probability is calculated using a spatiotemporal network on the multi-vehicle interaction scene map to obtain a spatiotemporal occupancy distribution. Conflict areas are identified within the spatiotemporal occupancy distribution, resulting in a conflict area distribution. Conflict types are then identified within the conflict area distribution to obtain a conflict feature description. Finally, vehicle collaborative scheduling optimization is performed based on the conflict feature description to generate a multi-vehicle collaborative operation scheme. In addition, after performing multi-vehicle path conflict detection and collaborative optimization scheduling on the joint prediction results to generate a multi-vehicle collaborative operation scheme, the process also includes: collecting operation monitoring data corresponding to the multi-vehicle collaborative execution of the multi-vehicle collaborative operation scheme, calculating prediction accuracy and operation efficiency on the operation monitoring data to obtain basic performance data, and conducting a mine vehicle operation safety assessment on the operation monitoring data to obtain safety assessment data; evaluating the prediction accuracy of the mine vehicle operation trajectory on the basic performance data to obtain accuracy assessment data, and evaluating the operation efficiency on the basic performance data to obtain efficiency assessment data; weighting and synthesizing the accuracy assessment data, safety assessment data, and efficiency assessment data on mine vehicle operation indicators to obtain a comprehensive performance assessment result; comparing the comprehensive performance assessment result with thresholds and classifying it into levels to obtain a performance level rating, and detecting whether the performance level corresponding to the performance level rating is greater than a preset level threshold; if the performance level corresponding to the performance level rating is less than the preset level threshold, then updating the parameters corresponding to the multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion.It also includes: collecting the actual execution results of multi-mine truck operation corresponding to the multi-vehicle collaborative operation scheme, comparing the actual execution results with the joint prediction results to obtain prediction deviation data; classifying and statistically analyzing the prediction deviation data for trajectory deviation and intention deviation to obtain deviation classification data, quantifying the degree of deviation and locating the source of deviation in the deviation classification data to obtain deviation distribution characteristics; adjusting the weight allocation ratio of the fusion of task features and vehicle interaction features based on the deviation distribution characteristics to obtain optimized weight configuration, and adjusting the query vector dimension and scheduling strategy weight of multi-branch channels to obtain optimized channel configuration, and updating the multi-vehicle joint trajectory prediction method based on the optimized weight configuration and optimized channel configuration.

[0049] In practical applications, multimodal query results are divided into multiple branches (i.e., independent computational paths for processing different types of prediction tasks in parallel). Query results are categorized according to functional attributes based on the different types of prediction tasks. Specifically, feature representation allocation is achieved by setting up three independent processing channels: a trajectory prediction channel specifically handles query information related to the vehicle's future position and trajectory; an intent decision channel handles query information related to driving behavior and operational intent; and a planning and coordination channel handles query information related to the consistency and coordination requirements of autonomous vehicle planning. Through index mapping and feature filtering operations, different types of features in the multimodal query results are assigned to their corresponding channels, resulting in trajectory query data, intent query data, and planning query data. This step, through specialized channel division, achieves parallel processing of multi-task prediction, avoids feature interference between different prediction tasks, and improves the specialization and accuracy of each prediction task.

[0050] The process involves extracting trajectory query vectors from the trajectory query data. A fully connected layer transforms the query data into a fixed-dimensional feature vector. Position and velocity prediction are then calculated using a regression network, which processes the extracted feature vectors to output a prediction result containing future position coordinates and velocity values, thus obtaining the trajectory prediction output. Next, the trajectory prediction output is used to generate trajectory coordinates for future time steps. The predicted position and velocity information is unfolded into a time series at preset time intervals, generating the coordinates of the mining truck at each future moment. Simultaneously, motion state estimation is performed, calculating motion parameters such as acceleration and steering angular velocity based on position changes at adjacent time points to form a complete motion state description, resulting in the mining truck trajectory prediction data. For example, for a mining truck heading to the loading point, the specific coordinates, speed, and steering angle for each second over the next 10 seconds are predicted. This step, through regression prediction and time series unfolding, transforms abstract query information into specific trajectory coordinates and motion parameters, achieving accurate quantitative prediction of the mining truck's future travel path.

[0051] The intent query data is processed by extracting intent query vectors. The query data is then transformed into vectors representing driving intentions using a feature extraction network. Driving behavior is then classified using a multi-classifier, which processes the intent vectors and outputs the classification probabilities for various driving behavior types, resulting in intent classification outputs. Next, the probability distributions of various combinations of driving intentions are calculated on the intent classification outputs. Considering the complex nature of driving intentions in mining operations, basic intent types are combined for calculation. For example, the probability distribution of a composite intent is calculated by combining "acceleration intent" with "heading to loading point intent." Probability multiplication and normalization are then used to obtain the probability distributions of various intent combinations, forming the mine truck intent decision data. A driving intent combination refers to a composite behavioral intent formed when multiple basic driving intentions exist simultaneously. For example, when a mine truck approaches a loading point, the output will show a combined prediction result with a probability of 0.7 for "deceleration intent," 0.6 for "right turn intent," and 0.8 for "preparing to load intent." This step, through classification prediction and probability combination calculation, achieves accurate identification and quantitative representation of complex operational intentions of mine trucks.

[0052] The process involves extracting the planned trajectory of the autonomous vehicle from the planning query data, separating its predetermined driving path and work plan information, and simultaneously extracting the predicted trajectory information of other vehicles. Through trajectory comparison analysis, the degree of overlap and deviation between the autonomous vehicle's planned trajectory and the predicted trajectories of other vehicles in the spatiotemporal dimensions is calculated to obtain consistency verification results. Subsequently, trajectory conflict identification is performed on the consistency verification results. Potential path conflicts are identified by calculating the intersection points and temporal overlap of different vehicle trajectories. Logical contradiction detection is also performed to check for contradictions in the prediction results that violate traffic rules or physical constraints, such as two vehicles predicting to occupy the same location at the same time, forming a coordination demand assessment. Consistency verification refers to the process of verifying the rationality of the logical relationship between different prediction results. This step, through comparative analysis and conflict detection, identifies inconsistencies and potential conflicts in multi-vehicle predictions, ensuring the logical rationality and practical feasibility of the prediction results.

[0053] Matching mine truck trajectory prediction data and mine truck intention decision data involves matching the predicted driving trajectory with the identified driving intention. For example, it checks whether a "left turn intention" corresponds to a "trajectory for turning left." The matching degree is evaluated by calculating the correlation between trajectory features and intention features. Simultaneously, logical relationship verification is performed to check for logical conflicts between trajectory prediction and intention decision, such as the contradiction of simultaneously predicting an "accelerate" trajectory and a "stop" intention. A coordinated prediction result is obtained through logical rule checks and consistency verification. Subsequently, the coordinated prediction result is integrated with the coordination requirement assessment. The two types of information are combined using a weighted average or priority selection method to form a result that satisfies both trajectory-intention... Figure 1The prediction results are then combined to achieve consistency and to meet the coordination needs of multiple vehicles. This step, through matching verification and logical validation, eliminates internal inconsistencies in the prediction results, ensuring the consistency and reliability of multi-vehicle collaborative prediction.

[0054] The joint prediction results are used to construct a global scene map of the target mine. The predicted trajectories, intent information, and mine environment information of all mining vehicles are integrated into a unified spatial coordinate system, forming a complete scene representation that includes both vehicle dynamics and environmental statics, resulting in a multi-vehicle interaction scene map. Subsequently, the vehicle occupancy probability is calculated using a spatiotemporal grid on the multi-vehicle interaction scene map. The mine space is divided into grid cells of fixed size, and the probability of each grid being occupied by a vehicle at different times is calculated. By statistically analyzing the frequency and temporal distribution of predicted trajectories passing through each grid, a probability distribution map is formed, yielding the spatiotemporal occupancy distribution. For example, if the mine is divided into a 5m × 5m spatial grid, and the vehicle occupancy probability of each grid is calculated at each 1-second time interval, this step, through scene integration and probability calculation, achieves a globally unified representation and quantitative analysis of the multi-vehicle movement state.

[0055] Conflict zone identification is performed on the spatiotemporal occupancy distribution. By setting an occupancy probability threshold, a high-risk area is marked when the occupancy probability of a vehicle in a spatiotemporal grid exceeds the threshold. When the occupancy probabilities of multiple vehicles overlap in the same spatiotemporal grid, a potential conflict area is identified, resulting in the conflict area distribution. Subsequently, conflict type identification is performed on the conflict area distribution. Based on the spatial location and temporal characteristics of the conflict areas, the specific type of conflict is analyzed, including different types such as frontal conflict, side conflict, and rear-end collision. The severity and impact range of the conflict are also identified, forming a structured description containing information such as conflict location, time, type, and severity, resulting in a conflict characteristic description. For example, at the loading point entrance, a side conflict risk is identified between two mining trucks within the next 30 seconds, with a conflict probability of 0.75. This step, through threshold judgment and type analysis, accurately identifies safety risk points and conflict characteristics in multi-vehicle operation.

[0056] Vehicle collaborative scheduling optimization is performed based on conflict characteristic descriptions. Corresponding scheduling strategies are formulated according to the identified conflict types and severity, adjusting vehicle speeds, route selections, or work sequences to avoid predicted conflicts. Specifically, a priority allocation method is used, determining passage priorities based on vehicle load status, task urgency, and location. High-priority vehicles maintain their original trajectories, while low-priority vehicles are adjusted to avoid them. Simultaneously, path replanning technology is employed to recalculate alternative routes for vehicles at risk of conflict, ensuring that the adjusted routes both avoid conflicts and efficiently complete the work tasks. Ultimately, a complete scheme is generated, including the adjusted travel trajectories of each vehicle, speed control commands, and work sequence arrangements, creating a multi-vehicle collaborative operation plan. This step, through intelligent scheduling and path optimization, transforms conflict warnings into specific coordinated operation commands, achieving safe and efficient collaborative operation of multiple vehicles in the mine.

[0057] After generating a multi-vehicle collaborative operation scheme, to further improve the collaborative operation capability of the multi-vehicle joint trajectory prediction method, operational monitoring data corresponding to the multi-vehicle collaborative execution scheme can be collected. This involves recording the actual position coordinates of the mining trucks every second using onboard GPS devices, collecting real-time driving speeds using vehicle speed sensors, and recording the start and end times of loading and unloading operations using onboard timers. Simultaneously, the video monitoring system at the mine monitoring center observes and records vehicle meeting situations, queue lengths, and other collaborative status information, storing all data synchronously as structured data files with timestamps. Then, the collected operational monitoring data is used to calculate prediction accuracy. Specifically, the predicted trajectory coordinates are mapped one-to-one with the actual trajectory coordinates at the same time, and the Euclidean distance of the position coordinates at each time moment is calculated as the position error. The average position error across all time moments is then calculated as the trajectory prediction accuracy index. Furthermore, operational efficiency is calculated by statistically analyzing the complete cycle time of each mining truck from leaving the loading point to returning to the loading point, calculating the average cycle time as a single-vehicle efficiency index, and statistically analyzing the number of service vehicles and idle time at the loading point to calculate equipment utilization, forming basic performance data. Simultaneously, the operation monitoring data is used to conduct a safety assessment of mining truck operation, measuring the minimum distance between adjacent vehicles. When the distance is less than the safety threshold, it is counted as a safety event. The number of safety events occurring per unit time is counted as a safety risk indicator to obtain safety assessment data.

[0058] The accuracy of mine car trajectory prediction is evaluated based on the performance baseline data. Key path nodes such as loading points, unloading points, and major turning points are selected, and the time deviation between predicted and actual arrival times and the spatial deviation between predicted and actual positions are calculated. The time deviation is obtained by subtracting the actual time from the predicted time, and the spatial deviation is obtained by calculating the straight-line distance between the predicted and actual coordinates. Statistical analysis is performed on the deviation values ​​of all key nodes to calculate the average deviation, maximum deviation, and standard deviation, forming the accuracy evaluation data. Simultaneously, the operational efficiency is evaluated based on the performance baseline data. The average cycle time after using the prediction method is compared with the average cycle time before use, and the percentage reduction in time is calculated as an efficiency improvement indicator. The utilization rate of loading points after using the prediction method is compared with the utilization rate before use, and the percentage increase in utilization rate is calculated. Multiple efficiency indicators are comprehensively calculated to obtain an overall efficiency improvement score, forming the efficiency evaluation data. For example, the original average cycle time of a mine car was 45 minutes, which was reduced to 40 minutes after using the prediction method, resulting in an efficiency improvement of 11.1%. This step accurately quantifies the performance of the prediction method in practical applications.

[0059] The accuracy assessment data, safety assessment data, and efficiency assessment data are weighted and synthesized using mining truck operation indicators. First, each assessment data point is standardized, converting indicators of different dimensions into standard scores from 0 to 100. The safety assessment score is multiplied by a weighting coefficient of 0.4, the accuracy assessment score by 0.35, and the efficiency assessment score by 0.25. The three weighted results are then summed to obtain a comprehensive score, forming the overall performance assessment result. The weighting coefficients are determined based on the actual needs of mining operations; safety has the highest weight because the losses from safety accidents far outweigh the efficiency losses, while accuracy has the second highest weight because prediction accuracy directly affects the reliability of scheduling decisions. This step, through quantitative weight allocation and mathematical calculations, transforms multi-dimensional performance indicators into a single comprehensive evaluation value.

[0060] The comprehensive performance evaluation results are compared against thresholds and classified into levels, with specific level classification standards set: a comprehensive score of 90-100 is excellent, 80-89 is good, 70-79 is acceptable, and 69 and below is unacceptable. The calculated comprehensive score is compared with these numerical ranges to determine the corresponding performance level, forming a performance level assessment. Then, it is checked whether the performance level corresponding to the assessment is greater than a preset level threshold, which is set to good (80 points). When the comprehensive score is below 80, a parameter update process is initiated, for example: adjusting the weight coefficient of the vehicle's planning intent in feature fusion by increasing it by 0.1 from the current value; increasing the length of the historical data time window for trajectory prediction by 1 second from the current length; and adjusting the feature vector dimension of each channel in the multi-branch channels according to the prediction accuracy performance of each channel. After the parameter update, the prediction process is re-executed and a performance evaluation is performed until the comprehensive score reaches or exceeds the preset threshold. This step, through clear threshold standards and specific parameter adjustment rules, achieves automatic optimization and performance assurance of the prediction method.

[0061] Furthermore, after generating the multi-vehicle collaborative operation scheme, to further improve the collaborative operation capability of the multi-vehicle joint trajectory prediction method, the actual execution results of the multi-mine truck operation corresponding to the multi-vehicle collaborative operation scheme can be collected. This involves using GPS positioning devices installed on the mine trucks to record the vehicle's real coordinate position every second, using vehicle speed sensors to collect the actual driving speed, and using operation time recorders to record the actual start and end times of loading and unloading. This data is then organized and stored according to vehicle number and timestamp. The actual execution results are compared with the joint prediction results. Specifically, the predicted and actual data are matched according to vehicle number and aligned according to timestamp, and then the differences are calculated item by item. Position deviation is obtained by calculating the Euclidean distance between the predicted and actual coordinates; speed deviation is calculated by subtracting the actual speed from the predicted speed; and time deviation is calculated by subtracting the actual arrival time from the predicted arrival time. All difference data are summarized to form the prediction deviation data. For example, the predicted position of mine truck 001 at 10:30:00 is (1200, 800), and the actual position is (1205, 795), resulting in a calculated position deviation of 7.07 meters. This step, through standardized data collection and precise numerical calculations, yielded quantitative deviation information between the prediction method and actual operation.

[0062] The prediction deviation data is statistically classified into trajectory deviation and intention deviation, with clear classification criteria: position deviation, speed deviation, and trajectory shape deviation are categorized into trajectory deviation, while driving behavior prediction errors and operational intention recognition errors are categorized into intention deviation. Statistical analysis is conducted using set thresholds: position deviations greater than 5 meters and speed deviations greater than 3 kilometers per hour are classified as trajectory deviation events, and discrepancies between predicted and actual behavior are classified as intention deviation events. The total number and frequency of each type of deviation event are statistically analyzed to obtain deviation classification data. The degree of deviation in the classification data is quantified by calculating the average value, maximum value, and number of times deviations exceed the threshold, as well as the false recognition rate and correct recognition rate of intention deviations. Simultaneously, the source of deviation is located by analyzing the environmental conditions at the time of deviation occurrence, identifying high-deviation areas and high-deviation periods. For example, the average trajectory deviation within a 50-meter radius of the loading point is found to be 8.2 meters, significantly higher than the 3.1 meters in other areas, identifying the loading point area as a high-deviation area and forming a deviation distribution characteristic. This step, through quantitative classification statistics and regional analysis, accurately identifies the distribution pattern and main influencing factors of prediction deviation.

[0063] Based on the deviation distribution characteristics, the weight allocation ratio of the fusion of task features and vehicle interaction features is adjusted, and specific adjustment rules are formulated: when trajectory deviation accounts for more than 70% of the total deviation, it indicates insufficient vehicle interaction prediction, so the weight of vehicle interaction features is increased by 0.05, and the weight of task features is correspondingly decreased by 0.05; when intent deviation accounts for more than 70% of the total deviation, it indicates inaccurate understanding of the task, so the weight of task features is increased by 0.05, and the weight of vehicle interaction features is correspondingly decreased by 0.05. For example, when the analysis finds that trajectory deviation accounts for 75% of the total deviation, the weight of vehicle interaction features is adjusted from 0.4 to 0.45, and the weight of task features is adjusted from 0.6 to 0.55, resulting in an optimized weight configuration. At the same time, the query vector dimension of multi-branch channels is finely adjusted. According to the performance of each channel in specific deviation types, when trajectory deviation is large, the query vector dimension of the trajectory prediction channel is increased by 25%, and when intent deviation is large, the query vector dimension of the intent decision channel is increased by 25%, rather than adjusting the scheduling strategy weight. For example, when trajectory deviation is dominant, the query vector dimension of the trajectory prediction channel is increased from 128 to 160 dimensions, forming an optimized channel configuration. Based on the optimized weight and channel configurations, the multi-vehicle joint trajectory prediction method is updated. By modifying the weight parameters of the feature fusion module and the dimension configuration of the query vector, the prediction accuracy of the method is recalibrated. This step, through targeted deviation feedback adjustments, achieves precise optimization of the prediction method, forming a complementary optimization mechanism with the overall adjustment based on comprehensive performance.

[0064] In this embodiment of the invention, the accuracy problem of multi-vehicle trajectory prediction in mining farms is solved by hierarchical intent fusion and collaborative prediction. In particular, in terms of vehicle planning intent fusion, vehicle interaction modeling, and collaborative decision-making, the unique characteristics and dynamics of mining operations are fully considered, thereby improving the accuracy of trajectory prediction. Furthermore, the adoption of multimodal query fusion and collaborative optimization strategies not only realizes the correlation calculation between intent recognition, trajectory prediction, and conflict detection, but also enhances the reliability of collaborative decision-making. In addition, through attention mechanisms and multi-branch prediction, vehicle behavioral intent and running trajectory are better identified, thereby achieving intelligent and accurate prediction of multi-vehicle collaborative operations in mining farms as a whole.

[0065] The above describes the multi-vehicle joint trajectory prediction method based on vehicle fusion in the embodiments of the present invention. The following describes the multi-vehicle joint trajectory prediction device based on vehicle fusion in the embodiments of the present invention. Please refer to [link to relevant documentation]. Figure 2 One embodiment of the multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion in this invention includes:

[0066] The data preprocessing module 201 is used to collect multi-vehicle status data and mine environment data in the target mine, and to preprocess and calibrate the lane-changing behavior of the multi-vehicle status data and mine environment data to obtain a standard multi-vehicle motion dataset, wherein the multi-vehicle status data includes the vehicle's own driving data and the status data of surrounding vehicles.

[0067] The feature extraction module 202 is used to extract the intention of the vehicle planning trajectory and task information in the standard multi-vehicle motion dataset to obtain the vehicle planning intention information, and to extract the mine environment features and construct the vehicle dynamic distribution in the standard multi-vehicle motion dataset to obtain the mine scene context representation.

[0068] The intent fusion module 203 is used to calculate attention weights and fuse features between the task features in the autonomous vehicle planning intent information and the vehicle interaction features in the mine scene context representation to obtain an intent-enhanced scene representation, and to construct trajectory query and intent query on the intent-enhanced scene representation to obtain multimodal query results.

[0069] The collaborative prediction module 204 is used to perform multi-vehicle collaborative prediction on the multimodal query results to obtain joint prediction results, and to perform multi-mine truck path conflict detection and collaborative optimization scheduling on the joint prediction results to generate a multi-vehicle collaborative operation scheme.

[0070] In this embodiment of the invention, the accuracy problem of multi-vehicle trajectory prediction in mining farms is solved by hierarchical intent fusion and collaborative prediction. In particular, in terms of vehicle planning intent fusion, vehicle interaction modeling, and collaborative decision-making, the unique characteristics and dynamics of mining operations are fully considered, thereby improving the accuracy of trajectory prediction. Furthermore, the adoption of multimodal query fusion and collaborative optimization strategies not only realizes the correlation calculation between intent recognition, trajectory prediction, and conflict detection, but also enhances the reliability of collaborative decision-making. In addition, through attention mechanisms and multi-branch prediction, vehicle behavioral intent and running trajectory are better identified, thereby achieving intelligent and accurate prediction of multi-vehicle collaborative operations in mining farms as a whole.

[0071] above Figure 2 The multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0072] Figure 3 This is a schematic diagram of the structure of a multi-vehicle joint trajectory prediction device based on vehicle fusion according to an embodiment of the present invention. The multi-vehicle joint trajectory prediction device 300 based on vehicle fusion can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the multi-vehicle joint trajectory prediction device 300 based on vehicle fusion. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the multi-vehicle joint trajectory prediction device 300 based on vehicle fusion.

[0073] The multi-vehicle joint trajectory prediction device 300 based on autonomous vehicle fusion may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3The illustrated structure of the multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion does not constitute a limitation on the multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0074] The present invention also provides a multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion. The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs each step of the multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion in the above embodiments.

[0075] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform each step of the multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion.

[0076] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0077] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0078] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0079] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion, characterized in that, The multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion includes: Collect multi-vehicle status data and mine environment data in the target mine, and preprocess and calibrate the multi-vehicle status data and mine environment data to obtain a standard multi-vehicle motion dataset, wherein the multi-vehicle status data includes the vehicle's own driving data and the status data of surrounding vehicles. Intent extraction is performed on the autonomous vehicle planning trajectory and task information in the standard multi-vehicle motion dataset to obtain autonomous vehicle planning intent information. In addition, mining environment features are extracted and vehicle dynamic distribution is constructed on the standard multi-vehicle motion dataset to obtain mining scene context representation. Attention weights are calculated and features are fused between the task features in the autonomous vehicle planning intent information and the vehicle interaction features in the mine scene context representation to obtain an intent-enhanced scene representation. Trajectory queries and intent queries are then constructed based on the intent-enhanced scene representation to obtain multimodal query results. The multi-vehicle collaborative prediction is performed on the multi-modal query results to obtain a joint prediction result. Then, the multi-mine truck path conflict detection and collaborative optimization scheduling are performed on the joint prediction result to generate a multi-vehicle collaborative operation scheme.

2. The multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion according to claim 1, characterized in that, The process involves collecting multi-vehicle status data and mine environment data from the target mine, preprocessing the multi-vehicle status data and mine environment data, and calibrating lane-changing behavior to obtain a standard multi-vehicle motion dataset. The multi-vehicle status data includes the vehicle's own driving data and surrounding vehicle status data, including: The system collects various driving data of its own mining truck in the target mine to obtain its own driving data, and collects multi-dimensional motion state data of the mining trucks around it to obtain the surrounding vehicle state data, and collects various static environmental information of the target mine to obtain the mine environment data. The vehicle's driving data and the surrounding vehicle status data are subjected to median filtering and weighted average filtering to obtain filtered motion data. The missing values ​​in the filtered motion data are then compensated by linear interpolation to obtain complete motion data. Lane change behavior detection is performed on the complete motion data to obtain lane change annotation data. The lane change annotation data and the mine environment data are then transformed using the mine global coordinate system to obtain a standard multi-vehicle motion dataset.

3. The multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion according to claim 2, characterized in that, The autonomous vehicle driving data includes the autonomous vehicle's planned trajectory and task information. The intention extraction process, which extracts the autonomous vehicle's planned trajectory and task information from the standard multi-vehicle motion dataset to obtain autonomous vehicle planning intention information, includes: The path point sequence is extracted and geometric features are calculated from the planned trajectory of the vehicle in the standard multi-vehicle motion dataset to obtain trajectory geometric feature data. Then, the trajectory geometric feature data is fitted with the expected speed curve and the key path node is identified to obtain trajectory motion feature data. The task information in the standard multi-vehicle motion dataset is hierarchically decomposed into mining operation process to obtain task planning data, and the task planning data is used to identify mining vehicle motion patterns to obtain motion pattern data. The task planning data and the motion mode data are labeled with various preset driving intention combinations to obtain intention labeling data. The intention labeling data is then encoded with intention features by a multi-layer embedding network to obtain vehicle planning intention information.

4. The multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion according to claim 1, characterized in that, The step of extracting mining environment features and constructing vehicle dynamic distribution from the standard multi-vehicle motion dataset to obtain a mining scene context representation includes: The road network topology and loading / unloading point extraction are performed on the mine environment data in the standard multi-vehicle motion dataset to obtain static environmental features. The static environmental features are then spatially encoded and functionally divided to obtain the mine environment features. Historical motion state extraction and load state labeling are performed on the standard multi-vehicle motion dataset to obtain vehicle temporal features. The motion parameter values ​​of the vehicle temporal features are then vectorized to obtain trajectory representation data. The trajectory representation data and the mine environment features are mapped to nodes and an adjacency matrix is ​​constructed to obtain a dynamic interaction graph structure. The mine truck interaction features and loading point queuing features are extracted from the dynamic interaction graph structure to obtain a mine scene context representation.

5. The multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion according to claim 1, characterized in that, The process of calculating attention weights and fusing features between the task features in the autonomous vehicle planning intent information and the vehicle interaction features in the mine scene context representation to obtain an intent-enhanced scene representation includes: The autonomous vehicle planning intent information is decomposed into task feature vectors, and the mine scene context representation is extracted into interaction features to obtain interaction feature vectors. The task feature vectors are used to construct a query matrix to obtain intent query representations, and the interaction feature vectors are used to construct a key-value matrix to obtain scene key-value representations. The relevance of the intent query representation and the scene key-value representation is calculated to obtain an attention score matrix, and the weight distribution of the attention score matrix is ​​calculated to obtain an attention weight matrix. The attention weight matrix and the interaction feature vector are weighted and summed and feature aggregated to obtain weighted interaction features. The weighted interaction features are then fused with the task feature vector to obtain an intent-enhanced scene representation.

6. The multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion according to claim 1, characterized in that, The construction of trajectory query and intent query on the intent-enhanced scene representation to obtain multimodal query results includes: The intent-enhanced scene representation is subjected to mine movement pattern recognition and trajectory type classification to obtain trajectory modality configuration. The intent-enhanced scene representation is subjected to mine operation decision type recognition and driving intent type classification to obtain intent modality configuration. Query features are constructed on the trajectory modality configuration to form a trajectory query tensor. Query features are also constructed on the intent modality configuration to form an intent query tensor. The mine workshop adjacency relationship is constructed and the operation area weight is calculated for the trajectory query tensor and the intent query tensor to obtain the mine interaction weight distribution. Then, the mine vehicle node information and loading point information are aggregated and fused for the mine interaction weight distribution to obtain enhanced query features. Cross-attention calculation and feature matching are performed on the enhanced query features and the intent-enhanced scene representation to obtain multimodal query results.

7. The multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion according to claim 1, characterized in that, The step of performing multi-vehicle collaborative prediction on the multimodal query results to obtain joint prediction results includes: The multimodal query results are divided into multi-branch channels and feature representations are assigned to obtain trajectory query data, intent query data and planning query data. The multi-branch channels include trajectory prediction channel, intent decision channel and planning coordination channel. Trajectory query vector extraction and position velocity prediction calculation are performed on the trajectory query data to obtain trajectory prediction output. Then, trajectory coordinates and motion state estimation are performed on the trajectory prediction output for future time steps to obtain mine car trajectory prediction data. Intent query vector extraction and driving behavior classification are performed on the intent query data to obtain intent classification output. Probability distribution calculation of multiple driving intent combinations is then performed on the intent classification output to obtain mining truck intent decision data. The planned trajectory of the vehicle is extracted from the planning query data and compared with the predicted trajectory of other vehicles to obtain the consistency test result. The consistency test result is then used to identify trajectory conflicts and detect logical contradictions to obtain the coordination requirement assessment. The vehicle trajectory prediction data and the vehicle intention decision data are matched and their logical relationships are verified to obtain a coordinated prediction result. The coordinated prediction result and the coordinated demand assessment are then fused to obtain a joint prediction result.

8. A multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion, characterized in that, The multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion includes: The data preprocessing module is used to collect multi-vehicle status data and mine environment data in the target mine, and to preprocess and calibrate the lane-changing behavior of the multi-vehicle status data and mine environment data to obtain a standard multi-vehicle motion dataset. The multi-vehicle status data includes the vehicle's own driving data and the status data of surrounding vehicles. The feature extraction module is used to extract the intention of the vehicle planning trajectory and task information in the standard multi-vehicle motion dataset to obtain the vehicle planning intention information, and to extract the mining environment features and construct the dynamic distribution of vehicles in the standard multi-vehicle motion dataset to obtain the mining scene context representation. The intent fusion module is used to calculate attention weights and fuse features between the task features in the autonomous vehicle planning intent information and the vehicle interaction features in the mine scene context representation to obtain an intent-enhanced scene representation. The module then constructs trajectory queries and intent queries based on the intent-enhanced scene representation to obtain multimodal query results. The collaborative prediction module is used to perform multi-vehicle collaborative prediction on the multimodal query results to obtain joint prediction results, and to perform multi-mine truck path conflict detection and collaborative optimization scheduling on the joint prediction results to generate a multi-vehicle collaborative operation scheme.

9. A multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion, characterized in that, The multi-vehicle joint trajectory prediction device based on autonomous vehicle fusion includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the multi-vehicle joint trajectory prediction device based on vehicle fusion to perform the various steps of the multi-vehicle joint trajectory prediction method based on vehicle fusion as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the multi-vehicle joint trajectory prediction method based on autonomous vehicle fusion as described in any one of claims 1-7.

Images