Mining unmanned transport vehicle collision prediction method based on adaptive expansion

Abstract

The invention relates to a mining unmanned transport vehicle collision prediction method based on adaptive expansion, belongs to the technical field of mining unmanned transport vehicle collision prediction, and solves the problems of insufficient accuracy and high calculation complexity in the prior art. Obtaining a driving scene recognition data set and a trajectory prediction data set, and further performing normalization and sliding window processing to obtain a driving scene recognition model training data set and a trajectory model training data set; s2, a track prediction module constructs a driving scene recognition model and a track prediction model, training of the driving scene recognition model and the track prediction model is completed by adopting the training data set in the step S1, track prediction is executed, and a predicted track is obtained; and S3, the collision prediction module performs collision prediction judgment based on the prediction trajectory to obtain a judgment result and outputs the judgment result.

Description

Technical Field

[0001] The invention relates to the technical field of collision prediction of unmanned mining transport vehicles, and in particular to a collision prediction method of unmanned mining transport vehicles based on adaptive expansion. Background Art

[0002] The demand for unmanned driving in closed scenarios such as open-pit mines is increasing. Compared with manned driving, unmanned driving has the advantages of higher safety and more scientific transportation management. In addition, the rapid development of unmanned driving technology in the mining field can greatly improve the production efficiency of equipment and reduce the loss caused by improper human operation. However, there is still considerable resistance to the development of unmanned driving in mining areas. Due to the complexity and harshness of the production environment in open-pit mines, unmanned transport vehicles for mining may collide with obstacles such as stones, earth piles, and cardboard boxes during driving, or collide with retaining walls when deviating from the road, causing damage to the vehicle. Therefore, if you want to achieve full intelligence in open-pit mines, you still need to focus on overcoming the problem of collision prediction when unmanned transport vehicles for mining are driving on unstructured roads in mining areas.

[0003] The existing content related to unmanned vehicle collisions is mainly divided into collision detection and collision prediction. Collision detection refers to the process of identifying whether a collision has occurred or is about to occur between objects at the current moment. It is usually based on geometric methods, physical methods, graphical methods, etc., and is judged by detecting the distance and intersection between objects; collision prediction refers to the process of predicting whether a collision will occur between moving objects in a certain period of time in the future. It is based on motion models, trajectory prediction, probability models, learning methods, etc., and evaluates the collision risk by predicting the future motion trajectory of the object.

[0004] Collision detection is very effective in simple environments because of its real-time and high reliability, but it cannot provide forward-looking warnings, especially in complex environments, where it has poor adaptability. Therefore, it is not suitable for the complex environment of open-pit mines. Collision prediction helps unmanned transport vehicles take preventive measures in advance by providing forward-looking information, thereby improving overall safety and decision-making quality. However, the complexity of risk assessment and reliance on data in collision prediction greatly increase the difficulty of implementing the system, and also place high demands on computing resources.

[0005] Therefore, there is a need in the art for a collision prediction method for unmanned mining transport vehicles that can provide high accuracy and low computational complexity. Summary of the invention

[0006] In order to solve the above problems, a collision prediction method for unmanned mine transport vehicles based on adaptive expansion is provided according to an embodiment of the present invention. Specifically, considering the daily driving stability requirements of unmanned mine transport vehicles, combined with the complex structure of unmanned mine transport vehicles and the road environment of open-pit mines, a collision prediction method for unmanned mine transport vehicles based on adaptive expansion is proposed. When a vehicle collision trend is detected, an early warning is given to the downstream modules of the system to enhance the safety and reliability of daily operations of unmanned mine transport vehicles.

[0007] According to one embodiment of the present invention, a method for predicting a collision of an unmanned mining transport vehicle based on adaptive expansion is provided, comprising the following steps:

[0008] Step S1, the data preprocessing module performs data collection and preprocessing to obtain a driving scene recognition data set and a trajectory prediction data set, and further performs normalization and sliding window processing to obtain a driving scene recognition model training data set and a trajectory model training data set;

[0009] Step S2, the trajectory prediction module constructs a driving scene recognition model and a trajectory prediction model, uses the training data set in step S1 to complete the training of the driving scene recognition model and the trajectory prediction model, performs trajectory prediction, and obtains a predicted trajectory;

[0010] Step S3, the collision prediction module performs collision prediction judgment based on the predicted trajectory, obtains the judgment result and outputs it;

[0011] Wherein, the step S1 specifically includes:

[0012] Step S1.1, collecting historical driving sensor data of different unmanned transport vehicles in different mining areas, classifying the collected driving sensor data and adding classification labels to obtain a driving scene recognition data set and a trajectory prediction data set;

[0013] Step S1.2, using Kalman filtering to clean the driving scene recognition data set and the trajectory prediction data set to obtain cleaned driving scene recognition data set and trajectory prediction data set;

[0014] Step S1.3, for the cleaned driving scene recognition data set and trajectory prediction data set, select features with high correlation with trajectory prediction and perform Z-score normalization respectively to obtain a normalized scene recognition data set and a normalized trajectory prediction data set, wherein the features with high correlation with trajectory prediction include 9 features: X coordinate in the UTM coordinate system, Y coordinate in the UTM coordinate system, heading angle heading, vehicle speed V, yaw angular velocity, longitudinal acceleration, lateral acceleration, lateral angle deviation and heading angle deviation;

[0015] Step S1.4, using a sliding window method to further process the normalized scene recognition data set and the normalized trajectory prediction data set, respectively, to obtain a driving scene recognition model training data set and a trajectory model training data set;

[0016] Wherein, the step S2 specifically includes:

[0017] Step S2.1, constructing a driving scene recognition model based on a GRU network, and training the driving scene recognition model based on the GRU network to obtain a trained driving scene recognition model;

[0018] Step S2.2, constructing a trajectory prediction model based on GA-seq2seqGRU and training it, wherein the trajectory prediction model has an encoding-decoding structure, including two independent GRU network loop bodies, one of which is used as an encoder and the other as a decoder;

[0019] Step S2.3, using the GA algorithm to optimize the hyperparameters of the trajectory prediction model to obtain an optimized trajectory prediction model;

[0020] Step S2.4, obtaining a predicted trajectory through the trained driving scene recognition model and the trained and optimized trajectory prediction model;

[0021] Wherein, the step S3 specifically includes:

[0022] Step S3.1, constructing a vehicle body box based on the obtained predicted trajectory;

[0023] Step S3.2, calculating the expansion coefficient expand to expand the vehicle body box to obtain the expanded vehicle body box;

[0024] Step S3.3, constructing an obstacle box based on the vertex information of the obstacle, and combining it with the expanded vehicle body box to determine whether there is a collision tendency between the unmanned mining transport vehicle and the obstacle as the judgment result;

[0025] Step S3.4, performing retaining wall collision prediction, determining whether there is a collision tendency between the mining unmanned transport vehicle and the retaining wall, as the determination result;

[0026] Step S3.5, the judgment results of the collision prediction between the unmanned mining transport vehicle and the obstacle and the retaining wall are fed back and output to the downstream safety module for decision-making.

[0027] Optionally, the step S2.1 specifically includes:

[0028] The 7 features of the trajectory sequence in the historical time domain of the driving scene recognition model training data set, as well as the lateral angle deviation and heading angle deviation of the last frame in the time domain, are input into the GRU network after passing through the FC (fully connected layer) and then processed by the ReLU activation function. The 7 features include the X coordinate in the UTM coordinate system, the Y coordinate in the UTM coordinate system, the heading angle, the vehicle speed V, the yaw angular velocity, the longitudinal acceleration, and the lateral acceleration.

[0029] The last hidden state is extracted from the output of the GRU network, and the hidden state is further extracted and dimensionally reduced through FC, so as to map the complex feature representation in the network to a lower dimensional space while retaining the most relevant features. Finally, after processing through the sigmoid function, the probability combination W = [q1, q2] of the two events of normal driving and abnormal deviation is obtained, where q1 is the probability of normal driving and q2 is the probability of abnormal deviation, and the trained driving scene recognition model is obtained.

[0030] Optionally, the step S2.2 specifically includes:

[0031] Construct a trajectory prediction model with an encoder-decoder structure, which includes two independent GRU network loops, one as an encoder and the other as a decoder. The loss function is selected as shown in the following formula, where y i and y′ i are the observed values ​​and predicted values ​​respectively, and n1 is the total number of predicted values.

[0032]

[0033] In the training phase of the trajectory prediction model, the encoder inputs 9 features of the sequence of the trajectory model training data set in the historical time domain after being processed in the above steps, and outputs a context vector;

[0034] Output the scene recognition result using the trained driving scene recognition model obtained in step S2.1;

[0035] The obtained scene recognition result is concatenated with the context vector into a trajectory prediction vector after passing through FC, and used as the input data stream of the decoder;

[0036] The input data stream in the decoder is as follows: t-1 and the hidden state h t-1 Update the hidden state h at the current moment t , and then according to the hidden state h at the current moment t and the decoder output vector q at the previous moment t-1 Update the decoder output q at the current moment t :

[0037] h t =f(h t-1 ,q t-1 )

[0038] q t =g(h t ,q t-1 )

[0039] Among them, f(·) represents nonlinear transformation, and g(·) similarly represents nonlinear transformation;

[0040] The output of the decoder at each moment needs to be adjusted in dimension by a fully connected layer to obtain a trajectory point. Such a loop structure is used to gradually generate future trajectories. The number of loops is related to the prediction time domain T o Related, and finally output the future prediction trajectory

[0041] O (t) =[X (t) ,Y (t) ,heading (t) ], t=(k+1,k+2,…)

[0042] Among them, X (t) Indicates the X coordinate of the future trajectory in the UTM coordinate system, Y (t) Indicates the Y coordinate of the future trajectory in the UTM coordinate system, heading (t) represents the heading angle of the future trajectory, k represents the current moment, and t represents the time frame.

[0043] Optionally, the step S2.3 specifically includes:

[0044] Step S2.3.1, population initialization, setting the parameter range and population number, and initializing the population;

[0045] Step S2.3.2, set the fitness function, which can be used to calculate the fitness score after the individual training is completed;

[0046] Step S2.3.3, selection operation, after all parent individuals are trained, select individuals with high ranking as offspring according to fitness scores;

[0047] Step S2.3.4, crossover operation, randomly select two individuals for crossover, exchange some genes to produce new offspring;

[0048] Step S2.3.5, mutation operation, randomly mutating individuals in the population, changing some gene values ​​in the gene string of the individual as offspring;

[0049] Step S2.3.6, iterative optimization, set the iteration rounds, repeat steps S2.3.3 to S2.3.5 until the maximum iteration rounds are met, and the optimal combination is obtained. The trajectory prediction model is trained using the optimal parameter combination to obtain the optimized trajectory prediction model.

[0050] Optionally, the step S3.1 specifically includes:

[0051] The spatial position information obtained includes the predicted trajectory, reference path, retaining walls and obstacles;

[0052] The predicted trajectory is downsampled to select the base point for constructing the vehicle box, including obtaining the X, Y and heading of the base point. X represents the X coordinate of the base point in the UTM coordinate system, Y represents the Y coordinate of the base point in the UTM coordinate system, and heading represents the heading angle of the base point.

[0053] The initial body box is constructed with the rear axle center of the mining unmanned transport vehicle as the initial origin, and the initial body box is transformed into the body coordinate system by rotation and translation based on the width and the length of the front and rear axles of the mining unmanned transport vehicle.

[0054]

[0055] in, is the corner point of the car body box in the car body coordinate system, (x traj ,y traj ) is the base point coordinate; w vel is the vehicle width; l F and l R are the front axle length and rear axle length of the vehicle respectively, Represents the x coordinate of the corner point after translation and rotation, It represents the y coordinate of the corner point after translation and rotation, i represents the corner point number, and the total number of corner points of the car box is 4.

[0056] Optionally, the step S3.2 specifically includes:

[0057] Set the expansion factor:

[0058]

[0059] Among them, W, W min and W max are the current road width, the minimum road width, and the maximum road width respectively; C, C min and C max are the current road curvature, the minimum road curvature, and the maximum road curvature, respectively; a and b are weight coefficients; r1 is the power exponent of width, r2 is the power exponent of curvature; k is the compensation coefficient;

[0060] Add the expansion coefficient expand as a scaling factor to expand the body box converted to the body coordinate system to obtain the corner point coordinates of the expanded body box i=1,2,3,4, thus obtaining the expanded body box.

[0061] Optionally, the step S3.3 specifically includes:

[0062] Construct an obstacle box based on the vertex information of the obstacle;

[0063] For the expanded car box, the separating axis theorem is used to predict the obstacle collision:

[0064]

[0065] Where n is the projection axis of the two polygons; is the vertex of the obstacle box, j represents the vertex number of the obstacle box, P(·) represents the projection operation, min traj Indicates the minimum value of the range of the expanded car box after projection on n, max traj Indicates the maximum value of the range of the expanded car box after projection on n, min obs Indicates the minimum value of the range after the obstacle box is projected on n, max obs Indicates the maximum value of the range interval after the obstacle box is projected on n;

[0066] Determine the two intervals [min traj , max traj ] and [min obs , max obs ] Whether the axis n overlaps, if there is an axis that makes the two intervals not overlap, it is judged that the vehicle box and the obstacle box do not collide, that is, it is judged that the unmanned transport vehicle for mining will not collide with the obstacle; otherwise, it is judged that there is a tendency for the unmanned transport vehicle for mining to collide with the obstacle.

[0067] Optionally, the step S3.4 specifically includes:

[0068] Get the wall point information O(x bd ,y bd );

[0069] Calculate the retaining wall points O(x bd ,y bd ) and the cross product of the vectors formed by the vertices of the expanded car box quadrilateral and its adjacent vertices

[0070]

[0071] in, and represents the coordinates of the corner points of the car box after expansion, and Cross(·) represents the vector cross product formula;

[0072] Analyze the obtained cross product results. When all the cross products have the same signs, define this retaining wall point as being inside the vehicle body box, and judge that there is a collision trend between the unmanned mining transport vehicle and the retaining wall; otherwise, judge that there is no collision risk between the unmanned mining transport vehicle and the retaining wall.

[0073] The technical solution adopted by the present invention is a collision prediction method for unmanned transport vehicles for mining based on adaptive expansion. Since the transportation path in the closed mining area is single and has a large amount of daily operation data, a database is constructed based on historical normal data and road deviation data, and a scene recognition model and a trajectory prediction model are built based on a gated recurrent unit (GRU). A vehicle box is constructed based on downsampling of the predicted trajectory, and the vehicle box is adaptively expanded considering the road width, curvature and vehicle information. Then, an obstacle box is constructed using obstacle information. Finally, based on the separating axis theorem, it is determined whether there is a collision with an obstacle, and based on the point-polygon relationship, it is determined whether there is a collision with a retaining wall point, so as to realize a full-cycle health monitoring and protection function. The steps of the technical solution are as follows:

[0074] The first part is data preprocessing: data cleaning, feature extraction and normalization of vehicle sensor data. Kalman filtering is used for data cleaning; key features are selected for network training; and Z-score method is used for normalization.

[0075] The second part, trajectory prediction: construct a driving scene recognition module and a trajectory prediction module to predict the future trajectory of unmanned mining transport vehicles.

[0076] The third part is collision prediction: using the predicted trajectory input by the upper model combined with vehicle information, road width, curvature, obstacle information and retaining wall point information, constructing the vehicle body box and obstacle box based on the adaptive expansion coefficient along the predicted trajectory, and judging whether there is a collision with obstacles and retaining walls based on the separating axis theorem and the relationship between points and polygons.

[0077] Compared with the prior art, a method for predicting collision of unmanned mining transport vehicles based on adaptive expansion provided in accordance with an embodiment of the present invention has at least the following beneficial effects:

[0078] 1. Aiming at the timeliness, high accuracy and unmanned requirements of collision prediction in open-pit mines, a collision prediction algorithm based on the combination of trajectory prediction and graphics method is proposed, which can effectively reduce the computational complexity.

[0079] 2. In view of the high uncertainty of trajectory prediction in different scenarios, a trajectory prediction model based on driving scene recognition is proposed, which splices scene recognition vectors to improve the accuracy of vehicle trajectory prediction in different scenarios and indirectly improve the accuracy of collision prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. The features and advantages of the present invention can be more clearly understood by referring to the drawings. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0081] Figure 1 It is a flowchart of a collision prediction method for unmanned mining transport vehicles based on adaptive expansion provided according to an embodiment of the present invention.

[0082] Figure 2 It is a longitudinal acceleration Kalman filter cleaning effect diagram of the collision prediction method for unmanned mining transport vehicles based on adaptive expansion provided according to an embodiment of the present invention.

[0083] Figure 3 It is a trajectory prediction module architecture diagram of a collision prediction method for unmanned mining transport vehicles based on adaptive expansion provided according to an embodiment of the present invention.

[0084] Figure 4 It is a schematic diagram of coordinate transformation, vehicle body box expansion and collision prediction judgment in an example of a collision prediction method for an unmanned mining transport vehicle based on adaptive expansion provided according to an embodiment of the present invention.

[0085] Figure 5 It is a schematic diagram of the collision prediction process effect in ROS\Rviz (three-dimensional visualization platform) using an example of the collision prediction method for unmanned mining transport vehicles based on adaptive expansion provided according to an embodiment of the present invention.

[0086] Figure 6 It is a schematic diagram of the collision prediction process effect in ROS\Rviz (three-dimensional visualization platform) using an example of the collision prediction method for unmanned mining transport vehicles based on adaptive expansion provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0087] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0088] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0089] A method for predicting a collision of an unmanned mining transport vehicle based on adaptive expansion according to an embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0090] In view of the fact that the existing graphics methods do not take into account the rapid changes in the road environment, combined with the characteristics of variable width and curvature of open-pit mine roads, an embodiment of the present invention provides a graphics method that takes into account an adaptive expansion coefficient.

[0091] like Figure 1 As shown, a method for predicting collision of unmanned mining transport vehicles based on adaptive expansion according to an embodiment of the present invention includes three processing modules: a data preprocessing module, a trajectory prediction module, and a collision prediction module. Specific descriptions are as follows.

[0092] Step S1, the data preprocessing module performs data collection and preprocessing to obtain a driving scene recognition data set and a trajectory prediction data set, and further performs normalization and sliding window processing to obtain a driving scene recognition model training data set and a trajectory model training data set. This step specifically includes the following steps.

[0093] Step S1.1, collect historical driving sensor data of different unmanned transport vehicles in different mining areas for driving scene recognition and trajectory prediction. The collected driving sensor data is scene classified and labeled, which can be divided into two categories: normal driving and abnormal deviation, to obtain a driving scene recognition data set and a trajectory prediction data set. The driving scene recognition data set and trajectory prediction data set after adding classification labels are the total data set.

[0094] Step S1.2, use Kalman filtering to clean the total data set to obtain a cleaned data set. The cleaning effect is as follows: Figure 2 The cleaned data set includes a cleaned driving scene recognition data set and a cleaned trajectory prediction data set.

[0095] Step S1.3, for the cleaned data set, select the features with high correlation with trajectory prediction for Z-score normalization. Here, 9 feature values ​​including X, Y, heading, V, yaw rate, longitudinal acceleration, lateral acceleration, lateral angle deviation and heading angle deviation are selected as key features. Among them, X represents the X coordinate in the UTM coordinate system (UNIVERSAL TRANSVERSE MERCATORPROJECTION), Y represents the Y coordinate in the UTM coordinate system, heading represents the heading angle, and V represents the vehicle speed. For the cleaned driving scene recognition data set, the 7 features except the lateral angle deviation and heading angle deviation are normalized to obtain the normalized scene recognition data set. For the cleaned trajectory prediction data set, all 9 features are normalized to obtain the normalized trajectory prediction data set.

[0096] Step S1.4, use the sliding window method to further process the normalized scene recognition dataset and the normalized trajectory prediction dataset. Assume that the historical time domain is T p , the prediction time domain is T o , set the sliding window length to T p , the sampling step is 1, the normalized driving scene recognition data set is processed to obtain the driving scene recognition model training data set; the sliding window length is set to T p +T o , the sampling step is 1, and the normalized trajectory prediction dataset is processed to obtain the trajectory model training dataset.

[0097] Step S2, the trajectory prediction module constructs a driving scene recognition model and a trajectory prediction model, uses the training data set in step S1 to complete the training of the driving scene recognition model and the trajectory prediction model, performs trajectory prediction, and obtains a predicted trajectory. Among them, the driving scene recognition model is first constructed based on the GRU (gated recurrent unit) network, the historical trajectory sequence features are input into the driving scene recognition model, and the probability combination W of the two events of normal driving probability and abnormal deviation probability is output. The output result of the driving scene recognition model is combined and the trajectory prediction model is constructed based on GA-seq2seqGRU (based on genetic algorithm sequence to sequence gated recurrent unit), and the historical trajectory sequence features are still input to output the future predicted trajectory O. (t) , driving scene recognition model and trajectory prediction model such as Figure 3 The step S2 specifically includes the following steps.

[0098] Step S2.1, construct a driving scene recognition model based on the GRU network, and train the driving scene recognition model based on the GRU network to obtain the trained driving scene recognition model. In the training process and the use process, the 7 features of the trajectory sequence in the historical time domain and the lateral angle deviation and heading angle deviation of the last frame in the time domain of the driving scene recognition model training data set processed by the above steps are input, and they are spliced ​​after passing through the fully connected layer (FC) and processed by the ReLU activation function as the input of the GRU network. The last hidden state is extracted from the output of the GRU network, and the hidden state is further extracted and reduced in dimension through FC. The purpose is to map the complex feature representation in the network to a lower dimensional space while retaining the most relevant features. Finally, after processing through the sigmoid function (S-shaped growth curve function), the probability combination of normal driving and abnormal deviation events W = [q1, q2] is obtained, where q1 is the normal driving probability and q2 is the abnormal deviation probability. The cross entropy loss function is used for the training of the driving scene recognition model. The trained driving scene recognition model is saved for trajectory prediction model training and actual prediction application.

[0099] Step S2.2: construct a trajectory prediction model based on GA-seq2seqGRU and perform training. Figure 3 As shown in the figure, the trajectory prediction model has an encoder-decoder structure, which includes two independent GRU network loops, one as an encoder and the other as a decoder. The loss function is selected as shown in formula (1), where y i and y′ i are the observed values ​​and predicted values ​​respectively, and n1 is the total number of predicted values.

[0100]

[0101] In the training phase of the trajectory prediction model, the encoder inputs the nine features of the sequence of the trajectory model training data set in the historical time domain after the above steps, outputs the context vector, uses the driving scene recognition model trained in step S2.1 to output the scene recognition result, and concatenates the scene recognition result with the context vector after passing FC to form a trajectory prediction vector as the input of the decoder. The data flow in the decoder is shown in formula (2). The decoder needs to output the vector q according to the decoder output vector q at the previous moment. t-1 and the hidden state h t-1 Update the hidden state h at the current moment t Then according to the hidden state h at the current moment t and the decoder output vector q at the previous moment t-1 Update the decoder output q at the current moment t :

[0102]

[0103] Among them, f(·) represents a nonlinear transformation, and g(·) similarly represents a nonlinear transformation.

[0104] The output of the decoder at each moment needs to be adjusted in dimension by a fully connected layer to obtain a trajectory point. Using such a loop structure to gradually generate future trajectories, the number of loops is related to the prediction time domain T o Related, and finally output the future prediction trajectory:

[0105] O (t) =[X (t) ,Y (t) ,heading (t) ], t=(k+1,k+2,…)

[0106] Among them, X (t) Indicates the X coordinate of the future trajectory in the UTM coordinate system, Y (t) Indicates the Y coordinate of the future trajectory in the UTM coordinate system, heading (t) represents the heading angle of the future trajectory, k represents the current moment, and t represents the time frame.

[0107] Step S2.3, using the GA algorithm to optimize the hyperparameters of the trajectory prediction model to obtain an optimized trajectory prediction model, specifically includes the following steps.

[0108] Step S2.3.1, population initialization, setting the parameter range and population number, and initializing the population.

[0109] Step S2.3.2, set the fitness function, which can be used to calculate the fitness score after the individual training is completed.

[0110] Step S2.3.3, selection operation, after all parent individuals are trained, individuals with high rankings are selected as offspring based on fitness scores.

[0111] Step S2.3.4, crossover operation, randomly select two individuals for crossover, exchange some genes to produce new offspring, such as Figure 2 shown.

[0112] Step S2.3.5, mutation operation, randomly mutates individuals in the population and changes certain gene values ​​in the individual gene string as offspring.

[0113] Step S2.3.6, iterative optimization, set the iteration rounds, repeat steps S2.3.3 to S2.3.5 until the maximum iteration rounds are met and the optimal combination is obtained. The fitness function of the GA algorithm is the same as formula (1). Set the iteration rounds to solve the hyperparameters, use the optimal parameter combination to train the model, and save the trained trajectory prediction model for practical application.

[0114] Step S2.4, obtaining a predicted trajectory through the trained driving scene recognition model and the trained and optimized trajectory prediction model.

[0115] Step S3, the collision prediction module performs collision prediction judgment, obtains the judgment result and outputs it. Based on the predicted trajectory, the expansion coefficient is calculated by considering the road information and vehicle information to construct the vehicle body box (vehicle body two-dimensional rectangular box), and the obstacle box (obstacle two-dimensional rectangular box) is constructed. The collision with obstacles and retaining walls is judged based on the separating axis theorem and the position relationship between points and polygons, such as Figure 4 This step specifically includes the following steps.

[0116] Step S3.1, construct a vehicle body box based on the obtained predicted trajectory. The obtained spatial position information, including the predicted trajectory, reference path, retaining wall and obstacle, is converted from the unified UTM coordinate system (UNIVERSAL TRANSVERSE MERCATORPROJECTION) to the vehicle body coordinate system with the rear axle center of the vehicle as the initial origin. The base point for constructing the vehicle body box is selected by downsampling the obtained predicted trajectory. The vehicle body box is constructed based on the X, Y and heading of the base point combined with the width and the front and rear half-axle lengths of the mining unmanned transport vehicle. The specific method is to construct the initial vehicle body box with the rear axle center of the mining unmanned transport vehicle as the initial origin, and then transform the initial vehicle body box to the vehicle body coordinate system by rotation and translation. As shown in formula (3), is the corner point of the car body box in the car body coordinate system:

[0117]

[0118] Among them, θ is the heading angle of the base point; (x traj ,y traj ) is the base point coordinate; w vel is the vehicle width; l F and l R are the front axle length and rear axle length of the vehicle respectively, Represents the x coordinate of the corner point after translation and rotation, It represents the y coordinate of the corner point after translation and rotation, i represents the corner point number, and the total number of corner points of the car box is 4.

[0119] Step S3.2, calculate the expansion coefficient expand to expand the body box and obtain the expanded body box. Due to the changeable unstructured road environment in the open-pit mine area, in order to ensure the driving safety of unmanned transport vehicles for mining, the concept of body box expansion is introduced here, and the expansion coefficient expand is added to appropriately expand the body box. The function of this expansion coefficient is proportional scaling. Since the risk of unmanned transport vehicles for mining is generally lower in sections with small curvature and the driving speed is faster than that in sections with large curvature, it is necessary to maintain a low false detection rate and the body box should be slightly expanded; when driving on sections with large curvature, the vehicle faces a higher risk, and the body box should continue to increase the degree of expansion to improve safety. At the same time, due to the restriction of road width, the expansion area can be appropriately increased in wide sections, while the expansion needs to be limited in narrow sections to ensure a low false detection rate. As shown in formula (4), the expansion coefficient is:

[0120]

[0121] Among them, W, W min and W max are the current road width, the minimum road width, and the maximum road width respectively; C, C min and C max are the current road curvature, the minimum road curvature and the maximum road curvature respectively; a and b are weight coefficients; r1 is the power exponent of width, r2 is the power exponent of curvature; k is the compensation coefficient.

[0122] Add the expansion coefficient expand as a scaling factor to expand the body box converted to the body coordinate system to obtain the corner point coordinates of the expanded body box i=1,2,3,4, thus obtaining the expanded body box.

[0123] Step S3.3, construct an obstacle box based on the vertex information of the obstacle, and combine it with the expanded vehicle body box to determine whether there is a collision trend between the unmanned mining transport vehicle and the obstacle as the judgment result. The separating axis theorem is used as the basic method for obstacle collision prediction:

[0124]

[0125] Where n is the projection axis of the two polygons; is the vertex of the obstacle box, j represents the vertex number of the obstacle box, P(·) represents the projection operation, min traj Indicates the minimum value of the range of the expanded car box after projection on n, max traj Indicates the maximum value of the range of the expanded car box after projection on n, min obs Indicates the minimum value of the range after the obstacle box is projected on n, maxobs Indicates the maximum value of the range interval after the obstacle box is projected on n.

[0126] Next, determine the two intervals [min traj , max traj ] and [min obs , max obs ] Whether the axis n overlaps, if there is an axis that makes the above condition not true, that is, there is an axis that makes the two intervals not overlap, then it is judged that the vehicle box and the obstacle box do not collide, that is, the unmanned transport vehicle for mining will not collide with the obstacle. Otherwise, it is judged that there is a tendency for the unmanned transport vehicle for mining to collide with the obstacle.

[0127] Step S3.4, perform retaining wall collision prediction, and determine whether there is a collision trend between the unmanned mining transport vehicle and the retaining wall, as the judgment result. The positional relationship between the dependent point and the polygon is calculated using the vector cross product method. The specific method is as shown in formula (6), and the retaining wall point O(x bd ,y bd ) and the cross product of the vectors formed by the vertices of the expanded car box quadrilateral and its adjacent vertices,

[0128]

[0129] in, and represents the coordinates of the corner points of the car box after expansion, and Cross(·) represents the vector cross product formula. The expanded expression of the cross product function is:

[0130] Cross(A,B,C)=(B x -A x )(C y -A y _-)B y -A y _(C x -A x )

[0131] Among them, A, B, C, A x , B x , C x , A y , B y , and C y Represents the variable in the cross product function.

[0132] Analyze the obtained cross product results. When all cross product signs are the same, define this retaining wall point as being inside the vehicle box, and judge that there is a collision trend between the unmanned mining transport vehicle and the retaining wall; otherwise, judge that there is no collision risk between the unmanned mining transport vehicle and the retaining wall. In this implementation, the retaining wall point is existing data and can be obtained from the upstream module.

[0133] Step S3.5, the judgment results of the collision prediction between the unmanned mining transport vehicle and the obstacle and the retaining wall are fed back and output to the downstream safety module for decision-making. Figure 5 and Figure 6 As shown, the collision prediction process effect in ROS\Rviz (3D visualization platform) is exemplified, wherein the gray long curve is the reference path point set; the gray short curve is the predicted trajectory; the black curve is the retaining wall point set; the black solid square is the obstacle; and the black hollow square is the vehicle box.

[0134] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application.

[0135] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0136] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0137] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A collision prediction method for unmanned mining transport vehicles based on adaptive expansion, characterized in that: The following steps are involved: Step S1, the data preprocessing module performs data collection and preprocessing to obtain a driving scene recognition data set and a trajectory prediction data set, and further performs normalization and sliding window processing to obtain a driving scene recognition model training data set and a trajectory model training data set; Step S2, the trajectory prediction module constructs a driving scene recognition model and a trajectory prediction model, uses the training data set in step S1 to complete the training of the driving scene recognition model and the trajectory prediction model, performs trajectory prediction, and obtains a predicted trajectory; Step S3, the collision prediction module performs collision prediction judgment based on the predicted trajectory, obtains the judgment result and outputs it; Wherein, the step S1 specifically includes: Step S1.1, collecting historical driving sensor data of different unmanned transport vehicles in different mining areas, classifying the collected driving sensor data and adding classification labels to obtain a driving scene recognition data set and a trajectory prediction data set; Step S1.2, using Kalman filtering to clean the driving scene recognition data set and the trajectory prediction data set to obtain cleaned driving scene recognition data set and trajectory prediction data set; Step S1.3, for the cleaned driving scene recognition data set and trajectory prediction data set, select features with high correlation with trajectory prediction and perform Z-score normalization respectively to obtain a normalized scene recognition data set and a normalized trajectory prediction data set, wherein the features with high correlation with trajectory prediction include 9 features: X coordinate in the UTM coordinate system, Y coordinate in the UTM coordinate system, heading angle heading, vehicle speed V, yaw angular velocity, longitudinal acceleration, lateral acceleration, lateral angle deviation and heading angle deviation; Step S1.4, using a sliding window method to further process the normalized scene recognition data set and the normalized trajectory prediction data set, respectively, to obtain a driving scene recognition model training data set and a trajectory model training data set; Wherein, the step S2 specifically includes: Step S2.1, constructing a driving scene recognition model based on a GRU network, and training the driving scene recognition model based on the GRU network to obtain a trained driving scene recognition model; Step S2.2, constructing a trajectory prediction model based on GA-seq2seqGRU and training it, wherein the trajectory prediction model has an encoding-decoding structure, including two independent GRU network loop bodies, one of which serves as an encoder and the other as a decoder; Step S2.3, using the GA algorithm to optimize the hyperparameters of the trajectory prediction model to obtain an optimized trajectory prediction model; Step S2.4, obtaining a predicted trajectory through the trained driving scene recognition model and the trained and optimized trajectory prediction model; Wherein, the step S3 specifically includes: Step S3.1, constructing a vehicle body box based on the obtained predicted trajectory; Step S3.2, calculating the expansion coefficient expand to expand the vehicle body box to obtain the expanded vehicle body box; Step S3.3, constructing an obstacle box based on the vertex information of the obstacle, and combining it with the expanded vehicle body box to determine whether there is a collision tendency between the unmanned mining transport vehicle and the obstacle as the judgment result; Step S3.4, performing retaining wall collision prediction, determining whether there is a collision tendency between the mining unmanned transport vehicle and the retaining wall, as the determination result; Step S3.5, the judgment results of the collision prediction between the unmanned mining transport vehicle and the obstacle and the retaining wall are fed back and output to the downstream safety module for decision-making.

2. The method for predicting collision of unmanned mining transport vehicles based on adaptive expansion according to claim 1 is characterized in that: The step S2.1 specifically includes: The 7 features of the trajectory sequence in the historical time domain of the driving scene recognition model training data set, as well as the lateral angle deviation and heading angle deviation of the last frame in the time domain, are input to the GRU network after FC respectively, and then processed by the ReLU activation function. The 7 features include the X coordinate in the UTM coordinate system, the Y coordinate in the UTM coordinate system, the heading angle heading, the vehicle speed V, the yaw angular velocity, the longitudinal acceleration, and the lateral acceleration; The last hidden state is extracted from the output of the GRU network, and the hidden state is further extracted and dimensionally reduced through FC, so as to map the complex feature representation in the network to a lower dimensional space while retaining the most relevant features. Finally, after processing through the sigmoid function, the probability combination W = [q1, q2] of the two events of normal driving and abnormal deviation is obtained, where q1 is the probability of normal driving and q2 is the probability of abnormal deviation, and the trained driving scene recognition model is obtained.

3. The method for predicting collision of unmanned mining transport vehicles based on adaptive expansion according to claim 2 is characterized in that: The step S2.2 specifically includes: Construct a trajectory prediction model with an encoder-decoder structure, which includes two independent GRU network loops, one as an encoder and the other as a decoder. The loss function is selected as shown in the following formula, where y i and y′ i are observed values ​​and predicted values ​​respectively, and n1 is the total number of predicted values; In the training phase of the trajectory prediction model, the encoder inputs 9 features of the sequence of the trajectory model training data set in the historical time domain after being processed in the above steps, and outputs a context vector; Output the scene recognition result using the trained driving scene recognition model obtained in step S2.1; The obtained scene recognition result is concatenated with the context vector into a trajectory prediction vector after passing through FC, and used as the input data stream of the decoder; The input data stream in the decoder is as follows: t-1 and the hidden state h t-1 Update the hidden state h at the current moment t , and then according to the hidden state h at the current moment t and the decoder output vector q at the previous moment t-1 Update the current decoder output q t : h t =f(h t-1 ,q t-1 ) q t =g(h t ,q t-1 ) Among them, f(·) represents nonlinear transformation, and g(·) similarly represents nonlinear transformation; The output of the decoder at each moment needs to be adjusted in dimension by a fully connected layer to obtain trajectory points. Such a loop structure is used to gradually generate future trajectories. The number of loops is related to the prediction time domain T o Related, and finally output the future prediction trajectory O (t) =[X (t) ,Y (t) ,heading (t) ],t=(k+1,k+2,…) Among them, X (t) Indicates the X coordinate of the future trajectory in the UTM coordinate system, Y (t) Indicates the Y coordinate of the future trajectory in the UTM coordinate system, heading (t) represents the heading angle of the future trajectory, k represents the current moment, and t represents the time frame.

4. The method for predicting collision of unmanned mining transport vehicles based on adaptive expansion according to claim 3 is characterized in that: The step S2.3 specifically includes: Step S2.3.1, population initialization, setting the parameter range and population number, and initializing the population; Step S2.3.2, set the fitness function, which can be used to calculate the fitness score after the individual training is completed; Step S2.3.3, selection operation, after all parent individuals are trained, select individuals with high ranking as offspring according to fitness scores; Step S2.3.4, crossover operation, randomly select two individuals for crossover, exchange some genes to produce new offspring; Step S2.3.5, mutation operation, randomly mutating individuals in the population, changing some gene values ​​in the gene string of the individual as offspring; Step S2.3.6, iterative optimization, set the iteration rounds, repeat steps S2.3.3 to S2.3.5 until the maximum iteration rounds are met, and the optimal combination is obtained. The trajectory prediction model is trained using the optimal parameter combination to obtain the optimized trajectory prediction model.

5. The method for predicting collision of unmanned mining transport vehicles based on adaptive expansion according to claim 4 is characterized in that: The step S3.1 specifically includes: The spatial position information obtained includes the predicted trajectory, reference path, retaining walls and obstacles; The predicted trajectory is downsampled to select the base point for constructing the vehicle box, including obtaining the X, Y and heading of the base point. X represents the X coordinate of the base point in the UTM coordinate system, Y represents the Y coordinate of the base point in the UTM coordinate system, and heading represents the heading angle of the base point. The initial body box is constructed with the rear axle center of the mining unmanned transport vehicle as the initial origin, and the initial body box is transformed into the body coordinate system by rotation and translation based on the width and the length of the front and rear axles of the mining unmanned transport vehicle. in, is the corner point of the car body box in the car body coordinate system, (x traj ,y traj ) is the base point coordinate; w vel is the vehicle width; l F and l R are the length of the front and rear axles of the vehicle, respectively. Represents the x coordinate of the corner point after translation and rotation, It represents the y coordinate of the corner point after translation and rotation, i represents the corner point number, and the total number of corner points of the car box is 4.

6. The method for predicting collision of unmanned mining transport vehicles based on adaptive expansion according to claim 5 is characterized in that: The step S3.2 specifically includes: Set the expansion factor: Among them, W, W min and W max are the current road width, the minimum road width, and the maximum road width respectively; C, C min and C max are the current road curvature, the minimum road curvature, and the maximum road curvature, respectively; a and b are weight coefficients; r1 is the power exponent of width, r2 is the power exponent of curvature; k is the compensation coefficient; Add the expansion coefficient expand as a scaling factor to expand the body box converted to the body coordinate system to obtain the corner point coordinates of the expanded body box Thus, the expanded body box is obtained.

7. The method for predicting collision of unmanned mining transport vehicles based on adaptive expansion according to claim 6 is characterized in that: The step S3.3 specifically includes: Construct an obstacle box based on the vertex information of the obstacle; For the expanded car box, the separating axis theorem is used to predict the obstacle collision: Where n is the projection axis of the two polygons; is the vertex of the obstacle box, j represents the vertex number of the obstacle box, P(·) represents the projection operation, min traj Indicates the minimum value of the range of the expanded car box after projection on n, max traj Indicates the maximum value of the range of the expanded car box after projection on n, min obs Indicates the minimum value of the range after the obstacle box is projected on n, max obs Indicates the maximum value of the range interval after the obstacle box is projected on n; Determine the two intervals [min traj , max traj ] and [min obs , max obs ] Whether the axis n overlaps, if there is an axis that makes the two intervals not overlap, it is judged that the vehicle box and the obstacle box do not collide, that is, it is judged that the unmanned transport vehicle for mining will not collide with the obstacle; otherwise, it is judged that there is a tendency for the unmanned transport vehicle for mining to collide with the obstacle.

8. The method for predicting collision of unmanned mining transport vehicles based on adaptive expansion according to claim 7, characterized in that: The step S3.4 specifically includes: Get the wall point information O(x bd ,y bd ); Calculate the retaining wall points O(x bd ,y bd ) and the cross product of the vectors formed by the vertices of the expanded car box quadrilateral and its adjacent vertices in, and represents the coordinates of the corner points of the car box after expansion, and Cross(·) represents the vector cross product formula; Analyze the obtained cross product results. When all the cross products have the same signs, define this retaining wall point as being inside the vehicle body box, and judge that there is a collision trend between the unmanned mining transport vehicle and the retaining wall; otherwise, judge that there is no collision risk between the unmanned mining transport vehicle and the retaining wall.

Images