Method and device for detecting steering performance of vehicle in mining area

Abstract

The invention relates to the technical field of automobiles, and discloses a mining area vehicle steering performance detection method and device, and the method comprises the steps: synchronously collecting vehicle end motion state time sequence data and weather time sequence data of a vehicle; constructing a dual-channel CNN, and training a first channel by taking the vehicle end motion state data as an input value and the influence amplitude of the vehicle end motion state on the vehicle steering angle as a label; training a second channel by taking the weather data as an input value and the influence amplitude of weather on the steering angle of the vehicle as a label; generating a fusion matrix; after labeling a label of a steering angle at the next acquisition moment on the fusion matrix, inputting the label into the LSTM for training, and generating a steering angle prediction model; and inputting the vehicle end motion state data and the weather data of the target vehicle into the steering angle prediction model to obtain a target steering angle. The method not only considers the vehicle end motion state characteristic data and combines the weather influence characteristic data to predict the steering angle, and improves the precision of the steering angle.

Description

Technical Field

[0001] The present application belongs to the field of automobile technology and relates to a method and device for detecting the steering performance of a mining vehicle. Background Art

[0002] Vehicles rely on a variety of sensors and algorithms to achieve precise control and operation. Among them, the accuracy and followability of the steering system are crucial to the stability and safety of the vehicle. However, due to the large size of mining vehicles, there is no commonly used four-wheel alignment to accurately calibrate their steering angles.

[0003] The Chinese patent application number CN202010597248.6 proposes a tire force online estimation method based on a neural network, whose input is the vertical load of each tire, the wheel slip angle, the longitudinal slip rate and the front and rear wheel angles, and the corresponding labels of each set of input data are the body state information longitudinal acceleration, lateral acceleration and yaw angular acceleration, and the output signal is the longitudinal force and lateral force of each wheel estimated by the algorithm. Four neural network parallel structures and gradient descent algorithms are used to realize the online estimation of tire forces of each wheel. The method designed by the present invention realizes real-time monitoring of wheel forces, which not only avoids the use of expensive wheel sensors, but also avoids the complex experiments required to obtain wheel models. The designed algorithm results are accurate and effective, which has a positive role in promoting vehicle active safety.

[0004] The road environment where vehicles operate in mining areas is full of mud, and the road adhesion coefficient will change back and forth, greatly affecting the vehicle feedback turning angle. Weather factors will affect the road adhesion coefficient. Existing technologies tend to use vehicle models to calculate theoretical turning angles, but only consider the vehicle-side motion state data to calculate the turning angle, which has low accuracy. Summary of the invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] In order to solve the problems existing in the related art, the embodiments of the present disclosure provide a method and device for detecting the steering performance of a mining vehicle, so as to solve the problem of low accuracy in identifying the predicted turning angle of a mining vehicle in the prior art.

[0007] In some embodiments, a method for detecting steering performance of a mining vehicle is provided, the method comprising: S10, synchronously collecting vehicle-side motion state time series data and weather time series data; S20, construct a dual-channel CNN, use the pre-processed vehicle-side motion state data as input values, and the influence of the vehicle-side motion state on the vehicle steering angle as a label to train the first channel; use the pre-processed weather data as input values, and the influence of the weather on the vehicle steering angle as a label to train the second channel; S30, integrating the vehicle steering angles at n consecutive acquisition moments to form a vehicle-side motion state feature matrix, and integrating the impact of the weather on the vehicle steering angle at the corresponding acquisition moment to form a weather feature matrix, and adjusting the weights of the weather feature matrix and the vehicle-side motion state feature matrix based on a cross-modal attention fusion mechanism to generate a fusion matrix; S40, constructing an LSTM, marking the label of the steering angle at the next acquisition moment on the fusion matrix, and inputting it into the LSTM for training, so that the input value is the fusion matrix, and the output value is the steering angle prediction model; S50, inputting the vehicle-side motion state data and weather data of the target vehicle into a steering angle prediction model to obtain a target steering angle.

[0008] Preferably, the characteristics of the vehicle-side motion state time series data include at least one or more of the vehicle wheelbase, vehicle speed, angular velocity, vehicle driving direction, and front wheel feedback angle; the characteristics of the weather time series data include at least one or more of the precipitation, snowfall, road surface humidity, and road surface ice and snow layer thickness arranged along the time series.

[0009] Preferably, constructing LSTM includes: Input fusion matrix; When the LSTM hidden layer is transferred, the unit state, forget gate, input gate and output gate are added, which act together on the hidden layer unit of the original RNN. The calculation formula is as follows: The forget gate determines how much of the cell state at the previous moment is retained at the current moment:

[0010] The input gate determines the input of the network at the current moment and how much is saved to the cell state:

[0011]

[0012] The update of the cell state combines the information from the forget gate and the input gate:

[0013] The output gate determines what information is output, and the output value is scaled by the tanh function:

[0014]

[0015] Map the LSTM output to the steering angle:

[0016] In the above formula, represents the forget gate, represents the input gate, represents the unit state at time t, are candidate memory cells, is the input at time t, represents the output gate, , , They represent the weight of the forget gate, the weight of the input gate, the weight of the candidate unit state, the weight of the output gate, and the weight of the LSTM output mapped to the actual corner. , , , , They represent the forget gate bias, input gate bias, candidate unit state bias, output gate bias, and the bias of LSTM output mapped to the actual corner. represents the output at time t, is the activation function, tanh represents the hyperbolic tangent activation function, and Y is the theoretical rotation angle of the output.

[0017] Preferably, a loss function is introduced in the training of the steering angle prediction model, including: Using mean square error as the loss function,

[0018] is the theoretical turning angle output by the model, is the front wheel feedback angle in the training data, s represents the index of the current sample data, and N represents the total number of samples in the training data set.

[0019] Use the training set to train the CNN-LSTM model, use the validation set to validate the model, and adjust the weight and bias parameters.

[0020] Preferably, before S20, the method further includes: S11, preprocessing the vehicle-side motion state time series data and weather time series data, the preprocessing process at least includes: filtering outliers in the data, filtering out noise from outliers in the data, normalizing the data and dividing the data, wherein the data is divided into a training set and a validation set.

[0021] Preferably, the formula for normalization is:

[0022] Where D represents the result of normalizing the collected data. Represents the collected raw data, Represents the maximum value in the original data. Indicates the minimum value in the original data.

[0023] Preferably, in S30, the vehicle steering angles at consecutive n acquisition moments are integrated to form a vehicle-side motion state feature matrix, and the impact of weather on the vehicle steering angle at the corresponding acquisition moment is integrated to form a weather feature matrix. Based on the cross-modal attention fusion mechanism, the weights of the weather feature matrix and the vehicle-side motion state feature matrix are adjusted to generate a fusion matrix, including: Assume the vehicle-side motion state characteristic matrix is:

[0024] Among them, U represents the vehicle-side motion state feature matrix, represents the j-th vehicle-side motion state feature data at the i-th moment, i=1,2,…,n, n represents the total number of acquisition moments, and p represents the total number of vehicle-side motion state features; Assume that the weather impact feature matrix is:

[0025] Among them, V represents the weather impact feature matrix, represents the impact of abnormal weather at the i-th moment on the j-th vehicle-side motion state feature matrix, and q is the total number of weather impact features; The principle of generating the fusion matrix is: generating the similarity of each element in the vehicle-side motion state feature matrix and the weather impact feature matrix, and the calculation formula is:

[0026] in, Represents the element value of the i-th row and j-th column in the similarity matrix, Represents the element value of the i-th row and j-th column of the vehicle-side motion state feature matrix, Represents the element value in the i-th row and j-th column of the weather impact feature matrix.

[0027] Use the softmax function to generate attention weights based on the similarity matrix:

[0028]

[0029] in, represents the attention weight of the vehicle-side motion state feature matrix, Indicates the attention weight of the feature matrix of the vehicle's motion state affected by weather, represents the similarity matrix, Represents the transposed matrix of the similarity matrix. Each element in the feature matrix is ​​converted into a probability through softmax. These probabilities reflect the relative possibility of each category. These probability values ​​are between 0 and 1, and the sum of all outputs is 1.

[0030] The generated fusion matrix is:

[0031] in, Represents a fusion matrix. This fusion matrix is ​​generated by fusing the vehicle-side motion state feature matrix and the weather feature matrix.

[0032] In some embodiments, a device for detecting the steering performance of a mining vehicle is provided, comprising: A data acquisition module is configured to synchronously acquire vehicle-side motion state time series data and weather time series data; The dual-channel construction module is configured to construct a dual-channel CNN, using the pre-processed vehicle-side motion state data as input values ​​and the magnitude of the impact of the vehicle-side motion state on the vehicle steering angle as a label to train the first channel; using the pre-processed weather data as input values ​​and the magnitude of the impact of the weather on the vehicle steering angle as a label to train the second channel; A fusion matrix generation module is configured to integrate the vehicle steering angles at n consecutive acquisition moments to form a vehicle-side motion state feature matrix, and integrate the impact of weather on the vehicle steering angle at the corresponding acquisition moment to form a weather feature matrix, and based on the cross-modal attention fusion mechanism, adjust the weights of the weather feature matrix and the vehicle-side motion state feature matrix to generate a fusion matrix; The model generation module is configured to construct an LSTM, mark the label of the steering angle at the next acquisition moment on the fusion matrix, and input it into the LSTM for training, so that the input value is the fusion matrix, and the output value is the steering angle prediction model; The module for obtaining the target steering angle is configured to input the vehicle-side motion state data and weather data of the target vehicle into the steering angle prediction model to obtain the target steering angle.

[0033] In some embodiments, a mining vehicle steering performance detection device is provided, and the data background executes the above method, including: a correction module configured to perform feedback correction on the steering angle prediction model.

[0034] Compared with the prior art, the method and device for detecting the steering performance of a mining vehicle provided by the embodiment of the present disclosure can achieve the following technical effects: The disclosed embodiment collects and preprocesses vehicle-side motion state data and weather data, uses dual-channel CNN to extract the vehicle-side motion state feature matrix and weather impact feature matrix, generates a fusion matrix through a cross-modal attention mechanism, and then builds an LSTM model for training to generate a steering angle prediction model based on CNN-LSTM. Compared with the prior art, it not only considers the vehicle-side motion state feature data, but also combines the weather impact feature data to predict the steering angle, thereby improving the accuracy of the steering angle.

[0035] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein: Figure 1 It is a flow chart of a method for detecting the steering performance of a mining vehicle provided by an embodiment of the present disclosure; Figure 2 It is a schematic diagram of a mining vehicle steering performance detection device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0038] The following description and accompanying drawings fully illustrate specific embodiments of the present invention so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The examples represent possible changes only. Unless explicitly required, separate components and functions are optional, and the order of operations may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. The scope of the embodiments of the present invention includes the entire scope of the claims, and all available equivalents of the claims. In this article, each embodiment may be represented individually or generally by the term "invention", which is only for convenience, and if more than one invention is disclosed in fact, it is not intended to automatically limit the scope of the application to any single invention or inventive concept. In this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, without requiring or implying any actual relationship or order between these entities or operations. Moreover, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method or automobile including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or automobile. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method or automobile including the element. The various embodiments herein are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the methods, products, etc. disclosed in the embodiments, since they correspond to the method part disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0039] Existing technologies tend to use vehicle models to calculate theoretical turning angles, but only consider vehicle-side motion state data to calculate the turning angle, which has low accuracy.

[0040] In order to solve the problems existing in the related art, the embodiments of the present disclosure provide a method and device for detecting the steering performance of a mining vehicle, so as to solve the problem of low accuracy in identifying the predicted turning angle of a mining vehicle in the prior art.

[0041] See also Figure 1 , is a method for detecting the steering performance of a mining vehicle in an embodiment of the present disclosure. The method comprises: S10, synchronously collect vehicle-side motion status time series data and weather time series data.

[0042] Furthermore, the characteristics of the vehicle-side motion state time series data include vehicle wheelbase, vehicle speed, angular velocity, vehicle driving direction, and front wheel feedback angle; the characteristics of the weather time series data include at least precipitation, snowfall, road surface humidity, and road surface ice and snow layer thickness arranged along the time series.

[0043] It should be understood that the vehicle-side motion state time series data in the present application corresponds to the vehicle-side motion state data, and the weather time series data corresponds to the weather data.

[0044] S11, preprocessing the vehicle-side motion state time series data and weather time series data, the preprocessing process at least includes: filtering outliers in the data, filtering out noise from outliers in the data, normalizing the data and dividing the data, wherein the data is divided into a training set and a validation set.

[0045] The formula for normalization is:

[0046] Where D represents the result of normalizing the collected data. Represents the collected raw data, Represents the maximum value in the original data. Indicates the minimum value in the original data.

[0047] The data is divided into a training set and a validation set, for example, the ratio of the training set to the validation set is 4:1. The training set is used to train the model parameters, and the validation set is used to verify the accuracy of the calculated theoretical rotation angle.

[0048] Furthermore, the preprocessing process may also include: time series construction: dividing the data into sample sequences according to the time step t, and inputting them into the CNN model.

[0049] S20, construct a dual-channel CNN, use the pre-processed vehicle-side motion state data as input values, and the impact of the vehicle-side motion state on the vehicle steering angle as a label to train the first channel; use the pre-processed weather data as input values, and the impact of the weather on the vehicle steering angle as a label to train the second channel.

[0050] S30, integrating the vehicle steering angles at n consecutive collection moments to form a vehicle-side motion state feature matrix, and integrating the impact of the weather on the vehicle steering angle at the corresponding collection moment to form a weather feature matrix, and based on the cross-modal attention fusion mechanism, adjusting the weights of the weather feature matrix and the vehicle-side motion state feature matrix to generate a fusion matrix.

[0051] Specifically, the vehicle-side motion state characteristic matrix is ​​assumed to be:

[0052] Among them, U represents the vehicle-side motion state feature matrix, Represents the j-th vehicle-side motion state feature data at the ith moment, i=1,2,…,n, n represents the total number of collection moments, and p is the total number of vehicle-side motion state features.

[0053] For example, when the vehicle-side motion state characteristics are vehicle wheelbase, vehicle speed, angular velocity, vehicle driving direction, and front wheel feedback angle, p is 5.

[0054] Assume that the weather impact feature matrix is:

[0055] Among them, V represents the weather impact feature matrix, It represents the impact of abnormal weather at the i-th moment on the j-th vehicle-side motion state feature matrix, and q is the total number of weather impact features.

[0056] For example, when the weather impact characteristics are precipitation, snowfall, road surface humidity, and road surface ice and snow layer thickness, q is 4.

[0057] Weather affects vehicle speed, angular velocity and front wheel feedback angle, especially in rainy and snowy days, which affects the adhesion coefficient of the road surface and thus affects the vehicle's driving speed, angular velocity and front wheel feedback angle.

[0058] The principle of generating the fusion matrix is: generating the similarity of each element in the vehicle-side motion state feature matrix and the weather impact feature matrix, and the calculation formula is:

[0059] in, Represents the element value of the i-th row and j-th column in the similarity matrix, Represents the element value of the i-th row and j-th column of the vehicle-side motion state feature matrix, Represents the element value in the i-th row and j-th column of the weather impact feature matrix.

[0060] Use the softmax function to generate attention weights based on the similarity matrix:

[0061]

[0062] in, represents the attention weight of the vehicle-side motion state feature matrix, Indicates the attention weight of the feature matrix of the vehicle's motion state affected by weather, represents the similarity matrix, Represents the transposed matrix of the similarity matrix. Through softmax, each element in the feature matrix can be converted into a probability, which reflects the relative possibility of each category. These probability values ​​are between 0 and 1, and the sum of all outputs is 1.

[0063] The generated fusion matrix is:

[0064] in, Represents a fusion matrix. This fusion matrix is ​​generated by fusing the vehicle-side motion state feature matrix and the weather feature matrix.

[0065] It should be noted that the fusion matrix fuses the vehicle-side motion state feature matrix and the weather feature matrix to generate a comprehensive matrix, which reflects the correlation between different weather and vehicle-side motion state feature data. For example, the higher the attention weight, the greater the correlation.

[0066] S40, constructing an LSTM, marking the label of the steering angle at the next acquisition moment on the fusion matrix, and inputting it into the LSTM for training, so that the input value is the fusion matrix, and the output value is the steering angle prediction model.

[0067] Input fusion matrix; When the LSTM hidden layer is transferred, the unit state, forget gate, input gate and output gate are added, which act together on the hidden layer unit of the original RNN. The calculation formula is as follows: The forget gate determines how much of the cell state at the previous moment is retained at the current moment:

[0068] The input gate determines the input of the network at the current moment and how much is saved to the cell state:

[0069]

[0070] The update of the cell state combines the information from the forget gate and the input gate:

[0071] The output gate determines what information is output, and the output value is scaled by the tanh function:

[0072]

[0073] Map the LSTM output to the steering angle:

[0074] In the above formula, represents the forget gate, represents the input gate, represents the unit state at time t, are candidate memory cells, is the input at time t, represents the output gate, , , They represent the weight of the forget gate, the weight of the input gate, the weight of the candidate unit state, the weight of the output gate, and the weight of the LSTM output mapped to the actual corner. , , , , They represent the forget gate bias, input gate bias, candidate unit state bias, output gate bias, and the bias of LSTM output mapped to the actual corner. represents the output at time t, is the activation function, tanh represents the hyperbolic tangent activation function, and Y is the theoretical rotation angle of the output.

[0075] S50, inputting the vehicle-side motion state data and weather data of the target vehicle into a steering angle prediction model to obtain a target steering angle.

[0076] The loss function is introduced in the training of the steering angle prediction model, including: Using mean square error as the loss function,

[0077] is the theoretical turning angle output by the model, is the front wheel feedback angle in the training data, s represents the index of the current sample data, and N represents the total number of samples in the training data set.

[0078] S60, feedback correction is performed on the steering angle prediction model, including: Apply the steering angle prediction model to the detection vehicle; The real-time vehicle motion state data and real-time weather data are input into the steering angle prediction model to obtain the theoretical steering angle; Compare the theoretical turning angle with the actual front wheel turning angle and calculate the error; The error threshold and time threshold are set. If the error continues to exceed the preset error threshold within the preset time threshold, it is determined that the steering angle feedback is incorrect, and the steering angle prediction model is debugged and corrected.

[0079] For example, the error threshold is 2 degrees and the time threshold is 2 seconds. If the error continues to exceed the error threshold of 2 degrees within the time threshold, the steering angle feedback is judged to be incorrect, and the corresponding fault code is generated and reported to the automatic driving system, and the relevant data is recorded for subsequent analysis.

[0080] See also Figure 2 The embodiment of the present disclosure also discloses a mining vehicle steering performance detection device, including: A data acquisition module is configured to synchronously acquire vehicle-side motion state time series data and weather time series data; The dual-channel construction module is configured to construct a dual-channel CNN, using the pre-processed vehicle-side motion state data as input values ​​and the magnitude of the impact of the vehicle-side motion state on the vehicle steering angle as a label to train the first channel; using the pre-processed weather data as input values ​​and the magnitude of the impact of the weather on the vehicle steering angle as a label to train the second channel; A fusion matrix generation module is configured to integrate the vehicle steering angles at n consecutive acquisition moments to form a vehicle-side motion state feature matrix, and integrate the impact of weather on the vehicle steering angle at the corresponding acquisition moment to form a weather feature matrix, and based on the cross-modal attention fusion mechanism, adjust the weights of the weather feature matrix and the vehicle-side motion state feature matrix to generate a fusion matrix; The model generation module is configured to construct an LSTM, mark the label of the steering angle at the next acquisition moment on the fusion matrix, and input it into the LSTM for training, so that the input value is the fusion matrix, and the output value is the steering angle prediction model; The module for obtaining the target steering angle is configured to input the vehicle-side motion state data and weather data of the target vehicle into the steering angle prediction model to obtain the target steering angle.

[0081] Furthermore, the mining area vehicle steering performance detection device further includes: The correction module is configured to perform feedback correction on the steering angle prediction model.

[0082] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method or automobile including the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0083] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the above-described devices, devices and units may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0084] The flowchart and block diagram in the accompanying drawings show the possible architecture, functions and operations of the device, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by dedicated hardware-based devices that perform the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for detecting the steering performance of a mining vehicle, characterized in that: include: S10, synchronously collecting vehicle-side motion state time series data and weather time series data; S20, construct a dual-channel CNN, use the pre-processed vehicle-side motion state data as input values, and the impact of the vehicle-side motion state on the vehicle steering angle as a label to train the first channel; The second channel is trained using the preprocessed weather data as input and the magnitude of the weather's effect on the vehicle's steering angle as a label. S30, integrating the vehicle steering angles at n consecutive acquisition moments to form a vehicle-side motion state feature matrix, and integrating the impact of the weather on the vehicle steering angle at the corresponding acquisition moment to form a weather feature matrix, and adjusting the weights of the weather feature matrix and the vehicle-side motion state feature matrix based on a cross-modal attention fusion mechanism to generate a fusion matrix; S40, constructing an LSTM, marking the label of the steering angle at the next acquisition moment on the fusion matrix, and inputting it into the LSTM for training, so that the input value is the fusion matrix, and the output value is the steering angle prediction model; S50, inputting the vehicle-side motion state data and weather data of the target vehicle into a steering angle prediction model to obtain a target steering angle.

2. The method for detecting the steering performance of mining vehicles according to claim 1, characterized in that: The characteristics of the vehicle-side motion state time series data include at least one or more of the vehicle wheelbase, vehicle speed, angular velocity, vehicle driving direction, and front wheel feedback angle; the characteristics of the weather time series data include at least one or more of the precipitation, snowfall, road surface humidity, and road surface ice and snow layer thickness arranged along the time series.

3. The method for detecting the steering performance of mining vehicles according to claim 2, characterized in that: Construct LSTM, including: Input fusion matrix; When the LSTM hidden layer is transferred, the unit state, forget gate, input gate and output gate are added, which act together on the hidden layer unit of the original RNN. The calculation formula is as follows: The forget gate determines how much of the cell state at the previous moment is retained at the current moment: The input gate determines the input of the network at the current moment and saves it to the cell state: The update of the cell state combines the information from the forget gate and the input gate: The output gate determines the output information, and the output value is scaled by the tanh function: Map the LSTM output to the steering angle: In the above formula, represents the forget gate, represents the input gate, represents the unit state at time t, are candidate memory cells, is the input at time t, represents the output gate, , , They represent the weight of the forget gate, the weight of the input gate, the weight of the candidate unit state, the weight of the output gate, and the weight of the LSTM output mapped to the actual corner. , , , , They represent the forget gate bias, input gate bias, candidate unit state bias, output gate bias, and the bias of LSTM output mapped to the actual corner. represents the output at time t, is the activation function, tanh represents the hyperbolic tangent activation function, and Y is the theoretical rotation angle of the output.

4. The method for detecting the steering performance of mining vehicles according to claim 1, characterized in that: The loss function is introduced in the training of the steering angle prediction model, including: Using mean square error as the loss function, is the theoretical turning angle output by the model, is the front wheel feedback angle in the training data, s represents the index of the current sample data, and N represents the total number of samples in the training data set; Use the training set to train the CNN-LSTM model, use the validation set to validate the model, and adjust the weight and bias parameters.

5. The method for detecting the steering performance of mining vehicles according to claim 1, characterized in that: Before S20, the method further includes: S11, preprocessing the vehicle-side motion state time series data and weather time series data, the preprocessing process at least includes: filtering outliers in the data, filtering out noise from outliers in the data, normalizing the data and dividing the data, wherein the data is divided into a training set and a validation set.

6. The method for detecting the steering performance of mining vehicles according to claim 5, characterized in that: The formula for normalization is: Where D represents the result of normalizing the collected data. Represents the collected raw data, Represents the maximum value in the original data. Indicates the minimum value in the original data.

7. The method for detecting the steering performance of mining vehicles according to claim 1, characterized in that: S30, integrating the vehicle steering angles at n consecutive acquisition moments to form a vehicle-side motion state feature matrix, and integrating the impact of the weather on the vehicle steering angle at the corresponding acquisition moment to form a weather feature matrix, and adjusting the weights of the weather feature matrix and the vehicle-side motion state feature matrix based on the cross-modal attention fusion mechanism to generate a fusion matrix, including: Assume the vehicle-side motion state characteristic matrix is: Among them, U represents the vehicle-side motion state feature matrix, represents the j-th vehicle-side motion state feature data at the i-th moment, i=1,2,…,n, n represents the total number of acquisition moments, and p is the total number of vehicle-side motion state features; Assume that the weather impact feature matrix is: Among them, V represents the weather impact feature matrix, represents the impact of abnormal weather at the i-th moment on the j-th vehicle-side motion state feature matrix, and q is the total number of weather impact features; The principle of generating the fusion matrix is: generating the similarity of each element in the vehicle-side motion state feature matrix and the weather impact feature matrix, and the calculation formula is: in, Represents the element value of the i-th row and j-th column in the similarity matrix, Represents the element value of the i-th row and j-th column of the vehicle-side motion state feature matrix, Represents the element value of the i-th row and j-th column of the weather impact feature matrix; Use the softmax function to generate attention weights based on the similarity matrix: in, represents the attention weight of the vehicle-side motion state feature matrix, Indicates the attention weight of the feature matrix of the vehicle's motion state affected by weather, represents the similarity matrix, Represents the transposed matrix of the similarity matrix; each element in the feature matrix is ​​converted into a probability through softmax. The probability reflects the relative possibility of each category. The probability value is between 0 and 1, and the sum of all outputs is 1; The generated fusion matrix is: in, Represents a fusion matrix; the fusion matrix is ​​generated by fusing the vehicle-side motion state feature matrix and the weather feature matrix.

8. The method for detecting the steering performance of mining vehicles according to claim 1, characterized in that: Also includes: S60, feedback correction is performed on the steering angle prediction model, including: Apply the steering angle prediction model to the detection vehicle; The real-time vehicle motion state data and real-time weather data are input into the steering angle prediction model to obtain the theoretical steering angle; Compare the theoretical turning angle with the actual front wheel turning angle and calculate the error; The error threshold and time threshold are set. If the error continues to exceed the preset error threshold within the preset time threshold, it is determined that the steering angle feedback is incorrect, and the steering angle prediction model is debugged and corrected.

9. A mining vehicle steering performance detection device, characterized in that: include: A data acquisition module is configured to synchronously acquire vehicle-side motion state time series data and weather time series data; The dual-channel construction module is configured to construct a dual-channel CNN, using the pre-processed vehicle-side motion state data as input values ​​and the magnitude of the impact of the vehicle-side motion state on the vehicle steering angle as a label to train the first channel; The second channel is trained using the preprocessed weather data as input and the magnitude of the weather's effect on the vehicle's steering angle as a label. A fusion matrix generation module is configured to integrate the vehicle steering angles at n consecutive acquisition moments to form a vehicle-side motion state feature matrix, and integrate the impact of weather on the vehicle steering angle at the corresponding acquisition moment to form a weather feature matrix, and based on the cross-modal attention fusion mechanism, adjust the weights of the weather feature matrix and the vehicle-side motion state feature matrix to generate a fusion matrix; The model generation module is configured to construct an LSTM, mark the label of the steering angle at the next acquisition moment on the fusion matrix, and input it into the LSTM for training, so that the input value is the fusion matrix, and the output value is the steering angle prediction model; The module for obtaining the target steering angle is configured to input the vehicle-side motion state data and weather data of the target vehicle into the steering angle prediction model to obtain the target steering angle.

10. The mining vehicle steering performance detection device according to claim 9, characterized in that: Also includes: The correction module is configured to perform feedback correction on the steering angle prediction model.

Images