Modeling method of digital twin data-driven model of autonomous shovel digging process of loader

Abstract

The application discloses a kind of loader autonomous shovel excavating process digital twin data-driven model modeling method, comprising: by obtaining the three-dimensional information of pile before autonomous shovel excavating, the optimal shovel trajectory and simulation operating resistance are obtained by simulation optimization using digital twin mechanism model;Obtain the in-service running data of trajectory, speed and pile after autonomous shovel excavating, calculate and obtain the predicted operating resistance using operating resistance prediction model, and the residual between in-service running data as input, predicted operating resistance and simulation operating resistance as output, establish autonomous shovel excavating process dataset, and according to shovel loading operation section identification result, obtain training set and test set, construct operating resistance residual prediction model based on improved SSA-LSTM algorithm, and output operating resistance residual;Finally, fusion digital twin mechanism model and data-driven model, both hybrid drive constitutes digital twin, realizes the error correction of simulation operating resistance, ensures the accuracy of digital twin model output.

Description

Technical Field

[0001] This invention relates to the field of digital twin simulation modeling, and in particular to a digital twin data-driven modeling method for the autonomous digging process of a loader. Background Technology

[0002] In the current field of simulation modeling, purely mechanistic modeling often simplifies certain factors in terms of modeling accuracy, leading to discrepancies between the resulting model and reality. Furthermore, mechanistic simulations are limited by solution speed, consume enormous computer resources, and are difficult to deploy on edge devices in actual loader operation sites. They must rely on human understanding of the data for adjustments; otherwise, they cannot learn from data and experience from the physical world. In addition, data-driven models based on data struggle to embed physical laws and domain knowledge. Sensor data collected during operation suffers from dispersion, finiteness, high noise, and potential homogeneity. Current sensor measurements have inherent limitations, resulting in models lacking interpretability. This is especially true when dealing with nonlinear, multidisciplinary, and multi-scale physical systems like the autonomous digging process of loaders, where models exhibit poor accuracy, stability, and severely insufficient generalization ability, failing to efficiently utilize this data.

[0003] Since current methods based solely on mechanism simulation or data-driven approaches are insufficient to meet the future development needs of autonomous, intelligent, and unmanned construction machinery, digital twin fusion modeling has been proposed. This approach largely relies on the integration of data-driven models and mechanism models. Because loaders are complex pieces of equipment with intricate components, simulating their working mechanisms in an equivalent manner is challenging. Currently, there is no readily available high-fidelity dynamic modeling and simulation system for loader digging operations.

[0004] Therefore, taking the mechanism model as the core, combining the characteristics of high-precision sensing data with the mechanism model of the system in a reasonable and effective manner, and using data-driven model building methods to process data using theories such as statistics, machine learning, and artificial intelligence, mathematical models are established by mining the inherent information in the data and expressing the system's operating state to characterize the complex fitting relationship between input and output parameters. The data-driven model is used to correct and supplement the parameters of the mechanism model, thereby achieving model fusion and establishing a high-fidelity, dynamic model. This is an important development trend and the main future direction of simulation modeling. Summary of the Invention

[0005] This invention proposes a digital twin data-driven modeling method for the autonomous digging process of a loader. It aims to address the shortcomings of traditional digital twin modeling methods based on single-mechanism or single-data approaches. Using a mechanistic model as the core and leveraging industrial artificial intelligence technologies such as machine learning, this invention proposes a control digital twin modeling method based on the fusion of mechanism and data. This approach fully utilizes existing mechanistic knowledge and data to accurately characterize the properties of components. Furthermore, the model has real-time data-driven update capabilities, allowing for changes to the simulation output of the mechanistic model to more closely approximate the actual operating resistance of the loader (predicted operating resistance), thus establishing a high-fidelity, dynamic model of the loader's autonomous digging process.

[0006] The present invention adopts the following technical solution:

[0007] A digital twin data-driven modeling method for the autonomous digging process of a loader includes:

[0008] Step 1: Obtain the three-dimensional information of the material pile before autonomous digging, and use the digital twin mechanism model to simulate and optimize to obtain the optimal digging trajectory and simulated operation resistance;

[0009] Step 2: Obtain in-service operation data after autonomous excavation, and calculate the predicted operation resistance using the operation resistance prediction model; the in-service operation data includes trajectory information, speed information, and material pile information;

[0010] Step 3: Using the in-service operating data as input, the residual between the predicted operating resistance and the simulated operating resistance is used as output to establish an autonomous digging process dataset. Based on the identification results of the loading operation section, the data training set and test set of the loader's autonomous digging stage are obtained.

[0011] Step 4: Construct a working resistance residual prediction model based on the improved SSA-LSTM algorithm. Input the training set and test set of the data obtained during the autonomous digging stage of the loader, and output the predicted value of the working resistance residual.

[0012] Step 5: Integrate the digital twin mechanism model and the digital twin data-driven model, and use the two to drive the formation of a digital twin. Then, use the predicted residual value of the working resistance to correct the error of the simulated working resistance.

[0013] Preferably, step 1 specifically includes:

[0014] Step 1.1: Use a binocular vision camera and a lidar to acquire the stacking status information of the material pile during the digging operation. Use multi-source data fusion technology to fuse the depth information of the lidar and the image color and texture information of the binocular vision camera to extract the three-dimensional information of the material pile.

[0015] Step 1.2: Using the three-dimensional information of the material pile during the autonomous digging process of the loader, construct a trajectory model of the loader's full bucket digging based on the three-dimensional information of the material pile according to different operating methods, so as to output possible digging trajectory information based on the input of the three-dimensional information of the material pile in front of the loader.

[0016] Step 1.3 analyzes the coupling mechanism of mechanics, hydraulics, control, load, and environment in the autonomous digging process of the loader. Using the unified language of Modelica, a digital twin coupling mechanism model of the loader's mechanics, hydraulics, control, load, and environment is constructed on the Mworks platform, achieving a bidirectional mapping between the physical model and the digital twin model, as follows:

[0017] Step 1.3.1: Construct the Modelica mechanical model of the loader. At the same time, graphically represent the established digital model and expose the necessary interfaces for drag-and-drop system-level modeling, thereby realizing the construction of the mechanical end of the coupled model.

[0018] Step 1.3.2: Determine the parameters of the hydraulic power components, open the corresponding parameter input interfaces for each component of the established hydraulic system, and then complete the setting of hydraulic system parameters; based on the interface information of the above mechanical model, perform drag-and-drop system-level modeling to complete the construction of the hydraulic end model;

[0019] Step 1.3.3: Based on the established digital model of the hydraulic end system, perform stability analysis, model each component of the control system, and then construct the control system using drag-and-drop system-level modeling.

[0020] Step 1.3.4 will ultimately integrate the above processes to construct a complete mechanical, electrical, and hydraulic coupling model of the loader.

[0021] Step 1.4: Analyze the coupling mechanism between the bucket and the material involved in the autonomous digging process of the loader, perform bucket dynamics analysis and bucket-material interaction analysis, and construct the dynamic model of the loader's loading mechanism, as follows:

[0022] Step 1.4.1: Based on the complete mechanical, electrical, and hydraulic coupling model of the loader built in the Mworks platform using the Modelica language, the speed, position, and attitude information of the bucket from various sensors during the loader's loading operation are acquired, and this information is input into the coupling control terminal to drive the mechanical end model.

[0023] Step 1.4.2: Based on the material characteristics and terrain parameters, the target material model is constructed using the discrete element method (DEM) and Modelica language, with corresponding interfaces provided between the model and the loader's mechanical model. This leads to the construction of a bucket structure dynamics model and a bucket-material interaction model. Simultaneously, more accurate three-dimensional information of the stockpile surface is obtained by combining the disparity map obtained from the binocular camera and the lidar disparity map. Through the interface information, a load-environment coupling model is constructed.

[0024] Step 1.4.3: Based on the above construction foundation, the mechanical-electrical-hydraulic coupling model is integrated with the load-environment coupling graphical component model to realize the construction of a digital twin virtual model of the loader's autonomous digging process;

[0025] Step 1.5: Based on the three-dimensional information of the material pile before digging, obtain possible digging trajectory information, and then optimize the trajectory of the loader's autonomous digging process on a multi-domain coupled simulation platform, and output the corresponding optimal trajectory and the corresponding simulated working resistance.

[0026] Preferably, step 3 specifically includes:

[0027] Step 3.1: Use the predicted operating resistance output by the resistance prediction model as the true value F of the loader's operating resistance. z The resistance of the loader's autonomous digging simulation operation, which is output by synchronous simulation of the digital twin mechanism model, is set to F. f The two outputs are subtracted simultaneously to obtain the residual data F of the operating resistance. c ,as follows

[0028] F c =F z -F f

[0029] Step 3.2: The in-service operating sensor data, including cylinder pressure, displacement, and speed, after autonomous excavation are used as the prediction feature column [A1, A2, ..., A...]. k ], k is the number of resistance residual prediction features; F c The residual time series dataset of operational resistance was used as the target column for prediction. The two datasets were merged to generate the time series dataset D loaded from the host excavation process. c As input, namely D c For [A1, A2, ..., A k F c The identification of the excavation operation sections mainly includes five operation stages: unloaded forward movement, loading, fully loaded reverse movement, fully loaded forward movement and unloading, and unloaded reverse movement; then, the aforementioned time series data D is obtained. c Next, it is divided into training and testing sets, and then the host-based shovel process time series dataset D is loaded. c=[D c1 D c2 …, D cz Divide into training set D cr =[D c1 D c2 …, D cm ] and test set D ce =[D cm+1 D cm+2 …, D cz ], and satisfy m < n, where D c1 For [A] 11 A 12 …, A 1k F c1 ];D c2 For [A] 21 A 22 …, A 2k F c2 ];D cz For [A] z1 A z2 …, A zk F cz ];

[0030] The extracted training and test sets are preprocessed, and the dataset is divided into several LSTM data subsets to complete the construction of the dataset; the preprocessing includes data filtering, feature selection and normalization.

[0031] Preferably, in step 4, the method for constructing the operation resistance residual prediction model based on the improved SSA-LSTM algorithm is as follows:

[0032] Step 4.1, modify the SSA sparrow optimization algorithm; assume that the constructed LSTM network model has n LSTM layers, optimize the network hyperparameters in the LSTM network model using the improved SSA algorithm; among them, the network hyperparameters include the number of neurons in each layer, the random inactivation rate of Dropout neurons, and the batch size of data processed by the network in each batch.

[0033] Step 4.1.1: First, add the Sobol sequence to optimize the initial parameters and population, so that the initial individuals are evenly distributed;

[0034] Based on the actual situation of each hyperparameter, let the range of hyperparameter values ​​required to obtain the optimal solution be [x]. min x max The random number K generated by the Sobol sequence. n ∈[0,1], define the initial position x of the population. n ,as follows:

[0035] xn =x min +K n ·(x max -x min )

[0036] Step 4.1.2, construct the sparrow population as follows:

[0037]

[0038] Where d represents the dimension of the problem to be optimized, and n represents the size of the sparrow population;

[0039] Step 4.1.3: Construct the fitness function for the sparrow population. The fitness function for all sparrow populations is expressed in the following form:

[0040]

[0041] in, This represents the set of all population fitness functions; This indicates the fitness of the first population; This indicates the fitness of the second population; This represents the fitness of the nth population. This represents the various hyperparameters of the LSTM network within the set range of values. When the fitness is optimal, the selected LSTM network hyperparameters can minimize the difference between the predicted value of the loader's operating resistance residual at each time step and the actual value of the loader's operating resistance residual.

[0042] Update the optimal position and iteratively optimize to reach the best fitness.

[0043] Step 4.1.4, update the location of the discoverers; when R2 < ST, there are no predators around the foraging area, and the discoverers can search for food extensively; when R2 ≥ ST, predators appear, and all discoverers need to fly to the safe area.

[0044]

[0045] Where t represents the current iteration number, Let α represent the position of the i-th sparrow in the j-th dimension in the t-th generation, α∈(0,1], itermax represents the maximum number of iterations, R2 represents the alarm value, ST represents the safety threshold, Q is a random number that follows a normal distribution, and L is a 1×dim matrix, where dim represents the dimension.

[0046] Step 4.1.5, follower position update; when i > n / 2, the fitness of the i-th joiner is low, and it is not qualified to compete with the discoverer for food, so it needs to fly to other areas to forage; otherwise, the joiner will be in the optimal individual X.p Find food nearby;

[0047]

[0048] in, This represents the location of the individual with the worst fitness in generation t. A represents the position of the individual with the best fitness in generation t+1; A represents a 1×dim matrix, where each element is randomly preset to -1 or 1. + =A T (AA T ) -1 ;

[0049] Step 4.1.6, Update the location of the vigilant; when fi > fg, the individual is on the periphery of the population and needs to take anti-predation behaviors, constantly changing positions to obtain higher fitness; when fi = fg, the individual is in the center of the population and will constantly approach nearby companions to stay away from the danger zone.

[0050]

[0051] in, Let f(t) represent the global optimal position in the t-th iteration; β controls the step size, which follows a normal distribution with a mean of 0 and a variance of 1; K∈[-1, 1]; ε is a constant; fi represents the fitness value of the current individual, and fg and fw represent the fitness values ​​of the current global best and worst individuals, respectively;

[0052] Step 4.1.7: Perturb the current optimal solution to generate a new solution and update the optimal position until the maximum number of iterations is reached. Otherwise, repeat the process after the discoverer's position is updated.

[0053] After the termination condition is met, the optimized hyperparameters of the LSTM neural network are obtained.

[0054] Step 4.2: After the SSA algorithm reaches the maximum number of iterations, the optimal LSTM network model parameters are input into the LSTM network model, and then training is performed to predict the residuals.

[0055] An LSTM network consists of three types of gated units: cell state and forget gate, input gate, and output gate; at time t, the cell state C is... t-1 Hidden state h t-1 and x t The output is the cell state C. t and hidden layer state h t The update process for an LSTM cell can be described as follows:

[0056]

[0057] Among them, W f b f W i b i W c b c For network weights; f t i t , There are three types of gate control units: forget gate, input gate, and output gate; (h1, h2, ..., h t ) represents the hidden layer state; x t For input information, including the time series dataset D of the host excavation process. c Except for F c Feature data of in-service operating sensor data [A1, A2, ..., A] other than the target column for predicting operating resistance residuals. k ];b f b i b o b c These are the biases of the function, respectively; C t and C t-1 This is the output layer of the hidden layer;

[0058] Input the above training set D cr and test set D ce Time series data are fed into the constructed LSTM network for training and testing, and finally a trained operation resistance residual prediction model based on the improved SSA-LSTM algorithm is obtained, which outputs the predicted operation resistance residual.

[0059] Preferably, step 5 specifically includes:

[0060] The predicted residual of the work resistance is used as compensation. In a data-driven manner, the simulated work resistance output of the digital twin mechanism model is corrected, and the simulation output of the mechanism model is changed to continuously approach the predicted work resistance until the error reaches a preset value.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] This invention presents a digital twin data-driven modeling method for the autonomous digging process of a loader. It proposes a control digital twin modeling method based on the fusion of mechanism and data, using a mechanistic model as the core and leveraging industrial artificial intelligence technologies such as machine learning. This approach can fully utilize existing mechanistic knowledge and data to accurately characterize the properties of components. Furthermore, the model has real-time data-driven update capabilities, modifying the simulation output of the mechanistic model to more closely approximate the actual operating resistance of a loader. This establishes a high-fidelity, dynamic model of the loader's autonomous digging process, which is of great significance for the development of autonomous, intelligent, and unmanned construction machinery. Attached Figure Description

[0063] Figure 1 A flowchart illustrating the digital twin data-driven modeling method for the autonomous digging process of a loader, according to an embodiment of the present invention;

[0064] Figure 2 This is a roadmap for predicting and modeling the residual working resistance during the autonomous digging process of a loader, as described in this invention.

[0065] Figure 3 This is a roadmap for digital twin data-driven modeling technology for the autonomous digging process of a loader based on industrial artificial intelligence, as described in this invention. Detailed Implementation

[0066] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0067] See Figure 1 As shown in this embodiment, a digital twin data-driven modeling method for the autonomous digging process of a loader includes:

[0068] Step 1: Obtain the three-dimensional information of the material pile before autonomous digging, and use the digital twin mechanism model to simulate and optimize to obtain the optimal digging trajectory and simulated operation resistance;

[0069] Step 2: Obtain in-service operation data after autonomous excavation, and calculate the predicted operation resistance using the operation resistance prediction model; the in-service operation data includes trajectory information, speed information, and material pile information;

[0070] Step 3: Using the in-service operating data as input, the residual between the predicted operating resistance and the simulated operating resistance is used as output to establish an autonomous digging process dataset. Based on the identification results of the loading operation section, the data training set and test set of the loader's autonomous digging stage are obtained.

[0071] Step 4: Construct a working resistance residual prediction model based on the improved SSA-LSTM algorithm. Input the training set and test set of the data obtained during the autonomous digging stage of the loader, and output the predicted value of the working resistance residual.

[0072] Step 5: Integrate the digital twin mechanism model and the digital twin data-driven model, and use the two to drive the formation of a digital twin. Use the predicted residual value of the working resistance to correct the error of the simulated working resistance; ensure the accuracy of the output of the digital twin model.

[0073] For details, see Figure 2 The diagram shown illustrates the technical roadmap for establishing a prediction model of the operational resistance residual during the autonomous digging process of a loader, as provided in this embodiment of the invention. The specific steps are as follows:

[0074] (1) First, the autonomous digging operation and in-service data perception of the loader are used to obtain the material pile information before digging as the basis for constructing the load-environment coupling model of the digital twin mechanism model.

[0075] Simultaneously, the sensor data acquired after digging, including the trajectory, speed, and material pile information after autonomous digging, are used as input to the loader's autonomous digging operation resistance prediction model. The predicted operation resistance output by the operation resistance prediction model is used as the true value F of the loader's operation resistance. z The digital twin mechanism model is used for synchronous simulation to output the resistance F of the loader's autonomous digging operation. f The two outputs are then subtracted to obtain the residual data F of the operating resistance. c ,as follows:

[0076] F c =F z -F f

[0077] (2) The in-service operating sensor data such as hydraulic cylinder pressure, displacement, and speed after autonomous excavation are used as the prediction feature column, F c The residual time series dataset of operational resistance was used as the target column for prediction. The two datasets were merged to generate the host excavation process dataset D. c As input, the excavation operation segment is identified, mainly including five operation stages: unloaded forward movement, loading, fully loaded backward movement, fully loaded forward movement and unloading, and unloaded backward movement; then, the time series data D for each stage is obtained. c The dataset was divided into training and testing sets, and then the host-based data mining process dataset D was used. c =[D c1 D c2 …, D cz Divide into training set D cr =[D c1， Dc2…,D cm ] and test set D ce =[D cm+1 D cm+2 …, D cz ], and satisfy m < n, where D c1 For [A] 11 A 12 …, A 1k F c1 ];D c2 For [A] 21 A 22…, A 2k F c2 ];D cz For [A] z1 A z2 …, A zk F cz ];

[0078] The extracted dataset D, loaded from the host machine, represents the shovel process. c The training and test sets are preprocessed, mainly by data filtering, feature selection and normalization, and the dataset is divided into several LSTM data subsets to complete the construction of the dataset.

[0079] (3) Then, the SSA sparrow optimization algorithm is modified by introducing Sobol sequences, Sin chaos, and the Bird Swarm Algorithm (BSA). Assume the constructed LSTM network model has n LSTM layers, and the number of neurons in each layer of the LSTM network model is Neurons1, ..., Neurons... n The dropout neuron random inactivation rate and the batch size are neural network hyperparameters used as target parameters for improving the SSA algorithm.

[0080] (3.1) First, the initial parameters and population are optimized by adding Sobol sequences, etc., to prevent the original SSA basic algorithm from getting trapped in local optima and having a slow convergence speed, so that the initial individuals are evenly distributed.

[0081] Based on the actual situation of each hyperparameter, let the range of hyperparameter values ​​required to obtain the optimal solution be [x]. min x max The random number K generated by the Sobol sequence. n ∈[0,1], defines the initial position of the population.

[0082] x n =x min +K n ·(x max -x min )

[0083] (3.2) Construct a sparrow population.

[0084]

[0085] Where d represents the dimension of the problem to be optimized, and n represents the size of the sparrow population.

[0086] (3.3) Construct the fitness function of the sparrow population.

[0087]

[0088] Among them, among them, This represents the set of all population fitness functions; This indicates the fitness of the first population; This indicates the fitness of the second population; This represents the fitness of the nth population. This represents the various hyperparameters of the LSTM network within the set range of values. When the fitness is optimal, the selected LSTM network hyperparameters can minimize the difference between the predicted value of the loader's operating resistance residual at each time step and the actual value of the loader's operating resistance residual.

[0089] Update the optimal position and iterate to find the best fitness.

[0090] (3.4) Location update of discoverers. When R2 < ST, there are no predators around the foraging area, and discoverers can search for food extensively; when R2 ≥ ST, predators appear, and all discoverers need to fly to the safe area.

[0091]

[0092] Where t represents the current iteration number, Let α represent the position of the i-th sparrow in the j-th dimension in the t-th generation, α∈(0,1], itermax represents the maximum number of iterations, R2 represents the alarm value, ST represents the safety threshold, Q is a random number following a normal distribution, and L is a 1×dim matrix, where dim represents the dimension.

[0093] (3.5) Follower position update. When i > n / 2, the fitness of the i-th follower is low, and it is not qualified to compete with the discoverer for food, so it needs to fly to other areas to forage; otherwise, the follower will be in the optimal individual X. p Looking for food nearby.

[0094]

[0095] in, This represents the location of the individual with the worst fitness in generation t. Let A represent the position of the individual with the best fitness in generation t+1. Let A be a 1×dim matrix, where each element is randomly preset to -1 or 1. + =A T (AA T ) -1 .

[0096] (3.6) Update of vigilant position. When fi > fg, the individual is on the periphery of the population and needs to take anti-predation behaviors to continuously change positions to obtain higher fitness; when fi = fg, the individual is in the center of the population and will continuously approach nearby companions to move away from the danger zone.

[0097]

[0098] in, Let f(t) represent the global optimal position in the t-th iteration; β controls the step size, which follows a normal distribution with a mean of 0 and a variance of 1; K∈[-1, 1]; ε is a constant used to avoid the denominator being 0. fi represents the fitness value of the current individual, and fg and fw represent the fitness values ​​of the current global best and worst individuals, respectively.

[0099] (3.7) Perturb the current optimal solution to generate a new solution and update the optimal position until the maximum number of iterations is reached. Otherwise, repeat the process after the position update of the discoverer.

[0100] After the termination condition is met, the optimized hyperparameters of the LSTM neural network are output.

[0101] (4) After the SSA algorithm reaches the maximum number of iterations, it outputs the optimal LSTM network model parameters. These parameters are then input into the LSTM network model for training and prediction of residuals.

[0102] An LSTM network consists of three gating units: a cell state gate, a forget gate, an input gate, and an output gate. At time t, there are three inputs: the cell state C. t-1 Hidden state h t-1 and the input x at time t t The output is the cell state C. t and hidden layer state h t The update process for an LSTM cell can be described as follows.

[0103]

[0104] Among them, W f b f W i b i W c b c For network weights; f t i t , There are three types of gate control units: forget gate, input gate, and output gate; (h1, h2, ..., h t ) represents the hidden layer state; x t The input information is the time series dataset D of the host excavation process.c Except for F c The in-service operating sensor data, including cylinder pressure, displacement, and velocity, are characteristics outside the target column for predicting operating resistance residuals [A1, A2, ..., A]. k Taking model training as an example, in a batch of size 1, the training data at time t-1 will be divided into feature variables and target variables. The feature variables are the first set of data D in the training set. c1 For [A] 11 A 12 …, A 1k The target variable is the residual of the loader's operating resistance [F]. c1 At time t-1, the feature variables and target variable data are simultaneously input into the LSTM, and through the calculations of various gates, a cell state C is output. t-1 with h t-1 These two variables will be used as input variables in the calculation at the next time step. At time t, the output C at time t-1 will be... t-1 with h t-1 and the feature data [A] in the second set of training data Dc2 input at time t. 21 A 22 …, A 2k The inputs are combined and fed into the LSTM model. Through calculations at various gates, C is finally obtained. t with h t Then input the next time step, repeating the above process until the entire batch is finished; b f b i b o b c These are the biases of the function, respectively; C t and C t-1 This is the output layer of the hidden layer.

[0105] Input the above training set D cr and test set D ce Time series data is fed into the constructed LSTM network for machine training and testing, and finally a work resistance residual prediction model based on the improved SSA-LSTM algorithm is constructed, which outputs the predicted work resistance residual.

[0106] See Figure 3 The diagram shows a roadmap for digital twin data-driven modeling technology for the autonomous digging process of a loader based on industrial artificial intelligence, provided in an embodiment of the present invention. The specific steps are as follows:

[0107] (1) Use binocular vision camera and lidar to acquire the stacking status information of material pile during shoveling operation, and use multi-source data fusion technology to fuse the depth information of lidar and image color and texture information to extract the three-dimensional information of material pile.

[0108] (2) Using the three-dimensional information of the material pile after the loader autonomously digs, a trajectory model of the loader full bucket digging based on the three-dimensional information of the material pile is constructed according to different operation methods, so that possible digging trajectory information can be output based on the input of the three-dimensional information of the material pile before digging.

[0109] (3) Analyze the coupling mechanism of mechanical, hydraulic, control, load and environment in the autonomous digging process of the loader. Construct a digital twin coupling mechanism model of mechanical, hydraulic, control, load and environment of the loader based on Modelica on the Mworks platform through the unified language of Modelica, and realize the bidirectional mapping between the physical model and the digital twin model.

[0110] (3.1) Construct the Modelica mechanical model of the loader, and at the same time, graphically represent the established digital model and open the necessary interfaces for drag-and-drop system-level modeling, thereby realizing the construction of the mechanical end of the coupled model.

[0111] (3.2) Determine the parameters of the hydraulic power components, open the corresponding parameter input interfaces for each component, and then complete the parameter settings of the hydraulic system. Based on the interface information of the above mechanical model, perform drag-and-drop system-level modeling to complete the construction of the hydraulic end model.

[0112] (3.3) Based on the established digital model of the hydraulic end system, stability analysis is performed, and the components of the control system are modeled. Then, the control system is constructed using drag-and-drop system-level modeling.

[0113] (3.4) Finally, the above processes will be integrated to construct a complete mechanical, electrical and hydraulic coupling model of the loader.

[0114] (4) Analyze the coupling mechanism between the bucket and the material involved in the autonomous digging process of the loader, conduct bucket dynamics analysis and bucket-material mutual operation analysis, and construct a dynamic model of the loader's digging mechanism.

[0115] (4.1) In addition, based on the construction of the electromechanical-hydraulic coupling model using the Modelica language on the Mworks platform, information such as the speed, position, and attitude of the bucket during the digging process is obtained from various sensors such as speed and angle during the loader's digging operation. This information is then input into the coupling control end to drive the mechanical end model.

[0116] (4.2) Based on the material characteristics parameters and terrain parameters, the target material model is constructed using the discrete element theory and Modelica language. Then, the bucket structure dynamics model and the interaction model between the bucket and the material are constructed. At the same time, the parallax map obtained by the binocular camera and the parallax map of the lidar are combined to obtain more accurate three-dimensional information of the material pile surface. Through the interface information, the coupling model of bucket-material (load-environment) is constructed.

[0117] (4.3) Based on the above construction foundation, the mechanical-electrical-hydraulic coupling model and the load-environment coupling graphical component model are integrated to realize the construction of a digital twin virtual model of the loader's autonomous digging process.

[0118] (5) Based on the three-dimensional information of the material pile before digging, obtain possible digging trajectory information, and then optimize the trajectory of the loader's autonomous digging process on a multi-domain coupled simulation platform, and finally output the corresponding optimal trajectory and the corresponding simulation operation resistance.

[0119] (6) Using the trajectory, speed, and material pile information after autonomous digging in (2) as input, and the residual between the predicted working resistance and the simulated working resistance as output, establish a dataset for the autonomous digging process. Based on the identification results of the loading operation section, obtain the training and testing sets of the data for the autonomous digging stage of the loader. Figure 2 The process constructs a prediction model for operational resistance residuals based on an improved SSA-LSTM (Sparrow Algorithm-Optimized Long Short-Term Memory Neural Network) algorithm, and finally outputs the operational resistance residuals.

[0120] (7) The predicted residual of the working resistance output described in (6) is used as compensation. In a data-driven manner, the simulated working resistance output of the digital twin mechanism model constructed in (5) is further corrected. The simulation output of the mechanism model is changed to make it closer to the actual working resistance of the loader. The constructed digital twin model is closer to the actual loader loading operation. Finally, a digital twin data-driven model of the loader's autonomous digging process is obtained.

[0121] According to another aspect of the present invention, a digital twin data-driven modeling system for the autonomous digging process of a loader includes:

[0122] The simulation operation resistance acquisition module is used to acquire three-dimensional information of the material pile before autonomous digging, and to obtain the optimal digging trajectory and simulation operation resistance by using a digital twin mechanism model for simulation optimization.

[0123] The predicted operating resistance acquisition module is used to acquire in-service operating data after autonomous excavation and calculate the predicted operating resistance using an operating resistance prediction model; the in-service operating data includes trajectory information, speed information, and material pile information;

[0124] The data training set and test set acquisition module is used to take the in-service operation data as input, predict the residual between the working resistance and the simulated working resistance as output, establish an autonomous digging process dataset, and acquire the data training set and test set of the loader's autonomous digging stage based on the identification results of the loading operation section.

[0125] The operation resistance residual prediction output module is used to build an operation resistance residual prediction model based on the improved SSA-LSTM algorithm. It takes as input the training set and test set of data obtained during the autonomous digging stage of the loader and outputs the operation resistance residual prediction value.

[0126] The error correction module is used to integrate the digital twin mechanism model and the digital twin data-driven model. The two are mixed to form a digital twin, and the error correction is performed on the simulated operation resistance using the predicted value of the operation resistance residual.

[0127] This embodiment provides a specific implementation of a digital twin data-driven modeling system for the autonomous digging process of a loader. The same digital twin data-driven modeling method for the autonomous digging process of a loader will not be described again in this embodiment.

[0128] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A digital twin data-driven modeling method for the autonomous digging process of a loader, characterized in that, include: Step 1: Obtain the three-dimensional information of the material pile before autonomous digging, and use the digital twin mechanism model to simulate and optimize to obtain the optimal digging trajectory and simulated operation resistance; Step 2: Obtain in-service operation data after autonomous excavation, and calculate the predicted operation resistance using the operation resistance prediction model; the in-service operation data includes trajectory information, speed information, and material pile information; Step 3: Using the in-service operating data as input, the residual between the predicted operating resistance and the simulated operating resistance is used as output to establish an autonomous digging process dataset. Based on the identification results of the loading operation section, the data training set and test set of the loader's autonomous digging stage are obtained. Step 4: Construct a working resistance residual prediction model based on the improved SSA-LSTM algorithm. Input the training set and test set of the data obtained during the autonomous digging stage of the loader, and output the predicted value of the working resistance residual. Step 5: Integrate the digital twin mechanism model and the digital twin data-driven model, and use the two to drive the formation of a digital twin. Then, use the predicted value of the work resistance residual to correct the error of the simulated work resistance. Step 3 specifically includes: Step 3.1: Use the predicted operating resistance output by the resistance prediction model as the actual value of the loader's operating resistance. The resistance of the loader's autonomous digging simulation operation, which is output by synchronously simulating the digital twin mechanism model, is set as... The two outputs are subtracted simultaneously to obtain the residual data of the operating resistance. ,as follows ； Step 3.2, taking the in-service operation sensing data including cylinder pressure, displacement, and speed after autonomous excavation as the prediction feature columns , …, , being the number of prediction features of the resistance residual; The working resistance residual time series data set is used as the prediction target column, and the two are combined to generate the time series data set of the loading autonomous machine excavation process As the input, that is being , …, to identify the excavation operation segments, mainly including: 5 operation stages of forward movement with no load, loading, backward movement with full load, forward movement with full load for unloading, and backward movement with no load; then after obtaining the above-mentioned time series data set , it is divided into a training set and a test set. After that, the time series data set of the loading autonomous machine excavation process = , …, is split into a training set = , …, and a test set = , …, , and m < n is satisfied, where being , …, ; being , …, ; being , …, ; The extracted training and test sets are preprocessed, and the dataset is divided into several LSTM data subsets to complete the construction of the dataset; the preprocessing includes data filtering, feature selection and normalization.

2. The digital twin data-driven modeling method for the autonomous digging process of a loader according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Use a binocular vision camera and a lidar to acquire the stacking status information of the material pile during the digging operation. Use multi-source data fusion technology to fuse the depth information of the lidar and the image color and texture information of the binocular vision camera to extract the three-dimensional information of the material pile. Step 1.2: Using the three-dimensional information of the material pile during the autonomous digging process of the loader, construct a trajectory model of the loader's full bucket digging based on the three-dimensional information of the material pile according to different operating methods, so as to output possible digging trajectory information based on the input of the three-dimensional information of the material pile in front of the loader. Step 1.3 analyzes the coupling mechanism of mechanics, hydraulics, control, load, and environment in the autonomous digging process of the loader. Using the unified language of Modelica, a digital twin coupling mechanism model of the loader's mechanics, hydraulics, control, load, and environment is constructed on the Mworks platform, achieving a bidirectional mapping between the physical model and the digital twin model, as follows: Step 1.3.1: Construct the Modelica mechanical model of the loader. At the same time, graphically represent the established digital model and expose the necessary interfaces for drag-and-drop system-level modeling, thereby realizing the construction of the mechanical end of the coupled model. Step 1.3.2: Determine the parameters of the hydraulic power components, open the corresponding parameter input interfaces for each component of the established hydraulic system, and then complete the setting of hydraulic system parameters; based on the interface information of the above mechanical model, perform drag-and-drop system-level modeling to complete the construction of the hydraulic end model; Step 1.3.3: Based on the established digital model of the hydraulic end system, perform stability analysis, model each component of the control system, and then construct the control system using drag-and-drop system-level modeling. Step 1.3.4 will ultimately integrate the above processes to construct a complete mechanical, electrical, and hydraulic coupling model of the loader. Step 1.4: Analyze the coupling mechanism between the bucket and the material involved in the autonomous digging process of the loader, perform bucket dynamics analysis and bucket-material interaction analysis, and construct the dynamic model of the loader's loading mechanism, as follows: Step 1.4.1: Based on the complete mechanical, electrical, and hydraulic coupling model of the loader built in the Mworks platform using the Modelica language, the speed, position, and attitude information of the bucket from various sensors during the loader's loading operation are acquired, and this information is input into the coupling control terminal to drive the mechanical end model. Step 1.4.2: Based on the material characteristics and terrain parameters, the target material model is constructed using the discrete element method (DEM) and Modelica language, with corresponding interfaces provided between the model and the loader's mechanical model. This leads to the construction of a bucket structure dynamics model and a bucket-material interaction model. Simultaneously, more accurate three-dimensional information of the stockpile surface is obtained by combining the disparity map obtained from the binocular camera and the lidar disparity map. Through the interface information, a load-environment coupling model is constructed. Step 1.4.3: Based on the above construction foundation, the mechanical-electrical-hydraulic coupling model is integrated with the load-environment coupling graphical component model to realize the construction of a digital twin virtual model of the loader's autonomous digging process; Step 1.5: Based on the three-dimensional information of the material pile before digging, obtain possible digging trajectory information, and then optimize the trajectory of the loader's autonomous digging process on a multi-domain coupled simulation platform, and output the corresponding optimal trajectory and the corresponding simulated working resistance.

3. The digital twin data-driven modeling method for the autonomous digging process of a loader according to claim 2, characterized in that, In step 4, the method for constructing the operation resistance residual prediction model based on the improved SSA-LSTM algorithm is as follows: Step 4.1, modify the SSA sparrow optimization algorithm; assume the constructed LSTM network model has The LSTM layer optimizes the network hyperparameters in the LSTM network model using an improved SSA algorithm; the network hyperparameters include the number of neurons in each layer. Dropout neuron random inactivation rate and Batch Size: the number of data processed per batch by the network; Step 4.1.1: First, add the Sobol sequence to optimize the initial parameters and population, so that the initial individuals are evenly distributed; Based on the actual situation of each hyperparameter, let the range of hyperparameter values ​​required to obtain the optimal solution be []. , Random numbers generated by Sobol sequences ∈[0,1], defines the initial position of the population. ,as follows: ； Step 4.1.2, construct the sparrow population as follows: ； in, This represents the dimension of the problem to be optimized. Indicates the population size of sparrows; Step 4.1.3: Construct the fitness function for the sparrow population. The fitness function for all sparrow populations is expressed in the following form: ； in, This represents the set of all population fitness functions; This indicates the fitness of the first population; This indicates the fitness of the second population; This represents the fitness of the nth population. This represents the various hyperparameters of the LSTM network within the set range of values. When the fitness is optimal, the selected LSTM network hyperparameters can minimize the difference between the predicted value of the loader's operating resistance residual at each time step and the actual value of the loader's operating resistance residual. Update the optimal position and iteratively optimize to reach the best fitness. Step 4.1.4, update the location of the discoverers; when R2 < ST, there are no predators around the foraging area, and the discoverers can search for food extensively; when R2 ≥ ST, predators appear, and all discoverers need to fly to the safe area; ； in, Indicates the current iteration number. Indicates the first The middle generation Only sparrows in the first The position of the dimension , Represents the maximum number of iterations. Indicates the alarm value. Indicates the safety threshold. It is a random number that follows a normal distribution. yes The matrix, Indicates dimension; Step 4.1.5, update the position of the follower; when At that time, the first Individuals with low fitness are not qualified to compete with the discoverer for food and need to fly to other areas to forage; otherwise, the individual will be the best candidate. Find food nearby; ； in, This represents the location of the individual with the worst fitness in generation t. Indicates the first The location of the individual with the best fitness in the generation; express A matrix, where each element is randomly preset to either -1 or 1. = ; Step 4.1.6, Guardian location update; when At this time, the individual is on the periphery of the population and needs to engage in anti-predation behavior, constantly changing location to gain higher fitness; when At this time, the individual is in the center of the population, and it will continuously approach nearby companions to move away from the danger zone; ； in, Indicates the first The global optimal position in the next iteration; The step size is controlled to follow a normal distribution with a mean of 0 and a variance of 1; ; It is a constant; This represents the fitness value of the current individual. and These represent the fitness values ​​of the best and worst individuals in the global dataset, respectively. Step 4.1.7: Perturb the current optimal solution to generate a new solution and update the optimal position until the maximum number of iterations is reached. Otherwise, repeat the process after the discoverer's position update. After the termination condition is met, the optimized hyperparameters of the LSTM neural network are obtained. Step 4.2: After the SSA algorithm reaches the maximum number of iterations, the optimal LSTM network model parameters are input into the LSTM network model, and then training is performed to predict the residuals. An LSTM network is composed of three types of gating units: cell state and forget gate, input gate, and output gate; Cellular state at all times Hidden state and The output is the cell state. and hidden layer states The update process for an LSTM cell can be described as follows: ； in, , , , , , Network weights; , , It includes three types of gate control units: forget gate, input gate, and output gate; , , ..., () represents the hidden layer state; Input information includes a time-series dataset of the host excavation process. Except Feature data of in-service operating sensor data for cylinder pressure, displacement, and velocity, outside the target column for predicting operating resistance residuals. , …, ]; , , , These are the biases of the function, respectively; This is the output layer of the hidden layer; Input the above training set and test set Time series data are fed into the constructed LSTM network for training and testing, and finally a trained operation resistance residual prediction model based on the improved SSA-LSTM algorithm is obtained, which outputs the predicted operation resistance residual.

4. The digital twin data-driven modeling method for the autonomous digging process of a loader according to claim 3, characterized in that, Step 5 specifically includes: The predicted residual of the work resistance is used as compensation. In a data-driven manner, the simulated work resistance output of the digital twin mechanism model is corrected, and the simulation output of the mechanism model is changed to continuously approach the predicted work resistance until the error reaches a preset value.

Images