Steel grabbing machine tail end track correction method based on domain adaptation technology

Abstract

The invention relates to a steel grabbing machine tail end track correction method based on a domain adaptation technology, which belongs to the field of automatic loading and unloading, and comprises the following steps: collecting a steel grabbing machine tail end motion track, and generating a mature hand data set and a software planning data set; preprocessing the matured hand data set to generate a matured hand input sample set and a matured hand output reference set; preprocessing the software planning data set to generate a software planning input sample set; s2, performing domain adaptation processing on the two input sample sets, and changing domain adaptation parameters to obtain r different input sample set domain adaptation outputs; s3, constructing a convolutional network for training based on r domain-adapted matured hand input sample sets and matured hand output reference sets to obtain r corresponding different prediction models; and S4, inputting the r software planning input sample sets subjected to domain adaptation into the corresponding prediction models for fusion, and finally obtaining a corrected trajectory of the tail end trajectory.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic loading and unloading, and relates to a method for correcting the end trajectory of a steel grabber based on domain adaptation technology. Background Art

[0002] With the development of artificial intelligence technology, automatic loading and unloading has become an important application field. In the process of automatic loading and unloading of a steel grabber, reasonable correction of the end trajectory of the steel grabber can greatly improve the loading and unloading efficiency. At present, there are few reports on the research results in this field, and the main difficulties are as follows:

[0003] (1) It is difficult for traditional software to accurately simulate the actual operation of a human skilled operator in planning the path trajectory.

[0004] (2) Deep learning-related methods rely on a large amount of training data, but the data scenarios of skilled operators do not match the software planning scenarios, resulting in poor model generalization ability.

[0005] (3) The observed trajectory generally uses fixed-time interval sampling, and it is difficult to utilize the multi-scale and multi-resolution depth information of sampling points with different time intervals. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a method for correcting the end trajectory of a steel grabber based on domain adaptation technology. By performing domain adaptation processing on the skilled operator scenario and the software planning scenario, the distributions of the skilled operator scenario and the software planning scenario are made consistent, and sequences of sampling points with different time intervals are fused to utilize the multi-scale and multi-resolution sampling point information to more accurately correct the end trajectory.

[0007] To achieve the above object, the present invention provides the following technical solution:

[0008] A method for correcting the end trajectory of a steel grabber based on domain adaptation technology, comprising the following steps:

[0009] S1: Collect the end movement trajectories of the steel grabber in the skilled operator scenario and the software planning scenario to generate corresponding skilled operator scenario datasets and software planning scenario datasets; preprocess the skilled operator scenario datasets to generate skilled operator input sample sets and skilled operator output reference sets; preprocess the software planning scenario datasets to generate software planning input sample sets;

[0010] S2: Perform domain adaptation processing on the skilled operator input sample sets and the software planning input sample sets, and change the domain adaptation parameters to obtain r different domain adaptation outputs of the skilled operator input sample sets and the software planning input sample sets;

[0011] S3: Based on the r domain-adapted skilled operator input sample sets and the skilled operator output reference sets, construct a convolutional network for training to obtain r corresponding different prediction models;

[0012] S4: Input the software planning input sample set after adaptation of r domains into the corresponding prediction model for fusion, and finally obtain the corrected trajectory of the end trajectory.

[0013] Furthermore, in the step S1, the expert scenario data set S includes F frame position coordinates formed by equally spaced sampling of the movement trajectories of N expert operators over a period of time, that is where 1 frame of data records the spatial positions of N operators at a certain moment, that is where the spatial position identifier of operator n in frame sequence f is is the frame sequence number, 1 ≤ n ≤ N is the operator number, and (x, y, z) are the coordinate points of the end trajectory in the three-dimensional space;

[0014] The time period [Δt, T + Δt] includes the T0 + T1 frame continuous position coordinates of N operators. The data of the first T0 frames constructs a multi-dimensional training input sample The data of the subsequent T1 frames constructs a multi-dimensional training output reference sample where the input sample of the nth operator output reference sample

[0015] Change the starting time Δt, that is, Δt = mT fra , 0 ≤ m ≤ (M - 1), where T fra is the time interval between adjacent frames, and a total of M input samples are obtained and M multi-dimensional output reference samples

[0016] Similarly, the software planning data set constructs K multi-dimensional input samples and K multi-dimensional output reference samples

[0017] Furthermore, the preprocessing of the software planning scenario data set includes automated trajectory segmentation, specifically including:

[0018] Divide the loading and unloading actions into 10 different stages for specific analysis. Based on the time series data of various parameters of the collected loading action trajectories, discover the sensitive parameters for stage division;

[0019] By comparing and analyzing the parameters collected in the actual loading and unloading actions of multiple grab cranes, it is concluded that the most sensitive and efficient parameters for stage division are the body rotation angle parameter and the end height parameter of the stick;

[0020] Normalize the body rotation angle parameter and the end height parameter of the stick to the interval [0, 1];

[0021] Perform median filtering and smoothing on the normalized parameter data;

[0022] Calculate the slope of each smoothed parameter data and normalize it to the interval [0, 1];

[0023] Perform median filtering and smoothing on the normalized parameter slopes for the data of different parameters used for stage division;

[0024] Set thresholds according to the physical meanings of the loading and unloading actions in different stages, and perform automated trajectory stage division corresponding to the specific values and slope values of the sensitive parameters. A total of 10 different stages are divided, with 11 time nodes t1 - t11.

[0025] Furthermore, the setting of thresholds according to the physical meanings of the loading and unloading actions in different stages, and performing automated trajectory stage division corresponding to the specific values and slope values of the sensitive parameters, a total of 10 different stages are divided, with 11 time nodes t1 - t11, specifically including:

[0026] 1) When the slope of the boom tip height parameter is lower than the negative threshold and the slope of the vehicle body rotation angle parameter is lower than the threshold, it is recorded as t1, indicating that the tip starts to descend until the material pile;

[0027] 2) When the slope of the boom tip height parameter is lower than the negative threshold and the slope of the vehicle body rotation angle parameter is lower than the threshold, it is recorded as t2, indicating that the bucket has been shoveled into the material pile and the excavation action starts;

[0028] 3) When the slope of the boom tip height parameter is higher than the negative threshold and the slope of the vehicle body rotation angle parameter is lower than the threshold, it is recorded as t3, indicating that the excavation is completed and the boom tip starts to rise;

[0029] 4) When the slope of the vehicle body rotation angle parameter is higher than the threshold, it is recorded as t4, indicating that the vehicle body starts to rotate and the tip also rises vertically at the same time;

[0030] 5) When the slope of the boom tip height parameter is lower than the threshold and the slope of the vehicle body rotation angle parameter is higher than the threshold, it is recorded as t5, indicating that the tip has risen to the highest point, and at this time the vehicle body has not finished rotating;

[0031] 6) When the slope of the vehicle body rotation angle parameter is lower than the threshold, it is recorded as t6, indicating that the tip is at the highest point of the unloading point and the vehicle body has rotated above the unloading point;

[0032] 7) When the slope of the boom tip height parameter is lower than the negative threshold and the slope of the vehicle body rotation angle parameter is higher than the negative threshold, it is recorded as t7, indicating that the tip starts to descend and locates to the unloading point;

[0033] 8) If the slope of the boom tip height parameter is higher than the negative threshold and the slope of the vehicle rotation angle parameter is higher than the negative threshold, it is denoted as t8, indicating that the bucket starts to unload. During unloading, the tip height is raised for some action samples. After unloading, the final tip height is at a suitable position to clear the baffle.

[0034] 9) If the slope of the vehicle rotation angle parameter is lower than the negative threshold, it is denoted as t9, indicating that the tip height is already at a suitable height and the vehicle starts to rotate until the tip crosses the baffle.

[0035] 10) If the slope of the boom tip height parameter is lower than the negative threshold and the slope of the vehicle rotation angle parameter is higher than the negative threshold, it is denoted as t10. The tip has crossed the baffle and the tip height starts to decrease. The tip continues to rotate until it is above the material pile.

[0036] 11) If the slope of the vehicle rotation angle parameter is higher than the negative threshold, it is denoted as t11, indicating that the tip has reached above the material pile and is ready to start a new grasping action.

[0037] Further, step S2 specifically includes the following steps:

[0038] S21: Extract the expert input sample set and the software planning input sample set of the x-component X x , Y x ; Consider X x and Y x as the source domain and the target domain respectively for domain adaptation processing. The domain adaptation methods for the spatial x, y, and z components are exactly the same. The domain adaptation process of the x-component is described as follows:

[0039] The source domain data set is constructed as an MN×T0 matrix:

[0040]

[0041] The target data set is constructed as a KN×T0 matrix:

[0042]

[0043] S22: Construct the kernel matrix K, the maximum mean discrepancy measure matrix M, and the central matrix H as follows:

[0044] The kernel matrix K is composed of the source domain matrix X x and the target domain matrix Y x concatenated, that is where the MN+MK-dimensional row vector is the F-norm of the matrix

[0045] That is, Central matrix where \(E\) is an \(MN + MK\)-dimensional identity matrix and \(1\) is an \(MN + MK\)-dimensional all-ones row vector;

[0046] S23: Perform eigenvalue decomposition on the matrix \((KMK+\mu E)\) -1 \(KHK\) and extract the first \(d\) principal eigenvectors to construct the transfer matrix \(W\), where \(\mu\) is a balancing factor;

[0047] S24: Extract the first \(MN\) column vectors and the last \(MK\) column vectors of \(W\) T to respectively form the domain-adapted \(MN\times d\) source domain matrix:

[0048]

[0049] The domain-adapted \(MK\times d\) target domain matrix is:

[0050]

[0051] Similarly, obtain and

[0052] Combine and into the domain-adapted training input sample set where

[0053]

[0054] Furthermore, step S3 includes the following steps:

[0055] S31: Rearrange the samples in the expert input sample set to construct the set where where

[0056]

[0057] S32: Input to the \(L\)-layer convolutional neural network for convolution, and output

[0058] S33: Use the average displacement error between the model output during training and the expert output reference set as the loss function for calculating the training error, and use the stochastic gradient descent algorithm to learn the weight values. The training termination condition is to meet the maximum number of training epochs, and save the trained prediction model;

[0059] S34: Modify the number \(d\) of principal eigenvectors, and the dimension of the input samples for training changes accordingly, thereby generating different models. Repeat steps S31 - 34 to generate \(r\) different models.

[0060] Further, in step S32, is a 2×N-dimensional matrix. Input into an L-layer convolutional network to obtain the first-layer output and higher-layer outputs Correspondingly obtain the first-layer output and higher-layer outputs are the weights of convolutional kernels of different layers, and K is the scale of the convolutional kernel.

[0061] Further, step S4 specifically includes the following steps:

[0062] S41: Combine and into a domain-adapted test input sample set where

[0063]

[0064] S42: Rearrange the samples in the software planning input sample set to construct a set where Input into the corresponding model to obtain r different corrected trajectory representations of manipulator n as:

[0065]

[0066] The final corrected trajectory of the steel grabber Tra = α1Tra(1) + α2Tra(2) + … + α r Tra(r), where

[0067] The beneficial effects of the present invention are as follows: 1) The present invention makes the data distributions of the expert scenario and the software planning scenario approximately consistent through domain adaptation processing, transfers the experience and skills of the skilled operator to the software planning trajectory, thereby improving the loading and unloading efficiency; 2) The present invention considers the multi-scale and multi-resolution depth information of the observed trajectory, maps the observed trajectory into training input trajectories of different scales by changing the domain adaptation parameters, trains the corresponding prediction models and then fuses them, thereby utilizing the deeper-level information of the observed trajectory. The above two points make the final corrected trajectory more accurate.

[0068] Other advantages, objectives and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail and preferably below in conjunction with the accompanying drawings, where:

[0070] Figure 1 is the first flowchart of the method for correcting the end trajectory of a steel grabber based on domain adaptation technology according to an embodiment of the present invention;

[0071] Figure 2 is the second flowchart of the method for correcting the end trajectory of a steel grabber based on domain adaptation technology according to an embodiment of the present invention;

[0072] Figure 3 is the flowchart of the automatic trajectory segmentation based on the sensitive parameters of the loading and unloading actions of a steel grabber according to an embodiment of the present invention. Specific Embodiments

[0073] The following uses specific specific examples to illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention schematically. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0074] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention schematically. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0075] In the following description, a large number of details are explored to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand. <{

[0076] Embodiment 1:

[0077] As Figure 1 shown, the present invention provides a method for correcting the end trajectory of a steel grabber based on domain adaptation technology, Figure 2 which more intuitively shows the process of the entire correction method. This method includes the following steps:

[0078] S1. Collect the end - motion trajectories of the grab crane in the expert operation scenarios and software planning scenarios, generate the corresponding expert datasets and software planning datasets, pre - process each training dataset to generate an expert input sample set and an expert output reference set, and pre - process the software planning dataset to generate a software planning input sample set.

[0079] Any training dataset S contains F frame position coordinates formed by equally - spaced sampling of the motion trajectories of N expert operators over a relatively long period of time, that is where 1 frame of data records the spatial positions of N operators at a certain moment, that is where the spatial position identifier of operator n in frame sequence f is 1 ≤ f ≤ F is the frame number, 1 ≤ n ≤ N is the operator number, and (x, y, z) are the three - dimensional space coordinate points of the end - trajectory;

[0080] The time period [Δt, T + Δt] contains T0 + T1 consecutive position coordinates of N operators. The first T0 frames of data can be used to construct a multi - dimensional training input sample The last T1 frames of data can be used to construct a multi - dimensional training output reference sample where the input sample of the nth operator The output reference sample

[0081] Change the starting time Δt, that is Δt = mT fra , 0 ≤ m ≤ (M - 1), where T fra is the time interval between adjacent frames, and a total of M input samples and M multi - dimensional output reference samples

[0082] Similarly, the software planning dataset can be used to construct K multi - dimensional input samples and K multi - dimensional output reference samples

[0083] The pre - processing part also includes automatic trajectory segmentation, such as Figure 3As shown. Based on the study of the physical meaning of the actual loading and unloading actions of the steel grabber, the loading and unloading actions can be divided into 10 different stages for specific analysis. Based on the time-series data of various parameters of the collected loading action trajectories, sensitive parameters for stage division are explored. By comparing and analyzing nearly twenty parameters collected from the actual loading and unloading actions of multiple steel grabbers, the most sensitive and efficient parameters for stage division are obtained: the body rotation angle parameter and the end height parameter of the dipper stick. Therefore, the body rotation angle parameter and the end height parameter of the dipper stick are normalized to the interval [0,1] to better control the data range in subsequent processing. Median filtering is performed on the normalized parameter data with a window size of 200, that is, the median of the data within the window is taken to represent the data within the window, and the filtered result is obtained to remove the noise in the signal. Smoothing processing is performed on the data after median filtering with a window size of 20, that is, the average value of the data within the window is taken to represent the data within the window, further smoothing the signal and reducing jitter, as well as weakening high-frequency noise. Slope calculations are performed on the smoothed parameter data of each parameter, also normalized to the interval [0,1]. Median filtering is performed on the normalized parameter slopes with a window size of 100, and then smoothing processing is performed on the parameter slopes after median filtering with a window size of 20 to further smooth the signal and reduce jitter, obtaining the data of different parameters for stage division. Finally, thresholds are set according to the physical meaning of the loading and unloading actions in different stages, corresponding to the specific values and slope values of the sensitive parameters for automatic trajectory stage division, which are divided into 10 different stages and a total of 11 time nodes t1 - t11:

[0084] 1) When the slope of the end height parameter of the dipper stick is lower than the negative threshold and the slope of the body rotation angle parameter is lower than the threshold, it is recorded as t1, indicating that the end starts to descend until the material pile.

[0085] 2) When the slope of the end height parameter of the dipper stick is lower than the negative threshold and the slope of the body rotation angle parameter is lower than the threshold, it is recorded as t2, indicating that the bucket has been shoveled into the material pile and the excavation action starts.

[0086] 3) When the slope of the end height parameter of the dipper stick is higher than the negative threshold and the slope of the body rotation angle parameter is lower than the threshold, it is recorded as t3, indicating that the excavation is completed and the end of the dipper stick starts to rise.

[0087] 4) When the slope of the body rotation angle parameter is higher than the threshold, it is recorded as t4, indicating that the body starts to rotate and the end also rises vertically at the same time.

[0088] 5) When the slope of the end height parameter of the dipper stick is lower than the threshold and the slope of the body rotation angle parameter is higher than the threshold, it is recorded as t5, indicating that the end has risen to the highest point. Since it is necessary to cross the baffle, the body has not completed rotating at this time.

[0089] 6) When the slope of the body rotation angle parameter is lower than the threshold value, it is denoted as t6, indicating that the end is at the highest point of the unloading point and the body has rotated above the unloading point.

[0090] 7) When the slope of the bucket end height parameter is lower than the negative threshold value and the slope of the body rotation angle parameter is higher than the negative threshold value, it is denoted as t7, indicating that the end starts to descend and is positioned at the unloading point.

[0091] 8) When the slope of the bucket end height parameter is higher than the negative threshold value and the slope of the body rotation angle parameter is higher than the negative threshold value, it is denoted as t8, indicating that the bucket starts to unload. Due to the characteristics of the bucket, some action samples will raise the end height while unloading. After unloading is completed, the end height finally reaches a suitable position that can cross the baffle.

[0092] 9) When the slope of the body rotation angle parameter is lower than the negative threshold value, it is denoted as t9, indicating that the end height has reached a suitable height and the body starts to rotate until the end crosses the baffle.

[0093] 10) When the slope of the bucket end height parameter is lower than the negative threshold value and the slope of the body rotation angle parameter is higher than the negative threshold value, it is denoted as t10. The end has crossed the baffle, the end height starts to descend, and the end continues to rotate until it is above the material pile.

[0094] 11) When the slope of the body rotation angle parameter is higher than the negative threshold value, it is denoted as t11, indicating that the end has reached above the material pile and is ready to start a new grasping action.

[0095] In this embodiment, the frame interval is 1 millisecond. Observe a complete loading and unloading action of the sample, and realize automatic trajectory segmentation. Based on the collected sensitive parameters, set the threshold values according to the physical meanings of the loading and unloading actions in different stages, and divide a complete loading and unloading action into 10 stages for specific analysis.

[0096] S2. Perform domain adaptation processing on the trained expert scenario sample set and the software planning input sample set, and change the domain adaptation parameter (the number of principal feature vectors d). A total of r different domain adaptation outputs of the expert scenario input sample set and the software planning input sample set are obtained. The domain adaptation processing specifically includes the following steps:

[0097] S21. Extract the input sample set of the skilled operator and the software planning input sample set of the x component X x , Y x . Regard X x and Y x as the source domain and the target domain respectively for domain adaptation processing. Since the domain adaptation methods for the x, y, and z component domains are exactly the same, the following uses the domain adaptation processing process of the x component as an example for description.

[0098] The source domain dataset can be constructed as an \(M_N\times T_0\) matrix

[0099]

[0100] The target dataset can be constructed as a \(K_N\times T_0\) matrix

[0101]

[0102] S22. Construct the kernel matrix \(K\), the maximum mean discrepancy measure matrix \(M\), and the central matrix \(H\) as follows:

[0103] The kernel matrix \(K\) is the concatenation of the source domain matrix \(X\) x and the target domain matrix \(Y\) x That is where the row vector of dimension \(M_N + M_K\)

[0104] is the F-norm of the matrix The central matrix where \(E\) is the identity matrix of dimension \(M_N + M_K\) and \(1\) is the row vector of all \(1\)'s of dimension \(M_N + M_K\);

[0105] S23. Perform eigenvalue decomposition on the matrix \((K M K+\mu E)\) -1 \(K H K\) and extract the first \(d\) principal eigenvectors to construct the transfer matrix \(W\), where \(\mu\) is the balancing factor;

[0106] S24. Extract the first \(M_N\) column vectors and the last \(M_K\) column vectors of \(W\) T \(K\) to form

[0107] the source domain matrix of dimension \(M_N\times d\) after domain adaptation

[0108]

[0109] the target domain matrix of dimension \(M_K\times d\) after domain adaptation

[0110]

[0111] Similarly, we can obtain and

[0112] Combine and into the training input sample set after domain adaptation

[0113] where

[0114] In this embodiment, the balance factor μ in the domain adaptation algorithm is 0.01, the number of main feature vectors d in the first domain adaptation is 12, the first d is 10, and the third d is 6, that is, the parameter changes three times, r = 3.

[0115] S3. Construct a convolutional network based on the r domain-adapted expert scenario sample sets and the expert output reference sets for training to obtain r corresponding different prediction models. The specific steps are as follows:

[0116] S31. Rearrange the samples in the expert input sample set to construct a set where

[0117]

[0118] S32. Input into the convolutional neural network of L layers for convolution, and output as a 2×N-dimensional matrix. Input into the L-layer convolutional network to obtain the first-layer output and the higher-layer output correspondingly obtain the first-layer output and the higher-layer output are the weights of the convolutional kernels of different layers, and K is the scale of the convolutional kernel.

[0119] S33. Design a loss function that uses the average displacement error between the model output and the expert output reference set during the training process as the training error, and use the stochastic gradient descent algorithm to learn the weight values. The training termination condition is to meet the maximum number of training epochs, and save the trained prediction model. By changing the number of main feature vectors d, the dimension of the training input samples changes accordingly, and different models can be generated. Repeating the above steps can generate r different models.

[0120] In this embodiment, the activation function where a = 0.25, the scale of the convolutional kernel is 2, the number of graph convolutional layers is 1 layer, and the total number of spatio-temporal convolutional layers L = 5. The loss function is the average displacement error ADE, the number of training samples per batch is 128, and the model is trained for 100 epochs using stochastic gradient descent (SGD) with a learning rate of 0.01.

[0121] S4. Input the r domain-adapted software planning input sample sets into the corresponding prediction models for fusion to finally obtain the corrected trajectory of the end effector. Specifically, and are combined into the domain-adapted test input sample set where Rearranging and constructing a set from the samples in the software planning input sample set in the sample set Rearranging and constructing a set where inputting into the corresponding model to obtain r different corrected trajectories of the manipulator n, expressed as:

[0122]

[0123] The final corrected trajectory of the steel grabber Tra = α1Tra(1) + α2Tra(2) + … + α r Tra(r), where

[0124] In this embodiment, the fusion parameters are α1 = 0.2, α2 = 0.3, and α3 = 0.5.

[0125] Finally, calculate the average displacement error ADE and the final displacement error FDE values between the optimal corrected trajectory and the ideal trajectory in the software planning to evaluate the classification effect. The expressions for the average displacement error ADE and the final displacement error FDE are as follows:

[0126]

[0127] where are the corrected coordinates and the ideal coordinates at the t-th frame respectively; are the corrected coordinates and the ideal coordinates at the last frame of the trajectory respectively.

[0128] Embodiment 2:

[0129] An electronic device, including a memory and a processor;

[0130] The memory is used to store a computer program;

[0131] The processor is used to implement the method as described in Embodiment 1 when executing the computer program.

[0132] Embodiment 3:

[0133] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method as described in Embodiment 1 is implemented.

[0134] Embodiment 4:

[0135] A computer program product, including a computer program, and when the computer program is executed by a processor, the method as described in Embodiment 1 is implemented.

[0136] In the above embodiments, the reference in the specification to "this embodiment" means that the specific features, structures, or characteristics described in connection with the embodiment are included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0137] In the above embodiments, although the present invention has been described in connection with specific embodiments of the present invention, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. Embodiments of the present invention are intended to cover all such alternatives, modifications, and variations that fall within the broad scope of the appended claims.

[0138] For the computer-readable storage medium in this embodiment, those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to a computer program. The foregoing computer program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disc that can store program code.

[0139] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication therebetween. The memory is used to store a computer program, the communication interface is used for communication, and the processor and the transceiver are used to run the computer program to cause the electronic terminal to execute each step of the above method.

[0140] In this embodiment, the memory may include a random access memory (Random Access Memory, abbreviated as RAM), and may also include non-volatile memory, such as at least one disk memory.

[0141] The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0142] The present invention can be used in numerous general-purpose or special-purpose computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on.

[0143] The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in a distributed computing environment, where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for correcting the end trajectory of a steel grabber based on domain adaptation technology, characterized in that: It includes the following steps: S1: Collect the end motion trajectories of the grab crane in the expert operation scenario and the software planning scenario, generate the corresponding expert operation scenario dataset and software planning scenario dataset; preprocess the expert operation scenario dataset to generate the expert operation input sample set and the expert operation output reference set; Preprocess the software planning scenario dataset to generate the software planning input sample set; S2: Perform domain adaptation processing on the expert operation input sample set and the software planning input sample set, and change the domain adaptation parameters to obtain r different domain adaptation outputs of the expert operation input sample set and the software planning input sample set; S3: Based on the r domain-adapted expert operation input sample sets and the expert operation output reference set, construct a convolutional network for training to obtain r corresponding different prediction models; S4: Input the r domain-adapted software planning input sample sets into the corresponding prediction models for fusion to finally obtain the corrected trajectory of the end trajectory.

2. The method for correcting the trajectory of the end of the steel grabber based on the domain adaptation technology according to claim 1, wherein: In the step S1, the expert scenario dataset S contains F frames of position coordinates formed by equally spaced sampling of the movement trajectories of N expert operators over a period of time, that is where one frame of data records the spatial positions of N operators at a certain moment, that is where the spatial position identifier of operator n in frame sequence f is S fn = [f n x y z], 1 ≤ f ≤ F is the frame number, 1 ≤ n ≤ N is the operator number, and (x, y, z) are the three-dimensional space coordinate points of the end trajectory; The time period [Δt, T+Δt] contains the continuous position coordinates of T0+T1 frames of N operators. The data of the first T0 frames constructs a multi-dimensional training input sample. The data of the subsequent T1 frames constructs a multi-dimensional training output reference sample. Among them, the input sample of the nth operator Output reference sample Change the starting time Δt, i.e., Δt = mT fra , 0 ≤ m ≤ (M - 1), where T fra is the time interval between adjacent frames, and a total of M input samples are obtained and M multi-dimensional output reference samples Similarly, the software planning data set constructs K multi-dimensional input samples according to the above method and K multi-dimensional output reference samples 3. The method for correcting the end trajectory of a steel grabber based on domain adaptation technology according to claim 1, characterized in that: The preprocessing of the software planning scenario dataset includes automated trajectory segmentation, specifically including: Divide the loading and unloading actions into 10 different stages for specific analysis. Based on the time-series data of each parameter of the collected loading action trajectory, discover the sensitive parameters for the stage division; Through comparative analysis of the parameters collected in the actual loading and unloading actions of multiple grab cranes, it is concluded that the most sensitive and efficient parameters for stage division are the body rotation angle parameter and the boom end height parameter; Normalize the body rotation angle parameter and the boom end height parameter to the interval [0, 1]; Perform median filtering and smoothing processing on the normalized parameter data; Calculate the slope of each smoothed parameter data and normalize it to the interval [0, 1]; Perform median filtering and smoothing processing on the normalized parameter slopes for the data of different parameters used for stage division; Set thresholds according to the physical meanings of the loading and unloading actions in different stages, and perform automated trajectory stage division corresponding to the specific values and slope values of the sensitive parameters. A total of 10 different stages are divided, with a total of 11 time nodes t1 - t11.

4. The method for correcting the end trajectory of a steel grabber based on domain adaptation technology according to claim 3, wherein: The setting of thresholds according to the physical meanings of the loading and unloading actions in different stages, and performing automated trajectory stage division corresponding to the specific values and slope values of the sensitive parameters, a total of 10 different stages are divided, with a total of 11 time nodes t1 - t11, specifically including: 1) When the slope of the boom end height parameter is lower than the negative threshold and the slope of the body rotation angle parameter is lower than the threshold, it is recorded as t1, indicating that the end starts to descend until the material pile; 2) When the slope of the boom end height parameter is lower than the negative threshold and the slope of the body rotation angle parameter is lower than the threshold, it is recorded as t2, indicating that the bucket has been shoveled into the material pile and the excavation action starts; 3) When the slope of the boom end height parameter is higher than the negative threshold and the slope of the body rotation angle parameter is lower than the threshold, it is recorded as t3, indicating that the excavation is completed and the boom end starts to rise; 4) When the slope of the body rotation angle parameter is higher than the threshold, it is recorded as t4, indicating that the body starts to rotate and the end also rises vertically at the same time; 5) When the slope of the height parameter at the end of the arm is lower than the threshold and the slope of the body rotation angle parameter is higher than the threshold, it is denoted as t5, indicating that the end has been lifted to the highest point, and at this time, the body has not completed rotation; 6) When the slope of the body rotation angle parameter is lower than the threshold, it is denoted as t6, indicating that the end is at the highest point of the unloading point, and the body has rotated above the unloading point; 7) When the slope of the height parameter at the end of the arm is lower than the negative threshold and the slope of the body rotation angle parameter is higher than the negative threshold, it is denoted as t7, indicating that the end starts to descend and is positioned at the unloading point; 8) When the slope of the height parameter at the end of the arm is higher than the negative threshold and the slope of the body rotation angle parameter is higher than the negative threshold, it is denoted as t8, indicating that the bucket starts to unload, and in some action samples, the end height is lifted while unloading. After unloading is completed, the end height finally reaches a suitable position to jump over the baffle; 9) When the slope of the body rotation angle parameter is lower than the negative threshold, it is denoted as t9, indicating that the end height has reached a suitable height, and the body starts to rotate until the end crosses the baffle; 10) When the slope of the height parameter at the end of the arm is lower than the negative threshold and the slope of the body rotation angle parameter is higher than the negative threshold, it is denoted as t10. The end has crossed the baffle, the end height starts to descend, and the end continues to rotate until above the material pile; 11) When the slope of the body rotation angle parameter is higher than the negative threshold, it is denoted as t11, indicating that the end has reached above the material pile and is ready to start a new grasping action.

5. The method for correcting the trajectory of the end of the steel grabber based on the domain adaptation technology according to claim 1, wherein: Step S2 specifically includes the following steps: S21: Extract the input sample set of proficient workers and the input sample set of software planning for the x-component X x , Y x ; Consider X x and Y x as the source domain and the target domain respectively for domain adaptation processing. The domain adaptation methods for the spatial x, y, and z components are exactly the same. The domain adaptation process for the x-component is described as follows: The source domain dataset is constructed as an MN×T0 matrix: The target dataset is constructed as a KN×T0 matrix: S22: Construct the kernel matrix K, the maximum mean discrepancy measure matrix M, and the central matrix H as follows: The kernel matrix K is composed of the source domain matrix X x and the target domain matrix Y x stitched together, that is where the row vector of dimension MN + MK is the F-norm of the matrix that is the central matrix where E is the identity matrix of dimension MN + MK, and 1 is the all-ones row vector of dimension MN + MK; S23: Perform eigenvalue decomposition on the matrix \((KMK + \mu E)\) -1 KHK, and extract the first d principal eigenvectors to construct the transition matrix W, where \(\mu\) is the balance factor; S24: Extract W T The first MN column vectors and the last MK column vectors of K respectively form an MN×d source domain matrix after domain adaptation: The MK×d target domain matrix after domain adaptation is: Similarly obtained and Combine and into a domain - adapted training input sample set wherein 6. The method for correcting the end trajectory of the steel grabber based on the domain adaptation technology according to claim 5, wherein: The said step S3 includes the following steps: S31: Input the expert input sample set The samples Re - arrange and construct a set Wherein S32: Input Perform convolution on the convolutional neural network of L layers and output S33: Use the average displacement error between the model output during training and the reference set of expert outputs as the loss function for calculating the training error, and use the stochastic gradient descent algorithm to learn the weight values. The training termination condition is to meet the maximum number of training epochs, and save the trained prediction model; S34: Modify the number d of the main eigenvectors, and the dimension of the input samples for training changes accordingly, thereby generating different models. Repeat steps S31 - 34 to generate r different models.

7. The method for correcting the end trajectory of a steel grabber based on domain adaptation technology according to claim 6, characterized in that: In the step S32, is a 2×N dimensional matrix. Input into an L-layer convolutional network to obtain the first-layer output and the higher-layer outputs correspondingly obtain the first-layer output and the higher-layer outputs are the weights of the convolutional kernels of different layers, and K is the scale of the convolutional kernel.

8. The method for correcting the trajectory of the end of the steel grabber based on the domain adaptation technology according to claim 7, characterized in that: The said step S4 specifically includes the following steps: S41: Combine and into the domain - adapted test input sample set where S42: Input the software planning input sample set The samples in it are rearranged and constructed into a set where r different corrected trajectories of the operating hand obtained by inputting into the corresponding model are expressed as: The final grab crane correction trajectory Tra = α1Tra(1) + α2Tra(2) + … + α r Tra(r), where

Images