Model online training system, method, device and equipment for unmanned excavator and storage medium

Abstract

The invention provides an online model training system, method, device and equipment for an unmanned excavator and a storage medium, in the online model training system, method and device for the unmanned excavator, high-performance computing equipment and the unmanned excavator are deployed in the same local area network, so that the high-performance computing equipment can communicate with the unmanned excavator, a trajectory tracking model of the unmanned excavator is deployed on the high-performance computing equipment, and online training is carried out on the trajectory tracking model, that is, the high-performance computing equipment completes reasoning and training for trajectory tracking of the unmanned excavator, and because reasoning and training are completed by the high-performance computing equipment, the computing power of the unmanned excavator is not occupied, and the accuracy of trajectory tracking of the unmanned excavator is improved. Therefore, the influence on the real-time trajectory tracking task of the unmanned excavator can be reduced.

Description

Technical Field

[0001] The present application relates to the field of automatic control technology, and in particular to an online model training system, method, device, equipment and storage medium for an unmanned excavator. Background Art

[0002] With the popularization of intelligence and the improvement of AI capabilities, in the field of excavators, learning methods are gradually being used to perform motion control tasks on unmanned excavators. Trajectory tracking is a relatively basic task in the motion control of unmanned excavators, and many learning-based methods have begun to be tried on this task. Learning-based methods can be divided into online learning and offline learning. In the online learning mode, data is continuously updated, and the trained model can be tested directly on the excavator for further training based on the results; offline learning generally collects a bunch of data and trains in a fixed data set. The quality of the model needs to be tested in a simulation environment. Because it is generally difficult to build a realistic simulation environment to test the model, the online learning mode is chosen for trajectory tracking.

[0003] Solutions based on online learning generally involve the learning of neural networks. However, the computing power of the embedded systems currently deployed for unmanned excavators is usually not high. If the online learning of neural networks is also deployed on the excavator, the excavator needs to use its own computing power to implement the training and reasoning process. This will result in a longer training and reasoning time, which will in turn affect the real-time trajectory tracking task of the unmanned excavator. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a model online training system, method, apparatus, device and storage medium for an unmanned excavator to reduce the impact on the real-time trajectory tracking task of the excavator.

[0005] In a first aspect, an embodiment of the present application provides a model online training system for an unmanned excavator, the model online training system comprising: a high-performance computing device and an unmanned excavator, the high-performance computing device and the unmanned excavator being deployed in the same local area network;

[0006] After acquiring the status information of the target part, the unmanned excavator sends the status information to the high-performance computing device through the local area network;

[0007] After receiving the state information, the high-performance computing device performs vector splicing on the target point and the state information to obtain a splicing vector, wherein the target point is determined on a target trajectory selected from a trajectory library;

[0008] The high-performance computing device inputs the splicing vector into a trajectory tracking model currently configured for the unmanned excavator to obtain action information for instructing the target part of the unmanned excavator to perform a target action;

[0009] The high-performance computing device uses the action information and the state information as a data pair to update and train the trajectory tracking model;

[0010] The high-performance computing device is configured to update the trajectory tracking model after training for the unmanned excavator.

[0011] In a second aspect, an embodiment of the present application provides a model online training method for an unmanned excavator, wherein the online training method is configured on a high-performance computing device, wherein the high-performance computing device and the unmanned excavator constitute a model online training system, wherein the high-performance computing device and the unmanned excavator are deployed in the same local area network, and wherein the method comprises:

[0012] After receiving the status information sent by the unmanned excavator through the local area network, vector splicing is performed on the target point and the status information to obtain a splicing vector, wherein the target point is determined on the target trajectory selected from the trajectory library;

[0013] Inputting the splicing vector into a trajectory tracking model currently configured for the unmanned excavator to obtain action information for instructing a target part of the unmanned excavator to perform a target action;

[0014] Using the action information and the state information as a data pair to update and train the trajectory tracking model;

[0015] A trajectory tracking model after training is updated for the unmanned excavator configuration.

[0016] In a third aspect, an embodiment of the present application provides a model online training device for an unmanned excavator, wherein the online training device is arranged on a high-performance computing device, wherein the high-performance computing device and the unmanned excavator constitute a model online training system, wherein the high-performance computing device and the unmanned excavator are deployed in the same local area network, and wherein the device comprises:

[0017] A splicing unit, configured to perform vector splicing on a target point and the status information after receiving the status information sent by the unmanned excavator through the local area network to obtain a splicing vector, wherein the target point is determined on a target trajectory selected from a trajectory library;

[0018] A training unit, used for inputting the splicing vector into a trajectory tracking model currently configured for the unmanned excavator to obtain action information for instructing a target part of the unmanned excavator to perform a target action;

[0019] An updating unit, configured to update and train the trajectory tracking model by taking the action information and the state information as a data pair;

[0020] A configuration unit is used to configure and update the trajectory tracking model after training for the unmanned excavator.

[0021] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the online training method as described in any one of the second aspects.

[0022] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the online training method as described in any one of the second aspects are executed.

[0023] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0024] In the present application, a high-performance computing device and an unmanned excavator are deployed in the same local area network, so that the high-performance computing device and the unmanned excavator can communicate, wherein a trajectory tracking model of the unmanned excavator is deployed on the high-performance computing device, and the trajectory tracking model is trained online, that is: the high-performance computing device completes the reasoning and training of the trajectory tracking of the unmanned excavator, and the unmanned excavator only needs to perform corresponding control according to the result inferred by the high-performance computing device to achieve trajectory tracking, and the trajectory tracking model used by the high-performance computing device is trained by online learning, so the trajectory tracking model used for reasoning is the currently updated trajectory tracking model, which is beneficial to improve the correctness of the unmanned excavator control. Since the reasoning and training are completed by the high-performance computing device, it does not occupy the computing power of the unmanned excavator itself, which is beneficial to reduce the impact on the real-time trajectory tracking task of the unmanned excavator.

[0025] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A model online training system for an unmanned excavator provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a sin curve and a cos curve provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of another sin curve and cos curve provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of a step curve adopted by an embodiment of the present application;

[0031] Figure 5 A schematic diagram of another step curve provided in an embodiment of the present application;

[0032] Figure 6 Another model online training system for unmanned excavators provided in an embodiment of the present application;

[0033] Figure 7 A flowchart of an online model training method for an unmanned excavator provided in an embodiment of the present application;

[0034] Figure 8 A schematic diagram of the structure of an online model training device for an unmanned excavator provided in an embodiment of the present application;

[0035] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0037] Figure 1 An online model training system for an unmanned excavator provided in an embodiment of the present application comprises: a high-performance computing device and an unmanned excavator, wherein the high-performance computing device and the unmanned excavator are deployed in the same local area network; Figure 1 The interactive process of the model online training system shown includes:

[0038] Step 101: After acquiring the status information of the target part, the unmanned excavator sends the status information to the high-performance computing device through the local area network.

[0039] Step 102: After receiving the state information, the high-performance computing device performs vector splicing on the target point and the state information to obtain a splicing vector, wherein the target point is determined on a target trajectory selected from a trajectory library.

[0040] Step 103: the high-performance computing device inputs the splicing vector into a trajectory tracking model currently configured for the unmanned excavator to obtain action information for instructing the target part of the unmanned excavator to perform a target action;

[0041] Step 104: The high-performance computing device uses the action information and the state information as a data pair to update and train the trajectory tracking model.

[0042] Step 105: The high-performance computing device updates the trajectory tracking model after training for the unmanned excavator configuration.

[0043] Specifically, Figure 1As shown, the unmanned excavator obtains the status information of its corresponding parts according to the target parts set in the trajectory tracking task (for example, the boom, forearm and cabin of the unmanned excavator), and then sends the status information to the high-performance computing device through the local area network, so that the high-performance computing device uses the status information to perform online training on the trajectory tracking model configured for the unmanned excavator, and uses the trajectory tracking model configured for the unmanned excavator to perform inference to obtain the action information for controlling the target part of the unmanned excavator to perform the target action.

[0044] The high-performance computing device is provided with a trajectory library, which includes trajectories and basic trajectories corresponding to the types of work that the unmanned excavator currently needs to learn (such as loading, leveling, leveling and soil dumping tasks), wherein the trajectories corresponding to the types of work collect the real operation data generated by the operator when operating the excavator. Since a large number of trajectories are required when training the trajectory tracking model, and the trajectories need to cover various behavioral trajectories that the excavator may use, so that a relatively complete model can be trained, but such trajectories are numerous and relatively difficult to collect. Since various behavioral trajectories can be understood as the composition of many basic trajectories, for example, a curved trajectory can be formed by a part of the trajectory such as a step trajectory, a sinusoidal curve trajectory, etc., some basic trajectories can be used to expand the trajectory library, so that the trajectory library can include relatively rich behavioral trajectories, and then the trained model can better track the basic trajectories, thereby tracking complex trajectories and improving the generalization of the trajectory tracking model.

[0045] Among them, the basic trajectory includes a step trajectory, a sin curve, and a cos curve (inverse function curve), and the basic trajectory is a trajectory that each target part of the excavator needs to reach. The generation process of the basic trajectory is as follows:

[0046] If a sin curve or cos curve is generated, assuming that random(a,b) represents a random floating point number between a and b, and the length of the generated trajectory is L, that is, there are L trajectory points:

[0047] Step 1: Test the reasonable position limits of the excavator's boom, arm, and cabin, such as boom_pos_limit: [-0.5, 1.65], arm_pos_limit: [-0.8, 1.2], swing_pos_limit: [-pi, pi].

[0048] Step 2: Randomly initialize the phase phase = random (0, 2pi) and randomly initialize the period num to 1 or 2.

[0049] Step 3: Randomly initialize the amplitude and offset; if the current joint is the cockpit, amplitude = random (0, pi), offset = random (-0.5pi, 0.5pi); if the current joint is the upper arm or the lower arm, the processing method is the same, taking the upper arm as an example:

[0050] amplitude=(random(0,0.9)+0.1)*(boom_pos_limit[1]-boom_pos_limit[0]-0.02) / 2;

[0051] offset=boom_pos_limit[0]+amplitude+random(0, (boom_pos_limit[1]-0.01)-(boom_pos_limit[0]+0.01+amplitude*2)).

[0052] Step 4: Generate the basic trajectory point index t, and evenly generate L points between 0 and 2pi*num. L is the number of trajectory points on a trajectory and can be set freely, that is, t = {t1, t2, …, tL}.

[0053] Step 5: Generate a sin curve or a cos curve according to the randomly generated phase phase, subscript t, amplitude amplitude and offset offset, that is: target_traj = amplitude*sin(t+phase)+offset, or, target_traj = amplitude*cos(t+phase)+offset.

[0054] Step 6: Standardize the generated trajectory to make it conform to a reasonable curve within the reach of the excavator. If the joint is the cockpit, each trajectory point is processed as follows: target_traj_point = (target_traj_point + pi)% (2*pi) - pi; if it is the boom or forearm, the processing is similar. Taking the boom as an example, each trajectory point is processed as follows: target_traj_point = max(min(target_traj_point, boom_pos_limit[1]), boom_pos_limit[0]).

[0055] The sin curve and cos curve generated by the above method are as follows Figure 2 and Figure 3 As shown, Figure 2 A schematic diagram of a sin curve and a cos curve provided in an embodiment of the present application, Figure 3 A schematic diagram of another sin curve and cos curve provided in an embodiment of the present application.

[0056] If a step trajectory is generated, assuming that random(a,b) represents randomly generating a random floating point number between a and b, random_int(a,b) represents randomly generating a random integer between a and b, and the generated trajectory length is L, that is, there are L trajectory points:

[0057] Step 1: Test the reasonable position limits of the excavator's boom, arm, and cabin, such as boom_pos_limit: [-0.5, 1.65], arm_pos_limit: [-0.8, 1.2], swing_pos_limit: [-pi, pi].

[0058] Step 2: Randomly generate the number num of horizontal segments in a trajectory and randomly select one from 1, 2, and 3.

[0059] Step 3: Randomly generate the values ​​of the horizontal segment. The processing methods of the boom, forearm and cockpit are the same. Here we take the boom as an example:

[0060] js=random(boom_pos_limit[0]+0.02;

[0061] boom_pos_limit[1]-0.02);

[0062] If num is 1, one js is generated, if num is 2, two js are generated, and so on.

[0063] Step 4: Randomly generate the step position step_pos.

[0064] If num is 1, there is no step position;

[0065] If num is 2, there is a step position: step_pos = random_int(L / 2-10, L / 2+10);

[0066] If num is 3, there are two step positions: step_pos1 = random_int(L / 3-10, L / 3+10), step_pos2 = random_int(L / 3*2-10, L / 3*2+10).

[0067] Step 5: Generate a step curve according to the randomly generated num, js and step_pos.

[0068] If num is 1, target_traj = [js] * L;

[0069] If num is 2, target_traj = [js[0]]*step_pos+[js[1]]*(L-step_pos);

[0070] If num is 3, target_traj = [js[0]]*step_pos1+[js[1]]*(step_pos2

[0071] -step_pos1)+[js[2]]*(L-step_pos2).

[0072] The step trajectory generated by the above method is as follows Figure 4 and Figure 5 As shown, Figure 4 A schematic diagram of a step curve adopted in an embodiment of the present application, Figure 5 A schematic diagram of another step curve provided in an embodiment of the present application.

[0073] After a trajectory library with relatively rich trajectory forms can be constructed in the above method, after the high-performance computing device receives the status information, a target trajectory is randomly selected from the constructed trajectory library, and then a target point is selected from the trajectory. When selecting the target point, if the trajectory points on the target trajectory are tracked point by point each time, when the current position of the target part of the excavator exceeds the currently tracked trajectory point, the target part will appear to have a trajectory retreat. In order to avoid this situation, the following solution can be used to select the target point:

[0074] The target trajectory includes the trajectory points {T0, T1, ..., Tn}, and the current position of each target part of the excavator is S. Here, both T and S are three-dimensional vectors, which are used for the positions of the boom, forearm and cockpit. Then, the end position (the three-dimensional vector of the x-axis, y-axis and z-axis) is calculated according to the three joint positions, and the distance from S represented by the end position to each trajectory point T in the target trajectory is calculated, which are {d0, d1, ..., dn} respectively. Finally, the next trajectory point corresponding to the trajectory point with the smallest end distance is selected as the current target point to be tracked, that is, T = arg(i)min{d0, d1, ...di, di+1, ..., dn}+1. Selecting the target point in the manner described above can effectively avoid the trajectory from retracing.

[0075] After obtaining the target point, the high-performance computing device performs vector splicing on the target point and the state information to obtain a splicing vector, and then inputs the obtained splicing vector into the trajectory tracking model currently configured for the unmanned excavator. The trajectory tracking model can output action information for instructing the target part of the unmanned excavator to perform the target action based on the splicing vector. At this time, the high-performance computing device can send the action information to the unmanned excavator so that the unmanned excavator can control the corresponding part according to the obtained action information, and it is also necessary to use the action information and the state information as a data pair to update and train the trajectory tracking model. After the trajectory tracking model is trained, the trained trajectory tracking model is configured to the unmanned excavator, that is, the high-performance computing device subsequently uses the updated trajectory tracking model to output action information.

[0076] In the present application, a high-performance computing device and an unmanned excavator are deployed in the same local area network, so that the high-performance computing device and the unmanned excavator can communicate, wherein a trajectory tracking model of the unmanned excavator is deployed on the high-performance computing device, and the trajectory tracking model is trained online, that is: the high-performance computing device completes the reasoning and training of the trajectory tracking of the unmanned excavator, and the unmanned excavator only needs to perform corresponding control according to the result inferred by the high-performance computing device to achieve trajectory tracking, and the trajectory tracking model used by the high-performance computing device is trained by online learning, so the trajectory tracking model used for reasoning is the currently updated trajectory tracking model, which is beneficial to improve the correctness of the unmanned excavator control. Since the reasoning and training are completed by the high-performance computing device, it does not occupy the computing power of the unmanned excavator itself, which is beneficial to reduce the impact on the real-time trajectory tracking task of the unmanned excavator.

[0077] In a feasible implementation scheme, when the unmanned excavator sends the status information, it sends its own safety information to the high-performance computing device through the local area network; after receiving the safety information, the high-performance computing device performs vector splicing on the target point and the status information when the safety information indicates that the status of the unmanned excavator is safe.

[0078] Specifically, when the unmanned excavator obtains status information, it also needs to judge its own safety information, that is: it can judge its own safety information through the obtained status information. When the safety information indicates that the status of the unmanned excavator is safe, it means that the unmanned excavator is not currently performing any dangerous actions, or that the various parts of the unmanned excavator have not exceeded the limit. For example, the boom of the unmanned excavator has not been raised too high and exceeded the limit, and the boom has not been raised too high to overturn the excavator or the boom has not touched the bottom and overturned the excavator. The extension of the forearm of the unmanned excavator has not exceeded the limit, and the forearm has not been extended too long to cause mechanical damage, etc. Since the trajectory tracking model is a high-performance computing device using the unmanned excavator's If the status information is used for training, if the status information cannot meet the safety requirements, then the trained trajectory tracking model cannot meet the requirements. If the trajectory tracking model trained in this way is used, it will cause unforeseen safety accidents. Therefore, when the unmanned excavator sends status information, it is also necessary to send its own safety information to the high-performance computing device. When the safety information indicates that the status of the unmanned excavator is safe, it means that the status information can be used as training data. At this time, the high-performance computing device will vector splice the target point and the status information. When the safety information indicates that the status of the unmanned excavator is unsafe, the high-performance computing device will abandon the use of the status information as training data.

[0079] In a feasible implementation scheme, when the high-performance computing device receives the status information and the safety information indicates that the status of the unmanned excavator is safe, the high-performance computing device stores the action information and the status information as a data pair in a data storage device. When the number of data pairs stored in the data storage device meets the threshold requirement, the high-performance computing device inputs the data pair into the trajectory tracking model for update training. The above method can control the training time of the trajectory tracking model so that the trajectory tracking model is trained intensively in a certain time period.

[0080] In a feasible implementation scheme, after obtaining the action information, the high-performance computing device sends the action information to the unmanned excavator through the local area network; after receiving the action information, the unmanned excavator controls the target part to perform the target action according to the action information.

[0081] Specifically, after the high-performance computing device inputs the splicing vector into the trajectory tracking model, the trajectory tracking model will output action information. On the one hand, the high-performance computing device stores the action information and the corresponding status information as a data pair in the data storage device. On the other hand, it is also necessary to send the action information to the unmanned excavator through the local area network so that the unmanned excavator can control the corresponding part to perform corresponding actions according to the action information.

[0082] In a feasible implementation scheme, after receiving the action information, the unmanned excavator adds the control information of the part to be controlled to the action information to obtain a collective control instruction; the unmanned excavator controls the corresponding part according to the collective control instruction and a preset action control cycle, so that the corresponding part performs the corresponding action.

[0083] Specifically, when the unmanned excavator is performing task tracking, it not only needs to control the movement of the target parts (such as the upper arm, lower arm, cockpit, etc.), but also needs to control the movement of other parts (such as oil pipes, buckets, etc.). Since the movement information only includes the control of some parts of the unmanned excavator, in order to enable the unmanned excavator to control the movement of other parts at the same time, after the unmanned excavator obtains the movement information, it is necessary to add the control information of other parts to be controlled to the movement information to obtain a collective control instruction. The collective control instruction can control all parts of the unmanned excavator that need to be controlled at the current moment. Specifically, the control information and movement information of the parts to be controlled can be expanded into a collective control instruction through the movement conversion module.

[0084] Furthermore, in order to enable the unmanned excavator to coordinately control various parts at the same time, a frequency control module is also provided on the unmanned excavator for controlling the frequency of issuing instructions. After the action conversion module expands the control information and action information of the part to be controlled into a collective control instruction, the collective control instruction is sent to the frequency control module. The frequency control module sends the collective control instruction to the lower computer used to control the unmanned excavator according to the preset action control cycle, so that the lower computer controls the part corresponding to the received collective control instruction so that the corresponding part performs the corresponding action.

[0085] In a feasible implementation scheme, the unmanned excavator collects its own position execution information according to a preset collection cycle; the unmanned excavator determines its own current safety information based on the position execution information; when the safety information indicates that there is a risk, the unmanned excavator performs an emergency stop operation and returns to its initial safe position.

[0086] Specifically, the frequency control module set on the unmanned excavator monitors its various parts according to the preset collection cycle, so as to obtain the part execution information of each part, and then the unmanned excavator determines its current safety information according to the execution information of each part, for example: whether the upper arm is lifted too high, or whether the lower arm is extended too long, or whether the current posture of the excavator is correct, etc. When the safety information indicates that there is a risk, it means that the unmanned excavator currently has a part that exceeds the limit or a dangerous action. At this time, the unmanned excavator needs to perform an emergency stop operation and restore to the initial safe position to prevent the unmanned excavator from having greater safety hazards. In addition, regardless of whether the safety information indicates whether there is a risk, the unmanned excavator will send the safety information to the high-performance computing device so that the high-performance computing device can determine whether the status information obtained this time is available based on the safety information.

[0087] The current safety information of the unmanned excavator can be determined in the following ways based on the part execution information:

[0088] After obtaining the part execution information, the frequency control module sends the part execution information to the abnormality detection module. The abnormality detection module determines the safety information of the excavator itself based on the part execution information. The boom_state of the upper arm and the arm_state of the lower arm are used as examples for explanation:

[0089] The limit positions of the boom and the forearm can be determined through a large number of field tests. First, the boom is lifted up and pressed down at different speeds to obtain the values ​​{boom_up1, boom_up2, …, boom_upn} at which the excavator is about to move when the boom is lifted up and the values ​​{boom_dowm1, boom_down2, …, boom_downn} at which the bucket is about to hit the ground when it is fully released. Then, the minimum value of {boom_up1, boom_up2, …, boom_upn} is taken and the maximum value is calculated. The lower 0.05 is used as the upward limit of the boom, that is, boom_up_limit = min{boom_up1, boom_up2,…, boom_upn}-0.05. Similarly, the maximum value of {boom_dowm1, boom_down2,…, boom_downn} is taken 0.05 higher as the downward limit of the boom, that is, boom_down_limit = max{boom_dowm1, boom_down2,…, boom_downn}+0.05.

[0090] The forearm also opens or retracts at different speeds, and the reasonable limit positions of the forearm at different speeds are obtained, {arm_up1, arm_up2, ..., arm_upn} and {arm_dowm1, arm_down2, ..., arm_downn}, and then the minimum value of the forearm at different speeds minus 0.05 is taken as the limit value of the forearm, that is, arm_up_limit = min{arm_up1, arm_up2, ..., arm_upn}-0.05, and the maximum value of the forearm at different speeds plus 0.05 is taken as the limit value of the forearm, that is, arm_down_limit = max{arm_dowm1, arm_down2, ..., arm_downn}+0.05. Since all positions of the cockpit are accessible and safe, there is no restriction on this cabin.

[0091] If the boom is raised and boom_state is greater than boom_up_limit, it is judged as exceeding the limit. If the boom is lowered and boom_state is less than boom_down_linit, it is also judged as exceeding the limit. The same is true for the forearm. If it exceeds the limit, it is determined that there is a risk in the safety information.

[0092] The time interval between the acquisition cycle of the frequency control module and the preset action control cycle can be the same or different. When determining the duration of the acquisition cycle, the time difference {dt1, dt2, ..., dt1000} from the frequency control module receiving joint_states to receiving action information can be counted multiple times, and then the maximum value is taken plus a certain fault tolerance e, that is, dt = max{dt1, dt2, ..., dt1000} + e is used as the final value, that is: the value of dt is obtained through multiple tests, the frequency control module obtains joint_states at a certain frequency, and publishes pwm_command at a certain frequency, and each time the time interval from obtaining joint_states to publishing pwm_command is also required to be certain. However, the time from obtaining information to receiving action information calculated by the model is uncertain. When the frequency control module receives joint_states, it is t0, and then pwm_command is sent to the outside at t0+dt.

[0093] In a feasible implementation scheme, the unmanned excavator parses the part execution information to obtain the status information; the unmanned excavator sends the status information and the security information to the high-performance computing device via the local area network.

[0094] Specifically, the unmanned excavator can obtain the part execution information of each part of itself through the frequency control module, and then send the part execution information to the abnormality detection module, so that the abnormality detection module can determine the safety information of the unmanned excavator based on the part execution information, and then the abnormality detection module sends the safety information and the part execution information to the state conversion module. The state conversion module obtains the state information of the corresponding part from the part execution information according to the set part that needs to be trained, and sends the safety information and the obtained state information to the high-performance computing device through the local area network, so that the high-performance computing device can determine whether the state information is available based on the safety information.

[0095] In a feasible implementation manner, the state information includes: position information and speed information of the boom, arm and cabin of the unmanned excavator at the current time point.

[0096] It should be noted that the type of status information can be set according to actual needs with specific target parts, and is not specifically limited here.

[0097] In order to explain the overall process of the online model training system for unmanned excavators, an example is given below. Figure 6 Another model online training system for an unmanned excavator provided in an embodiment of the present application, wherein the unmanned excavator includes a frequency control module, an anomaly detection module and a state / action conversion module, the high-performance computing device includes a target selection module, a state construction module, a trajectory tracking module configured with a trajectory tracking model, a data storage module and a model training module, and the state / action conversion module can communicate with the module selection module and the trajectory tracking module respectively through a local area network, such as Figure 6 As shown in the figure, the workflow of the model online training system is as follows:

[0098] 1. The frequency control module collects the execution information of each part of the unmanned excavator that needs to be detected according to the preset collection cycle;

[0099] 2. The frequency control module sends the collected part execution information to the abnormality detection module;

[0100] 3. The anomaly detection module determines the safety information of the unmanned excavator based on the part execution information;

[0101] 4. The anomaly detection module sends the security information and part execution information to the state / action conversion module;

[0102] 5. The state / action conversion module obtains the state information corresponding to the target part from the part execution information according to the target part to be trained;

[0103] 6. The state / action conversion module sends the state information and security information to the target selection module via the local area network;

[0104] The following description will be made by taking the case where the state of the unmanned excavator indicated by the safety information is safe as an example.

[0105] 7. When the state of the unmanned excavator indicated by the safety information is safe, the target selection module selects a target trajectory from the trajectory library and selects a target point from the target trajectory.

[0106] 8. The target selection module sends the state information and target point to the state construction module;

[0107] 9. The state construction module performs vector splicing on the target point and state information to obtain a splicing vector;

[0108] 10. The state building module takes the concatenated vector as input and feeds it into the trajectory tracking module;

[0109] 11. The trajectory tracking module sends the output action information to the state / action conversion module, and sends the action information and state information in the form of data pairs to the data storage module;

[0110] 12. The data storage module determines whether the number of currently stored data pairs meets the threshold requirement. When the number of stored data pairs meets the threshold requirement, the data pairs are sent to the model training module;

[0111] 13. The model training module uses the data to update the trajectory tracking module configured in the trajectory tracking module, and configures the trajectory tracking model after the update training as the trajectory tracking model of the unmanned excavator.

[0112] 14. After receiving the action information, the state / action conversion module combines the control information and action information of the part to be controlled into a collective control instruction;

[0113] 15. The state / action conversion module sends the collective control instruction to the frequency control module;

[0114] 16. The frequency control module sends the collective control instruction to the lower computer of the unmanned excavator according to the preset action control cycle, so that the lower computer can control various parts of the excavator according to the collective control instruction.

[0115] Figure 7 A flow chart of a model online training method for an unmanned excavator provided in an embodiment of the present application, wherein the online training method is configured on a high-performance computing device, wherein the high-performance computing device and the unmanned excavator constitute a model online training system, and wherein the high-performance computing device and the unmanned excavator are deployed in the same local area network, such as Figure 7As shown, the online training method includes the following steps:

[0116] Step 701: after receiving the status information sent by the unmanned excavator through the local area network, vector splicing is performed on the target point and the status information to obtain a splicing vector, wherein the target point is determined on a target trajectory selected from a trajectory library.

[0117] Step 702: Input the splicing vector into the trajectory tracking model currently configured for the unmanned excavator to obtain action information for instructing the target part of the unmanned excavator to perform the target action.

[0118] Step 703: Use the action information and the state information as a data pair to perform update training on the trajectory tracking model.

[0119] In a feasible implementation, the high-performance computing device also receives safety information sent by the unmanned excavator through the local area network, and when the safety information indicates that the state of the unmanned excavator is safe, vector splicing is performed on the target point and the state information.

[0120] In a feasible implementation, the high-performance computing device stores the data pairs in a data storage device, and when the number of the data pairs stored in the data storage device meets a threshold requirement, the splicing vector is input into the trajectory tracking model.

[0121] In a feasible implementation scheme, after obtaining the action information, the action information is sent to the unmanned excavator through the local area network, so that after receiving the action information, the unmanned excavator controls the target part to perform the target action according to the action information.

[0122] about Figure 7 The principle description of the related content shown can be referred to Figure 1-Figure 6 The relevant explanations in will not be repeated in detail here.

[0123] In one possible embodiment, Figure 8 A schematic diagram of the structure of a model online training device for an unmanned excavator provided in an embodiment of the present application, wherein the online training device is arranged on a high-performance computing device, wherein the high-performance computing device and the unmanned excavator constitute a model online training system, and wherein the high-performance computing device and the unmanned excavator are deployed in the same local area network, such as Figure 8 As shown, the device comprises:

[0124] A splicing unit 81 is used for performing vector splicing on a target point and the status information after receiving the status information sent by the unmanned excavator through the local area network to obtain a splicing vector, wherein the target point is determined on a target trajectory selected from a trajectory library;

[0125] A training unit 82, configured to input the splicing vector into a trajectory tracking model currently configured for the unmanned excavator, to obtain action information for instructing a target part of the unmanned excavator to perform a target action;

[0126] An updating unit 83, configured to update and train the trajectory tracking model by taking the action information and the state information as a data pair;

[0127] The configuration unit 84 is used to configure and update the trajectory tracking model after training for the unmanned excavator.

[0128] In a feasible embodiment, the device further comprises:

[0129] A receiving unit, used for receiving the safety information sent by the unmanned excavator through the local area network;

[0130] When the splicing unit is used to perform vector splicing on the target point and the state information, it includes:

[0131] When the safety information indicates that the state of the unmanned excavator is safe, vector splicing is performed on the target point and the state information.

[0132] In a feasible embodiment, the device further comprises:

[0133] a storage unit, configured to store the data pair in a data storage device;

[0134] When the training unit is used to input the splicing vector into a trajectory tracking model currently configured for the unmanned excavator, it includes:

[0135] When the number of the data pairs stored in the data storage device meets a threshold requirement, the splicing vector is input into the trajectory tracking model.

[0136] In a feasible embodiment, the device further comprises:

[0137] The sending unit is used to send the action information to the unmanned excavator through the local area network after obtaining the action information, so that the unmanned excavator controls the target part to perform the target action according to the action information after receiving the action information.

[0138] about Figure 8The principle description of the related content shown can be referred to Figure 1-Figure 6 The relevant explanations in will not be repeated in detail here.

[0139] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application includes: a processor 91, a storage medium 92 and a bus 93, wherein the storage medium 92 stores machine-readable instructions executable by the processor 91. When the electronic device runs the above-mentioned model online training method, the processor 91 communicates with the storage medium 92 through the bus 93, and the processor 91 executes the machine-readable instructions to execute Figure 7 The method steps are shown.

[0140] For specific implementation methods, steps and principles, see Figure 1-Figure 6 The description will not be repeated in detail here.

[0141] Embodiment 4 of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned model online training method steps are executed.

[0142] For specific implementation methods, steps and principles, see Figure 1-Figure 6 The description will not be repeated in detail here.

[0143] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0146] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0147] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0148] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A model online training system for unmanned excavators, characterized in that: The model online training system includes: a high-performance computing device and an unmanned excavator, wherein the high-performance computing device and the unmanned excavator are deployed in the same local area network; After acquiring the status information of the target part, the unmanned excavator sends the status information to the high-performance computing device through the local area network; After receiving the state information, the high-performance computing device performs vector splicing on the target point and the state information to obtain a splicing vector, wherein the target point is determined on a target trajectory selected from a trajectory library; The high-performance computing device inputs the splicing vector into a trajectory tracking model currently configured for the unmanned excavator to obtain action information for instructing the target part of the unmanned excavator to perform a target action; The high-performance computing device uses the action information and the state information as a data pair to update and train the trajectory tracking model; The high-performance computing device is configured to update the trajectory tracking model after training for the unmanned excavator.

2. The online training system according to claim 1, characterized in that: When sending the status information, the unmanned excavator sends its own security information to the high-performance computing device through the local area network; After receiving the safety information, the high-performance computing device performs vector splicing on the target point and the status information when the safety information indicates that the status of the unmanned excavator is safe.

3. The online training system according to claim 1, characterized in that: The high performance computing device stores the data pair in a data storage device; When the number of the data pairs stored in the data storage device meets the threshold requirement, the high-performance computing device inputs the data pairs into the trajectory tracking model for update training.

4. The online training system according to claim 1, characterized in that: After obtaining the action information, the high-performance computing device sends the action information to the unmanned excavator through the local area network; After receiving the action information, the unmanned excavator controls the target part to perform the target action according to the action information.

5. The online training system according to claim 4, characterized in that: After receiving the action information, the unmanned excavator adds the control information of the part to be controlled to the action information to obtain a collective control instruction; The unmanned excavator controls the corresponding parts according to the collective control instructions and a preset action control cycle, so that the corresponding parts perform corresponding actions.

6. The online training system according to claim 2, characterized in that: The unmanned excavator collects its own position execution information according to a preset collection cycle; The unmanned excavator determines its own current safety information according to the part execution information; When the safety information indicates that there is a risk, the unmanned excavator performs an emergency stop operation and returns to an initial safe position.

7. The online training system according to claim 6, characterized in that: The unmanned excavator analyzes the part execution information to obtain the status information; The unmanned excavator sends the status information and the safety information to the high-performance computing device through the local area network.

8. The online training system according to claim 1, characterized in that: The state information includes: position information and speed information of the boom, arm and cabin of the unmanned excavator at the current time point.

9. A model online training method for unmanned excavators, characterized in that: The online training method is configured on a high-performance computing device, the high-performance computing device and the unmanned excavator constitute a model online training system, the high-performance computing device and the unmanned excavator are deployed in the same local area network, and the method includes: After receiving the status information sent by the unmanned excavator through the local area network, vector splicing is performed on the target point and the status information to obtain a splicing vector, wherein the target point is determined on the target trajectory selected from the trajectory library; Inputting the splicing vector into a trajectory tracking model currently configured for the unmanned excavator to obtain action information for instructing a target part of the unmanned excavator to perform a target action; Using the action information and the state information as a data pair to update and train the trajectory tracking model; A trajectory tracking model after training is updated for the unmanned excavator configuration.

10. The online training system according to claim 9, characterized in that: The method further comprises: Receiving security information sent by the unmanned excavator through the local area network; The performing vector splicing on the target point and the state information includes: When the safety information indicates that the state of the unmanned excavator is safe, vector splicing is performed on the target point and the state information.

11. The online training method as claimed in claim 9, characterized in that: The method further comprises: storing the data pair in a data storage device; The step of inputting the splicing vector into a trajectory tracking model currently configured for the unmanned excavator comprises: When the number of the data pairs stored in the data storage device meets a threshold requirement, the splicing vector is input into the trajectory tracking model.

12. The online training method according to claim 9, characterized in that: The method further comprises: After obtaining the action information, the action information is sent to the unmanned excavator through the local area network, so that after receiving the action information, the unmanned excavator controls the target part to perform the target action according to the action information.

13. A model online training device for unmanned excavators, characterized in that: The online training device is arranged on a high-performance computing device, the high-performance computing device and the unmanned excavator constitute a model online training system, the high-performance computing device and the unmanned excavator are deployed in the same local area network, and the device comprises: A splicing unit, configured to perform vector splicing on a target point and the status information after receiving the status information sent by the unmanned excavator through the local area network to obtain a splicing vector, wherein the target point is determined on a target trajectory selected from a trajectory library; A training unit, used for inputting the splicing vector into a trajectory tracking model currently configured for the unmanned excavator to obtain action information for instructing a target part of the unmanned excavator to perform a target action; An updating unit, configured to update and train the trajectory tracking model by taking the action information and the state information as a data pair; A configuration unit is used to configure and update the trajectory tracking model after training for the unmanned excavator.

14. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the online training method as described in any one of claims 9 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the online training method according to any one of claims 9 to 12 are executed.

Images