Mechanical equipment and operation control method thereof

Abstract

The invention provides mechanical equipment and an operation control method thereof. The method comprises the steps that observation data, used for trajectory tracking control, of the mechanical equipment are acquired; searching a plurality of candidate action combinations corresponding to executed actions of the mechanical equipment at the current moment from a plurality of action combinations constructed for the mechanical equipment; determining a plurality of predicted trajectories of the mechanical equipment under the plurality of candidate action combinations based on the executed action at the current moment, the plurality of candidate action combinations and the observation data; and controlling the mechanical equipment to work at the next moment according to the target action combination corresponding to the optimal prediction trajectory, wherein the optimal prediction trajectory is the prediction trajectory with the highest matching degree with the target planning trajectory in the plurality of prediction trajectories. According to the method and the device, the generalization of the algorithm can be improved, and the migration among different mechanical equipment is facilitated.

Description

Technical Field

[0001] The present application relates to the field of engineering machinery, and in particular to a mechanical device and an operation control method thereof. Background Art

[0002] Engineering machinery and equipment, for example, excavators can dig to fill or move materials, and loaders can be used to shovel, load, unload, and transport bulk materials such as soil and stone to perform operations such as gathering and loading.

[0003] Take the excavator as an example. It is a typical hydraulic control system with high time delay, significant time-varying characteristics, and severe joint coupling in its motion characteristics. The control problem of such highly nonlinear systems is still a big problem. At present, most industrial controls use PID control methods. On the robotic arm, high tracking accuracy (such as current, speed, and position) is achieved through three-loop control. However, this method requires complex PID parameter tuning, which is time-consuming and labor-intensive and has poor portability. Changing an excavator requires a lot of repeated operations.

[0004] Trajectory control is a common problem in robotic systems, requiring the robot to move along a specific trajectory. A simplified trajectory control method is to track each point of the trajectory point by point, but this is likely to cause motion lag because the subsequent trajectory is not considered. Summary of the invention

[0005] In view of this, the embodiments of the present application provide at least one mechanical device and an operation control method thereof to overcome at least one of the above-mentioned defects.

[0006] In a first aspect, an exemplary embodiment of the present application provides an operation control method for a mechanical device, the method comprising: obtaining observation data of the mechanical device for trajectory tracking control, the mechanical device comprising a plurality of movable joints, and the observation data comprising a joint position and a joint speed of each joint; searching for a plurality of candidate action combinations corresponding to an executed action of the mechanical device at a current moment from a plurality of action combinations constructed for the mechanical device; determining a plurality of predicted trajectories of the mechanical device under the plurality of candidate action combinations based on the executed action at the current moment, the plurality of candidate action combinations, and the observation data; and controlling the mechanical device to perform operation at the next moment according to a target action combination corresponding to an optimal predicted trajectory, the optimal predicted trajectory being a predicted trajectory among the plurality of predicted trajectories that has the highest matching degree with a target planned trajectory.

[0007] In a second aspect, an embodiment of the present application also provides a mechanical device, including a processor, configured to: obtain observation data of the mechanical device for trajectory tracking control, the mechanical device including multiple movable joints, and the observation data including the joint position and joint speed of each joint; search for multiple candidate action combinations corresponding to the executed actions of the mechanical device at the current moment from multiple action combinations constructed for the mechanical device; determine multiple predicted trajectories of the mechanical device under the multiple candidate action combinations based on the executed actions at the current moment, the multiple candidate action combinations and the observation data; control the mechanical device to perform operation according to the target action combination corresponding to the optimal predicted trajectory at the next moment, the optimal predicted trajectory being a predicted trajectory among the multiple predicted trajectories that has the highest match with the target planned trajectory.

[0008] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the operation control method of the above-mentioned mechanical equipment are executed.

[0009] The mechanical equipment and operation control method thereof provided in the embodiments of the present application can improve the generalization of the algorithm and facilitate migration between different mechanical equipment.

[0010] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 A flowchart showing a method for controlling a mechanical device according to an exemplary embodiment of the present application;

[0013] Figure 2 A flowchart showing the steps of constructing multiple action combinations provided by an exemplary embodiment of the present application;

[0014] Figure 3 A schematic diagram of a single joint action provided by an exemplary embodiment of the present application is shown;

[0015] Figure 4 A flowchart showing the steps of determining a multi-step single joint action of a joint provided by an exemplary embodiment of the present application;

[0016] Figure 5 A schematic diagram of a trajectory tracking control process provided by an exemplary embodiment of the present application is shown;

[0017] Figure 6 A flowchart showing the steps of determining the optimal prediction trajectory provided by an exemplary embodiment of the present application;

[0018] Figure 7 A control structure schematic diagram of a mechanical device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0020] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0021] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "Including A, B and / or C" means including any one, any two, or any three of A, B, and C.

[0022] It should be understood that in the embodiments of the present application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined according to A. Determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0023] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0024] Engineering machinery and equipment, for example, excavators can dig to fill or move materials, and loaders can be used to shovel, load, unload, and transport bulk materials such as soil and stone to perform operations such as gathering and loading.

[0025] Take the excavator as an example. It is a typical hydraulic control system with high time delay, significant time-varying characteristics, and severe joint coupling in its motion characteristics. The control problem of such highly nonlinear systems is still a big problem. At present, most industrial controls use PID control methods. On the robotic arm, high tracking accuracy (such as current, speed, and position) is achieved through three-loop control. However, this method requires complex PID parameter tuning, which is time-consuming and labor-intensive and has poor portability. Changing an excavator requires a lot of repeated operations.

[0026] Trajectory control is a common problem in robotic systems, requiring the robot to move along a specific trajectory. A simplified trajectory control method is to track each point of the trajectory point by point, but this is likely to cause motion lag because the subsequent trajectory is not considered.

[0027] In related technologies, the trajectory control of mechanical equipment usually adopts the following control methods:

[0028] PID control method: Use PID to track the trajectory point by point, calculate the speed of each point in the trajectory through interpolation, and track the speed and position of each point in the trajectory as the target, which can achieve high tracking accuracy. However, this control method requires manual calibration and lacks a standard process. It still relies more on manual experience for trial and error. In addition, there are problems with the transferability of this method. If you change an excavator for trajectory tracking, you need to repeat the above process, which is time-consuming and labor-intensive. In addition, PID has a better effect on single-point control, but a poor effect on tracking some complex curves.

[0029] MPC control method: Build an explicit robot model, and use the optimization method to obtain the optimal action according to the target trajectory. Only a few steps are executed each time, and then re-planning is performed to reduce the impact of errors. However, this control method requires the establishment of an explicit motion model of the excavator, and also requires manual calibration. In addition, the optimization calculation of MPC is time-consuming and real-time calculation is difficult. The effect of its trajectory control is improved compared to PID control.

[0030] RL control method: Trajectory planning through reinforcement learning is also a popular approach. Data is obtained online or offline, and a reasonable reward function is set to allow the robot to learn the desired actions in the exploration and utilization strategy. This control method essentially builds a model of the excavator, but only requires data collection and does not require manual calibration. However, the RL control method is unstable, cannot guarantee optimality, cannot guarantee the control effect, and has poor interpretability, which limits its application in safety-first issues such as excavators.

[0031] In response to the above problems, the present application proposes a mechanical equipment and an operation control method thereof, which solves the problems of poor generalization, manual calibration and high cost of traditional PID control methods in at least one aspect; on the other hand, it solves the problems of large computational complexity and difficulty in real-time use of MPC control methods; on yet another hand, it solves the problems of unstable training and poor interpretability of RL control methods.

[0032] To facilitate understanding of the present application, the mechanical equipment and the operation control method thereof provided in the embodiments of the present application are introduced in detail below.

[0033] See also Figure 1 , which is a flow chart of the operation control method of the mechanical equipment provided in the exemplary embodiment of the present application, which is generally applied to the controller of the mechanical equipment. The above control method can be integrated into the on-board controller of the mechanical equipment, or it can also be run in an independent controller on the mechanical equipment, and the independent controller communicates with the on-board controller of the mechanical equipment and / or the driving mechanism to realize trajectory tracking and / or shoveling operation process, but the present application is not limited to this.

[0034] Reference Figure 1 In step S101, observation data of the mechanical equipment for trajectory tracking control is obtained.

[0035] Exemplarily, the above-mentioned mechanical equipment may include but is not limited to an excavator or a loader. For example, an excavator can dig or move materials by digging, and a loader is a kind of engineering mechanical equipment mainly used for loading, unloading, handling and transporting various materials and goods.

[0036] The mechanical equipment provided by the exemplary embodiment of the present application has multiple movable joints. For example, the multiple movable joints may include but are not limited to: a cabin, a boom, an arm and a bucket. The cabin can be driven to control the rotation of the cabin, and the shoveling action can be performed by driving the boom, and / or the arm and / or the bucket. Here, the driving methods for the above multiple joints may include but are not limited to: wire rope dragging, motor drive, hydraulic drive, and the present application does not limit this.

[0037] In an exemplary embodiment of the present application, the observation data of the mechanical equipment may include, but are not limited to, the joint position and joint speed of each joint. Exemplarily, a sensor may be provided at each joint to obtain observation data from the provided sensor. For example, a heading angle sensor may be provided at the cockpit of the mechanical equipment to obtain the rotation angle and rotation speed of the cockpit as the joint position and joint speed of the cockpit. In addition, an inclination sensor may be provided at each arm, forearm, and bucket to obtain the inclination value and rotation speed corresponding to the arm, forearm, and bucket as the joint position and joint speed corresponding to the arm, forearm, and bucket.

[0038] Preferably, the observation data used for trajectory tracking control may include but is not limited to observation data for multiple joints at the current moment and observation data at historical moments before the current moment. That is, in the trajectory tracking control process for the mechanical equipment, the historical state of the mechanical equipment is taken into account to help quickly migrate the trajectory tracking control scheme to different mechanical equipment.

[0039] In an optional example, the historical moment before the current moment may refer to the previous moment. Figure 5 Taking the trajectory tracking control flow diagram shown in FIG. 1 as an example, assuming that the current time is t, the observation data used for this round of trajectory prediction may include the observation data Q at the current time t t and the observed data Q at historical time t-1 t- 1 .

[0040] It should be understood that the observation data at the current time t and the historical time tN can also be selected for trajectory prediction, where N is a natural number greater than 1, and this application does not impose any restrictions on this.

[0041] In step S102 , multiple candidate action combinations corresponding to the actions that have been executed by the mechanical device at the current moment are searched from the multiple action combinations constructed for the mechanical device.

[0042] At present, trajectory tracking control for mechanical equipment usually involves the following two types:

[0043] One is based on Cartesian space, for example, tracking the end trajectory of the mechanical equipment, illustratively, it can refer to the tooth tip trajectory of the bucket used to track and control the mechanical equipment.

[0044] The other is based on joint space, for example, based on the joint positions of multiple joints included in the mechanical device to form the trajectory of the mechanical device. The operation control method of the mechanical device of the present application is a trajectory tracking control scheme based on joint space.

[0045] In the embodiment of the present application, the observation data of the mechanical device is used to characterize the state of the mechanical device from the joint side, such as the observation data is used to characterize the state of multiple joints of the mechanical device, and the state of the joint is characterized by the joint position and the joint speed. The action of the mechanical device is used to characterize the control of the mechanical device from the drive side, such as the action is used to characterize the control amount of the drive mechanism corresponding to each joint, and for example, taking the drive mechanism as a hydraulic system, the action refers to the control amount used to drive the joint movement in the hydraulic system.

[0046] In a preferred embodiment of the present application, multiple action combinations can be pre-constructed and stored in the video memory, so that the subsequent processing for determining the optimal predicted trajectory can be performed in the video memory. Here, considering that the number of action combinations is very large, by performing trajectory prediction processing in the video memory and directly calling multiple action combinations from the video memory, the computing efficiency can be effectively improved. Figure 2-Figure 4 Let's introduce the process of building multiple action combinations.

[0047] Figure 2 A flowchart showing the steps of constructing multiple action combinations provided by an exemplary embodiment of the present application.

[0048] like Figure 2 As shown, in step S201, for each movable joint of the mechanical device, a multi-step single-joint action of the joint is determined.

[0049] Taking the above-mentioned mechanical equipment including four joints of a cockpit, a boom, an arm and a bucket as an example, for a single joint, the multi-step single-joint actions of the joint can be predetermined.

[0050] See also Figure 3 The single joint action schematic diagram shown in FIG. 1 shows the action space of a single joint. In this example, the action space is represented by a two-dimensional coordinate system, the abscissa of the action space represents the number of steps executed in time sequence, and the ordinate of the action space represents the action of the joint, that is, the control amount used to drive the joint movement. For each joint, the control amount used by the driving mechanism to drive the joint movement can be normalized so that the action of the joint can be normalized to the range of [-1, 1].

[0051] Figure 3 The curve shown represents the future multi-step motion of the joint, with 0 as the starting point and straight lines with different slopes as the motion, which represent the change of the motion over time.

[0052] Refer to the following Figure 4 To introduce the steps to determine the multi-step single joint action of a joint, each joint of the mechanical device can be Figure 4 The processing steps are used to determine the corresponding multi-step single-joint actions.

[0053] In an embodiment of the present application, the multi-step single-joint action of each joint may include an initial action and a subsequent action, for example, one initial action and multiple subsequent actions. In the trajectory tracking control of the present application, since only the first step action is executed subsequently, the determination of the initial action is more important.

[0054] like Figure 4 As shown, in step S301, the first data point projected by the joint in the action space is determined.

[0055] by Figure 3 Taking the example shown as an example, the first data point may refer to the data point corresponding to the starting step number 0 in the action space. The initial control amount of the first data point is 0.5. It should be understood that it may also be 0.2, 0.6, etc.

[0056] In step S302, the action within the neighborhood of the first data point is determined as the initial action of the joint.

[0057] Here, multiple initial actions can be determined from the neighborhood according to the preset traversal precision. It should be understood that the finer the traversal precision, the larger the amount of data of the action combination. Correspondingly, the coarser the traversal precision, the smaller the amount of data. The traversal precision can be determined according to actual needs. When determining the above multiple actions and subsequently forming action combinations, the coupling of the joints of the mechanical equipment must also be considered.

[0058] See also Figure 3In the example shown, it is assumed that the first data point is at (0, 0.5). In this example, the neighborhood of the first data point is represented by a black box in the schematic diagram (e.g., the corresponding step interval 0.5-1.5). It should be understood that the above neighborhood can refer to a step interval after the step 0 of the first data point. Those skilled in the art can select a step interval as the domain of the first data point according to actual needs. In this example, the step interval near the step 1 represented by the black box is used as the neighborhood. This selection method is only an example, and the present application is not limited thereto. Assuming that 5 data points are searched in the neighborhood according to the preset traversal fineness, the corresponding 5 initial actions are determined. Here, the traversal precision can refer to the accuracy of constructing the action curve for a single joint. For example, taking the joint of the mechanical equipment as a cockpit, assuming that the movable range of the cockpit is 0 to 360 degrees, illustratively, when 60 degrees is used as the traversal precision, it can be determined that starting from the joint position corresponding to the first data point, multiple joint positions can be obtained within the movable range of 0 to 360 degrees according to the traversal precision of 60 degrees. The control quantities corresponding to these multiple joint positions in the drive system are mapped to the joint space of the cockpit, and multiple data points searched in the neighborhood of the first data point can be obtained. It should be understood that those skilled in the art can determine the traversal precision according to actual needs, and can adopt a unified traversal precision for all joints, or can select different traversal precisions for different joints. This application does not limit this.

[0059] In step S303, the action slope is determined. Here, the action slope may refer to the slope of a line segment from a first data point to a second data point where the initial action is projected on the action space of the joint.

[0060] As described above, after multiple initial actions are determined in the action space, for each initial action, the second data point corresponding to the initial action is connected with the first data point to determine the slope of the connected line segment, that is, multiple initial actions correspond to multiple line segments, and each line segment corresponds to a slope.

[0061] Following the above Figure 3 In the example shown, the second data points corresponding to the five initial actions are respectively connected to the first data points to form five line segments, such as the green, orange, blue, red, and purple line segments shown in the figure. Further, the slope of each line segment is determined as the action slope.

[0062] In step S304, in the motion space of the joint, the first data point is taken as a starting point and extended according to the motion slope to obtain a motion curve.

[0063] Preferably, the range of motion of the joint is constrained in the joint motion space, such as normalized to the range of [-1, 1] as mentioned above, see Figure 3In the example shown, in the joint motion space, starting from the first data point, the motion is extended according to the motion slope, and the part beyond the motion range is limited to the boundary value of the motion range. For example, taking the green motion curve as an example, when extending to the upper boundary limit according to the motion slope, its motion is limited to the upper boundary limit of 1, and the orange motion curve is similar. For the purple motion curve, when extending to the lower boundary limit according to the motion slope, its motion is limited to the lower boundary limit of -1, to indicate that after step 3, the control amount of the drive mechanism corresponding to the joint remains unchanged and is maintained at the control amount after -1 anti-normalization.

[0064] In step S305 , a subsequent motion of the joint is determined based on a third data point falling on the motion curve.

[0065] Here, the subsequent actions of the joints may include multi-step actions, e.g. Figure 3 In the example shown, based on each step on the horizontal axis, an action corresponding to the step can be found from an action curve to form a subsequent action of the joint. Figure 3 An action curve in the figure constitutes a multi-step single-joint action of a joint.

[0066] To reduce the amount of calculation, the overall action can be modeled as a line segment with a fixed slope, and the values ​​beyond the boundary range can be clamped at the boundary.

[0067] return Figure 2 In step S202, multiple single-joint actions of each joint are combined to form multiple action combinations.

[0068] Here, a preset number of steps can be set for the action combination, and each action combination includes the same preset number of steps. At this time, each action combination includes multiple actions with a preset number of steps, and each action is composed of a single joint action combination of each joint under the step number.

[0069] For example, taking an action combination consisting of 3 steps as an example, for the 4 movable joints of the mechanical device, each step of the action includes 4 actions of the 4 joints under the number of steps. Figure 3 In the single joint action diagram shown, taking joint A as an example, based on an action curve in the single joint action diagram of joint A, an action at the step number is determined. Correspondingly, based on an action curve in the single joint action diagram of joint B, an action at the step number is determined. By analogy, the four action combinations determined for the four joints at the step number form one step action in the action combination. With reference to the above process, other step actions in the action combination can be determined. Here, each joint corresponds to multiple action curves. By combining multiple action curves of multiple joints, a large number of action combinations can be formed.

[0070] A series of action combinations are pre-generated in the joint space to ensure that these action combinations cover the entire joint space. The number of action combinations affects the real-time performance of the overall algorithm. At the same time, when combining actions for each joint, the coupling of the joints of the mechanical equipment must also be considered.

[0071] In the embodiment of the present application, each candidate action combination includes multiple steps of actions to be performed after the action executed by the mechanical device at the current moment, which are predicted for the mechanical device. Exemplarily, in the above manner, for any action of the mechanical device, it is possible to predict multiple steps of actions to be performed after the mechanical device can connect with the any action. Here, there are multiple situations of actions that can be connected with the any action, and one situation can be formed into a candidate action combination.

[0072] For example, the multiple action combinations constructed above can form an action combination sequence. At this time, according to the execution action at the current moment, an action combination sequence that meets the requirements can be searched from the preloaded action combination sequence as a candidate action combination, such as intercepting it from the preloaded action combination sequence, requiring that the starting point is the execution action at the current moment.

[0073] In order to speed up the calculation, a fixed slope can be used when forming the action curve in the action space. In addition, the action combination can be pre-generated and placed in the video memory. Later, it can be read directly from the video memory, avoiding switching data from the internal memory to the video memory during real-time calculation, which can significantly speed up the calculation efficiency.

[0074] return Figure 1 In step S103, based on the execution action at the current moment, multiple candidate action combinations and the acquired observation data, multiple predicted trajectories of the mechanical equipment under multiple candidate action combinations are determined.

[0075] In a preferred embodiment of the present application, a pre-trained world model is used to determine the predicted trajectory corresponding to each candidate action combination. Preferably, in order to improve the prediction accuracy, the world model is a single-step prediction world model, so as to obtain the predicted trajectory corresponding to each candidate action combination through multiple iterations, thereby predicting the execution of the mechanical device under a given action. In the embodiment of the present application, the trajectory of the mechanical device may refer to the joint position of each joint.

[0076] At present, the core of trajectory tracking control is to build a dynamic transfer model of the robot and obtain the optimal action based on the model. There are many ways to obtain the model. One is to build an explicit model through differential equations. The advantage is that the optimal action can be obtained through optimization methods. Another way is to build an implicit model. Considering that the motion characteristics of mechanical equipment (such as excavators) are relatively complex, the world model used in the embodiment of this application is obtained through neural network training and belongs to an implicit model.

[0077] The mechanical equipment trajectory control method based on the world model proposed in this application combines the trained world model with the trajectory search to find the action execution corresponding to the optimal trajectory. It can solve the problems of manual calibration required for mechanical equipment trajectory control, low computational efficiency, and poor interpretability of results.

[0078] In addition, the above control process also takes into account the time delay characteristics of the mechanical equipment to make the result more accurate. Moreover, by considering the historical state of the mechanical equipment, the generalization of the model is improved, so that the control scheme can be quickly migrated to different mechanical equipment.

[0079] For each candidate action combination, the following processing is performed at each iteration: the executed action at the current moment, the candidate action combination, and the observation data at the current moment and the historical moment are input into the world model to obtain a one-step predicted action for the mechanical equipment.

[0080] Combination Figure 5 The trajectory tracking control flow diagram shown in FIG. 1 is an exemplary diagram. For time t, the executed action at the current time refers to A t ,The candidate action combination includes 3 steps, such as,A t The corresponding candidate action combination is A t+ 1 , A t+ 2 , A t+ 3 , the observation data at the current time t is Q t , the observation data at historical time t-1 is Q t- 1 .

[0081] Based on this, at time t, A can be t , A t+ 1 , A t+ 2 , A t+ 3 , Q t , Q t- 1 Input to the world model, at this point, the world model outputs a one-step prediction, such as S't+ 1 , determine the predicted trajectory S' of this step t+ 1 The corresponding predicted action, such as A' t+ 1 , the predicted action is used as the execution action in the next iteration. For example, the predicted action A' t+ 1 , A t+ 1 , A t+ 2 , A t+ 3 , Q t , Q t- 1 Input to the world model to output the next step prediction, such as S' t+ 2 Alternatively, you can also use A t , A' t+ 1 , A t+ 2 , A t+ 3 , Q t , Q t- 1 Input to the world model to output the next step prediction S' t+ 2 , and the iteration is terminated when the number of steps of the predicted action output by the world model reaches the preset number of steps, thereby obtaining the predicted trajectory corresponding to the candidate action combination.

[0082] To speed up the computational efficiency, the pre-generated actions can be normalized (which can be completed and stored in advance), and then the normalized candidate action combinations can be batch-input into the world model to iteratively generate future states. Finally, the state output by the world model (i.e., the predicted trajectory) can be denormalized to obtain the result.

[0083] Undertake Figure 5 For example, taking the action combination including 3 steps as an example, at time t, a predicted trajectory S' is output through the world model for a candidate action combination t+ 1 , S' t+ 2 , S' t+ 3 , S' t+ 4 .

[0084] In an optional embodiment, the world model for trajectory prediction of mechanical equipment in the embodiment of the present application can be obtained by neural network training. Exemplarily, the world model can be obtained by the following training method: obtain training data of mechanical equipment, which can be obtained based on the historical actual operation data of mechanical equipment. For example, the training data includes multiple training samples, each training sample includes training observation data of mechanical equipment, training action combination and operation trajectory, the training observation data can be composed of observation data of adjacent moments in the historical actual operation data of mechanical equipment, such as observation data at time t and time t+1, the training action combination can be composed of multiple actions at adjacent moments in the historical actual operation data of mechanical equipment, and the training observation data in a training sample corresponds to the time of the training action combination, such as the training action combination can include the action at time t+1 and the actions at time t+2, t+3, t+4. Correspondingly, the operation trajectory in the training sample can refer to the actual motion trajectory at time t+2, t+3, t+4, t+5 in the historical actual operation data of mechanical equipment.

[0085] After obtaining the above-mentioned multiple training samples, the training observation data and training action combination in each training sample is used as the input of the initial neural network, and the operation trajectory in each training sample is used as the output of the initial neural network. The initial neural network is trained to obtain a world model that meets the trajectory prediction requirements.

[0086] return Figure 1 In step S104, the mechanical equipment is controlled to perform operation according to the target action combination corresponding to the optimal predicted trajectory at the next moment.

[0087] Here, the predicted trajectories obtained by different candidate action combinations are different, and it is necessary to find a predicted trajectory that is closest to the target planning trajectory as the target trajectory. That is, the optimal predicted trajectory is the predicted trajectory with the highest matching degree with the target planning trajectory among multiple predicted trajectories.

[0088] Exemplarily, the target planning trajectory can be given by the planning module, for example, the planning module uses any existing trajectory algorithm to determine a trajectory in real time / predetermined. In addition, the target planning trajectory can also be a trajectory manually specified according to the trajectory tracking task, and this application does not limit the method for determining the target planning trajectory.

[0089] Combine the following Figure 6 Let's introduce the process of searching for the best prediction trajectory from multiple prediction trajectories. It should be understood that Figure 6 The optimization method shown is only an optional example, and the present application is not limited thereto. A trajectory that has the highest matching degree with the target planning trajectory may also be determined by other methods.

[0090] Figure 6 A flowchart showing the steps of determining an optimal prediction trajectory provided by an exemplary embodiment of the present application.

[0091] like Figure 6 As shown, in step S401, for each predicted trajectory, the mean square error value of each trajectory point on the predicted trajectory relative to the target planned trajectory is calculated.

[0092] Here, the method for calculating the mean square error is common knowledge in the art, and this application will not elaborate on this part.

[0093] In an optional example, the sum of the mean square error values ​​of all trajectory points on a predicted trajectory may be determined as the matching index of the predicted trajectory to screen out the optimal predicted trajectory.

[0094] In addition to the above methods, we can also consider that the hydraulic system of the mechanical equipment has a time delay characteristic and introduce a weighted mean square error. The later the trajectory point is, the higher the corresponding weight value is.

[0095] For example, in step S402, for each predicted trajectory, a weight value is set for each trajectory point on the predicted trajectory according to the action execution order.

[0096] Here, the weight value corresponding to the trajectory point at the front of the action execution order is larger, and the weight value corresponding to the trajectory point at the back of the action execution order is smaller. For example, the weight value can be decreased by the exponential power of e.

[0097] In step S403, for each predicted trajectory, a matching index for characterizing the matching degree of the predicted trajectory is calculated based on the mean square error value and the corresponding weight value of each trajectory point on the predicted trajectory.

[0098] In step S404, an optimal predicted trajectory is determined according to the matching indexes of all predicted trajectories.

[0099] Here, a prediction trajectory corresponding to the minimum matching index may be determined as the optimal prediction trajectory.

[0100] In a preferred embodiment of the present application, the target action combination corresponding to the optimal predicted trajectory is stored in the memory for being called.

[0101] Preferably, in an embodiment of the present application, the mechanical device is controlled to perform a further action at the next moment. For example, the first action in the target action combination is extracted from the memory, and the mechanical device is controlled to perform the first action at the next moment. Compared with directly executing all actions, this can reduce the impact of system errors.

[0102] Here, after controlling the mechanical device to perform the first action at the next moment (e.g., t+1), the first action performed at the next moment is used as the above-mentioned executed action at the current moment (e.g., t), and the above-mentioned processing flow in the embodiment of the present application is repeated to obtain the optimal predicted trajectory corresponding to the next moment (e.g., t+2), and the mechanical device is controlled to perform the first action in the action combination corresponding to the optimal predicted trajectory at t+2, and so on and so forth until the controlled mechanical device completes the movement of the target planned trajectory.

[0103] In an embodiment of the present application, the observation data includes joint speed and joint position, the movement of the joint refers to the control amount of the hydraulic system used to drive the joint movement, and the trajectory of the mechanical equipment includes the joint position of each joint. It can be understood that in the operation control scheme of the present application, the joint position of each joint of the mechanical equipment can be predicted based on the combination of joint speed, joint position, and the control amount under the drive system (i.e., predicted trajectory), and then the control amount for each joint in the drive system can be inferred based on the predicted joint position of each joint, so as to control the drive system to control each joint to move to the predicted joint position according to the control amount.

[0104] Based on the same application concept, the embodiments of the present application also provide mechanical equipment corresponding to the methods provided in the above embodiments. Since the principles of solving the problems by the mechanical equipment in the embodiments of the present application are similar to the operation control methods in the above embodiments of the present application, the implementation of the equipment can refer to the implementation of the method, and the repeated parts will not be repeated.

[0105] Figure 7 The control structure diagram of the mechanical device provided by the exemplary embodiment of the present application is shown in FIG. Figure 7 As shown, the mechanical device 200 includes a controller 210, which is configured to perform the following processes:

[0106] Obtain observation data of mechanical equipment for trajectory tracking control;

[0107] Searching for multiple candidate action combinations corresponding to the execution actions of the mechanical device at the current moment from the multiple action combinations constructed for the mechanical device;

[0108] For each candidate action combination, based on the execution action at the current moment, the candidate action combination and the observation data, determine the predicted trajectory of the mechanical device under the candidate action combination;

[0109] The mechanical equipment is controlled to perform operation according to the target action combination corresponding to the optimal predicted trajectory at the next moment, and the optimal predicted trajectory is a predicted trajectory with the highest matching degree with the target planned trajectory among multiple predicted trajectories.

[0110] In a possible implementation of the present application, the mechanical device includes multiple movable joints, and the observation data for trajectory tracking control includes observation data for the multiple joints at the current moment and the historical moments before the current moment, and the observation data includes the joint position and joint speed of each joint. Exemplarily, the joint position and joint speed of each joint can be detected by a sensor 220 arranged at each joint.

[0111] In one possible implementation of the present application, the controller 210 is also configured to construct the multiple action combinations in the following manner: for each movable joint of the mechanical device, determine a multi-step single-joint action of the joint after a preset action; combine the multiple single-joint actions of each joint to form a plurality of action combinations, each action combination including a plurality of actions with a preset number of steps, and each action is composed of a combination of single-joint actions of each joint at the preset number of steps.

[0112] In one possible implementation of the present application, the multi-step single-joint action of each joint includes an initial action and a subsequent action, wherein the controller 210 is further configured to determine the initial action of each joint in the following manner: determine the first data point where the preset action is projected on the action space of the joint, and determine the action within the neighborhood of the first data point as the initial action of the joint.

[0113] In one possible implementation of the present application, the controller 210 is also configured to determine the subsequent action of each joint in the following manner: determine the action slope, where the action slope is the slope of a line segment from a first data point to a second data point where the initial action is projected on the action space of the joint; in the action space of the joint, take the first data point as the starting point and extend according to the action slope to obtain an action curve; and determine the subsequent action of the joint based on a third data point falling on the action curve.

[0114] In a possible implementation of the present application, the joints of the mechanical device are driven by a hydraulic system 240, and the movement of the joint refers to the control amount of the hydraulic system 240 used to drive the joint movement, wherein the horizontal axis of the action space represents the number of steps, and the vertical axis of the action space represents the movement range of the joint.

[0115] In one possible implementation of the present application, the controller 210 is further configured to form an action curve in the following manner: in the action space of the joint, starting from the first data point, extending according to the action slope, and limiting the part exceeding the action range to the boundary value of the action range.

[0116] In a possible implementation manner of the present application, each candidate action combination includes multiple steps of actions predicted for the mechanical device to be performed after the execution action at the current moment.

[0117] In one possible implementation of the present application, the controller 210 is further configured to determine the predicted trajectory corresponding to each candidate action combination in the following manner: using a pre-trained world model to obtain the predicted trajectory corresponding to the candidate action combination through multiple iterations, and performing the following processing in each iteration: inputting the execution action, the candidate action combination and the observation data into the world model to obtain a one-step predicted action for the mechanical equipment, wherein the predicted action output by the world model in the current iteration is used as the execution action in the next iteration, and the iteration terminates when the number of steps of the predicted action output by the world model reaches a preset number of steps.

[0118] In a possible implementation of the present application, the controller 210 is further configured to determine the optimal predicted trajectory in the following manner: for each predicted trajectory, calculating the mean square error value of each trajectory point on the predicted trajectory relative to the target planned trajectory; for each predicted trajectory, setting a weight value for each trajectory point on the predicted trajectory according to the order of action execution, wherein the trajectory point with an earlier action execution order has a larger weight value; for each predicted trajectory, calculating a matching index for characterizing the matching degree of the predicted trajectory based on the mean square error value and the corresponding weight value of each trajectory point on the predicted trajectory; and determining an optimal predicted trajectory based on the matching indexes of all predicted trajectories.

[0119] In a possible implementation manner of the present application, the controller 210 is further configured to: store the constructed multiple action combinations in the video memory 230, so as to perform an operation for determining the optimal predicted trajectory in the video memory 230.

[0120] In a possible implementation of the present application, the controller 210 is further configured to: store the target action combination corresponding to the optimal predicted trajectory into a memory, wherein the controller 210 extracts the first step action in the target action combination from the memory; and control the mechanical device to perform the first step action at the next moment.

[0121] In a possible implementation manner of the present application, the controller 210 is further configured to: based on the first step action performed at the next moment, continue to perform trajectory prediction for the mechanical device until the mechanical device is controlled to complete the movement of the target planned trajectory.

[0122] Based on the above equipment, the generalization of the algorithm can be improved and it is helpful to migrate between different mechanical equipment.

[0123] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the operation control method of the mechanical equipment in any of the above embodiments can be executed, which are as follows:

[0124] Acquire observation data of a mechanical device for trajectory tracking control, wherein the mechanical device includes a plurality of movable joints, and the observation data includes a joint position and a joint speed of each joint; search for a plurality of candidate action combinations corresponding to an executed action of the mechanical device at a current moment from a plurality of action combinations constructed for the mechanical device; determine a plurality of predicted trajectories of the mechanical device under the plurality of candidate action combinations based on the executed action at the current moment, the plurality of candidate action combinations, and the observation data; and control the mechanical device to perform an operation at the next moment according to a target action combination corresponding to an optimal predicted trajectory, wherein the optimal predicted trajectory is a predicted trajectory among the plurality of predicted trajectories that has the highest matching degree with a target planned trajectory.

[0125] Based on the above-mentioned computer-readable storage medium, the generalization of the algorithm can be improved and the migration between different mechanical devices can be facilitated.

[0126] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0127] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0129] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0130] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for controlling the operation of a mechanical device, characterized in that: The method comprises: Acquiring observation data of a mechanical device for trajectory tracking control, wherein the mechanical device includes a plurality of movable joints, and the observation data includes a joint position and a joint velocity of each joint; Searching for multiple candidate action combinations corresponding to the actions that have been executed by the mechanical device at the current moment from the multiple action combinations constructed for the mechanical device; Determine a plurality of predicted trajectories of the mechanical device under the plurality of candidate action combinations based on the executed actions at the current moment, the plurality of candidate action combinations, and the observation data; The mechanical equipment is controlled to perform operation according to the target action combination corresponding to the optimal predicted trajectory at the next moment, and the optimal predicted trajectory is a predicted trajectory with the highest matching degree with the target planned trajectory among the multiple predicted trajectories.

2. The method according to claim 1, characterized in that The observation data for trajectory tracking control includes observation data of the multiple joints at a current moment and at historical moments before the current moment.

3. The method according to claim 1, characterized in that The multiple action combinations are constructed in the following way: For each movable joint of the mechanical device, determining a multi-step single-joint action of the joint; Multiple single-joint actions of each joint are combined to form multiple action combinations, each action combination includes multiple actions with a preset number of steps, and each action is composed of a combination of single-joint actions of each joint under the number of steps.

4. The method according to claim 3, characterized in that The multi-step single-joint action of each joint includes the initial action and the follow-up action. The initial motion of each joint is determined as follows: Determine the first data point of the joint projection in the action space, The action within the neighborhood of the first data point is determined as the initial action of the joint.

5. The method according to claim 4, characterized in that The subsequent action of each joint is determined by: determining a motion slope, wherein the motion slope is a slope of a line segment from a first data point to a second data point where the initial motion is projected on a motion space of the joint; In the motion space of the joint, take the first data point as the starting point and extend it according to the motion slope to obtain the motion curve; Based on a third data point falling on the motion curve, a subsequent motion of the joint is determined.

6. The method according to claim 5, characterized in that Each joint of the mechanical device is driven by a hydraulic system, and the movement of the joint refers to the control amount of the hydraulic system used to drive the joint movement. The action space is represented by a two-dimensional coordinate system, the abscissa of the action space represents the number of steps, and the ordinate of the action space represents the control amount used to drive the joint movement.

7. The method according to claim 6, characterized in that The action curve is formed by: In the motion space of the joint, starting from the first data point, the motion is extended according to the motion slope, and the portion exceeding the motion range is limited to the boundary value of the motion range.

8. The method according to claim 1, characterized in that Each candidate action combination includes a plurality of actions predicted for the mechanical device to be performed after the action is performed at the current moment.

9. The method according to claim 1, characterized in that: The predicted trajectory corresponding to each candidate action combination is determined in the following way: Using a pre-trained world model, a predicted trajectory corresponding to the candidate action combination is obtained through multiple iterations, and the following processing is performed in each iteration: the executed action, the candidate action combination and the observation data are input into the world model to obtain a one-step predicted action for the mechanical equipment, wherein the predicted action output by the world model in the current iteration is used as the execution action in the next iteration, and the iteration is terminated when the number of steps of the predicted action output by the world model reaches a preset number of steps.

10. The method according to claim 1, characterized in that The optimal prediction trajectory is determined by: For each predicted trajectory, calculate the mean square error value of each trajectory point on the predicted trajectory relative to the target planned trajectory; For each predicted trajectory, a weight value is set for each trajectory point on the predicted trajectory according to the action execution order, wherein the trajectory point with an earlier action execution order has a larger weight value; For each predicted trajectory, based on the mean square error value and the corresponding weight value of each trajectory point on the predicted trajectory, a matching index used to characterize the matching degree of the predicted trajectory is calculated; According to the matching indexes of all predicted trajectories, an optimal predicted trajectory is determined.

11. The method according to claim 3, characterized in that Also includes: The constructed multiple action combinations are stored in a video memory, so that an operation for determining an optimal prediction trajectory is performed in the video memory.

12. The method according to claim 11, characterized in that Also includes: The target action combination corresponding to the optimal predicted trajectory is stored in the memory, The step of controlling the mechanical equipment to perform operation according to the target action combination corresponding to the optimal predicted trajectory at the next moment includes: Extracting the first action in the target action combination from the memory; Control the mechanical device to perform the first step action at the next moment.

13. The method according to claim 12, characterized in that Also includes: Based on the first step action executed at the next moment, the trajectory prediction for the mechanical device continues to be executed until the mechanical device is controlled to complete the movement of the target planned trajectory.

14. A mechanical device, characterized in that: Includes a controller, configured to: Acquiring observation data of a mechanical device for trajectory tracking control, wherein the mechanical device includes a plurality of movable joints, and the observation data includes a joint position and a joint velocity of each joint; Searching for multiple candidate action combinations corresponding to the actions that have been executed by the mechanical device at the current moment from the multiple action combinations constructed for the mechanical device; Determine a plurality of predicted trajectories of the mechanical device under the plurality of candidate action combinations based on the executed actions at the current moment, the plurality of candidate action combinations, and the observation data; The mechanical equipment is controlled to perform operation according to the target action combination corresponding to the optimal predicted trajectory at the next moment, and the optimal predicted trajectory is a predicted trajectory with the highest matching degree with the target planned trajectory among the multiple predicted trajectories.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are executed.

Images