Communication method, device and communication middleware for multi-robot collaborative operation

Abstract

The present application relates to the field of robot communication technology, and discloses a communication method for multi-robot collaborative operation. The communication method includes: obtaining the first state parameter and the first timestamp of the robot collected by each controller; the first state parameter and the first timestamp correspond one to one; synchronizing the timestamps of the multiple first state parameters and the first timestamp to obtain multiple second state parameters with the same timestamp; encapsulating the multiple second state parameters into a state parameter frame, and sending the state parameter frame to the upper software; obtaining the control instruction frame corresponding to the state parameter frame through the upper software; according to the control instruction frame, feeding back the corresponding control instruction to each controller, so that each controller can control its corresponding robot. The use of this communication method can improve the collaborative operation effect of different types of robots. The present application also discloses a communication device and communication middleware for multi-robot collaborative operation.

Description

Technical Field

[0001] The present application relates to the field of robot communication technology, for example, to a communication method, device and communication middleware for multi-robot collaborative operation. Background Art

[0002] At present, multiple robots have the ability to work together, such as multiple handling robots working together, or rotorcraft clusters working together. During the collaborative operation of robots, the controllers of each robot are usually connected to a central controller, which coordinates multiple robots to enable them to work together.

[0003] In the process of implementing the embodiments of the present application, it is found that there are at least the following problems in the related technology:

[0004] The robots of the collaborative handling robots or collaborative rotorcraft are of the same type, and each robot controller has the same sampling frequency for collecting its own state parameters (such as motion state, force state, etc.). After the time of each robot is synchronized, the central controller can obtain the state parameters of each robot at the same time, and coordinate the robots accordingly.

[0005] When different types of robots work together, each robot controller has a different sampling frequency for collecting its own state parameters. Even if each robot achieves time synchronization, the state parameters of each robot obtained by the central controller are unlikely to be at the same time. The central controller coordinates each robot based on its state parameters at different times, resulting in poor coordination of the robots, which in turn leads to poor collaborative working effects of different types of robots. Summary of the invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] The embodiments of the present application provide a communication method, device and communication middleware for multi-robot collaborative operation to synchronize the state parameters of different types of robots into state parameters at the same time, thereby improving the coordination effect of each robot and improving the collaborative operation effect of different types of robots.

[0008] In some embodiments, each robot corresponds to a controller, and multiple robots collaborate to complete a motion scene. The communication method for multi-robot collaborative operation includes: obtaining the first state parameter and first timestamp of the robot collected by each controller; the first state parameter and the first timestamp correspond one-to-one; performing timestamp synchronization on multiple first state parameters and first timestamps to obtain multiple second state parameters with the same timestamp; encapsulating multiple second state parameters into a state parameter frame, and sending the state parameter frame to the upper-level software; obtaining a control instruction frame corresponding to the state parameter frame through the upper-level software; and according to the control instruction frame, feeding back the corresponding control instruction to each controller, so that each controller can control its corresponding robot.

[0009] Optionally, timestamp synchronization is performed on multiple first state parameters and first timestamps, including: determining the data collection moment according to a preset sampling frequency; determining the second state parameters of the robot corresponding to the one controller at the data collection moment according to the historical state parameters and the first historical timestamp sent by the one controller before the data collection moment; wherein the historical state parameters include the first state parameters, the first historical timestamp includes the first timestamp, and the first timestamp is the timestamp sent by the one controller that is closest to the data collection moment; the one controller is any one of the multiple controllers.

[0010] Optionally, based on historical state parameters and a first historical timestamp sent by a controller before the data collection moment, the second state parameters of the robot corresponding to the controller at the data collection moment are determined, including: when the preset sampling frequency is the same as the sampling frequency at which the controller collects state parameters of its corresponding robot, the first state parameters sent by the controller are used as the second state parameters at the data collection moment.

[0011] Alternatively, based on the historical state parameters and the first historical timestamp sent by a controller before the data collection moment, determining the second state parameters of the robot corresponding to the controller at the data collection moment, including: when the preset sampling frequency is the same as the sampling frequency at which the controller collects the state parameters of its corresponding robot, obtaining the first fitting curve or the first neural network model corresponding to the historical state parameters and the historical timestamp sent by the controller before the data collection moment, substituting the data collection moment into the first fitting curve or the first neural network model, and obtaining the second state parameters of the data collection moment output by the first fitting curve or the first neural network model; wherein the first fitting curve or the first neural network model is used to represent the correspondence between the timestamp and the state parameter.

[0012] Alternatively, based on the historical state parameters and historical timestamps sent by a controller before the data collection moment, determining the second state parameters of the robot corresponding to the controller at the data collection moment, including: when the preset sampling frequency is different from the sampling frequency at which the controller collects the state parameters of its corresponding robot, obtaining a first fitting curve or a first neural network model corresponding to the historical state parameters and historical timestamps sent by the controller before the data collection moment, substituting the data collection moment into the first fitting curve or the first neural network model, and obtaining the second state parameters of the data collection moment output by the first fitting curve or the first neural network model; wherein the first fitting curve or the first neural network model is used to represent the correspondence between timestamps and state parameters.

[0013] Optionally, according to the control instruction frame, the corresponding control instruction is fed back to each controller, including: obtaining a preset control frequency, and determining the control moment of multiple robots according to the preset control frequency; the preset control frequency is less than or equal to a preset sampling frequency; according to the historical control instructions and the second historical timestamp obtained by the upper-level software before the control moment, multiple first control instructions corresponding to the control moment are determined, and each first control instruction is sent to the corresponding controller, so that each controller can control its corresponding robot; wherein the historical control instructions include the second control instruction in the control instruction frame, the second historical timestamp includes the second timestamp corresponding to the second control instruction, and the second timestamp is the timestamp sent by the upper-level software that is closest to the control moment.

[0014] Optionally, determining multiple first control instructions corresponding to the control moment based on historical control instructions and a second historical timestamp obtained by the upper-layer software before the control moment includes: using the second control instruction in the control instruction frame as the first control instruction at the control moment.

[0015] Alternatively, based on the historical control instructions and the second historical timestamp obtained by the upper-layer software before the control moment, multiple first control instructions corresponding to the control moment are determined, including: obtaining a second fitting curve or a second neural network model corresponding to the historical control instructions and the second historical timestamp obtained before the control moment, substituting the control moment into the second fitting curve or the second neural network model, and obtaining the first control instruction at the control moment output by the second fitting curve or the second neural network model; wherein the second fitting curve or the second neural network is used to represent the correspondence between the timestamp and the control instruction.

[0016] Optionally, each first control instruction is sent to a corresponding controller, including: obtaining multiple groups of first state parameters and first control instructions with corresponding relationships related to the one controller within a second set time length; calculating the movement direction of the robot corresponding to the one controller according to each group of first state parameters and first control instructions; if the two movement directions of the robot calculated by two adjacent groups of first state parameters and first control instructions of the first timestamp are different, determining that the robot changes its movement direction once; if the number of times the robot's movement direction changes within the second set time length is less than or equal to a set number threshold, determining that the first control instruction corresponding to the one controller is a non-easy-to-jitter instruction, and sending the first control instruction to the one controller.

[0017] Optionally, sending each first control instruction to the corresponding controller also includes: if the first control instruction to be sent causes the robot to change its movement direction once, and the number of times the robot's movement direction changes within the second set time period is more than the set number threshold, then determining that the first control instruction corresponding to the one controller is an easy-to-jitter instruction; re-determining the first control instruction corresponding to the one controller based on the historical control instructions and the second historical timestamp obtained before the control moment, or, using a stationary instruction as the first control instruction corresponding to the one controller, and sending the first control instruction to the one controller.

[0018] Optionally, multiple first control instructions corresponding to the control moment are determined based on historical control instructions and second historical timestamps obtained by the upper-layer software before the control moment, including: determining the short-term delay duration of the communication link corresponding to the one controller based on historical status parameters and first historical timestamps sent by the one controller within a first set time period before the sending moment; the sending moment is the moment when the first control instruction is sent to the one controller; if the short-term delay duration is shorter than the average delay duration of the communication links corresponding to all controllers, then determining the first control instruction corresponding to the one controller at the moment before the control moment based on the historical control instructions and second historical timestamps obtained before the control moment; if the short-term delay duration is longer than the average delay duration of the communication links corresponding to all controllers, then determining the first control instruction corresponding to the one controller at the moment after the control moment based on the historical control instructions and second historical timestamps obtained before the control moment.

[0019] In some embodiments, each robot corresponds to a controller, and multiple robots collaborate to complete a motion scene. The communication device for multi-robot collaborative operation includes a first communication module, a synchronization module, a second communication module, a third communication module, and a fourth communication module.

[0020] The first communication module is used to obtain the first state parameter and the first timestamp of the robot collected by each controller; the first state parameter and the first timestamp correspond one to one; the synchronization module is used to synchronize the timestamps of multiple first state parameters and the first timestamp to obtain multiple second state parameters with the same timestamps; the second communication module is used to encapsulate the multiple second state parameters into a state parameter frame, and send the state parameter frame to the upper-level software; the third communication module is used to obtain the control instruction frame corresponding to the state parameter frame through the upper-level software; the fourth communication module is used to feedback the corresponding control instruction to each controller according to the control instruction frame, so that each controller can control its corresponding robot.

[0021] In some embodiments, the communication middleware for multi-robot collaborative work includes a processor and a memory storing program instructions, and the processor is configured to execute the communication method for multi-robot collaborative work provided by the above embodiments when executing the program instructions.

[0022] The communication method, device, and communication middleware for robot collaborative operation provided in the embodiments of the present application can achieve the following technical effects:

[0023] After obtaining the state parameters and timestamps of the robots sent by different controllers, multiple state parameters and timestamps are synchronized. The timestamps of the synchronized state parameters are the same. Multiple state parameters with the same timestamp can represent the state of each robot at the same time. The upper-level software (or central controller) coordinates each robot based on such state parameters to obtain better coordination effect, thereby improving the collaborative operation effect of different types of robots.

[0024] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] One or more embodiments are exemplarily described by corresponding drawings, and these exemplary descriptions and drawings do not limit the embodiments. Elements with the same reference numerals in the drawings are regarded as similar elements, and wherein:

[0026] Figure 1 This is a schematic diagram of the communication relationship of multiple robots working together provided by an embodiment of the present application;

[0027] Figure 2 It is a flowchart of a communication method for multi-robot collaborative operation provided by an embodiment of the present application;

[0028] Figure 3 It is a flowchart of a communication method for multi-robot collaborative operation provided by an embodiment of the present application;

[0029] Figure 4 This is a schematic diagram of the distribution of data collection time and control time on the time axis provided by an embodiment of the present application;

[0030] Figure 5 It is a flowchart of a communication method for multi-robot collaborative operation provided by an embodiment of the present application;

[0031] Figure 6 is a schematic diagram of a communication device for multi-robot collaborative operation provided by an embodiment of the present application;

[0032] Figure 7 It is a schematic diagram of a communication device for multi-robot collaborative operation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to be able to understand the features and technical contents of the embodiments of the present application in more detail, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, in the absence of these details, one or more embodiments can still be implemented. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0034] The terms "first", "second", etc. in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present application described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0035] Unless otherwise stated, the term "plurality" means more than two.

[0036] In the embodiment of the present application, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0037] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0038] Figure 1 This is a schematic diagram of the communication relationship for collaborative operation of multiple robots provided in an embodiment of the present application.

[0039] Combination Figure 1As shown, the multi-robot collaborative operation system is a virtual motion system, and users can use the virtual motion system to complete running and other sports. The virtual motion system includes a six-axis platform robot 11 and an exoskeleton robot 12, the six-axis platform robot 11 corresponds to a controller 13, and the exoskeleton robot 12 corresponds to a controller 14. The controller 13 and the controller 14 are both connected to a universal communication interface (UCI) 15, and the UCI 15 is connected to the upper software 16. The upper software 16 includes the relevant parameters of the motion scene, and based on the relevant parameters of the motion scene, the coordination state of each robot is coordinated. The controller 13 can control the action of the six-axis platform robot 11, and the controller 14 can control the action of the exoskeleton robot 12. The virtual motion system can provide users with virtual motion scenes, and the six-axis platform robot 11 and the exoskeleton robot 12 can feedback the force state and motion state in the virtual motion scene to the user, providing users with a virtual motion experience.

[0040] The virtual sports system also includes some peripheral supporting equipment 17 and corresponding other sensors 18, such as relevant sensors for detecting the user's motion state or physiological parameters, and relevant supporting equipment for providing a virtual environment.

[0041] The upper-layer software 16 can be implemented in a terminal, for example, it can be implemented in a control terminal of the virtual motion system, and the control terminal can be fixedly mounted on the six-axis platform robot 11, or fixedly mounted on the exoskeleton robot 12, or fixedly mounted on other support frames, and can also be a mobile control terminal; alternatively, the upper-layer software 17 can be implemented in a server, for example, a user can set relevant parameters in the server in the control terminal, and then the server runs the upper-layer software 16.

[0042] The six-axis platform robot 11 may be a universal treadmill, with the help of which the user may realize multi-directional running motion.

[0043] In an embodiment of the present application, the state parameters of the robots collected by the controllers of different robots at different sampling frequencies are synchronized into state parameters at the same moment (same timestamp), and these state parameters are sent to the upper-level software, so that the upper-level software coordinates multiple robots based on the state parameters of different robots at the same moment, thereby improving the collaborative operation effect between multiple robots.

[0044] In an embodiment of the present application, the time of each robot, UCI and upper-layer software is synchronized.

[0045] Figure 2It is a flow chart of a communication method for multi-robot collaborative operation provided by an embodiment of the present application. The communication method for multi-robot collaborative operation can be implemented in communication middleware, such as UCI, and can also be implemented in other software, hardware, or a combination of the two similar to communication middleware. In the application scenario of the communication method, there are multiple robots, each robot corresponds to a controller, and multiple robots collaborate to complete a sports scene, such as a running sports scene.

[0046] Combination Figure 2 As shown, the communication method for multi-robot collaborative operation includes:

[0047] S201. Obtain a first state parameter and a first timestamp of the robot collected by each controller.

[0048] The first state parameter and the first timestamp correspond one to one.

[0049] The robot itself is equipped with multiple sensors. For example, the exoskeleton robot is equipped with sensors for detecting the angle data and force data of each joint of the robot. Multiple sensors collect state parameters such as the angle data and force data of each joint of the robot at a certain sampling frequency. Multiple sensors are connected to the robot's controller. Each sensor uploads its collected state parameters to the controller. The controller controls the robot according to the robot's state parameters to enable the robot to complete the expected action.

[0050] After receiving the state parameters of the robot collected by the sensor, the controller adds a timestamp to each state parameter; or, encapsulates all state parameters and then adds a timestamp. The timestamp can be used to indicate the time when the state parameter occurs.

[0051] The first state parameter is a state parameter collected by the robot controller, and the first timestamp is a timestamp added by the robot controller to obtain the state parameter of the robot. There are multiple robots, multiple controllers, and multiple first state parameters and first timestamps.

[0052] The first state parameter and the first timestamp correspond one-to-one, which means that the state parameter collected by a controller and the timestamp added to the state parameter correspond one-to-one. It can be that each state parameter collected by a sensor set in the robot corresponds to a timestamp, or it can be that the state parameters collected by all sensors set in the robot are encapsulated as a whole and correspond to a timestamp.

[0053] In the embodiment of the present application, UCI is used as a carrier for implementing the communication method for multi-robot collaborative operation, and the communication method for multi-robot collaborative operation is exemplarily described.

[0054] After obtaining the first state parameter and the first timestamp, each controller sends the first state parameter and the first timestamp to the UCI, and the UCI can obtain the first state parameter and the first timestamp of the robot collected by each controller.

[0055] The controllers of different robots have different sampling frequencies for collecting the state parameters of the robots through sensors, and the first timestamps of the first state parameters of the robots collected by the controllers are different.

[0056] S202: Perform timestamp synchronization on a plurality of first state parameters and a first timestamp to obtain a plurality of second state parameters with the same timestamp.

[0057] Multiple second state parameters with the same timestamp can be used to represent the state parameters of multiple robots at the same moment.

[0058] S203: Encapsulate the plurality of second status parameters into a status parameter frame, and send the status parameter frame to the upper-layer software.

[0059] The state parameter frame may include a check value to ensure the integrity of the state parameter frame. Those skilled in the art may select a suitable data check algorithm according to actual needs, which will not be described in detail here.

[0060] S204. Obtain a control instruction frame corresponding to the state parameter frame through upper-layer software.

[0061] The state parameter frame includes a plurality of second state parameters and a robot or controller corresponding to each second state parameter; the control instruction frame includes a plurality of control instructions and a robot or controller corresponding to each control instruction.

[0062] The upper layer software can coordinate multiple robots to complete collaborative work. Those skilled in the art can select appropriate upper layer software according to actual needs.

[0063] S205. Feedback corresponding control instructions to each controller according to the control instruction frame, so that each controller can control its corresponding robot.

[0064] After obtaining the state parameters and timestamps of the robots sent by different controllers, multiple state parameters and timestamps are synchronized. The timestamps of the synchronized state parameters are the same. Multiple state parameters with the same timestamp can represent the state of each robot at the same time. The upper-level software (or central controller) coordinates each robot based on such state parameters to obtain better coordination effect, thereby improving the collaborative operation effect of different types of robots.

[0065] In addition, compared with the scenarios of multiple handling robots working together and the scenarios of rotorcraft clusters working together, in the sports scenarios of multiple robots working together, the coordination requirements for the real-time actions of each robot are relatively high. Each robot needs to work together in real time to smoothly complete the sports scene. Otherwise, the sports scene will be stuck and not smooth, resulting in a poor user experience.

[0066] By adopting the scheme in the embodiment of the present application, multiple first state parameters and first timestamps are timestamped and synchronized to obtain multiple second state parameters with the same timestamp. The upper-level software then coordinates the multiple robots based on the multiple second state parameters with the same timestamp. This can achieve the goal of losing a certain amount of accuracy (a certain amount of accuracy will be lost in the process of synchronizing the multiple first state parameters and the first timestamp, and technical personnel in this field need to ensure that the accuracy lost in the timestamp synchronization process meets the requirements of robot collaborative work) so that the upper-level software can match the collaborative state between the robots at the next moment based on the state parameters of the robots at the same moment, and obtain control instructions corresponding to each robot / controller. The control instructions obtained in this way are conducive to the smooth transition of the collaborative state of the multi-robot system from the current moment to the next moment, thereby improving the fluency of the motion scene and the user experience.

[0067] The following is an exemplary description of the timestamp synchronization process.

[0068] Optionally, timestamp synchronization is performed on multiple first state parameters and first timestamps, including: determining the data collection time according to a preset sampling frequency; determining the second state parameters of a robot corresponding to a controller at the data collection time according to historical state parameters and the first historical timestamp sent by a controller before the data collection time; wherein the historical state parameters include the first state parameters, the first historical timestamp includes the first timestamp, and the first timestamp is a timestamp sent by a controller that is closest to the data collection time; a controller is any one of multiple controllers.

[0069] The preset sampling frequency is greater than or equal to the minimum sampling frequency that ensures that the motion scene can be smoothly realized. In the motion scene, the faster the movement speed of each robot, the higher the minimum sampling frequency that ensures that the motion scene can be smoothly realized; in the motion scene, the smaller the value representing the accuracy of each robot, the higher the minimum sampling frequency that ensures that the motion scene can be smoothly realized; in the motion scene, the more complex the movement of each robot, the higher the minimum sampling frequency that ensures that the motion scene can be smoothly realized. Those skilled in the art can determine the preset sampling frequency that conforms to the actual motion scene according to the actual motion scene.

[0070] The data collection time refers to the time determined by the preset sampling frequency and the last data collection time. For example, the time interval (sampling period) between the last data collection time and the current data collection time is calculated by the preset sampling frequency, and the current data collection time is determined by the sum of the last data collection time and the time interval.

[0071] The above historical state parameters and the first historical timestamp are in one-to-one correspondence. The above historical state parameters and the first historical timestamp may be multiple state parameters and timestamps sent by each controller before the data collection moment, and having the smallest time interval with the data collection moment. The smallest time interval with the data collection moment refers to any other state parameter and timestamp sent by each controller before the data collection moment, whose time interval with the data collection moment is longer than the time interval between any timestamp in the first historical timestamp and the data collection moment.

[0072] The above-mentioned historical state parameters and first historical timestamps may be state parameters and timestamps selected at intervals from a plurality of state parameters and timestamps sent by each controller before the data collection moment in the order of the timestamps from the last to the first.

[0073] In the embodiment of the present application, the term "first state parameter" refers to a state parameter sent by any controller without timestamp synchronization, and the term "second state parameter" refers to a state parameter that has been timestamp synchronized.

[0074] Furthermore, determining a second state parameter of a robot corresponding to a controller at the data collection moment based on historical state parameters and a first historical timestamp sent by a controller before the data collection moment may include: when a preset sampling frequency is the same as a sampling frequency at which a controller collects state parameters of its corresponding robot, using the first state parameter sent by a controller as the second state parameter at the data collection moment.

[0075] For example, if the sampling frequencies of the state parameters of the robots collected by the controllers of multiple robots through sensors are in multiple relationship, the lower sampling frequency can be used as the preset sampling frequency; or, if the sampling frequencies of the state parameters of the robots collected by the controllers of multiple robots through sensors are not in multiple relationship, any one of the sampling frequencies can be used as the preset sampling frequency. If the sampling frequency of the state parameters collected by a controller for its corresponding robot is the same as the preset sampling frequency, and the first timestamp obtained by the controller coincides with the data collection time, the first state parameter sent by the controller can be used as the second state parameter at the data collection time.

[0076] Even if the sampling frequency of a controller for collecting the state parameters of its corresponding robot is the same as the preset sampling frequency, but the first timestamp obtained by the controller does not coincide with the data collection moment, it is still necessary to perform timestamp synchronization on the first state parameter and the first timestamp sent by the controller.

[0077] At this time, the second state parameters of a robot corresponding to a controller at the data collection moment can be determined in the following way: when the preset sampling frequency is the same as the sampling frequency at which a controller collects the state parameters of its corresponding robot, the historical state parameters sent by a controller before the data collection moment and the first fitting curve or the first neural network model corresponding to the first historical timestamp are obtained, the data collection moment is substituted into the first fitting curve or the first neural network model, and the second state parameters at the data collection moment output by the first fitting curve or the first neural network model are obtained; wherein the first fitting curve or the first neural network model is used to represent the correspondence between the timestamp and the state parameters.

[0078] The first fitting curve or the first neural network model corresponding to the historical state parameters and the first historical timestamp sent by a controller before the data collection moment refers to the first fitting curve obtained by data fitting the historical state parameters and the first historical timestamp sent by a controller before the data collection moment, or the first neural network model obtained by training the first preset neural network model using the historical state parameters and the first historical timestamp sent by a controller before the data collection moment. Those skilled in the art select the data fitting method or the first preset neural network that meets the accuracy requirements of the actual motion scene based on experience.

[0079] The above-mentioned embodiment mentioned that the preset sampling frequency is greater than or equal to the minimum sampling frequency that ensures that the motion scene can be smoothly realized, that is, the preset sampling frequency is related to the motion scene, and the sampling frequency at which the controller of each robot collects the state parameters of its corresponding robot is related to the characteristics of the robot itself. In the case of different types of robots, the sampling frequency at which the controller of each robot collects the state parameters of its corresponding robot is difficult to be the same. Therefore, the phenomenon that the preset sampling frequency is different from the sampling frequency at which the controller collects the state parameters of its corresponding robot occurs more frequently.

[0080] In this case, the second state parameters of a robot corresponding to a controller at the data collection time can be determined in the following manner, including: when a preset sampling frequency is different from the sampling frequency at which a controller collects the state parameters of its corresponding robot, obtaining a first fitting curve or a first neural network model corresponding to a historical state parameter sent by a controller before the data collection time and a first historical timestamp, substituting the data collection time into the first fitting curve or the first neural network model, and obtaining the second state parameter at the data collection time output by the first fitting curve or the first neural network model; wherein the first fitting curve or the first neural network model is used to represent the correspondence between the timestamp and the state parameter.

[0081] Similarly, the first fitting curve or the first neural network model corresponding to the historical state parameters and the first historical timestamp sent by a controller before the data collection moment refers to the first fitting curve obtained by data fitting the historical state parameters and the first historical timestamp sent by a controller before the data collection moment, or the first neural network model obtained by training the first preset neural network model using the historical state parameters and the first historical timestamp sent by a controller before the data collection moment. Those skilled in the art select the data fitting method or the first preset neural network that meets the accuracy requirements of the actual motion scene based on experience.

[0082] Using the timestamp synchronization scheme provided by the above embodiment, multiple second state parameters of timestamp synchronization can be obtained. UCI encapsulates the multiple second state parameters into a state parameter frame and sends it to the upper software, and the upper software can obtain the control instruction frame corresponding to the state parameter frame. The state parameter frame reflects the state parameters of the whole formed by each robot, and the state parameters of the motion scene formed at a certain moment during the collaborative operation. For example, the upper software stores the state parameters of the motion scene at each moment, and can match the state parameters of the whole formed by each robot at the next moment of the motion scene, and calculate the control instructions corresponding to each robot based on the state parameter difference of the state parameter of the whole formed by each robot at a certain moment and the state parameter difference of the state parameter of the whole formed by each robot at the next moment. Those skilled in the art can choose to calculate the calculation method of the control instructions corresponding to each robot according to actual needs.

[0083] Of course, the above-listed methods for upper-layer software to obtain control instruction frames corresponding to state parameter frames are merely illustrative, and those skilled in the art may select a suitable method to determine control instruction frames corresponding to state parameter frames according to the requirements of application scenarios.

[0084] The control instruction frame contains multiple control instructions and the robot or controller corresponding to each control instruction. After the UCI obtains the control instruction frame, it can directly send the control instruction to the corresponding robot or controller based on the correspondence between the control instruction and the robot or controller.

[0085] Furthermore, corresponding control instructions may be fed back to each controller according to a preset control frequency.

[0086] Figure 3 It is a flow chart of a communication method for multi-robot collaborative operation provided in an embodiment of the present application, which is used to exemplify the process of feeding back corresponding control instructions to each controller according to a preset control frequency.

[0087] Combination Figure 3 As shown, the communication method for multi-robot collaborative operation includes:

[0088] S301. Obtain a first state parameter and a first timestamp of the robot collected by each controller.

[0089] The first state parameter and the first timestamp correspond one to one.

[0090] S302: Determine the data collection time according to a preset sampling frequency.

[0091] S303: Determine a second state parameter of a robot corresponding to a controller at the data collection time according to a historical state parameter and a first historical timestamp sent by a controller before the data collection time.

[0092] The historical state parameter includes a first state parameter, the first historical timestamp includes a first timestamp, the first timestamp is a timestamp sent by a controller that is closest to the data collection time, and a controller is any one of multiple controllers.

[0093] S304: Encapsulate the multiple second status parameters into a status parameter frame, and send the status parameter frame to the upper-layer software.

[0094] S305. Obtain a control instruction frame corresponding to the state parameter frame through upper-layer software.

[0095] S306: Obtain a preset control frequency, and determine the control time of the multiple robots according to the preset control frequency.

[0096] The preset control frequency is less than or equal to the preset sampling frequency.

[0097] The control frequency refers to the frequency at which the upper-level software sends control instructions to the controller.

[0098] The preset control frequency is greater than or equal to the minimum control frequency that ensures that the motion scene can appear smoothly. In the motion scene, the faster the movement speed of each robot, the higher the minimum control frequency that ensures that the motion scene can be smoothly realized; in the motion scene, the smaller the value representing the accuracy of each robot (the higher the accuracy), the higher the minimum control frequency that ensures that the motion scene can be smoothly realized; in the motion scene, the more complex the movement of each robot, the higher the minimum control frequency that ensures that the motion scene can be smoothly realized.

[0099] S307. Determine multiple first control instructions corresponding to the control moment based on the historical control instructions and the second historical timestamp obtained by the upper-level software before the control moment, and send each first control instruction to the corresponding controller, so that each controller can control its corresponding robot.

[0100] The historical control instruction includes the second control instruction in the control instruction frame, the second historical timestamp includes the second timestamp corresponding to the second control instruction, and the second timestamp is the timestamp sent by the upper-layer software and is closest to the control time.

[0101] In an embodiment of the present application, UCI performs timestamp synchronization on the first state parameter and the first timestamp sent by the controller, so that the upper-level software can obtain the state parameters of each robot at the same time, and generate a control instruction frame that can better coordinate each robot based on the state parameters of each robot at the same time. Afterwards, UCI performs timestamp synchronization on the second control instruction and the second timestamp received in the control instruction frame, and synchronizes the second control instruction and the second timestamp to the first control instruction corresponding to the control moment, so that the control instruction can reach the controller or robot at a relatively fixed frequency.

[0102] The relatively fixed frequency is related to the preset control frequency. When the communication delay between UCI and each controller is not considered, the relatively fixed frequency is the same as the preset control frequency. When the communication delay between UCI and the controller is considered, there is delay jitter in the process of obtaining the first state parameter and the first timestamp corresponding to each controller, that is, there is jitter in the communication delay between UCI and each controller, the delay size will change, and the relatively fixed frequency fluctuates above and below the preset control frequency.

[0103] The situation that the preset control frequency is less than or equal to the preset sampling frequency can be divided into two types: the preset control frequency is lower than the preset sampling frequency, and the preset control frequency is equal to the preset sampling frequency.

[0104] When the preset control frequency is lower than the preset sampling frequency, the robot is either not adjusted by the first control instruction or has been adjusted by the first control instruction between two adjacent data collection moments. When the robot is not adjusted by the first control instruction between two adjacent data collection moments, the second state parameter of the controller at the data collection moment determined by the UCI based on the historical state parameter sent by the controller before the data collection moment and the first historical timestamp is relatively accurate; when the robot has been adjusted by the first control instruction between two adjacent data collection moments, the accuracy of the second state parameter of the controller at the data collection moment determined by the UCI based on the historical state parameter sent by the controller before the data collection moment and the first historical timestamp is reduced.

[0105] That is, in the process of determining the second state parameters of a robot corresponding to a controller at the data collection time based on the historical state parameters and the first historical timestamp sent by a controller before the data collection time, the robot is adjusted by the first control instruction, which disturbs the free motion trajectory of the robot, causing the first state parameters before and after the disturbed moment to fail to conform to the simple motion model, thereby increasing the complexity of the actual motion model constituted by the historical state parameters and the first historical timestamp, and easily resulting in a reduction in the accuracy of the determined second state parameters when determining the second state parameters.

[0106] Controlling the robot according to a relatively fixed frequency can evenly distribute the time nodes when the robot's free motion trajectory is disturbed, reduce the complexity of the actual motion model composed of the historical state parameters and the first historical timestamp to a certain extent, and easily improve the accuracy of the determined second state parameters when determining the second state parameters.

[0107] The moment when the robot is adjusted by the first control instruction is likely to result in a slightly lower accuracy of the second state parameter determined later; the more historical state parameters collected before the robot obtains the second state parameter, the more likely it is that the second state parameter obtained by the robot will have a higher accuracy. This makes it easy for the accuracy of the second state parameter at multiple data collection moments to fluctuate according to a relatively stable regular cycle, so that the second state parameters at multiple data collection moments can present the motion state of each robot more stably as a whole.

[0108] Combination Figure 4 As shown, the data collection moments are COL1, COL2, COL3, COL4, COL5 and COL6 respectively; the moments when the controller controls the robot are CON1, CON2, CON3 and CON4 respectively.

[0109] After the control moment CON1, the robot's free motion state is disturbed, and the actual motion model formed by the state parameters and timestamps obtained thereafter and before the data collection moment COL2 is relatively complex, and the accuracy of the second state parameter obtained at the data collection moment COL2 is relatively poor. After the data collection moment COL2 and before the data collection moment COL3, the robot's self-use motion state is not disturbed, and the actual motion model formed by the state parameters and timestamps obtained during this period is relatively simple, and the accuracy of the second state parameter obtained at the data collection moment COL3 is relatively high; similarly, the accuracy of the second state parameter obtained at the data collection moment COL4 is relatively low, the accuracy of the second state parameter obtained at the data collection moment COL5 is relatively low, and the accuracy of the second state parameter obtained at the data collection moment COL6 is relatively high. In this way, the accuracy of the second state parameters at multiple data collection moments fluctuates according to a relatively stable regular period, so that the second state parameters at multiple data collection moments present the motion state of each robot relatively stably as a whole.

[0110] In addition, the accuracy of the second state parameters at multiple data collection moments fluctuates according to a relatively stable regular period, and the degree of matching between the control instruction frame (or the control instructions of each robot) obtained based on the second state parameters at each data collection moment and the actual state parameters of each robot also fluctuates according to a relatively stable regular period, making the robot control error more evenly distributed. Compared with using a non-relatively fixed frequency to control the robot, using a relatively fixed frequency to control the robot can enable the robot to move better as expected.

[0111] When the preset control frequency is equal to the preset sampling frequency, the robot is adjusted by the first control instruction between any two adjacent data collection moments. The UCI then determines the second state parameters of a controller at the data collection moment based on the historical state parameters sent by the controller before the data collection moment and the first historical timestamp. The accuracy of each second state parameter obtained is equivalent.

[0112] The accuracy of the second state parameters at each data collection moment is comparable, and the degree of matching between the control instruction frame (or the control instructions of each robot) obtained based on the second state parameters at each data collection moment and the actual state parameters of each robot is also comparable, so that the control error of the robot is evenly distributed.

[0113] The second state parameter with relatively stable accuracy and the control instruction frame with a relatively stable degree of matching with the actual state parameters of each robot can reduce the factors that affect the accuracy of each robot working together, especially in motion scenarios with relatively fast movement speeds and high requirements for the coordination of the movements of multiple robots. There are fewer factors that affect the accuracy of each robot working together, and technical personnel in this field can easily adjust the movement accuracy of each robot to present a more coordinated motion scene.

[0114] The following is an exemplary description of determining a plurality of first control instructions corresponding to a control moment according to the historical control instructions and the second historical timestamp.

[0115] Optionally, determining multiple first control instructions corresponding to the control moment according to historical control instructions obtained by upper-layer software before the control moment and the second historical timestamp includes: using the second control instruction in the control instruction frame as the first control instruction at the control moment.

[0116] The term "second control instruction" in the embodiment of the present application refers to a control instruction directly obtained by UCI through upper-level software and not adjusted or confirmed by UCI; the term "first control instruction" in the embodiment of the present application refers to a control instruction at the control moment obtained by UCI after obtaining a control instruction frame through upper-level software and then performing certain transformations, adjustments or confirmations to maintain the control instructions in the control instruction frame unchanged.

[0117] When the second timestamp in the control instruction frame is the same as the control moment or the difference between the two satisfies the ignore condition, the second control instruction in the control instruction frame can be used as the first control instruction at the control moment. The difference between the second timestamp and the control moment satisfies the ignore condition, which means that the difference between the two is less than or equal to the set ignore threshold. The higher the preset control frequency, the smaller the set ignore threshold, and the lower the preset control frequency, the larger the set ignore threshold can be. Those skilled in the art can determine the set ignore threshold that meets the conditions according to the preset control frequency.

[0118] When the difference between the second timestamp in the control instruction frame and the control moment does not meet the ignoring condition, the first control instruction at multiple moments can be determined in the following manner: obtain the second fitting curve or second neural network model corresponding to the historical control instructions obtained before the control moment and the second historical timestamp, substitute the control moment into the second fitting curve or the second neural network model, and obtain the first control instruction at the control moment output by the second fitting curve or the second neural network model; wherein the second fitting curve or the second neural network is used to represent the correspondence between the timestamp and the control instruction.

[0119] The second fitting curve or second neural network model corresponding to the historical control instructions and the second historical timestamp obtained before the control moment refers to the second fitting curve obtained by data fitting the historical control instructions and the second historical timestamp obtained before the control moment, or the second neural network model obtained by training the second preset neural network model using the historical control instructions and the second historical timestamp obtained before the control moment. Those skilled in the art can select a data fitting method or a second preset neural network that meets the accuracy requirements of the actual motion scene based on experience. Here, the second preset neural network can be the same as the aforementioned first preset neural network, or it can be different from the aforementioned first preset neural network.

[0120] In a specific application scenario, there is a certain delay in the communication process between UCI and the upper-layer software, and during data transmission, there is jitter in the communication delay between UCI and the upper-layer software, that is, the delay size will change.

[0121] When the robot's movement speed approaches zero, the jitter in the communication delay between the UCI and the upper-layer software will cause the robot to shake, which is manifested as the robot vibrating in the original position. This kind of robot jitter is likely to bring a bad experience to users. That is, there is a delay jitter in the process of obtaining the control command frame through the upper-layer software.

[0122] In the above technical scheme, even if the upper-level software sends control instruction frames to UCI according to the preset control frequency, due to the jitter in the communication delay between UCI and the upper-level software, the frequency at which UCI obtains the control instruction frames jitters around the preset control frequency. By calculating the first control instruction at the control moment, the jitter of the robot's control frequency can be reduced or even eliminated, thereby reducing or even avoiding the phenomenon of robot jitter.

[0123] In addition, the aforementioned embodiment mentioned that there is also jitter in the communication delay between UCI and each controller (there is delay jitter in the process of obtaining the first state parameter and the first timestamp corresponding to each controller). The aforementioned embodiment provides a technical solution for obtaining the second state parameter using the first state parameter and the first timestamp, which can also reduce or even eliminate the adverse effects caused by the jitter in the communication delay between UCI and each controller, so that the multiple second state parameters at the time of data collection can better reflect the motion state of each robot at the same time.

[0124] The jitter in the communication delay between UCI and each controller will also have an adverse effect on the process of UCI sending the first control instruction to each controller, that is, there is a delay jitter in the process of feeding back the corresponding control instruction to each controller. For example, UCI sends the first control instruction to each controller at the same time, but due to the jitter in the communication delay between UCI and each controller, each controller receives the first control instruction at different times, which in turn causes each controller to be unable to synchronously control the corresponding robot according to the motion scene, and ultimately causes the poor effect of the collaborative operation of each robot. The control instruction frame is calculated based on the state parameters of each robot at a certain moment and the state parameters of each robot at the next moment. Its purpose is to treat each robot as a whole and synchronously adjust the state parameters at a certain moment to the state parameters at the next moment. Due to the communication delay between UCI and each controller, each controller cannot simultaneously receive the first control instruction sent by UCI at the same time, which in turn causes each robot to be unable to synchronously control the corresponding robot, causing each robot to be unable to synchronously move from the state parameters at a certain moment to the state parameters at the next moment, reducing the collaborative operation effect of each robot.

[0125] To this end, the multiple first control instructions corresponding to the control time can be determined in the following manner:

[0126] Determine the short-term delay duration of a communication link corresponding to a controller according to a historical state parameter and a first historical timestamp sent by a controller within a first set time before a sending time; the sending time is the time when a first control instruction is sent to a controller;

[0127] If the short-term delay duration is shorter than the average delay duration of the communication links corresponding to all controllers, determining a first control instruction corresponding to a controller at a time before the control moment according to the historical control instructions obtained before the control moment and the second historical timestamp;

[0128] If the short-term delay is longer than the average delay of the communication links corresponding to all controllers, a first control instruction corresponding to a controller at a time after the control moment is determined based on the historical control instructions obtained before the control moment and the second historical timestamp.

[0129] Since the delays of the communication links between the UCI and each controller are different, the time at which the first control instruction sent by the UCI to each controller arrives at each controller is also different.

[0130] For the above problem, when the short-term delay of a controller is shorter than the average delay, a first control instruction corresponding to a controller is obtained at a moment before the control moment, and such a first control instruction will arrive at the controller in advance, and the controller will perform the action according to the first control instruction; after the control moment and at a time interval with the control moment of the average delay, the first control instruction arrives at other controllers, and the other controllers control their corresponding robots according to the corresponding first control instruction; the first control instruction corresponding to the moment before the control moment arrives at the controller first, and after the control moment and at a time interval with the average delay, the motion state of the robot corresponding to the controller and the motion state of the robots corresponding to other controllers are closer to the motion state of each robot at the same time required by the motion scene, so that each robot has a better collaborative work effect.

[0131] When the short-term delay of a controller is longer than the average delay, a first control instruction corresponding to a controller is obtained at a moment after the control moment, and such a first control instruction can arrive at the controller later, and the controller performs an action according to the first control instruction; the first control instruction arrives at other controllers at a moment after the control moment and with a time interval of the average delay time from the control moment, and the other controllers control their corresponding robots according to the corresponding first control instruction; the first control instruction corresponding to the moment after the control moment arrives at the controller later, and at a moment after the control moment and with a time interval of the average delay time from the control time, the motion state of the robot corresponding to the controller and the motion state of the robots corresponding to other controllers are closer to the motion state of each robot at the same time required by the motion scene, so that each robot has a better collaborative operation effect.

[0132] The first set time length may be greater than or equal to a sampling period of a controller collecting the state parameters of the corresponding robot, and less than two sampling periods; or, the first set time length may be the time length of the sampling periods of more than two controllers. In this case, different controllers may correspond to different first set time lengths, and those skilled in the art may obtain the first set time length corresponding to the sampling period according to the sampling period of each controller.

[0133] Alternatively, each controller may be set to the same first set duration. For example, the second state parameter corresponding to the data acquisition moment may be determined, the control moment corresponding to the control instruction frame corresponding to the second state parameter may be determined, the time difference between the data acquisition moment and the control moment may be calculated, and the first set duration positively correlated with the time difference may be determined. Alternatively, those skilled in the art may also determine the first set duration that conforms to the actual situation based on the need to calculate network delay.

[0134] In the above embodiment, when only one state parameter and one timestamp can be received within the first set time period (the historical state parameter only includes the first state parameter, and the first historical timestamp only includes the first timestamp), the time difference between the moment of receiving the first state parameter and the first timestamp sent by a controller and the first timestamp can be used as the short-term delay duration.

[0135] When more than two state parameters and timestamps can be received within the first set time period (the historical state parameters include more than two state parameters, and the first historical timestamp includes more than two timestamps), the time difference between the time when each state parameter and timestamp sent by a controller is received and the time stamp can be calculated, and the average of the two or more time differences can be used as the short-term delay duration.

[0136] The average delay duration is the average delay duration of the communication links corresponding to all controllers. For example, the short-term delay duration of the communication link corresponding to each controller can be calculated, and the average value of all short-term delay durations is taken as the average delay duration.

[0137] The average delay time of the communication links corresponding to all controllers can be calculated in real time.

[0138] Figure 5 It is a flow chart of a communication method for multi-robot collaborative operation provided in an embodiment of the present application.

[0139] Combination Figure 5 As shown, the communication method for multi-robot collaborative operation includes:

[0140] S501. Obtain a first state parameter and a first timestamp of the robot collected by each controller.

[0141] The first state parameter and the first timestamp correspond one to one.

[0142] S502: Determine the data collection time according to the preset sampling frequency.

[0143] S503: Determine a second state parameter of a robot corresponding to a controller at the data collection time according to a historical state parameter and a first historical timestamp sent by a controller before the data collection time.

[0144] The historical state parameter includes a first state parameter, the first historical timestamp includes a first timestamp, the first timestamp is a timestamp sent by a controller that is closest to the data collection time, and a controller is any one of multiple controllers.

[0145] S504: Encapsulate the plurality of second status parameters into a status parameter frame, and send the status parameter frame to the upper-layer software.

[0146] S505. Obtain a control instruction frame corresponding to the state parameter frame through upper-layer software.

[0147] S506: Obtain a preset control frequency, and determine the control time of the multiple robots according to the preset control frequency.

[0148] The preset control frequency is less than or equal to the preset sampling frequency.

[0149] S507 . Determine a plurality of first control instructions corresponding to the control moment according to the historical control instructions obtained by the upper-layer software before the control moment and the second historical timestamp.

[0150] The historical control instruction includes the second control instruction in the control instruction frame, the second historical timestamp includes the second timestamp corresponding to the second control instruction, and the second timestamp is the timestamp sent by the upper-layer software and is closest to the control time.

[0151] S508: Obtain multiple groups of first state parameters and first control instructions having corresponding relationships and related to a controller within a second set time period.

[0152] The second set duration is related to the duration required for the robot to experience jitter. The intuitive phenomenon of the robot jitter is that the robot vibrates in place; its actual control process is: when the actual position of the robot (actual displacement or actual angle) does not reach the set position (set displacement or set angle), the robot's actuator moves a minimum action unit in one direction (such as the minimum step angle of a stepper motor), and the robot's actual position exceeds the set position. The robot's actuator then moves in the opposite direction by one or more minimum action units, and the robot's actual position is switched again to not reach the set position, and so on. This jittering phenomenon of the robot can easily bring a bad experience to users.

[0153] The jitter of the robot usually occurs when the robot's speed approaches zero. The higher the transmission accuracy of the robot's mechanical structure, the shorter the second setting time; the lower the transmission accuracy of the robot's mechanical structure, the longer the second setting time.

[0154] Those skilled in the art may determine the second set time length that suits the actual situation according to the actual situation of the robot.

[0155] The first state parameter of this step may be the first state parameter whose first timestamp is closest to the data collection time.

[0156] The correspondence between the first state parameter and the first control instruction refers to the first control instruction being obtained by obtaining the second state parameter based on the first state parameter. Each time the second state parameter is calculated, the control instruction frame is obtained, and the first control instruction is obtained, a set of first state parameters and first control instructions having a correspondence relationship can be obtained. Each set of first state parameters and first control instructions having a pair relationship has a sequential order in terms of timing.

[0157] S509: Calculate a movement direction of the robot corresponding to a controller according to each group of first state parameters and first control instructions.

[0158] For example, in the defined positive direction, if the first state parameter corresponding to a controller is smaller than the state parameter indicated by the first control instruction, it can be determined that the robot corresponding to the controller moves in the positive direction; if the first state parameter corresponding to a controller is greater than the state parameter indicated by the first control instruction, it can be determined that the robot corresponding to the controller moves in the reverse direction.

[0159] The above positive direction can represent the angular rotation direction or the linear movement direction.

[0160] S510: If the two movement directions of the robot calculated by two sets of first state parameters and first control instructions having adjacent first timestamps are different, it is determined that the robot changes its movement direction once.

[0161] S511. If the number of times the robot's motion direction changes within the second set time period is less than or equal to the set number threshold, determine that a first control instruction corresponding to a controller is a non-easy-to-jitter instruction, and send the first control instruction to a controller.

[0162] The setting number threshold is related to the tolerance for robot vibration. The higher the tolerance for robot vibration, the more the robot is allowed to vibrate occasionally, and the larger the setting number threshold is; the lower the tolerance for robot vibration, the less the robot is allowed to vibrate occasionally, and the smaller the setting number threshold is. Those skilled in the art can determine the setting number threshold that meets the actual situation according to the actual requirements for the robot.

[0163] It is mentioned in the above-mentioned embodiment that the second set time length is related to the transmission accuracy of the robot's mechanical structure. In the step of combining the second set time length and the set number threshold to determine whether a first control instruction is a non-jitter-prone instruction, the second set time length is also positively correlated with the set number threshold. The larger the set number threshold, the longer the second set time length; the smaller the set number threshold, the shorter the second set time length.

[0164] By adopting the above technical solution, before the UCI sends the first control instruction to the controller, it makes a certain pre-judgment on whether the first control instruction is likely to cause the robot to shake, and sends a non-shake-prone instruction to the robot.

[0165] When the controller controls the robot, it sends control instructions to the robot based on some algorithms. These algorithms themselves are not likely to cause the robot to shake. In addition, the controller and the robot are matched to each other. The controller's algorithm is also matched to the minimum action unit of the robot's actuator and the transmission accuracy of the robot's mechanical structure. When the controller controls the robot normally, the probability of the robot shaking is relatively low.

[0166] During the process of upper-level software coordinating the robots, the upper-level software sends control instructions to the controllers of the robots based on some algorithms. These algorithms themselves are not likely to cause robot jitters. Moreover, the coordination algorithms selected by the upper-level software are usually also selected by technical personnel in this field based on the characteristics of the robots and controllers. The coordination algorithms are in line with the characteristics of the robots and controllers. During the normal coordination of the robots by the upper-level software, the probability of robot jitters is relatively low.

[0167] In the embodiment of the present application, in order to enable the upper software to coordinate each robot based on the state parameters of each robot at the same time, the historical state parameters and the first historical timestamp sent by a controller before the data collection time are used to determine the second state parameters of the robot corresponding to a controller at the data collection time, and the first state parameter is adjusted to the second state parameter in this process; correspondingly, in order to enable each robot to act more synchronously, the control time is determined, and the second control instruction in the control instruction frame is adjusted to the first control instruction. In the process of adjusting the first state parameter to the second state parameter, the state parameter of the robot has an error, and in the process of adjusting the second control instruction in the control instruction frame to the first control instruction, the control instruction has an error; usually, the controller feeds back the state parameters of each robot to the upper software through UCI, and the upper software obtains the control instruction for coordinating each robot, and feeds back the control instruction to each controller through UCI, which is a relatively short process. The above two errors (the error in the robot state parameter and the error in the robot control instruction) have little effect on the actual motion state of the robot, but when the robot's motion speed approaches zero, it is easy to cause the robot's control process to jitter (such as the actual control process corresponding to the aforementioned robot jitter phenomenon).

[0168] Therefore, in the above technical scheme, the jitter of the robot is calculated using the real first state parameter without error and the first control instruction with a certain error. The first state parameter and the first control instruction are parameters and instructions close to the robot side, and can more accurately determine whether the first control instruction is an easy-to-jitter instruction, and then send the non-easy-to-jitter instruction to a controller to reduce the jitter phenomenon of the robot.

[0169] The above technical solution exemplarily illustrates the process of determining that a first control instruction corresponding to a controller is a non-jitter-prone instruction.

[0170] Correspondingly, if the first control instruction to be sent causes the robot to change its movement direction once, and the number of times the robot's movement direction changes within a second set time period is greater than the set number threshold, then the first control instruction corresponding to a controller is determined to be an easy-to-jitter instruction; the first control instruction corresponding to a controller is re-determined based on the historical control instructions obtained before the control moment and the second historical timestamp, or, the stationary instruction is used as the first control instruction corresponding to a controller, and the first control instruction is sent to a controller.

[0171] In the above technical solution, the possibility of robot jitter is further reduced at the cost of losing the accuracy of the control command. The jitter of the robot usually occurs in the scene where the movement speed of the robot approaches zero. In the scene where the movement speed of the robot approaches zero, even if the control command loses a certain degree of accuracy, the adverse effect on the collaborative operation effect between the robots can be almost negligible. Therefore, the above technical solution can reduce the possibility of robot jitter.

[0172] Figure 6 1 is a schematic diagram of a communication device for multi-robot collaborative operation provided by an embodiment of the present application. The communication method for multi-robot collaborative operation can be implemented in the form of communication middleware, or in the form of other software, hardware, or a combination of the two similar to the communication middleware. In the application scenario of the communication device, there are multiple robots, each robot corresponds to a controller, and multiple robots collaborate to complete a sports scene, such as a running sports scene.

[0173] Combination Figure 6 As shown, the communication device for multi-robot collaborative operation includes a first communication module 61 , a synchronization module 62 , a second communication module 63 , a third communication module 64 and a fourth communication module 65 .

[0174] The first communication module 61 is used to obtain the first state parameter and the first timestamp of the robot collected by each controller; the first state parameter and the first timestamp correspond one to one;

[0175] The synchronization module 62 is used to perform timestamp synchronization on the multiple first state parameters and the first timestamp to obtain multiple second state parameters with the same timestamp;

[0176] The second communication module 63 is used to encapsulate the plurality of second state parameters into a state parameter frame, and send the state parameter frame to the upper layer software;

[0177] The third communication module 64 is used to obtain the control instruction frame corresponding to the state parameter frame through the upper layer software;

[0178] The fourth communication module 65 is used to feed back corresponding control instructions to each controller according to the control instruction frame, so that each controller can control its corresponding robot.

[0179] Optionally, the synchronization module 62 includes a first determination unit and a second determination unit; the first determination unit is used to determine the data collection time according to a preset sampling frequency; the second determination unit is used to determine the second state parameter of a robot corresponding to a controller at the data collection time according to the historical state parameters and the first historical timestamp sent by a controller before the data collection time; wherein the historical state parameters include the first state parameters, the first historical timestamp includes the first timestamp, and the first timestamp is the timestamp sent by a controller that is closest to the data collection time; a controller is any one of multiple controllers.

[0180] Optionally, the second determination unit is specifically configured to use a first state parameter sent by a controller as a second state parameter at the data collection moment when a preset sampling frequency is the same as a sampling frequency at which a controller collects state parameters of its corresponding robot.

[0181] Alternatively, the second determination unit can be specifically used to obtain a first fitting curve or a first neural network model corresponding to a historical state parameter sent by a controller before the data collection moment and a first historical timestamp when a preset sampling frequency is the same as a sampling frequency at which a controller collects state parameters of its corresponding robot, substitute the data collection moment into the first fitting curve or the first neural network model, and obtain the second state parameter at the data collection moment output by the first fitting curve or the first neural network model; wherein the first fitting curve or the first neural network model is used to represent the correspondence between the timestamp and the state parameter.

[0182] Alternatively, the second determination unit can also be specifically used to obtain a first fitting curve or a first neural network model corresponding to a historical state parameter sent by a controller before the data collection moment and a first historical timestamp when a preset sampling frequency is different from a sampling frequency at which a controller collects state parameters of its corresponding robot, substitute the data collection moment into the first fitting curve or the first neural network model, and obtain the second state parameter at the data collection moment output by the first fitting curve or the first neural network model; wherein the first fitting curve or the first neural network model is used to represent the correspondence between the timestamp and the state parameter.

[0183] Optionally, the fourth communication module 65 includes a third determination unit, a fourth determination unit and a sending unit; the third determination unit is used to obtain a preset control frequency, and determine the control moment of multiple robots according to the preset control frequency; the preset control frequency is less than or equal to the preset sampling frequency; the fourth determination unit is used to determine multiple first control instructions corresponding to the control moment according to the historical control instructions and the second historical timestamp obtained by the upper-level software before the control moment; the sending unit is used to send each first control instruction to the corresponding controller, so that each controller can control its corresponding robot; wherein the historical control instruction includes the second control instruction in the control instruction frame, the second historical timestamp includes the second timestamp corresponding to the second control instruction, and the second timestamp is the timestamp sent by the upper-level software that is closest to the control moment.

[0184] Optionally, the fourth determining unit is specifically configured to use the second control instruction in the control instruction frame as the first control instruction at the control moment.

[0185] Alternatively, the fourth determination unit can be specifically used to obtain a second fitting curve or a second neural network model corresponding to a historical control instruction obtained before the control moment and a second historical timestamp, substitute the control moment into the second fitting curve or the second neural network model, and obtain the first control instruction at the control moment output by the second fitting curve or the second neural network model; wherein the second fitting curve or the second neural network is used to represent the correspondence between the timestamp and the control instruction.

[0186] Optionally, the sending unit is specifically used to obtain multiple groups of first state parameters and first control instructions with corresponding relationships related to a controller within a second set time length; calculate the movement direction of a robot corresponding to a controller according to each group of first state parameters and first control instructions; if the two movement directions of the robot calculated by two adjacent groups of first state parameters and first control instructions of the first timestamp are different, it is determined that the robot changes its movement direction once; if the number of times the robot's movement direction changes within the second set time length is less than or equal to a set number threshold, it is determined that the first control instruction corresponding to a controller is a non-easy-to-jitter instruction, and the first control instruction is sent to a controller.

[0187] Optionally, the sending unit is also specifically used for: if the first control instruction to be sent causes the robot to change its movement direction once, and the number of times the robot's movement direction changes within a second set time period is more than a set threshold number, then determining that the first control instruction corresponding to a controller is an easy-to-jitter instruction; re-determining the first control instruction corresponding to a controller based on the historical control instructions obtained before the control moment and the second historical timestamp, or, taking the stationary instruction as the first control instruction corresponding to a controller, and sending the first control instruction to a controller.

[0188] Optionally, the fourth determination unit is also specifically used to: determine the short-term delay duration of a communication link corresponding to a controller based on historical status parameters and a first historical timestamp sent by a controller within a first set time period before a sending moment; the sending moment is the moment of sending a first control instruction to a controller; if the short-term delay duration is shorter than the average delay duration of the communication links corresponding to all controllers, then determine the first control instruction corresponding to a controller at the moment before the control moment based on the historical control instructions obtained before the control moment and the second historical timestamp; if the short-term delay duration is longer than the average delay duration of the communication links corresponding to all controllers, then determine the first control instruction corresponding to a controller at the moment after the control moment based on the historical control instructions obtained before the control moment and the second historical timestamp.

[0189] In some embodiments, a communication device for multi-robot collaborative work includes a processor and a memory storing program instructions, and the processor is configured to execute the communication method for multi-robot collaborative work provided by the aforementioned embodiments when executing the program instructions.

[0190] Figure 7 It is a schematic diagram of a communication device for multi-robot collaborative operation provided in an embodiment of the present application.

[0191] Combination Figure 7 As shown, a communication device for multi-robot collaborative operation provided by an embodiment of the present application includes:

[0192] The processor 71 and the memory 72 may also include a communication interface 73 and a bus 74. The processor 71, the communication interface 73, and the memory 72 may communicate with each other through the bus 74. The communication interface 73 may be used for information transmission. The processor 71 may call the logic instructions in the memory 72 to execute the communication method for multi-robot collaborative operation provided in the aforementioned embodiment.

[0193] In addition, the logic instructions in the above-mentioned memory 72 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0194] The memory 72 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present application. The processor 71 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 72, that is, implementing the methods in the above method embodiments.

[0195] The memory 72 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 72 may include a high-speed random access memory and may also include a non-volatile memory.

[0196] An embodiment of the present application provides a communication middleware for multi-robot collaborative operation, including the communication device for multi-robot collaborative operation provided by the aforementioned embodiment.

[0197] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the communication method for multi-robot collaborative operation provided by the aforementioned embodiment.

[0198] An embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the communication method for multi-robot collaborative operation provided by the aforementioned embodiment.

[0199] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0200] The technical solution of the embodiment of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method in the embodiment of the present application. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0201] The above description and the accompanying drawings fully illustrate the embodiments of the present application so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms as well. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or including (comprising) refer to the existence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or these groups. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same or similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0202] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0203] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units can be only a logical function division, and there can be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0204] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present application. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or code, and a part of a module, program segment or code includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, which can depend on the functions involved. Each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a dedicated hardware-based system that performs the function or action of the specification, or can be realized by a combination of special-purpose hardware and computer instructions.

Claims

1. A communication method for multi-robot collaborative operation, characterized in that: Each robot corresponds to a controller, and multiple robots collaborate to complete a motion scene. The communication method includes: Obtaining a first state parameter and a first timestamp of the robot collected by each controller; the first state parameter and the first timestamp correspond one to one; Performing time stamp synchronization on the plurality of first state parameters and the first time stamp to obtain a plurality of second state parameters with the same time stamp; encapsulating the plurality of second state parameters into a state parameter frame, and sending the state parameter frame to the upper layer software; Obtaining a control instruction frame corresponding to the state parameter frame through the upper layer software; Obtaining a preset control frequency, and determining control moments of a plurality of robots according to the preset control frequency; Determining a plurality of first control instructions corresponding to the control moment according to the historical control instructions and the second historical timestamp obtained by the upper-layer software before the control moment, specifically comprising: obtaining a second fitting curve or a second neural network model corresponding to the historical control instructions obtained before the control moment and the second historical timestamp, substituting the control moment into the second fitting curve or the second neural network model, and obtaining the first control instruction at the control moment output by the second fitting curve or the second neural network model; wherein the second fitting curve or the second neural network is used to represent the correspondence between the timestamp and the control instruction; Sending each first control instruction to a corresponding controller so that each controller can control its corresponding robot; The historical control instruction includes the second control instruction in the control instruction frame, the second historical timestamp includes the second timestamp corresponding to the second control instruction, and the second timestamp is the timestamp sent by the upper-layer software that is closest to the control time.

2. The communication method according to claim 1, characterized in that: Performing time stamp synchronization on a plurality of first state parameters and a first time stamp includes: Determine the data collection time according to the preset sampling frequency; Determine, based on a historical state parameter and a first historical timestamp sent by a controller before the data collection moment, a second state parameter of a robot corresponding to the controller at the data collection moment; Among them, the historical state parameters include a first state parameter, the first historical timestamp includes a first timestamp, and the first timestamp is a timestamp sent by the one controller that is closest to the data collection time; the one controller is any one of the multiple controllers.

3. The communication method according to claim 2, characterized in that: Determining a second state parameter of a robot corresponding to the controller at the data collection time according to a historical state parameter and a first historical timestamp sent by the controller before the data collection time includes: In the case where the preset sampling frequency is the same as the sampling frequency at which the one controller collects the state parameters of its corresponding robot, the first state parameter sent by the one controller is used as the second state parameter at the data collection moment; or, In the case where the preset sampling frequency is the same as the sampling frequency at which the controller collects the state parameters of the corresponding robot, the historical state parameters sent by the controller before the data collection moment and the first fitting curve or the first neural network model corresponding to the first historical timestamp are obtained, the data collection moment is substituted into the first fitting curve or the first neural network model, and the second state parameters at the data collection moment output by the first fitting curve or the first neural network model are obtained; wherein the first fitting curve or the first neural network model is used to represent the corresponding relationship between the timestamp and the state parameters; or, When the preset sampling frequency is different from the sampling frequency at which the controller collects the state parameters of its corresponding robot, obtain the first fitting curve or the first neural network model corresponding to the historical state parameters and the first historical timestamp sent by the controller before the data collection moment, substitute the data collection moment into the first fitting curve or the first neural network model, and obtain the second state parameters of the data collection moment output by the first fitting curve or the first neural network model; wherein the first fitting curve or the first neural network model is used to represent the correspondence between the timestamp and the state parameters.

4. The communication method according to claim 2, characterized in that: The preset control frequency is less than or equal to the preset sampling frequency.

5. The communication method according to claim 1, characterized in that: Determining a plurality of first control instructions corresponding to the control moment according to the historical control instructions and the second historical timestamp obtained by the upper layer software before the control moment, further comprising: The second control instruction in the control instruction frame is used as the first control instruction at the control moment.

6. The communication method according to claim 1, characterized in that: Sending each first control instruction to a corresponding controller includes: Obtaining a plurality of groups of first state parameters and first control instructions having corresponding relationships and related to the one controller within a second set time period; Calculating the movement direction of the robot corresponding to the one controller according to each group of first state parameters and first control instructions respectively; If the two movement directions of the robot calculated by the two sets of first state parameters and the first control instruction adjacent to the first timestamp are different, it is determined that the robot changes its movement direction once; If the number of times the robot's motion direction changes within the second set time period is less than or equal to the set number threshold, it is determined that the first control instruction corresponding to the one controller is a non-easy-to-jitter instruction, and the first control instruction is sent to the one controller.

7. The communication method according to claim 6, characterized in that: Also includes: If the first control instruction to be sent causes the robot to change its movement direction once, and the number of times the robot's movement direction changes within the second set time period is greater than the set number threshold, then determining that the first control instruction corresponding to the one controller is an easy-to-jitter instruction; Re-determine the first control instruction corresponding to the one controller based on the historical control instruction obtained before the control moment and the second historical timestamp, or use the static instruction as the first control instruction corresponding to the one controller, and send the first control instruction to the one controller.

8. The communication method according to claim 1, characterized in that: Determining a plurality of first control instructions corresponding to the control moment according to the historical control instructions and the second historical timestamp obtained by the upper layer software before the control moment, further comprising: Determine the short-term delay duration of the communication link corresponding to the one controller according to the historical state parameters and the first historical timestamp sent by the one controller within a first set time before the sending time; the sending time is the time when the first control instruction is sent to the one controller; If the short-term delay duration is shorter than the average delay duration of the communication links corresponding to all controllers, determining the first control instruction corresponding to the one controller at a time before the control moment according to the historical control instructions obtained before the control moment and the second historical timestamp; If the short-term delay duration is longer than the average delay duration of the communication links corresponding to all controllers, the first control instruction corresponding to the one controller at a time after the control moment is determined based on the historical control instructions and the second historical timestamp obtained before the control moment.

9. A communication device for multi-robot collaborative operation, characterized in that: Each robot corresponds to a controller, and multiple robots work together to complete a motion scene. The communication device includes: A first communication module is used to obtain a first state parameter and a first timestamp of the robot collected by each controller; the first state parameter and the first timestamp correspond one to one; A synchronization module, used for performing timestamp synchronization on the plurality of first state parameters and the first timestamp to obtain a plurality of second state parameters with the same timestamp; A second communication module, used for encapsulating the plurality of second state parameters into a state parameter frame, and sending the state parameter frame to the upper layer software; A third communication module, used for obtaining a control instruction frame corresponding to the state parameter frame through the upper layer software; a third determining unit, configured to obtain a preset control frequency and determine control moments of the plurality of robots according to the preset control frequency; A fourth determination unit is used to determine a plurality of first control instructions corresponding to a control moment according to the historical control instructions and the second historical timestamp obtained by the upper-layer software before the control moment; the fourth determination unit is specifically used to obtain a second fitting curve or a second neural network model corresponding to the historical control instructions and the second historical timestamp obtained before the control moment, substitute the control moment into the second fitting curve or the second neural network model, and obtain the first control instruction at the control moment output by the second fitting curve or the second neural network model; wherein the second fitting curve or the second neural network is used to represent the correspondence between the timestamp and the control instruction; A sending unit, used for sending each first control instruction to a corresponding controller, so that each controller can control its corresponding robot; The historical control instruction includes the second control instruction in the control instruction frame, the second historical timestamp includes the second timestamp corresponding to the second control instruction, and the second timestamp is the timestamp sent by the upper-layer software that is closest to the control time.

10. A communication middleware for multi-robot collaborative operation, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the communication method for multi-robot collaborative operation according to any one of claims 1 to 8 when executing the program instructions.

Images