Multi-device cooperative control method and device, autonomous mobile device and storage medium

Abstract

The embodiment of the invention provides a multi-device cooperative control method, device and system and a storage medium, and relates to an autonomous mobile device. A task trajectory is planned according to a cooperative task by responding to a master vehicle determined to be a queue and receiving the cooperative task, the queue including one of the master vehicles and at least one slave vehicle; the task trajectory and the execution starting time of the cooperative task are issued to the at least one slave vehicle, so that the at least one slave vehicle plans respective following trajectories according to the task trajectory, and the respective following trajectories are executed when the execution starting time of the task trajectory is reached; and when the execution starting time of the task track is reached, executing the task track, so that the master vehicle and the at least one slave vehicle are kept in a queue to synchronously execute the task, and cooperative operation of integrating a plurality of devices into a whole is realized, so that the application scene of cooperation of the plurality of devices is expanded.

Description

Technical Field

[0001] This application relates to the technical field of autonomous mobile devices, and more specifically, to a multi-device collaborative control method, device, system, and storage medium. Background Art

[0002] Autonomous mobile devices such as mobile robots are widely used in multiple fields. When a single robot is unable to carry large goods, multiple robots need to be used for multi-device collaboration and joint operation to carry large goods.

[0003] Regarding the collaborative cooperation of multiple mobile robots, the related technology adopts segmented forward movement and lateral movement of multiple mobile robots. In this way, the overall movement mode and movement scenario of multiple mobile robots are relatively limited, and true collaborative operation of multiple mobile robots integrated into one body has not been achieved. Summary of the Invention

[0004] Embodiments of this application provide a multi-device collaborative control method, device, system, and storage medium, enabling the master vehicle and at least one slave vehicle to maintain queue synchronization and execute tasks, achieving collaborative operation of multiple devices integrated into one body, thereby expanding the application scenarios of multi-device collaborative cooperation.

[0005] In a first aspect, an embodiment of this application provides a multi-device collaborative control method, which includes: in response to being determined as the master vehicle of the queue and receiving a collaborative task, planning a task trajectory according to the collaborative task, where the queue includes one master vehicle and at least one slave vehicle; publishing the task trajectory and the start execution time of the collaborative task to the at least one slave vehicle, so that the at least one slave vehicle plans its respective following trajectories according to the task trajectory and executes its respective following trajectories when reaching the start execution time of the task trajectory; when reaching the start execution time of the task trajectory, executing the task trajectory.

[0006] In a second aspect, an embodiment of this application provides a multi-device collaborative control method, which includes: after being determined as a slave vehicle in the queue, receiving the task trajectory and the start execution time of the task trajectory published by the master vehicle of the queue, where the queue includes one master vehicle and at least one slave vehicle; performing trajectory planning according to the task trajectory to obtain a following trajectory; when reaching the start execution time of the task trajectory, executing the following trajectory.

[0007] Thirdly, an embodiment of the present application provides a multi-device collaborative control method, which is applied to a multi-device collaborative control system. The multi-device collaborative control system includes a master vehicle and at least one slave vehicle, and the master vehicle and the at least one slave vehicle form a queue. The method includes: the master vehicle, in response to being determined as the master vehicle of the queue and receiving a collaborative task, plans a task trajectory according to the starting point and the ending point of the collaborative task, and publishes the task trajectory and the start execution time of the collaborative task to the at least one slave vehicle; the at least one slave vehicle plans a trajectory according to the task trajectory to obtain respective following trajectories of the at least one slave vehicle; when the start execution time of the task trajectory is reached, the master vehicle executes the task trajectory, and the at least one slave vehicle respectively executes its own following trajectory.

[0008] Fourthly, an embodiment of the present application provides a multi-device collaborative control device, which includes: a response module, configured to, in response to being determined as the master vehicle of the queue and receiving a collaborative task, plan a task trajectory according to the collaborative task, where the queue includes one master vehicle and at least one slave vehicle; a publishing module, configured to publish the task trajectory and the start execution time of the collaborative task to the at least one slave vehicle, so that the at least one slave vehicle plans its respective following trajectory according to the task trajectory and executes its respective following trajectory when the start execution time of the task trajectory is reached; an execution module, configured to execute the task trajectory when the start execution time of the task trajectory is reached.

[0009] Fifthly, an embodiment of the present application provides a multi-device collaborative control device, which includes: a receiving module, configured to, after being determined as a slave vehicle in the queue, receive the task trajectory published by the master vehicle of the queue and the start execution time of the task trajectory, where the queue includes one master vehicle and at least one slave vehicle; a planning module, configured to plan a trajectory according to the task trajectory to obtain a following trajectory; a following module, configured to execute the following trajectory when the start execution time of the task trajectory is reached.

[0010] Sixthly, an embodiment of the present application provides an autonomous mobile device, which includes: a memory and a processor. An application program is stored in the memory, and the application program is used to execute the method provided in the first aspect or the second aspect of the embodiment of the present application when called by the processor.

[0011] Seventhly, an embodiment of the present application provides a computer-readable storage medium, on which program codes are stored. The program codes are used to cause a processor to execute the method provided in the first aspect or the second aspect of the embodiment of the present application when called by the processor.

[0012] In the multi-device collaborative control method, device, system, and storage medium provided in the embodiments of the present application, the master vehicle publishes a task trajectory and the start execution time of the task trajectory to at least one slave vehicle. The at least one slave vehicle can plan its own following trajectory according to the task trajectory. When the start execution time of the task trajectory is reached, the master vehicle executes the task trajectory, and at the same time, the at least one slave vehicle executes its own following trajectory, so that the master vehicle and the at least one slave vehicle can maintain queue synchronization to execute tasks, achieving collaborative operation with multiple devices integrated into one, enhancing the autonomy and intelligence in multi-device collaborative operation, and thus expanding the application scenarios of multi-device collaborative cooperation. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, rather than all embodiments. Based on the embodiments of the present application, all other embodiments and drawings obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0014] Figure 1 is a structural block diagram of a multi-device collaborative control system provided in an embodiment of the present application;

[0015] Figure 2 is a flowchart of a multi-device collaborative control method provided in an embodiment of the present application;

[0016] Figure 3 is a flowchart of a multi-device collaborative control method on the master vehicle side of the queue provided in an embodiment of the present application;

[0017] Figure 4 is a schematic diagram of the formation of a queue provided in an exemplary embodiment of the present application;

[0018] Figure 5 is a schematic diagram of the formation of a queue provided in another exemplary embodiment of the present application;

[0019] Figure 6 is a schematic diagram of the formation of a queue provided in another exemplary embodiment of the present application;

[0020] Figure 7 is a flowchart of obstacle avoidance control in a multi-device collaborative control method provided in an embodiment of the present application;

[0021] Figure 8 is a schematic diagram of the rotation control process in a multi-device collaborative control method provided in an exemplary embodiment of the present application;

[0022] Figure 9It is a schematic diagram of the rotation control process in the multi-device collaborative control method provided by another exemplary embodiment of the present application;

[0023] Figure 10 It is a flowchart of the chassis control on the main vehicle side in the multi-device collaborative control method provided by an embodiment of the present application;

[0024] Figure 11 It is a flowchart of the jacking control on the main vehicle side in the multi-device collaborative control method provided by an embodiment of the present application;

[0025] Figure 12 It is a flowchart of the multi-device collaborative control method on the queue following vehicle side provided by an embodiment of the present application;

[0026] Figure 13 It is a schematic diagram of the process of the queue making a curve movement provided by an embodiment of the present application;

[0027] Figure 14 It is a flowchart of the chassis control on the following vehicle side in the multi-device collaborative control method provided by an embodiment of the present application;

[0028] Figure 15 It is a flowchart of the jacking control on the following vehicle side in the multi-device collaborative control method provided by an embodiment of the present application;

[0029] Figure 16 It is a flowchart of the multi-device collaborative control method provided by an embodiment of the present application;

[0030] Figure 17 It is a structural block diagram of the multi-device collaborative control device on the main vehicle side of the queue provided by an embodiment of the present application;

[0031] Figure 18 It is a structural block diagram of the multi-device collaborative control device on the following vehicle side of the queue provided by an embodiment of the present application;

[0032] Figure 19 It is a structural block diagram of the autonomous mobile device provided by an embodiment of the present application. Detailed implementation manners

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0034] See Figure 1 , Figure 1It is a structural block diagram of a multi-device collaborative control system provided by an embodiment of the present application. The multi-device collaborative control device may include a scheduling system, a main vehicle, and at least one slave vehicle. The main vehicle and at least one slave vehicle form a queue. The scheduling system is used to overall manage the scheduling of the main vehicle and the slave vehicles, and the main vehicle and the slave vehicles are used to execute operations according to the scheduling of the scheduling system.

[0035] The scheduling system can be used to schedule and manage the collaborative work of multiple autonomous mobile devices, so that multiple autonomous mobile devices can efficiently execute tasks issued by users in the same area without interfering with each other. The scheduling system includes a user interface (User Interface, abbreviated as UI), and tasks can be managed and controlled through the user interface. For example, tasks can be issued, cancelled, paused, and stopped, etc.

[0036] Both the main vehicle and at least one slave vehicle can be autonomous mobile devices. The autonomous mobile device may include, but is not limited to, an Automated Guided Vehicle (abbreviated as AGV) and an Autonomous Mobile Robot (abbreviated as AMR). The autonomous mobile device can adapt to various models such as differential, steering wheels, and Mecanum wheels, and supports functions such as multi-device collaborative execution of linear trajectories, curved trajectories, in-situ spinning, obstacle stopping, obstacle avoidance, state synchronization, and exception handling. It should be noted that the same autonomous mobile device can support both the main vehicle role and the slave vehicle role, and an autonomous mobile device can also only support the main vehicle role or the slave vehicle role, and no specific restrictions are made here.

[0037] The working process of the multi-device system collaborative control system is as follows: The scheduling system issues a task to the main vehicle. The main vehicle leads all the slave vehicles to execute the task issued by the scheduling system. Specifically, when the main vehicle receives the task issued by the scheduling system, it issues the task and the start execution time of the task to all the slave vehicles. When the start execution time of the task is reached, the main vehicle and all the slave vehicles maintain the formation of the queue and execute the task synchronously.

[0038] For example, referring to Figure 2 , the scheduling system issues a formation task to the main vehicle. The main vehicle plans the task trajectory of the main vehicle (i.e., the main vehicle trajectory) according to data such as kinematic constraints, obstacle information, formation profile, and path information. Slave vehicles 1 and 2 receive the task trajectory of the main vehicle and the start execution time of the task trajectory (i.e., the start execution time of the task), and respectively calculate their own following trajectories (slave vehicle trajectories) according to the formation model. When the start execution time of the task trajectory is reached, the main vehicle executes the task trajectory, slave vehicle 1 executes following trajectory 1, and slave vehicle 2 executes following trajectory 2. After slave vehicles 1 and 2 complete the execution of the following trajectories, they report an instruction indicating the completion of the task to the main vehicle. When the main vehicle determines that it has completed the execution of the task trajectory and both slave vehicles 1 and 2 have completed the execution of the following trajectories, it determines that the formation task is completed.

[0039] The multi-device collaborative control method in the embodiments of this application is applied to a multi-device collaborative control system. Specifically, the multi-device collaborative control method on the host vehicle side is applied to an autonomous mobile device serving as the host vehicle, or a multi-device collaborative control device on the host vehicle side. The multi-device collaborative control method on the slave vehicle side is applied to an autonomous mobile device serving as the slave vehicle, or a multi-device collaborative control device on the slave vehicle side. In the embodiments of this application, the host vehicle's operations of issuing trajectories, tasks, instructions, etc. to the slave vehicle can all adopt broadcast form or point-to-point form, and the embodiments of this application do not limit the communication method between the host vehicle and the slave vehicle.

[0040] See Figure 3 , Figure 3 is a flowchart of the multi-device collaborative control method on the host vehicle side provided by an embodiment of this application. This method may include steps S110 to S130.

[0041] Step S110: In response to being determined as the host vehicle of the queue and receiving a collaborative task, plan a task trajectory according to the collaborative task, where the queue includes one said host vehicle and at least one slave vehicle.

[0042] Before the handling task, the user can, through the scheduling system, determine the number of devices for the current collaborative task and the formation of the queue according to the size and shape of the goods for the current handling task. The scheduling system comprehensively considers factors such as the idle status, distance, and model of each autonomous mobile device, and notifies multiple autonomous mobile devices to form a queue for executing the current handling task. When issuing the notice, the host vehicle of the queue and the positions of each autonomous mobile device in the queue are specified simultaneously.

[0043] In the embodiments of this application, the formation of the queue may include, but is not limited to, a horizontal one-line shape (horizontal one-line queue), a vertical one-line shape (vertical one-line queue), and a triangular shape (triangular queue), etc. For example, assuming that three autonomous mobile devices are required for the current handling task, the formation of the queue can be seen in Figures 4 to 6 , Figure 4 shows a horizontal one-line queue, where the middle autonomous mobile device is designated as the host vehicle, and the two autonomous mobile devices on both sides are slave vehicles. Figure 5 shows a vertical one-line queue, where the middle autonomous mobile device is designated as the host vehicle, and the autonomous mobile devices in the front and back are slave vehicles. Figure 6 shows a triangular queue, where the frontmost autonomous mobile device is designated as the host vehicle, and the subsequent autonomous mobile devices are slave vehicles.

[0044] In some embodiments, after each autonomous mobile device arrives at the position, it feeds back to the scheduling system that it has arrived, and then the scheduling system issues the current collaborative task to the host vehicle. In other embodiments, when the scheduling system notifies each device to arrive at the position to form a queue, it can issue the collaborative task to the host vehicle at the same time.

[0045] In the embodiments of the present application, a collaborative task refers to a task that needs to be executed in a queue, that is, a task that requires multiple autonomous mobile devices to execute collaboratively. A collaborative task can be issued by a user through the user interface of the scheduling system. One collaborative task can be issued at a time, or multiple collaborative tasks can be issued at a time. For example, the collaborative task can be a transportation task.

[0046] In some embodiments, the collaborative task includes a task end point, that is, the end point that this transportation task needs to reach. For example, it is the goods unloading point. When the autonomous mobile device serving as the master vehicle receives the collaborative task, it can plan the task trajectory according to information such as the end point of the collaborative task, the current position of the master vehicle, obstacles in the global map, and kinematic constraints. The task trajectory is the trajectory between the current position of the master vehicle and the end point of the collaborative task. For example, multiple discrete points from the current position of the master vehicle around the obstacles in the global map to the task end point can be obtained, and the distance between each discrete point and the obstacle is greater than the distance between the geometric center of the queue profile and the obstacle. A fifth-order Bezier curve is used to fit the multiple discrete points, and curve smoothing is performed after fitting to obtain the task path. According to the maximum speed limit of each point on the task path and the kinematic constraints, the speed and pose of each point on the task path are planned, so as to obtain the task trajectory.

[0047] In some embodiments, if multiple collaborative tasks are received and the multiple collaborative tasks have their respective priorities, the multiple collaborative tasks can be executed in sequence according to the priorities. In other embodiments, if multiple collaborative tasks are received, the multiple collaborative tasks can be executed in sequence according to the order of receipt time of the multiple collaborative tasks from first to last. For the collaborative tasks received simultaneously, the execution order of the collaborative tasks received simultaneously can be randomly arranged, or the collaborative tasks received simultaneously can be arranged in sequence according to the distance between the task end point and the current position of the master vehicle from near to far.

[0048] Step S120: Publish the task trajectory and the start execution time of the collaborative task to at least one slave vehicle, so that at least one slave vehicle plans its respective following trajectory according to the task trajectory and executes its respective following trajectory when reaching the start execution time of the task trajectory.

[0049] In some embodiments, in order to ensure that the master vehicle and the slave vehicle execute the collaborative task synchronously, the time of the master vehicle and the slave vehicle can be synchronized first, so as to ensure that the timestamps of the master vehicle and the slave vehicle are consistent and achieve synchronous execution of the collaborative task.

[0050] After the task trajectory planning (and time synchronization) is completed, the task trajectory and the start execution time of the collaborative task can be published to at least one slave vehicle. After receiving the task trajectory and the start execution time of the collaborative task, the slave vehicle can plan its own following trajectory according to the task trajectory and execute the following trajectory when it reaches the start execution time of the task trajectory, so as to ensure that the master vehicle and the slave vehicle maintain the formation synchronization of the queue and execute the collaborative task, achieving the integration of multiple devices. For the specific description of the slave vehicle planning the following trajectory, please refer to step S220 in the following text.

[0051] In some embodiments, when the master vehicle completes the task trajectory planning, the moment obtained by adding N (N is a positive integer) seconds to the current moment can be determined as the start execution time of the collaborative task, that is, the start execution time of the collaborative task = the task trajectory planning completion moment + N seconds. For example, assuming N is 1, the start execution time of the collaborative task = the task trajectory planning completion moment + 1 second.

[0052] In some other embodiments, the start execution time of the collaborative task can be issued by the user through the scheduling system. It should be understood that when setting the start execution time of the collaborative task, a certain trajectory planning time needs to be reserved for the master vehicle and the slave vehicle.

[0053] Step S130: Execute the task trajectory when reaching the start execution time of the task trajectory.

[0054] During the execution of the task trajectory, global path collision detection can be performed respectively based on the global map and / or local collision detection can be performed based on the vehicle body coordinate system to ensure the safety of the master vehicle and the slave vehicle in the collaborative operation. The global map is the map in the entire collaborative operation scenario. The global map includes static obstacles and dynamic obstacles. The static obstacles can include the static obstacles added to the global map before the operation and the static obstacles updated to the global map by the autonomous mobile devices in the scenario during the operation. The dynamic obstacles can be the obstacles updated to the global map by the autonomous mobile devices in the scenario during the operation.

[0055] In some embodiments, performing global path collision detection based on the global map can include: The master vehicle can perform collision detection on the trajectory points on the task trajectory according to the outline of the queue and the obstacles (outline) in the global map. Specifically, the trajectory points on the task trajectory can be used as the center and expanded outward to the outline of the queue to determine whether the outline of the queue intersects with the obstacles (outline) in the global map. If the outline of the queue intersects with the obstacles (outline) in the global map, the trajectory point corresponding to the intersection of the outline of the queue and the obstacles (outline) is determined as the collision point. If there is any collision at any point, it is considered that there is a collision on the path that the queue of multi-device collaboration is going to travel.

[0056] In some embodiments, local collision detection based on the vehicle body coordinate system may include: The host vehicle may perform collision detection on the trajectory points on the task trajectory according to the expanded contour of the host vehicle. Specifically, the trajectory points on the task trajectory may be used as the center, and expanded outward to the expanded contour of the host vehicle to determine whether the expanded contour of the host vehicle intersects with the obstacles (contours) in the global map. If the expanded contour of the host vehicle intersects with the obstacles (contours) in the global map, the trajectory points corresponding to the intersection of the expanded contour of the host vehicle and the obstacles are determined as collision points. Only if there is a collision at any point, it is considered that there is a collision on the path that the multi-device collaborative queue is to travel. Among them, the expanded contour of the host vehicle = the contour of the host vehicle itself + the safety distance (around). The safety distance is used to protect the autonomous mobile device from colliding with obstacles. It should be understood that the safety distances around the host vehicle in the front, rear, left, and right directions may be the same or different, and the safety distance can be specifically set according to the actual operation scenario and safety requirements. For example, the safety distances in the front, rear, left, and right directions can all be 0.5 meters. Since the longitudinal movement of the autonomous mobile device is more intense than the lateral movement, the safety distances in the front and rear can also be set to 1 meter, while the safety distances on the left and right are 0.5 meters.

[0057] In some embodiments, local collision detection based on the vehicle body coordinate system may further include: At least one slave vehicle may perform collision detection on the trajectory points on the following trajectory according to the expanded contour of the slave vehicle. Specifically, for each slave vehicle, the slave vehicle may use the trajectory points on the following trajectory as the center, and expand outward to the expanded contour of the slave vehicle to determine whether the expanded contour of the slave vehicle intersects with the obstacles (contours) in the global map. If the expanded contour of the slave vehicle intersects with the obstacles (contours) in the global map, the trajectory points corresponding to the intersection of the expanded contour of the slave vehicle and the obstacles are determined as collision points. Only if there is a collision at any point, it is considered that there is a collision on the path that the multi-device collaborative queue is to travel. Among them, the expanded contour of the slave vehicle = the contour of the slave vehicle itself + the above safety distance. In this embodiment, after the slave vehicle discovers a collision point, it will synchronously feedback the collision point to the host vehicle. The host vehicle determines that a collision point is detected in response to receiving the collision point sent by any one of at least one slave vehicle.

[0058] After detecting a collision point by using any one of the collision detection methods provided in the above three embodiments, the host vehicle may determine that there is a collision in the multi-device collaborative operation queue. At this time, it is necessary to control the multiple devices in the queue to stop synchronously to ensure that after the multiple devices stop, each device can still maintain its respective position in the queue, that is, to ensure that the formation of the queue remains unchanged. That is to say, at this time, the host vehicle needs to perform parking trajectory planning.

[0059] During the execution of the task trajectory, the host vehicle can perform parking trajectory planning in response to detecting a collision point (or receiving a temporary stop instruction). When the host vehicle completes the parking trajectory planning, it can determine the start execution time of the parking trajectory as the time obtained by adding N (N is a positive integer) seconds to the current moment. That is, the start execution time of the parking trajectory = the completion time of the parking trajectory planning + N seconds. For example, assuming N is 1, the start execution time of the parking trajectory = the completion time of the parking trajectory planning + 1 second. After the host vehicle completes the parking trajectory planning, it can publish the parking trajectory and the start execution time of the parking trajectory to at least one slave vehicle, so that at least one slave vehicle can plan its own parking following trajectory according to the parking trajectory. When the start execution time of the parking trajectory is reached, the host vehicle executes the parking trajectory, and at least one slave vehicle executes its own parking following trajectory, thereby ensuring that the host vehicle and the slave vehicles maintain the formation synchronization of the queue for parking, avoiding the situation where a certain autonomous mobile device stops first or the parking positions of multiple autonomous mobile devices are not unified, resulting in chaos in the queue, and achieving the integration of multiple devices. Among them, for the specific description of the slave vehicle planning the parking following trajectory, please refer to step S220 in the following text.

[0060] Among them, the implementation method of the host vehicle for parking trajectory planning is as follows: The host vehicle can determine the parking point according to the collision point, and the distance between the parking point and the collision point in the direction of the task trajectory is the specified distance; perform parking trajectory planning according to the current trajectory point on the task trajectory and the parking point. Among them, the specified distance can be preset by relevant personnel according to actual needs. For example, the specified distance can be 1 meter. The host vehicle can plan a path from the current point to the point 1 meter in front of the collision point on the original straight or curved task path, and re-plan the speed for this path, so as to obtain a parking trajectory that can make the host vehicle accurately stop 1 meter in front of the obstacle.

[0061] After encountering an obstacle and stopping (or receiving a continue execution instruction), the host vehicle can perform obstacle avoidance trajectory planning. The host vehicle can obtain multiple discrete points according to the task trajectory and the obstacle, and the multiple discrete points bypass the obstacle from the current trajectory point of the task trajectory and then return to the task trajectory; perform obstacle avoidance trajectory planning according to the multiple discrete points. Specifically, the obstacle avoidance trajectory planning can include three parts, namely path search, path optimization, and speed planning. That is, the A* (A Star) algorithm can be used to search for multiple discrete points (discrete point set) that bypass the obstacle from the current trajectory point of the task trajectory and then return to the task trajectory, and sequentially connect the discrete points with a fifth-order Bezier curve. By continuously adjusting the tangent directions of the two endpoints (starting point and target point) and the intermediate control points of each segment of the Nessel curve, the discrete path is optimized to obtain a smooth obstacle avoidance path. According to the kinematic and dynamic constraints of the queue, speed planning is performed on the points on the obstacle avoidance path, so as to obtain the obstacle avoidance trajectory.

[0062] After the obstacle avoidance trajectory planning is completed, in order to avoid affecting the operations of other autonomous mobile devices in the operation scenario, the master vehicle needs to apply to the scheduling system for the passing permission of the area through which the obstacle avoidance trajectory passes, and the scheduling system determines whether to allow the queue to perform obstacle avoidance. Specifically, after the obstacle avoidance trajectory planning of the master vehicle is completed, it can report the area through which the obstacle avoidance trajectory passes to the scheduling system to apply for obstacle avoidance. The scheduling system judges whether the obstacle avoidance trajectory of the queue will affect other autonomous mobile devices in the area through which the obstacle avoidance trajectory passes according to the movement conditions of all autonomous mobile devices in the operation scenario. If the obstacle avoidance trajectory will affect other autonomous mobile devices, it notifies the master vehicle that obstacle avoidance is not allowed. If the obstacle avoidance trajectory will not affect other autonomous mobile devices, it notifies the master vehicle that obstacle avoidance is allowed.

[0063] When receiving the notification of allowing obstacle avoidance sent by the scheduling system, that is, when the obstacle avoidance trajectory is allowed to be executed, the master vehicle can determine the start execution time of the obstacle avoidance trajectory by adding N (N is a positive integer) seconds to the current time. That is, the start execution time of the obstacle avoidance trajectory = the completion time of the obstacle avoidance trajectory planning + N seconds. For example, assuming N is 1, the start execution time of the obstacle avoidance trajectory = the completion time of the obstacle avoidance trajectory planning + 1 second. The master vehicle publishes the obstacle avoidance trajectory and the start execution time of the obstacle avoidance trajectory to at least one slave vehicle, so that at least one slave vehicle plans its own obstacle avoidance following trajectory according to the obstacle avoidance trajectory. When reaching the start execution time of the obstacle avoidance trajectory, the master vehicle executes the obstacle avoidance trajectory, and at least one slave vehicle executes its own obstacle avoidance following trajectory, so as to ensure that the master vehicle and the slave vehicles maintain the formation of the queue and synchronously execute obstacle avoidance, achieving the integration of multiple devices. The specific description of the slave vehicle planning the obstacle avoidance following trajectory can be found in step S220 in the following text.

[0064] For example, refer to Figure 7 The shown obstacle avoidance process: During the execution of the collaborative task, the master vehicle and the slave vehicles respectively perform collision detection. When a collision point is detected, the parking task is executed: the master vehicle plans a parking trajectory, publishes the parking trajectory and the start execution time of the parking trajectory to the slave vehicles, and the slave vehicles plan a parking following trajectory according to the parking trajectory. When reaching the start execution time of the parking trajectory, the master vehicle and the slave vehicles synchronously park, and still perform collision detection during the parking process. After parking due to encountering an obstacle, the obstacle avoidance task is executed: the master vehicle plans an obstacle avoidance trajectory and applies to the scheduling system for the passing request of the area through which the obstacle avoidance trajectory passes. After the scheduling system notifies the master vehicle that obstacle avoidance is allowed, the master vehicle publishes the obstacle avoidance trajectory and the start execution time of the obstacle avoidance trajectory to the slave vehicles, and the slave vehicles plan an obstacle avoidance following trajectory according to the obstacle avoidance trajectory. When reaching the start execution time of the obstacle avoidance trajectory, the master vehicle and the slave vehicles synchronously perform obstacle avoidance, and still perform collision detection during the obstacle avoidance process. After the obstacle avoidance is completed, the obstacle avoidance task is deleted, and the collaborative task is continued: the master vehicle and the slave vehicles delete the obstacle avoidance task, the master vehicle continues to execute the collaborative task along the task trajectory, and the slave vehicles continue to execute the collaborative task along the following trajectory.

[0065] During the execution of the task trajectory, if there is a need for in-place rotation at the current trajectory point on the task trajectory, the master vehicle can obtain the type of in-place rotation and the target pose corresponding to the current trajectory point; and adopt a rotation method corresponding to the type of in-place rotation to control the master vehicle and at least one slave vehicle to rotate according to the target pose.

[0066] Among them, the types of in-place rotation can include the first type and the second type. The in-place rotation task of the first type is to carry the storage rack (shelf) and rotate around the geometric center of the queue together. The in-place rotation task of the second type is to rotate without carrying the storage rack together, that is, with different storage racks, the autonomous mobile device rotates in place. It should be noted that the second type is for symmetric queues, and symmetric queues can include but are not limited to a horizontal single queue, a vertical single queue, and a square queue composed of four autonomous mobile devices, etc.

[0067] Among them, if there is a need for in-place rotation at the trajectory point, the trajectory point stores a preset target pose, and the target pose is used to guide each device to perform in-place rotation.

[0068] The entire in-place rotation process is dominated by the master vehicle, and the master vehicle controls the slave vehicle to perform synchronous rotation by issuing a rotation control instruction and the start execution time of the rotation control instruction to the slave vehicle.

[0069] In some embodiments, if the type of in-place rotation is the first type, the master vehicle can control the master vehicle and at least one slave vehicle to carry the storage rack and rotate around the geometric center of the queue according to the target pose. For example, the rotation of the first type includes the following three rotations:

[0070] The first rotation: The master vehicle controls the master vehicle and at least one slave vehicle to rotate around their respective Z axes to their respective corresponding tangent directions. Among them, the tangent direction corresponding to the master vehicle or the slave vehicle is the tangent direction of the circle with the geometric center of the queue as the center at the master vehicle or the slave vehicle. Specifically, the master vehicle issues the first instruction and the execution time of the first instruction to the slave vehicle, and when the execution time of the first instruction is reached, the master vehicle and the slave vehicle synchronously execute the first rotation.

[0071] For example, see Figure 8 , assuming that the master vehicle and two slave vehicles form a triangular pyramid queue, and the formation of the current queue is as Figure 8 shown in the left formation, Figure 8 In the left formation, the directions indicated by the solid black arrows of each vehicle are the orientations of each device, that is, the orientation of the queue. After the master vehicle and the slave vehicle rotate around their respective Z axes to their respective corresponding tangent directions, the formation of the queue is as Figure 8 shown in the right formation, Figure 8 In the right formation, the directions indicated by the solid black arrows of each vehicle are the orientations of each device, and at the same time, they are also the tangent directions corresponding to each device.

[0072] Second rotation: The master vehicle controls the master vehicle and at least one slave vehicle to carry the storage rack and rotate around the geometric center of the queue to the target posture. Specifically, the master vehicle issues a second command and the execution time of the second command to the slave vehicle. When the execution time of the second command is reached, the master vehicle and the slave vehicle synchronously execute the second rotation.

[0073] Third rotation: The master vehicle controls the master vehicle and at least one slave vehicle to rotate around their respective own Z axes to the queue posture, in which the master vehicle and at least one slave vehicle have the same orientation. Specifically, the master vehicle issues a third command and the execution time of the third command to the slave vehicle. When the execution time of the third command is reached, the master vehicle and the slave vehicle synchronously execute the third rotation. The purpose of the third rotation is to make the orientations of the devices the same, maintain the queue posture, so as to keep the formation synchronization of the queue and continue to execute the collaborative task subsequently.

[0074] In some other embodiments, if the in-situ spin type is the second type, the master vehicle can control the master vehicle and at least one slave vehicle to rotate around their respective own Z axes to the target posture. Specifically, the master vehicle issues a single rotation command and the execution time of the single rotation command to the slave vehicle. When the execution time of the single rotation command is reached, the master vehicle and the slave vehicle synchronously execute the in-situ spin of the second type.

[0075] For example, refer to Figure 9 , assuming that the master vehicle and two slave vehicles form a horizontal queue, and the formation of the current queue is as shown in Figure 9 the left formation, Figure 9 in the left formation, the directions indicated by the black solid arrows are the orientations of the devices, that is, the orientation of the queue. After the master vehicle and the slave vehicles rotate around their respective own Z axes to the target posture, the formation of the queue is as shown in Figure 9 the right formation, Figure 9 in the right formation, the directions indicated by the black solid arrows are the orientations of the devices, and also the orientation of the queue. It should be noted that refer to Figure 9 , the position of slave vehicle 1 changes from the left side of the master vehicle to the right side of the master vehicle, and the position of slave vehicle 2 changes from the right side of the master vehicle to the left side of the master vehicle. Therefore, after the in-situ spin of the second type, the formation model needs to be updated, that is, the relative position of the slave vehicle relative to the master vehicle after the in-situ spin is updated, and the subsequent trajectory is calculated according to the new model. For this reason, the in-situ spin of the second type requires that the formation model is a symmetric model.

[0076] The above trajectory execution control all belongs to chassis control. The goal of the chassis control of the master vehicle is to ensure that the master vehicle can accurately track the trajectory issued by the planning layer, so that the master vehicle plays a leading role in the whole vehicle fleet. Refer to Figure 10, the specific process of the chassis control of the master vehicle is as follows: The input of the chassis control of the master vehicle comes from the external trajectory of the planning layer (including trajectories such as task trajectory, parking trajectory, and obstacle avoidance trajectory, etc.). The speed of the trajectory point is directly input to the master vehicle as a feedforward quantity, and the difference between the desired position and the actual position in the trajectory point is input to the longitudinal proportional integral (PI) controller and the lateral PI controller. Here, the longitudinal and lateral directions are defined with respect to the vehicle body coordinate system. Therefore, the deviation along the longitudinal direction of the autonomous mobile device is corrected by the longitudinal PI controller, and the deviation along the lateral direction and the heading angle of the autonomous mobile device is eliminated by the lateral PI controller. The outputs of the two PI controllers and the feedforward speed in the external trajectory are jointly input to the kinematic model of the autonomous mobile device, and then the speeds of each wheel of the autonomous mobile device are calculated and sent to the motor controller.

[0077] During the process of multi-device collaborative operation, the master vehicle will monitor the task status of each slave vehicle in real time, including task execution status, safety status, obstacle encounter status, etc., and the slave vehicle needs to report the above status to the master vehicle in real time. In addition to chassis control, the multi-device collaborative control method can also include the control of other specified tasks (such as lifting tasks). When the master vehicle receives a temporarily specified task, the master vehicle needs to publish the specified task and the start execution time of the specified task to the slave vehicle in real time to ensure that multiple vehicles execute the specified task simultaneously. Here, the specified task can include but is not limited to tasks such as lifting, canceling, abnormal, error clearing, and emergency stop.

[0078] During the process of executing the task trajectory, in response to receiving a specified task, the master vehicle can determine the start execution time of the specified task as the time obtained by adding N (N is a positive integer) seconds to the current moment. That is, the start execution time of the specified task = the moment of receiving the specified task + N seconds. For example, assuming N is 1, the start execution time of the specified task = the moment of receiving the specified task + 1 second. The specified task and the start execution time of the specified task are published to at least one slave vehicle. When the start execution time of the specified task is reached, at least one slave vehicle and the master vehicle execute the specified task synchronously, so as to ensure that the master vehicle and the slave vehicle execute the specified task synchronously and achieve the integration of multiple devices.

[0079] In some embodiments, the specified task includes a lifting task, and the lifting task includes a target lifting height, and the target lifting height is less than or equal to the maximum lifting height of the autonomous mobile device. The master vehicle can obtain the target lifting height in the lifting task; according to the difference between the target lifting height and the current lifting height of the master vehicle, control the master vehicle to lift to the target lifting height. During the lifting process of the master vehicle, the current lifting height of the master vehicle is published to the slave vehicle in real time, and the slave vehicle adjusts the lifting speed according to the difference between its own current lifting height and the current lifting height of the master vehicle, so as to ensure synchronization with the master vehicle's lifting.

[0080] For example, seeFigure 11 The specific process of the lifting control of the master vehicle is as follows: After receiving the lifting task, the motion controller of the master vehicle starts timing. When it reaches the start execution time of the lifting task, it starts to execute the lifting instruction. The difference between the current lifting height and the target lifting height of the master vehicle is used as the input of the lifting PI controller. The output of the PI controller and the preset speed are jointly input to the lifting model, and finally it is converted to the rotational speed of the lifting motor and sent to the lifting motor driver. The entire control process ensures that the lifting of the master vehicle starts to lift to the target height at the start execution time of the lifting task at a predetermined speed. Among them, the preset speed can be set in advance by relevant personnel. The preset speed should not be too large, as it is likely to cause the goods to fall off.

[0081] It should be understood that the implementation method of the lowering task opposite to the direction of the lifting task is similar to that of the lifting task, and the only difference is that the operating directions of the autonomous mobile devices in the two tasks are opposite, one is rising and the other is falling.

[0082] After the slave vehicle finishes executing the following trajectory, it can send an instruction to the master vehicle to indicate that the trajectory has been executed. The master vehicle responds to the completion of the task trajectory by the master vehicle and the completion of their respective following trajectories by at least one slave vehicle, and sends an instruction to the scheduling system to indicate the completion of the collaborative task. In some embodiments, after sending the instruction indicating the completion of the collaborative task, in response to receiving the disband instruction issued by the scheduling system, the master vehicle resumes its independent motion state, and at the same time sends a disband instruction to at least one slave vehicle to notify at least one slave vehicle to resume its independent motion state. In other embodiments, after sending the instruction indicating the completion of the collaborative task, in response to receiving a new collaborative task issued by the scheduling system, the master vehicle re-executes steps S110 to step S130.

[0083] Based on steps S110 to step S130, the master vehicle will send the task trajectory and the start execution time of the task trajectory to at least one slave vehicle. At least one slave vehicle can plan its own following trajectory according to the task trajectory. When it reaches the start execution time of the task trajectory, the master vehicle executes the task trajectory, and at the same time at least one slave vehicle executes its respective following trajectory, so that the master vehicle and at least one slave vehicle can maintain queue synchronization to execute the task, and can achieve collaborative operation of multiple devices integrated into one body, improving the autonomy and intelligence in the collaborative operation of multiple devices, and thus expanding the application scenarios of collaborative cooperation of multiple devices.

[0084] See Figure 12 , Figure 12 is a flowchart of a multi-device collaborative control method on the slave vehicle side of a queue according to an embodiment of the present application. The method includes steps S210 to step S230.

[0085] Step S210: After being determined as a following vehicle in the queue, receive the task trajectory published by the leading vehicle of the queue and the start execution time of the task trajectory, where the queue includes one leading vehicle and at least one following vehicle.

[0086] Step S220: Perform trajectory planning according to the task trajectory to obtain a following trajectory.

[0087] The following vehicle can determine the relative position relationship between itself and the leading vehicle in the queue; according to the positions of the trajectory points on the task trajectory and the relative position relationship, plan the positions of the trajectory points on the following trajectory. For example, through the queue model, the relative pose relationship between the following vehicle and the leading vehicle in the queue can be calculated, and the rotation matrix is constructed as follows:

[0088] P follower =P leader *T

[0089] where, P leader represents the pose of the leading vehicle in the queue, P follower represents the pose of the following vehicle in the queue, and T represents the rotation matrix.

[0090] The rotation matrix includes two processes of translation and rotation, and its final result is the product of the rotation matrix and the translation matrix. Since an autonomous mobile device such as a mobile robot only rotates along the Z-axis and does not consider rotation along the X-axis and Y-axis, the rotation matrix is:

[0091]

[0092] The translation matrix is:

[0093]

[0094] where, α is the rotation angle, x e 、y e 、z e are the moving distances on the X, Y, and Z axes respectively. Taking the triangular queue shown in Figure 6 as an example, the relative position relationship between following vehicle 1 and the leading vehicle is to translate 1 meter backward along the X-axis and 1 meter to the left along the Y-axis, and the pose does not rotate relative to the leading vehicle. The relative position relationship between following vehicle 2 and the leading vehicle is to translate 1 meter backward along the X-axis and 1 meter to the right along the Y-axis, and the pose does not rotate relative to the leading vehicle. Since there is no rotation relative to the leading vehicle, the rotation matrix can be not added. Therefore, only the corresponding X-axis and Y-axis offset amounts need to be updated to the translation matrix, and combined with the pose of the autonomous mobile device, the pose information of each of following vehicle 1 and following vehicle 2 in the following trajectory can be solved.

[0095] The follower vehicle can determine the angular velocity, curvature, and attitude direction corresponding to the trajectory points on the task trajectory; determine the distance between itself and the host vehicle in the direction perpendicular to the attitude direction; and plan the speed of the trajectory points on the following trajectory based on the angular velocity, curvature, and the distance.

[0096] The basis for calculating the speed of the follower vehicle is that the rotational angular velocities of multiple autonomous mobile devices in the queue are the same during the process of following the trajectory. Therefore, the different rotational speeds of each autonomous mobile device are due to different rotational radii during the process of tracking the curve. From the speed calculation formula V = W * R, when the angular velocity of the host vehicle and the rotational radius of the follower vehicle are obtained, the speed of the follower vehicle can be obtained. Taking the horizontal queue 1 as an example, the relationship between the rotational radii of two follower vehicles 1 and 2 and the rotational radius of the host vehicle can be as Figure 13 shown: When the queue follows the curve, the center of rotation around the curve moves from position 1 to position 2. Refer to Figure 13 , and it can be obtained that:

[0097] R1 = R + L

[0098] R2 = R - L

[0099] where, R1 is the rotational radius of follower vehicle 1 at position 2, R is the rotational radius of the host vehicle at position 2, R2 is the rotational radius of follower vehicle 2 at position 2, and L is the distance between the follower vehicle and the host vehicle in the direction perpendicular to the attitude direction of the trajectory points on the task trajectory (the orientation of the host vehicle at the trajectory point, such as Figure 13 the direction indicated by the black solid arrow). The rotational radius R of the host vehicle is the reciprocal of the curvature K of the host vehicle, and there is:

[0100] R = 1 / K

[0101] Therefore, the speeds of follower vehicles 1 and 2 at position 2 are respectively:

[0102] V1 = W * R1

[0103] V2 = W * R2

[0104] where, is the speed of follower vehicle 1 at position 2, and is the angular velocity of the host vehicle at position 2.

[0105] It should be understood that Figure 13 shown is the trajectory of the queue rotating to the right, that is, the case where the curvature of the (task and following) trajectory is less than 0. When the curvature of the trajectory is greater than 0, that is, when the queue rotates to the left, then follower vehicle 2 is located outside the rotation circle, and follower vehicle 1 is located inside the rotation circle. At this time, the calculation formulas for the rotational radii of the two follower vehicles are as follows:

[0106] R1 = R - L

[0107] R2 = R + L

[0108] Step S230: When the start execution time of the task trajectory is reached, execute the following trajectory.

[0109] During the execution of the following trajectory, the follower vehicle can obtain the real-time position of the leading vehicle, the real-time speed of the follower vehicle, and the communication delay. Among them, the communication delay is the difference between the moment when the leading vehicle publishes information (such as the real-time position) and the moment when the follower vehicle receives the information. According to the real-time position of the leading vehicle, the relative position relationship, the communication delay, and the real-time speed of the follower vehicle, determine the expected position of the follower vehicle; execute the following trajectory according to the expected position. Among them, the real-time speed of the follower vehicle includes the linear speed and the angular speed.

[0110] Since the follower vehicle needs to maintain the same formation of the queue in real time during the process of tracking the trajectory, the follower vehicle needs to track the position of the leading vehicle in real time. Therefore, the real-time position and real-time speed of the leading vehicle, the real-time target position of the leading vehicle in the corresponding follower vehicle model (the expected position of the follower vehicle), and the real-time position and real-time speed of the follower vehicle need to be published to the control module in real time. At the same time, due to the problem of communication delay, a time compensation amount needs to be added when calculating the expected position of the follower vehicle to ensure the accuracy of the tracking position and ensure that the formation of the queue remains unchanged.

[0111] Among them, the real-time position and real-time speed of the leading vehicle and the real-time position and speed of the follower vehicle can be directly obtained through their respective positioning modules and published in real time.

[0112] Among them, the real-time target position of the leading vehicle in the corresponding follower vehicle model is calculated according to the following expression:

[0113] P = P follower + P0

[0114] Among them, P follower can be calculated using the above expression P follower = P leader * T. P is the expected position of the follower vehicle, and P0 is the compensation amount generated by the communication delay. The compensation amount P0 = V current * t, V current is the real-time speed of the follower vehicle, and t is the communication delay. The compensation amount needs to be compensated separately in three aspects: the X-axis, the Y-axis, and the attitude. Therefore, when expanded, the X-axis position compensation is: P x = V x * t; the Y-axis position compensation is: P y = V y * t; the attitude compensation is: T x = W * t. Among them, V x is the speed on the X-axis, V y is the speed on the Y-axis, and W is the angular speed.

[0115] During the execution of following the trajectory, the slave vehicle can perform collision detection on the trajectory points on the following trajectory according to its own expanded contour; determine the trajectory points corresponding to the intersections of the expanded contour of the slave vehicle and the obstacles as collision points; and send the collision points to the master vehicle. For the specific description of the slave vehicle performing collision detection based on the vehicle body coordinate system, please refer to the part about collision detection in the above text and will not be elaborated here.

[0116] During the execution of the following trajectory, if the slave vehicle receives the parking trajectory issued by the master vehicle and the start execution time of the parking trajectory, the slave vehicle can plan a parking following trajectory according to the parking trajectory; and execute the parking following trajectory when reaching the start execution time of the parking trajectory, so as to ensure that the master vehicle and the slave vehicle keep the formation of the queue synchronized for parking, avoid the situation that a certain autonomous mobile device parks first or the parking positions of multiple autonomous mobile devices are not unified, resulting in chaos in the queue, and achieve the integration of multiple devices. The planning of the parking following trajectory is similar to the planning method of the following trajectory. For the specific description of the planning of the parking following trajectory, please refer to the part about the planning method of the following trajectory and will not be elaborated here.

[0117] During the execution of the following trajectory, if the slave vehicle receives the obstacle avoidance trajectory issued by the master vehicle and the start execution time of the obstacle avoidance trajectory, the slave vehicle can plan an obstacle avoidance following trajectory according to the obstacle avoidance trajectory; and execute the obstacle avoidance following trajectory when reaching the start execution time of the obstacle avoidance trajectory. Thus, it ensures that the master vehicle and the slave vehicle keep the formation of the queue synchronized for obstacle avoidance and achieve the integration of multiple devices. The planning of the obstacle avoidance following trajectory is similar to the planning method of the following trajectory. For the specific description of the planning of the obstacle avoidance following trajectory, please refer to the part about the planning method of the following trajectory and will not be elaborated here.

[0118] The above trajectory execution controls all belong to chassis control. The goal of the slave vehicle chassis control is to ensure that the slave vehicle can accurately maintain its position in the queue, so as to keep the formation of the entire queue unchanged during movement. Refer to Figure 14 , the specific process of the slave vehicle chassis control is as follows: The input of the slave vehicle chassis control comes from the following trajectory of the planning layer and the expected position of the slave vehicle calculated to maintain the formation (the real-time position of the master vehicle × rotation matrix + compensation amount generated by communication delay). The speed of the trajectory point is directly used as a feedforward quantity and input to the master vehicle. The difference between the expected position and the actual position of the slave vehicle in the queue is used as the input of the longitudinal PI controller and the lateral PI controller. Here, the longitudinal and lateral are defined according to the vehicle body coordinate system. Therefore, the deviation along the longitudinal direction of the slave vehicle is corrected by the longitudinal PI controller, and the deviation along the lateral direction and the heading angle of the slave vehicle is eliminated by the lateral PI controller. The outputs of the two PI controllers and the feedforward speed in the following trajectory are jointly input to the vehicle kinematics model, and then the speeds of each wheel are calculated and sent to the motor controller.

[0119] In addition to chassis control, the multi-device collaborative control method may also include the control of other specified tasks (such as lifting tasks). During the execution of following the trajectory, if the slave vehicle receives the specified task issued by the master vehicle and the start execution time of the specified task, the slave vehicle can execute the specified task when it reaches the start execution time of the specified task.

[0120] Exemplarily, refer to Figure 15 , the specific process of the lifting control of the slave vehicle is as follows: After the motion controller of the slave vehicle receives the lifting task, it starts timing. When it reaches the start execution time of the lifting task, it starts to execute the lifting task. The difference between the current lifting height of the slave vehicle and the current lifting height of the master vehicle is used as the input of the lifting PI controller. The output of the PI controller and the lifting speed are jointly input to the lifting model, and finally converted to the rotational speed of the lifting motor and sent to the lifting motor driver. The whole control process is to ensure that the slave vehicle and the master vehicle start lifting simultaneously and synchronously lift to the target lifting height in the lifting task. It should be noted that the starting lifting heights of the master vehicle and the slave vehicle are the same. During the lifting process, there may be errors between the slave vehicle and the master vehicle due to various reasons, resulting in different current lifting heights of the slave vehicle and the master vehicle. In the case where there is a difference in the current lifting heights of the slave vehicle and the master vehicle, the slave vehicle can adjust its own lifting speed according to this difference to synchronize the lifting heights of the slave vehicle and the master vehicle.

[0121] In some embodiments, the specified task includes a lifting task. The slave vehicle can obtain the current lifting height of the master vehicle; according to the difference between its own current lifting height and the current lifting height of the master vehicle, control its own lifting speed so that it synchronously lifts to the target lifting height with the master vehicle.

[0122] In response to the completion of following the trajectory, the slave vehicle can send an instruction to the master vehicle to indicate the completion of the collaborative task. In some embodiments, in response to the disband instruction issued by the master vehicle, the slave vehicle resumes its independent motion state. In other embodiments, in response to a new task trajectory issued by the master vehicle, steps S210 to S230 are re-executed.

[0123] Based on steps S210 to S230, the slave vehicle can plan its own following trajectory according to the received task trajectory and execute its own following trajectory when it reaches the start execution time of the task trajectory, so that the slave vehicle can keep the formation of the queue with the master vehicle and execute the task synchronously, achieving a collaborative operation where multiple devices are integrated into one, enhancing the autonomy and intelligence in the multi-device collaborative operation, and thus expanding the application scenarios of the collaborative cooperation of multiple devices.

[0124] Refer to Figure 16 , Figure 16 is the flowchart of the multi-device collaborative control method provided by another embodiment of the present application. This method includes steps S310 to S330.

[0125] Step S310: In response to being determined as the master vehicle of the queue and receiving a collaborative task, the master vehicle plans a task trajectory according to the collaborative task, and publishes the task trajectory and the start execution time of the collaborative task to at least one slave vehicle.

[0126] Step S320: At least one slave vehicle plans a trajectory according to the task trajectory to obtain the respective following trajectories of the at least one slave vehicle.

[0127] Step S330: When the start execution time of the task trajectory is reached, the master vehicle executes the task trajectory, and at least one slave vehicle respectively executes its respective following trajectory.

[0128] For the specific descriptions of Steps S310 to S330, please refer to Steps S110 to S130 and Steps S210 to S230, which will not be elaborated here.

[0129] Based on Steps S310 to S330, the master vehicle will publish the task trajectory and the start execution time of the task trajectory to at least one slave vehicle. The at least one slave vehicle can plan its own following trajectory according to the task trajectory. When the start execution time of the task trajectory is reached, the master vehicle executes the task trajectory, and at the same time, the at least one slave vehicle executes its respective following trajectory, so that the master vehicle and the at least one slave vehicle can maintain queue synchronization in task execution, enabling collaborative operation of multiple devices as an integrated unit, enhancing the autonomy and intelligence in the collaborative operation of multiple devices, and thus expanding the application scenarios of collaborative cooperation among multiple devices.

[0130] See Figure 17 , Figure 17 FIG. is a structural block diagram of a multi-device collaborative control device on the master vehicle side of a queue provided by an embodiment of the present application. The multi-device collaborative control device 100 can be applied to an autonomous mobile device. The multi-device collaborative control device 100 includes a response module 110, a publishing module 120, and an execution module 130.

[0131] The response module 110 is configured to, in response to being determined as the master vehicle of the queue and receiving a collaborative task, plan a task trajectory according to the collaborative task, where the queue includes one master vehicle and at least one slave vehicle. The specific working process of the response module 110 can be seen in Step S110.

[0132] The publishing module 120 is configured to publish the task trajectory and the start execution time of the collaborative task to the at least one slave vehicle, so that the at least one slave vehicle plans its respective following trajectory according to the task trajectory and executes its respective following trajectory when the start execution time of the task trajectory is reached. The specific working process of the publishing module 120 can be seen in Step S120.

[0133] An execution module 130, configured to execute the task trajectory when the start execution time of the task trajectory is reached. For the specific working process of the execution module 130, please refer to step S130.

[0134] See Figure 18 , Figure 18 FIG. is a structural block diagram of a multi-device collaborative control device on the slave vehicle side provided by an embodiment of the present application. The multi-device collaborative control device 200 can be applied to an autonomous mobile device. The multi-device collaborative control device 200 includes a receiving module 210, a planning module 220, and a following module 230.

[0135] The receiving module 210, the planning module 220, and the following module 230.

[0136] The receiving module 210 is configured to receive the task trajectory and the start execution time of the task trajectory published by the master vehicle of the queue after being determined to be a slave vehicle in the queue, where the queue includes one master vehicle and at least one slave vehicle. For the specific working process of the receiving module 210, please refer to step S210.

[0137] The planning module 220 is configured to perform trajectory planning according to the task trajectory to obtain a following trajectory. For the specific working process of the planning module 220, please refer to step S220.

[0138] The following module 230 is configured to execute the following trajectory when the start execution time of the task trajectory is reached. For the specific working process of the following module 230, please refer to step S230.

[0139] Those skilled in the art can clearly understand that the above devices provided by the embodiments of the present application can implement the corresponding methods provided by the embodiments of the present application. For the specific working processes of the above-described devices and modules, reference can be made to the corresponding processes in the methods of the embodiments of the present application, which will not be elaborated here.

[0140] In the embodiments provided by the present application, the coupling, direct coupling, or communication connection between the modules shown or discussed with each other can be indirect coupling or communication coupling through some interfaces, devices, or modules, and can be in electrical, mechanical, or other forms. The embodiments of the present application do not make specific limitations in this regard.

[0141] In addition, in each embodiment of the present application, the functional modules can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0142] See Figure 19 , Figure 19It is a structural block diagram of an autonomous mobile device provided by an embodiment of the present application. The autonomous mobile device 300 may include a memory 310 and a processor 320. An application program is stored in the memory 310, and the application program is configured to execute the method provided by the embodiment of the present application when called by the processor 320.

[0143] The processor 320 may include one or more processing cores. The processor 320 uses various interfaces and lines to connect various parts within the entire autonomous mobile device 300, and is used to run or execute instructions, programs, code sets or instruction sets stored in the memory 310, and to call and run or execute data stored in the memory 310, and execute various functions of the autonomous mobile device 300 and process data.

[0144] The processor 320 may be implemented in at least one hardware form of digital signal processing (DSP for short), field programmable gate array (FPGA for short), or programmable logic array (PLA for short). The processor 320 may integrate one or a combination of several of a central processing unit (CPU for short), a graphics processing unit (GPU for short), and a modem. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing display content; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated in the processor 320 and may be implemented separately through a communication chip.

[0145] The memory 310 may include a random access memory (RAM for short), or may also include a read-only memory (ROM for short). The memory 310 may be used to store instructions, programs, code, code sets or instruction sets. The memory 310 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for implementing at least one function, instructions for implementing the above method embodiments, etc. The data storage area may store data created during the use of the autonomous mobile device 300.

[0146] The embodiment of the present application also provides a computer-readable storage medium, on which program code is stored, and the program code is configured to execute the method provided by the embodiment of the present application when called by a processor.

[0147] A computer-readable storage medium may be an electronic memory such as a flash memory, an Electrically-Erasable Programmable Read-Only Memory (EEPROM for short), an Erasable Programmable Read-Only Memory (EPROM for short), a hard disk, or a ROM.

[0148] In some embodiments, the computer-readable storage medium includes a Non-Transitory Computer-Readable Storage Medium (Non-TCRSM for short). The computer-readable storage medium has a storage space for program code that executes any of the method steps in the above methods. These program codes can be read out from one or more computer program products or written into these one or more computer program products. The program codes can be compressed in a suitable form.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-device collaborative control method, characterized in that: include: In response to being determined as a master vehicle of a platoon and receiving a collaborative task, planning a task trajectory according to the collaborative task, wherein the platoon includes one master vehicle and at least one slave vehicle; Publishing the task trajectory and the start execution time of the collaborative task to the at least one slave vehicle, so that the at least one slave vehicle plans its own following trajectory according to the task trajectory and executes its own following trajectory when the start execution time of the task trajectory is reached; When the start execution time of the task trajectory is reached, the task trajectory is executed.

2. The method according to claim 1, characterized in that The method further comprises: During execution of the mission trajectory, in response to detecting a collision point, performing parking trajectory planning; publishing a parking trajectory and a start execution time of the parking trajectory to the at least one slave vehicle, so that the at least one slave vehicle plans a respective parking following trajectory according to the parking trajectory and executes the respective parking following trajectory when the start execution time of the parking trajectory is reached; When the start execution time of the parking trajectory is reached, the parking trajectory is executed.

3. The method according to claim 2, characterized in that The parking trajectory planning includes: determining a parking point according to the collision point, wherein the distance between the parking point and the collision point in the direction of the task trajectory is a specified distance; Parking trajectory planning is performed based on the current trajectory point on the task trajectory and the parking point.

4. The method according to claim 2, characterized in that After executing the parking trajectory, the method further includes: After stopping when encountering an obstacle, plan a trajectory around the obstacle; When the obstacle avoidance trajectory is allowed to be executed, publishing the obstacle avoidance trajectory and the start execution time of the obstacle avoidance trajectory to the at least one slave vehicle, so that the at least one slave vehicle plans its own obstacle avoidance following trajectory according to the obstacle avoidance trajectory and executes its own obstacle avoidance following trajectory when the start execution time of the obstacle avoidance trajectory is reached; When the start execution time of the obstacle avoidance trajectory is reached, the obstacle avoidance trajectory is executed.

5. The method according to claim 4, characterized in that The obstacle avoidance trajectory planning includes: Acquire a plurality of discrete points according to the task trajectory and the obstacle, wherein the plurality of discrete points go from a current trajectory point of the task trajectory around the obstacle and then return to the task trajectory; Obstacle avoidance trajectory planning is performed based on the multiple discrete points.

6. The method according to claim 4, characterized in that When the obstacle avoidance trajectory is allowed to be executed, publishing the obstacle avoidance trajectory and the start execution time of the obstacle avoidance trajectory to the at least one slave vehicle includes: Report the area passed by the obstacle avoidance trajectory to the dispatch system; In response to receiving the obstacle avoidance permission notification issued by the scheduling system, the obstacle avoidance trajectory and the start execution time of the obstacle avoidance trajectory are published to the at least one slave vehicle.

7. The method according to any one of claims 2 to 6, characterized in that: Before performing parking trajectory planning in response to detecting the collision point, the method further includes: performing collision detection on trajectory points on the task trajectory based on the outline of the queue and obstacles in the global map; The trajectory point corresponding to the intersection of the outline of the queue and the obstacle is determined as the collision point.

8. The method according to any one of claims 2 to 6, characterized in that: Before performing parking trajectory planning in response to detecting the collision point, the method further includes: performing collision detection on track points on the task track according to the expanded outline of the main vehicle; The trajectory point corresponding to the intersection of the expanded outline of the host vehicle and the obstacle is determined as the collision point.

9. The method according to any one of claims 2 to 6, characterized in that: Before performing parking trajectory planning in response to detecting the collision point, the method further includes: In response to receiving a collision point sent by any one of at least one slave vehicle, it is determined that a collision point is detected, and the at least one slave vehicle performs collision detection in the following manner: based on the expanded contour of the slave vehicle, collision detection is performed on the trajectory points on the follow-up trajectory; and the trajectory point corresponding to the intersection of the expanded contour of the slave vehicle and the obstacle is determined as the collision point.

10. The method according to claim 1, characterized in that The method further comprises: During the execution of the task trajectory, if a current trajectory point on the task trajectory has a spin-in-place requirement, obtaining the type of the spin-in-place requirement and the target posture corresponding to the current trajectory point; The master vehicle and the at least one slave vehicle are controlled to rotate according to the target posture using a rotation method corresponding to the in-place spin type.

11. The method according to claim 10, characterized in that The method of adopting a rotation method corresponding to the spin-in-place type and controlling the master vehicle and the at least one slave vehicle to rotate according to the target posture includes: If the in-place spin type is the first type, controlling the master vehicle and the at least one slave vehicle carrying the storage rack to rotate around the geometric center of the queue according to the target posture; If the in-place spin type is the second type, the master vehicle and the at least one slave vehicle are controlled to rotate around their respective Z axes to the target posture.

12. The method according to claim 11, characterized in that The controlling the master vehicle and the at least one slave vehicle carrying the storage rack to rotate around the geometric center of the queue according to the target posture includes: Controlling the master vehicle and the at least one slave vehicle to rotate about their respective Z axes to their respective corresponding tangential directions, wherein the tangential direction corresponding to the master vehicle or the slave vehicle is the tangential direction of a circle with the geometric center of the platoon as the center at the master vehicle or the slave vehicle; Controlling the master vehicle and the at least one slave vehicle to carry the storage rack together and rotate around the geometric center of the queue to a target posture; The master vehicle and the at least one slave vehicle are controlled to rotate around their respective Z axes to a queue posture, in which the master vehicle and the at least one slave vehicle have the same orientation.

13. The method according to claim 1, wherein The method further comprises: During the execution of the task trajectory, in response to receiving a designated task, publishing the designated task and a start execution time of the designated task to the at least one slave vehicle, so that the at least one slave vehicle executes the designated task when the start execution time of the designated task is reached; When the start execution time of the designated task is reached, the designated task is executed.

14. The method according to claim 13, characterized in that The designated task includes a jacking task, and controlling the main vehicle to perform the designated task includes: Obtaining a target lifting height in the lifting task; According to the difference between the target lifting height and the current lifting height of the main vehicle, the main vehicle is controlled to be lifted to the target lifting height.

15. The method according to claim 1, wherein When the start execution time of the task trajectory is reached, after executing the task trajectory, the method further includes: In response to the master vehicle completing the task trajectory and the at least one slave vehicle completing its respective follow-up trajectory, sending an instruction indicating completion of the collaborative task to the scheduling system; In response to receiving the disbanding instruction issued by the dispatching system, the vehicle resumes its own independent movement state, and issues the disbanding instruction to the at least one slave vehicle to notify the at least one slave vehicle to resume its own independent movement state.

16. A multi-device collaborative control method, characterized in that: include: After being determined as a slave vehicle in a queue, receiving a task trajectory and a start execution time of the task trajectory issued by a master vehicle of the queue, wherein the queue includes one master vehicle and at least one slave vehicle; Perform trajectory planning according to the task trajectory to obtain a following trajectory; When the start execution time of the task trajectory is reached, the following trajectory is executed.

17. The method according to claim 16, characterized in that The performing trajectory planning according to the task trajectory to obtain a following trajectory includes: Determine the relative position relationship between the vehicle itself and the host vehicle in the queue; The positions of the track points on the following track are planned according to the positions of the track points on the task track and the relative position relationship.

18. The method according to claim 17, characterized in that The method further comprises: During the process of following the trajectory, the real-time speed of the vehicle, the real-time position of the host vehicle, and the communication delay are obtained, wherein the communication delay is the difference between the time when the host vehicle publishes the real-time position and the time when the vehicle receives the real-time position; determining a desired position of the slave vehicle according to the real-time position, the relative position relationship, the communication delay, and the real-time speed; According to the desired position, the following trajectory is performed.

19. The method according to claim 16, wherein The performing trajectory planning according to the task trajectory to obtain a following trajectory includes: Determining the angular velocity, curvature, and attitude direction corresponding to the trajectory points on the task trajectory; determining a distance between the vehicle and the host vehicle in a direction perpendicular to the attitude direction; The speed of the track point on the following track is planned according to the angular velocity, the curvature and the distance.

20. The method according to any one of claims 16 to 19, characterized in that: The method further comprises: During the execution of the following trajectory, if a parking trajectory and a start execution time of the parking trajectory issued by the host vehicle are received, a parking following trajectory is planned according to the parking trajectory; When the start execution time of the parking trajectory is reached, the parking following trajectory is executed.

21. The method according to any one of claims 16 to 19, characterized in that: The method further comprises: During the execution of the following trajectory, collision detection is performed on the trajectory points on the following trajectory according to the expanded contour of the self; Determining the trajectory point corresponding to the intersection of the expanded outline of the slave vehicle and the obstacle as the collision point; The collision point is sent to the host vehicle.

22. The method according to any one of claims 16 to 19, characterized in that: The method further comprises: During the execution of the following trajectory, if an obstacle avoidance trajectory and a start execution time of the obstacle avoidance trajectory issued by the host vehicle are received, an obstacle avoidance following trajectory is planned according to the obstacle avoidance trajectory; When the start execution time of the obstacle avoidance trajectory is reached, the obstacle avoidance following trajectory is executed.

23. The method according to any one of claims 16 to 19, characterized in that: The method further comprises: During the process of following the trajectory, if a designated task issued by the host vehicle and a start time for executing the designated task are received, the designated task is executed when the start time for executing the designated task is reached.

24. The method according to claim 23, wherein The designated task includes a lifting task, and when the start execution time of the designated task is reached, executing the designated task includes: Obtaining the current lifting height of the main vehicle; The vehicle controls its own lifting speed according to the difference between its own current lifting height and the current lifting height of the main vehicle, so that the vehicle and the main vehicle are synchronously lifted to the target lifting height.

25. The method according to claim 16, wherein After executing the following trajectory when the start execution time of the task trajectory is reached, the method further includes: In response to completing the following trajectory, sending an instruction to the host vehicle indicating completion of the collaborative task; In response to the disbanding command issued by the host vehicle, the host vehicle resumes its own independent movement state.

26. A multi-device collaborative control method, characterized in that: Applied to a multi-device collaborative control system, the multi-device collaborative control system includes a master vehicle and at least one slave vehicle, the master vehicle and the at least one slave vehicle forming a platoon, the method comprising: The master vehicle, in response to being determined as the master vehicle of the queue and receiving the collaborative task, plans a task trajectory according to the collaborative task, and publishes the task trajectory and the start execution time of the collaborative task to the at least one slave vehicle; The at least one slave vehicle performs trajectory planning according to the task trajectory to obtain a respective following trajectory of the at least one slave vehicle; When the start execution time of the mission trajectory is reached, the master vehicle executes the mission trajectory, and the at least one slave vehicle respectively executes its own following trajectory.

27. A multi-device collaborative control device, characterized in that: include: a response module, configured to, in response to being determined as a master vehicle of a queue and receiving a collaborative task, plan a task trajectory according to the collaborative task, wherein the queue includes one master vehicle and at least one slave vehicle; a publishing module, configured to publish the task trajectory and the start execution time of the collaborative task to the at least one slave vehicle, so that the at least one slave vehicle plans its own following trajectory according to the task trajectory and executes its own following trajectory when the start execution time of the task trajectory is reached; The execution module is used to execute the task trajectory when the start execution time of the task trajectory is reached.

28. A multi-device collaborative control device, characterized in that: include: a receiving module, configured to receive, after being determined to be a slave vehicle in a queue, a task trajectory and a start execution time of the task trajectory issued by a master vehicle in the queue, wherein the queue comprises one master vehicle and at least one slave vehicle; A planning module, configured to perform trajectory planning according to the task trajectory to obtain a follow-up trajectory; The following module is used to execute the following trajectory when the start execution time of the task trajectory is reached.

29. An autonomous mobile device, characterized in that: include: A memory and a processor, wherein an application is stored on the memory, and the application is used to execute the method according to any one of claims 1 to 15 or the method according to any one of claims 16 to 25 when called by the processor.

30. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, and the program code is used to execute the method according to any one of claims 1 to 15 or the method according to any one of claims 16 to 25 when called by a processor.

Images