Robot swarm control method

Abstract

The application relates to a robot cluster control method, which collects local environment data through a robot cluster and sends the data to an edge terminal, while the edge terminal acquires global environment data. Based on deep analysis and processing of the global and local environment data, the edge terminal can accurately identify dynamic targets in the environment and their detailed dynamic information. Based on this information, the edge terminal determines a robot task and selects a target robot most suitable for executing the task from the robot cluster, avoiding blind allocation of the task. Further, the edge terminal generates task execution information for the target robot based on the robot task, so that the target robot can act according to the instruction. The application greatly improves the global planning capability of the multi-robot system, so that the robot cluster can quickly and efficiently respond in a complex environment, realize close collaborative work, and significantly improve the ability and execution efficiency of the multi-robot system in dealing with complex tasks.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a method for controlling a robot swarm. Background Technology

[0002] Multi-robot systems, with their ability to perform diverse tasks in complex environments, have demonstrated enormous application potential in numerous fields and have attracted widespread attention. Traditional multi-robot systems primarily employ a distributed architecture, in which each robot makes autonomous decisions based on locally collected information. The advantage of this approach lies in improved system robustness and scalability, as the failure of a single robot will not have a catastrophic impact on the entire system, and the system can be relatively easily expanded in functionality as the number of robots increases. However, distributed architectures also have significant drawbacks, namely poor global coordination capabilities. When faced with complex tasks requiring close collaboration among multiple robots, the lack of unified global planning makes it difficult for robots to achieve efficient cooperation, leading to low task execution efficiency or even task failure.

[0003] In recent years, hybrid control methods have emerged to compensate for the shortcomings of distributed architectures. These methods attempt to introduce a centralized processor onto a distributed architecture, sending data collected by distributed robots to the centralized processor for computation and decision-making, and then controlling the distributed robots to execute corresponding tasks, thereby achieving a certain degree of global planning. However, existing hybrid control methods still have shortcomings in global planning capabilities in practical applications, failing to accurately and efficiently handle dynamic information in complex environments, and struggling to meet the growing demand for multi-robot collaborative operations. Therefore, how to further improve the global planning capabilities of multi-robot systems and achieve more efficient and accurate robot swarm control has become an urgent technical problem to be solved. Summary of the Invention

[0004] Based on this, the purpose of this application is to provide a robot swarm control method to solve the problem of insufficient global planning capability of multi-robot systems in the prior art.

[0005] The robot swarm control method described in this application includes the following steps:

[0006] Several robots in a robot swarm collect several local environmental data and send the several local environmental data to an edge terminal;

[0007] The edge terminal acquires global environmental data and several local environmental data.

[0008] The edge terminal generates a global environment model based on the global environment data and the several local environment data; identifies dynamic targets and their dynamic information based on the global environment model; determines the robot task based on the dynamic information of the dynamic targets; and identifies the target robot in the robot cluster that will perform the robot task.

[0009] The edge terminal generates task execution information for the target robot based on the robot task; and sends the task execution information to the target robot.

[0010] The target robot performs the robot task based on the task execution information.

[0011] This application embodiment collects local environmental data through a robot swarm and sends it to an edge terminal, while the edge terminal simultaneously acquires global environmental data. Based on in-depth analysis and processing of the global and local environmental data, the edge terminal can accurately identify dynamic targets in the environment and their detailed dynamic information. Based on this information, the edge terminal determines the robot task and selects the most suitable target robot from the robot swarm to perform the task, avoiding blind task allocation. Furthermore, the edge terminal generates task execution information for the target robot based on the robot task, enabling the target robot to act according to instructions. This application significantly improves the global planning capability of multi-robot systems, enabling robot swarms to respond quickly and efficiently in complex environments, achieving close collaborative operation, effectively solving the problem of insufficient global coordination capability in existing technologies, and significantly improving the ability and execution efficiency of multi-robot systems to handle complex tasks.

[0012] To better understand and implement this application, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the robot swarm control method according to an embodiment of this application;

[0014] Figure 2 This is a schematic diagram illustrating the steps of a robot uploading local environmental data to an edge terminal based on preset features, as described in an embodiment of this application.

[0015] Figure 3 A schematic diagram illustrating the steps for constructing a global environment model for an edge terminal in this application embodiment;

[0016] Figure 4 This is a schematic diagram illustrating the steps of an edge terminal calling upon an idle robot to share computing power tasks in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Wherein, when the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0018] It should be understood that the embodiments described below do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0019] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application are also intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, in the description of this application, unless otherwise stated, “a plurality” means two or more. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items, for example, A and / or B, which can represent: A alone, A and B together, and B alone; the character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0020] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms, and these terms are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Depending on the context, the word "if" as used in this application can be interpreted as "when," "when," or "in response to determination."

[0021] This application provides a robot swarm control method to solve the problem of insufficient global planning capability in existing multi-robot systems.

[0022] Please refer to Figure 1 The robot swarm control method described in this application includes the following steps:

[0023] S101: Several robots in the robot cluster collect several local environmental data and send the several local environmental data to the edge terminal;

[0024] S102: The edge terminal acquires global environmental data and the several local environmental data;

[0025] S103: The edge terminal generates a global environment model based on the global environment data and the several local environment data; identifies dynamic targets and their dynamic information based on the global environment model; determines the robot task based on the dynamic information of the dynamic targets, and determines the target robot in the robot cluster that will perform the robot task.

[0026] S104: The edge terminal generates task execution information for the target robot based on the robot task; and sends the task execution information to the target robot;

[0027] S105: The target robot executes the robot task based on the task execution information.

[0028] This application embodiment collects local environmental data through a robot swarm and sends it to an edge terminal, while the edge terminal simultaneously acquires global environmental data. Based on in-depth analysis and processing of the global and local environmental data, the edge terminal can accurately identify dynamic targets in the environment and their detailed dynamic information. Based on this information, the edge terminal determines the robot task and selects the most suitable target robot from the robot swarm to perform the task, avoiding blind task allocation. Furthermore, the edge terminal generates task execution information for the target robot based on the robot task, enabling the target robot to act according to instructions. This application significantly improves the global planning capability of multi-robot systems, enabling robot swarms to respond quickly and efficiently in complex environments, achieving close collaborative operation, effectively solving the problem of insufficient global coordination capability in existing technologies, and significantly improving the ability and execution efficiency of multi-robot systems to handle complex tasks.

[0029] In step S101, several robots in the robot cluster collect several local environmental data and send the several local environmental data to the edge terminal.

[0030] In this context, a robot swarm is a group of multiple robots that can collaborate to complete a specific task. In this embodiment, each robot in the swarm has the ability to independently collect environmental data.

[0031] Local environmental data refers to the environmental information perceived by a single robot within its local area, such as the location and distance of surrounding obstacles, and the local terrain. This data reflects the environmental conditions within a limited range around the robot.

[0032] An edge terminal is a device with strong data processing and communication capabilities, typically deployed close to the data source, i.e., the robot cluster. It can quickly receive data sent by the robots, perform preliminary processing and analysis, and also interact with other systems or sensors. In this embodiment, the robot sends the collected local environmental data to the edge terminal for subsequent integration and processing.

[0033] In this step, several robots in the robot cluster use their onboard sensors, such as cameras, lidar, and ultrasonic sensors, to collect environmental data of their respective local areas. Then, they send this local environmental data to the edge terminal via wireless communication modules such as Wi-Fi, Bluetooth, and ZigBee.

[0034] In one embodiment, step S101, which involves several robots in a robot swarm collecting several local environmental data, includes:

[0035] In step S1011, several robots in the robot cluster collect corresponding local environmental data through the first hyper-fusion module they are equipped with; the first hyper-fusion module integrates at least two of the following: a visual sensor, a radar sensor, and an inertial measurement unit.

[0036] The first hyper-fusion module is a multi-functional sensor assembly integrated onto the robot, combining at least two of the following: a vision sensor, a radar sensor, and an inertial measurement unit (IMU). The vision sensor acquires visual information about the environment, such as the color and texture of objects, through image acquisition and analysis; the radar sensor uses the reflection of electromagnetic waves to detect information such as the distance and speed of objects; and the IMU measures the robot's motion parameters, such as acceleration and angular velocity. By fusing these sensors, more comprehensive and accurate data about the local environment surrounding the robot can be obtained.

[0037] In this embodiment, several robots in the robot swarm each activate their onboard first hyper-fusion module, in which various sensors work collaboratively. Visual sensors acquire image information of the surrounding environment, radar sensors emit and receive electromagnetic waves to obtain distance and velocity data of objects, and inertial measurement units monitor the robot's motion state in real time. By fusing the data collected by these different sensors, the robot can generate corresponding local environmental data and send this data out, providing a foundation for subsequent global environment construction.

[0038] In one embodiment, step S1011, which involves several robots in a robot cluster collecting corresponding local environmental data through their onboard first hyper-converged module, includes:

[0039] In step S10111, the plurality of robots respectively acquire first local image data, first local point cloud data, and first local motion data through the vision sensor, radar sensor, and inertial measurement unit of the first hyper-fusion module they are equipped with; and obtain a plurality of local environmental data based on the plurality of first local image data, the plurality of first local point cloud data, and the plurality of first local motion data.

[0040] The first local image data consists of image information collected by the robot's vision sensor within its local environment, reflecting the visual characteristics of objects in that area.

[0041] The first local point cloud data is local environmental data collected by the robot's radar sensors. It records the spatial position and distance information of surrounding objects in the form of points, which can construct the three-dimensional structure of the local environment.

[0042] The first local motion data is collected by the robot's inertial measurement unit and reflects the robot's own motion state, such as acceleration and angular velocity, which is used to determine the robot's motion trajectory and attitude changes.

[0043] In this step, several robots in the robot swarm are each equipped with a first hyper-fusion module. Each robot uses the vision sensors in this module to acquire first local image data of its surrounding environment, which includes the visual features of objects within that local area. Simultaneously, the robot scans its surrounding environment using radar sensors in the first hyper-fusion module to obtain first local point cloud data, which presents the position and shape information of surrounding objects in three-dimensional space. Furthermore, the robot also uses the inertial measurement unit in the first hyper-fusion module to acquire its own first local motion data, including acceleration and angular velocity, to reflect the robot's motion state and attitude changes. Finally, the robot performs comprehensive processing and analysis on the acquired first local image data, first local point cloud data, and first local motion data to obtain several local environmental data that comprehensively reflect the condition of its surrounding local environment.

[0044] In this embodiment, the robot collects local environmental data by being equipped with a first hyper-fusion module, fully utilizing the advantages of visual sensors, radar sensors, and inertial measurement units (IMUs). Visual sensors provide rich visual information, helping the robot accurately identify objects in the environment; radar sensors are not limited by lighting conditions and can work stably in various environments, providing reliable distance and spatial information; the IMU monitors the robot's motion status in real time, providing crucial information for its localization and navigation. Fusing the data from these three sensors yields local environmental data, making the collected data more comprehensive, accurate, and reliable. This multi-sensor fusion approach enhances the robot's perception of complex environments, enabling more precise acquisition of surrounding environmental information. This provides a high-quality data foundation for subsequent edge terminal operations such as global environment modeling, dynamic target recognition, and task allocation, thereby improving the performance and stability of the entire robot swarm control system. This allows it to better cope with complex and changing working environments and efficiently complete various tasks.

[0045] Please refer to Figure 2 In one embodiment, the step of sending the partial local environmental data collected by the several robots in the robot cluster in step S101 to the edge terminal includes:

[0046] Step S1012: Several robots in the robot cluster collect some local environmental data;

[0047] Step S1013: When the plurality of local environmental data are identified to have preset features, the plurality of robots send the plurality of local environmental data to the edge terminal.

[0048] Each robot has a pre-defined set of features, determined based on the specific application scenario and task requirements. For example, in a warehouse logistics scenario, these features might be specific types of goods, abnormally stacked items, or malfunctioning equipment. When the robot collects local environmental data containing these pre-defined features, it indicates a situation requiring special attention or action. At this point, the robot sends this local environmental data with pre-defined features to an edge terminal. The edge terminal, with its enhanced computing and processing capabilities, can further analyze and make decisions based on this data, coordinating the actions of the entire robot swarm.

[0049] Step S1014: When it is identified that the plurality of local environmental data does not have preset features, the plurality of robots determine the task to be executed based on the plurality of local environmental data and execute the task to be executed.

[0050] If the robot identifies that the collected local environmental data does not possess preset characteristics, it means that the current environment is in a normal state and there are no special circumstances requiring immediate reporting. In this case, the robot will determine the next task to be performed based on this local environmental data, according to its programming and algorithms. For example, in a warehouse, the robot may determine the task of moving goods from a specific shelf based on the storage location and transportation needs, and execute the task immediately to improve work efficiency and resource utilization.

[0051] In this embodiment, for local environmental data without preset characteristics, the robot can autonomously determine and execute tasks without relying on instructions from the edge terminal each time. This improves the robot's autonomous decision-making ability and response speed, enabling it to more flexibly handle various situations in daily work environments. The edge terminal only needs to process key data with preset characteristics, reducing its computational burden and allowing it to allocate more resources to complex global decision-making and coordination tasks. This distributed data processing and task execution method reduces dependence on a single node. Even if the edge terminal malfunctions or the network connection is interrupted, the robot can still autonomously execute certain tasks based on local environmental data, ensuring the basic operation of the system and improving the reliability and stability of the entire robot cluster.

[0052] In step S102, the edge terminal acquires global environmental data and several local environmental data.

[0053] The global environmental data encompasses macroscopic environmental information across the entire mission execution area, including large-scale terrain maps, weather conditions, and the location of global targets. This data can be acquired from external systems such as geographic information systems, weather forecasting systems, or sensors deployed in key locations.

[0054] In this step, while receiving some local environmental data sent by the robot, the edge terminal obtains global environmental data by connecting with external systems or reading its own stored preset data, thus gaining comprehensive information about the entire task environment.

[0055] In one embodiment, step S102, the step of the edge terminal acquiring global environmental data, includes:

[0056] Step S1021: The edge terminal collects global environmental data through the second hyper-converged module; wherein the second hyper-converged module integrates at least two of the following: a visual sensor, a radar sensor, and an inertial measurement unit.

[0057] The second hyperconverged module is a sensor combination module integrated on the edge terminal, which also integrates at least two of the following: visual sensors, radar sensors, and inertial measurement units. Its function is similar to that of the first hyperconverged module, but because it is deployed on the edge terminal, it may focus more on acquiring large-scale, macroscopic environmental information and can integrate and analyze data from multiple sources.

[0058] In this embodiment, the edge terminal activates its second hyper-fusion module. The visual sensor in this module performs a wide-area image scan of the task area, acquiring visual information such as terrain and obstacles; the radar sensor emits electromagnetic waves covering a large area to detect the distance and position of fixed obstacles; and the inertial measurement unit monitors the edge terminal's own motion state (if the edge terminal has mobility). By fusing the data collected by these sensors, the edge terminal can acquire global environmental data, thereby gaining a comprehensive understanding of the entire task environment.

[0059] This application's embodiments, through the design of a first hyper-fusion module and a second hyper-fusion module, enable the system to simultaneously acquire both local and global environmental information, avoiding the incomplete or inaccurate information problems that may arise from a single sensor or a single data source. This multi-source data fusion approach provides a rich and accurate data foundation for subsequent edge terminal tasks such as global environment model construction, dynamic target recognition, and robot task allocation. This significantly improves the global planning and dynamic adaptability of multi-robot systems in complex environments, enabling robot swarms to complete collaborative tasks more efficiently and accurately. It effectively solves the problems of incomplete and inaccurate environmental perception leading to system decision-making errors and low task execution efficiency in existing technologies.

[0060] In one embodiment, step S1021, the step of the edge terminal collecting global environmental data through the second hyperconverged module, includes:

[0061] In step S10211, the edge terminal collects first image data, first point cloud data, and first motion data through the visual sensor, radar sensor, and inertial measurement unit of the second hyper-fusion module, respectively; and obtains global environment data based on the first image data, the first point cloud data, and the first motion data.

[0062] In this step, the edge terminal is equipped with a second hyper-fusion module. The vision sensor within this module acquires images of a large area of ​​the environment, obtaining first image data. This data includes environmental information and the appearance features of objects within the environment. Simultaneously, the edge terminal uses the radar sensor in the second hyper-fusion module to perform a comprehensive scan of the surrounding environment, obtaining first point cloud data. This data presents the position and shape distribution of objects in the environment in three-dimensional space. Furthermore, the edge terminal uses the inertial measurement unit in the second hyper-fusion module to collect its own first motion data, including acceleration and angular velocity, to reflect its motion state and attitude changes. Finally, the edge terminal performs comprehensive processing and analysis based on the acquired first image data, first point cloud data, and first motion data, integrating the advantages of different sensor data to obtain global environmental data that comprehensively and accurately reflects the overall working environment.

[0063] In this embodiment, the edge terminal uses a second hyper-fusion module to collect global environmental data, fully leveraging the strengths of the visual sensor, radar sensor, and inertial measurement unit. The first image data provided by the visual sensor offers rich visual details of the global environment, aiding in the accurate identification of various objects and scenes. The radar sensor, unaffected by environmental factors such as lighting, collects first point cloud data that stably presents the spatial position and shape of objects, providing reliable spatial information for constructing an accurate global environment model. The first motion data collected by the inertial measurement unit can monitor the edge terminal's own motion state and attitude changes in real time, assisting in correcting errors during data collection and ensuring the accuracy of the global environmental data. By fusing the data from these three sensors to obtain global environmental data, the collected data becomes more comprehensive, accurate, and reliable, providing a high-quality data foundation for subsequent operations such as robot task planning and dynamic target tracking based on global environmental data. This improves the environmental adaptability and task execution efficiency of the entire robot system, enhancing its stability and reliability in complex environments.

[0064] In this embodiment, the robot and edge terminal significantly improve the quality and richness of environmental data through multi-sensor data acquisition, refined processing, and feature fusion. By combining local and global environmental data, the system can more accurately perceive environmental changes, rationally plan robot tasks, and achieve efficient collaborative operation of robot swarms in complex environments. Compared with traditional methods, this embodiment effectively solves problems such as unreasonable task planning and low robot collaboration efficiency caused by inaccurate or incomplete data, significantly improving the ability and execution efficiency of multi-robot systems to handle complex tasks, and providing strong support for the widespread application of multi-robot systems in more fields.

[0065] For step S103, the edge terminal generates a global environment model based on the global environment data and the several local environment data; identifies dynamic targets and their dynamic information based on the global environment model; determines the robot task based on the dynamic information of the dynamic targets; and determines the target robot in the robot cluster that will perform the robot task.

[0066] The global environment model is a digital representation of the entire task environment. It integrates global and local environmental data and is constructed using specific algorithms and models. This model can accurately describe the location, state, and interrelationships of various elements in the environment, providing a basis for subsequent decision-making.

[0067] Dynamic targets refer to objects in the environment that have the ability to move autonomously or whose motion state may change, such as moving obstacles, moving objects that need to be tracked or grasped.

[0068] Dynamic information refers to information related to the motion state, position changes, and behavior patterns of dynamic targets in the environment. Specifically, it can include information on the dynamic target's position, velocity, acceleration, and direction of motion as they change over time. This information reflects the motion state and trend of the dynamic target and is an important basis for edge terminals to determine robot tasks.

[0069] Robot tasks are specific work items that the edge terminal determines after identifying dynamic targets and their dynamic information based on the global environment model, and that need to be completed by the target robots in the robot cluster. It clarifies the goals that the robots need to achieve and the related work requirements.

[0070] In this step, the edge terminal uses preset data processing algorithms and models to fuse and analyze global environmental data and several local environmental data to generate a global environmental model. Through real-time monitoring and analysis of the global environmental model, the edge terminal can identify dynamic targets in the environment and accurately obtain the dynamic information of these targets. Based on the dynamic information of the targets, the edge terminal, in conjunction with preset task rules, determines the corresponding robot task and selects the most suitable target robot from the robot cluster to perform the task.

[0071] Please refer to Figure 3In one embodiment, the global environment data includes global multimodal perception data corresponding to several timestamps; the local environment data includes local multimodal perception data corresponding to several timestamps; the global multimodal perception data includes first image data, first point cloud data, and first motion data collected by a visual sensor, a radar sensor, and an inertial measurement unit, respectively; the local multimodal perception data includes first local image data, first local point cloud data, and first local motion data collected by a visual sensor, a radar sensor, and an inertial measurement unit, respectively.

[0072] Step S103, the step of the edge terminal generating a global environment model based on the global environment data and the plurality of local environment data, includes:

[0073] Step S1031: The edge terminal performs time synchronization processing on the first local image data, the first local point cloud data, and the first local motion data; the time-synchronized first local image data, the first local point cloud data, and the first local motion data are converted to the same spatial coordinate system of the first hyper-fusion module to obtain second local image data, second local point cloud data, and second local motion data; obstacle recognition and noise removal are performed on the second local point cloud data using a point cloud clustering algorithm, and the center points of the clustered point clouds are spatially associated with the center points of the target detection boxes in the second local image data; target features of the second local image data, obstacle size features of the second local point cloud data, and pose features of the second local motion data are extracted, and feature layer fusion is performed through a deep learning fusion network to obtain local fused data.

[0074] In this step, since the data collected by different sensors may differ in time, time synchronization processing aligns the data collected by different sensors according to a unified time standard to ensure the consistency of the data in the time dimension, so as to facilitate accurate data analysis and fusion in the future.

[0075] Point cloud clustering algorithms are used to group point cloud data, grouping points with similar characteristics together to form different clusters. In this embodiment, it is mainly used to identify obstacles and remove noise from the second local point cloud data, grouping points belonging to the same obstacle together while removing invalid noise points.

[0076] In image processing, the object detection algorithm draws a rectangle for the detected object in the image. The center point of this rectangle is called the center point of the object detection box, which is used to identify the approximate location of the object in the image.

[0077] Spatial association involves matching different types of data (such as the center point of a point cloud and the center point of a target detection box) in space to determine their correspondence in three-dimensional space, so as to perform more accurate data fusion and analysis.

[0078] Deep learning fusion networks are neural network models built using deep learning techniques for feature extraction and fusion of different types of data. They can automatically learn complex features from data and effectively integrate features from different sources, improving the data's expressive power and information utilization.

[0079] In step S1032, the edge terminal performs time synchronization processing on the first image data, the first point cloud data, and the first motion data; the time-synchronized first image data, the first point cloud data, and the first motion data are converted to the same spatial coordinate system of the second hyper-fusion module to obtain second image data, second point cloud data, and second motion data; the geometric features of the second point cloud data and the pixel features of the second image data are associated and fused through a deep learning fusion network to extract the contour features of the second image data, the geometric features of the second point cloud data, and the stability features of the second motion data, and feature layer fusion is performed to obtain global fused data.

[0080] Step S1033: The edge terminal integrates the local fusion data and the global fusion data through a Bayesian filtering framework to obtain integrated data; static features and dynamic features are extracted from the integrated data; wherein, the static features include the position and size of fixed obstacles and the boundary of passable areas, and the dynamic features include the real-time position, movement speed and movement trajectory of dynamic targets.

[0081] The Bayesian filtering framework is a probabilistic filtering method based on Bayes' theorem, used to estimate and predict the state of dynamic systems. In this embodiment, it is used to integrate local and global fused data, and estimate the true state of the system by continuously updating the probability distribution.

[0082] Static features refer to features in the environment that are fixed or change slowly, such as the location and size of fixed obstacles and the boundaries of passable areas. These features are relatively stable over a period of time.

[0083] Dynamic features refer to the features in the environment that change over time, such as the real-time position, speed, and trajectory of a dynamic target. These features change as the target moves.

[0084] Step S1034: The static features are integrated using a clustering algorithm to generate a static environment layer; the dynamic features are tracked and optimized using a Kalman filter algorithm to generate a dynamic target layer.

[0085] Clustering algorithms are unsupervised learning algorithms used to group objects in a dataset, ensuring high similarity among objects within the same group (cluster) and low similarity between objects in different groups. Here, it's used to integrate static features, classifying static data points with similar characteristics to form a static environment layer.

[0086] The Kalman filter algorithm is a highly efficient recursive filter capable of estimating and predicting the state of dynamic systems. Here, it is used to optimize the tracking of dynamic features, accurately predicting and updating the real-time state of the dynamic target based on its historical state and current observation data, thus generating a dynamic target layer.

[0087] This step employs a clustering algorithm to integrate static features, grouping static data points with similar characteristics, such as fixed obstacles in close proximity, to generate a static environment layer. This layer clearly displays the static structural information of the global environment. A Kalman filter algorithm is then used to track and optimize dynamic features. Based on the historical state of the dynamic target and current observation data, it accurately predicts and updates the real-time position, velocity, and trajectory of the dynamic target, generating a dynamic target layer. This layer reflects the motion state of the dynamic target in the environment in real time.

[0088] In this embodiment, to achieve accurate tracking of dynamic features, it is necessary to first define the state space of the dynamic target and update it based on observations from multiple sensors:

[0089] 1. State-space representation and prediction

[0090] The system's state space is used to represent the state of dynamic targets (such as trapped personnel or moving goods), and is defined as follows:

[0091] in,( This indicates the two-dimensional position of the target in the world coordinate system (unit: meters). Indicates movement speed (unit: meters per second). Indicates the angle of orientation (unit: radians). This represents the confidence level of the state estimate (range [0, 1]).

[0092] In the prediction phase, the state at the next moment is predicted based on the uniform kinematics model:

[0093]

[0094] in, This is the state transition function. The process noise follows a multidimensional normal distribution with a mean of zero, and its covariance matrix Q = diag[0.05, 0.05, 0.1, 0.02].

[0095] 2. Multi-sensor observation update

[0096] During the update phase, data is observed using sensors. The predicted state is corrected using a Kalman filter framework:

[0097]

[0098] in, For Kalman gain, For the observation matrix, To observe the noise covariance matrix, and These are the predicted and updated state covariance matrices, respectively.

[0099] In this embodiment, the Kalman filter algorithm achieves the optimal estimation of the dynamic target state through the following recursive formula:

[0100] 1. Prediction Phase:

[0101] State prediction:

[0102] Covariance prediction:

[0103] 2. Update Phase:

[0104] Kalman gain calculation:

[0105] Status Update:

[0106] Covariance update:

[0107] Parameter description:

[0108] : Prior state estimate (predicted value) at time step tt;

[0109] : Posterior state estimate (updated optimal value) at time step tt.

[0110] : State transition model, defined according to the target motion model (such as uniform motion model);

[0111] : Control input model, utut is the control vector (such as acceleration);

[0112] Prior estimation of covariance (prediction uncertainty);

[0113] Posterior estimation of covariance (updated uncertainty);

[0114] Process noise covariance represents the uncertainty in the state transition process;

[0115] : Observation model, which maps state vectors to the observation space;

[0116] : The actual observed value at time step tt;

[0117] Observation noise covariance represents the uncertainty of sensor measurements;

[0118] Kalman gain determines the weight of observations in state updates;

[0119] : Identity matrix.

[0120] Through iterative steps, the system can continuously output optimized real-time dynamic information such as the target's position, velocity, and trajectory based on the target's historical state and current multi-sensor observation data.

[0121] Step S1035: The static environment layer and the dynamic target layer are spatiotemporally correlated and fused to generate a global environment model containing static environment information and the real-time state of the dynamic target.

[0122] In this embodiment, the step of spatiotemporally fusing the static environment layer and the dynamic target layer aims to establish a unified environment model, in which the movement of the dynamic target and the static environment are mutually constrained. The fusion process is achieved through the following core calculations to establish the mutual constraint relationship between the dynamic target and the static environment:

[0123] 1. Spatial Association - Interaction Detection of Dynamic Targets and Static Obstacles:

[0124] For the predicted position of each dynamic target at time t Check its usage on the static environment layer G:

[0125] Where G(x, y) is the occupancy probability value of the static occupancy grid at coordinates (x, y) (range [0, 1]).

[0126] Among them, if (in If a preset occupancy threshold (e.g., 0.5) is set, then the dynamic target is considered to be in spatial conflict or interaction with the static obstacle.

[0127] 2. Temporal correlation – Walkability analysis of dynamic target trajectories:

[0128] Based on the motion state of a dynamic target, predict the set of trajectory points within the next time interval Δt. For each predicted point on the trajectory Calculate the conflict cost between it and the static environment:

[0129]

[0130] Static environment feasibility score for the entire trajectory It can be represented as:

[0131] Where H is the trajectory prediction step size.

[0132] 3. Fusion model generation:

[0133] Final global environment model It is a composite data structure that contains spatiotemporal correlation information: ,in:

[0134] : Static environment layer (occupies grid space).

[0135] Dynamic target layer (containing target state sequence) ).

[0136] Interactive layer, one with A grid of the same size, where each cell The value records the most recent Within a given time period, whether a dynamic target has entered this cell or its neighborhood can be represented as: .in, It is Euclidean distance. It is the distance attenuation parameter.

[0137] Through the spatiotemporal correlation fusion described above, the generated global environment model not only includes the instantaneous states of static structures and dynamic targets, but also reveals the interaction between the two (such as the feasible area of ​​dynamic targets and potential collision risks), providing a safer and more efficient decision-making basis for collaborative path planning and task execution of robot swarms.

[0138] Spatiotemporal correlation fusion is the process of associating and fusing static environment layers and dynamic target layers in time and space, enabling the global environment model to simultaneously contain static environment information and the real-time status of dynamic targets, thus reflecting the environmental situation more comprehensively and accurately.

[0139] This step involves spatiotemporally fusing the static environment layer and the dynamic target layer, linking the static environment information and the real-time state of the dynamic target in both time and space to form a global environment model containing comprehensive environmental information. This model can simultaneously reflect both the static structure and dynamic changes in the environment, providing a more accurate and comprehensive environmental basis for task planning and collaborative operations of robot swarms.

[0140] This embodiment generates an accurate and comprehensive global environment model through a series of complex and sophisticated data processing and fusion techniques. First, time synchronization processing is performed on global and local multimodal sensing data to ensure accurate data correspondence in the time dimension, laying the foundation for subsequent fusion and analysis. Data collected by different hyper-fusion modules is converted to the same spatial coordinate system, resolving the inconsistency in spatial coordinate systems between different sensors and modules, enabling accurate data fusion in the spatial dimension. Point cloud clustering algorithms and spatial association techniques effectively identify and eliminate noisy data, improving data quality and reliability. The application of deep learning fusion networks automatically extracts complex features from different types of data and performs efficient feature layer fusion, fully mining the potential information in the data and enhancing its expressive power and information utilization. A Bayesian filtering framework integrates local and global fused data, further optimizing the estimation of system state and improving data accuracy and consistency. The extraction and separate processing of static and dynamic features, using clustering and Kalman filtering algorithms to generate static environment layers and dynamic target layers respectively, makes the representation of static environment and dynamic targets clearer and more accurate. Finally, the static environment layer and the dynamic target layer are spatiotemporally correlated and fused. The resulting global environment model can simultaneously contain static environment information and the real-time status of dynamic targets, providing comprehensive and accurate environmental information support for robot clusters in task planning, path planning, and dynamic target tracking. This greatly improves the robot clusters' environmental perception, decision-making ability, and task execution efficiency in complex environments, and enhances the stability and reliability of the system.

[0141] Please refer to Figure 4 In one embodiment, step S103, where the edge terminal generates a global environment model based on the global environment data and the plurality of local environment data; identifies dynamic targets and their dynamic information based on the global environment model; determines a robot task based on the dynamic information of the dynamic targets, and determines the target robot in the robot cluster that performs the robot task, further includes:

[0142] Step S201: The edge terminal monitors its own computing load;

[0143] Computational load refers to the utilization of hardware resources such as processors and memory on an edge terminal during operation, typically measured by metrics such as CPU utilization and memory usage. The level of computational load reflects the current workload of the edge terminal; excessively high loads may affect its processing speed and response time.

[0144] Step S202: When the computational load exceeds a preset threshold, the edge terminal splits the task to be computed into several sub-computation tasks; wherein, the task to be computed refers to generating a global environment model based on the global environment data and the several local environment data; identifying dynamic targets and their dynamic information based on the global environment model; determining robot tasks based on the dynamic information of the dynamic targets; and determining the computational tasks of the target robot in the robot cluster that executes the robot tasks.

[0145] The preset threshold is a pre-defined value used to determine whether the computational load on the edge terminal is too high. When the computational load exceeds this preset threshold, it indicates that the edge terminal is currently under heavy workload, and corresponding measures need to be taken to distribute the workload to ensure the normal operation of the system.

[0146] The task to be computed in this embodiment is a comprehensive task, which includes multiple steps such as generating a global environment model based on global environment data and several local environment data, identifying dynamic targets and their dynamic information based on the global environment model, determining the robot task based on the dynamic information of the dynamic targets, and determining the target robot in the robot cluster that will perform the robot task.

[0147] In this step, when the computational load on the edge terminal exceeds a preset threshold, the task to be processed is split into several smaller tasks. These sub-tasks are part of the task to be processed and have relative independence and parallel processing capability.

[0148] Step S203: The edge terminal determines the robots in the robot cluster that are in an idle state, and assigns at least a portion of the sub-computation tasks to the idle robots.

[0149] In this context, idle robots refer to robots in the robot cluster that are not currently performing any specific tasks, and whose processors, sensors, and other hardware resources are idle. These robots have the ability to receive and process sub-computation tasks. This step assigns some sub-computation tasks to idle robots for processing.

[0150] Step S204: The idle robot processes the sub-computation task to obtain the calculation result and sends the calculation result to the edge terminal;

[0151] In step S205, the edge terminal receives the calculation result, completes the task to be calculated based on the calculation result, and determines the target robot in the robot cluster to execute the robot task.

[0152] In this embodiment, the edge terminal monitors its own computing load to grasp its processing capacity status in real time. When the computing load exceeds a preset threshold, the edge terminal breaks down the comprehensive task into several sub-tasks. This breakdown makes the originally complex and large task more manageable and assignable. Next, the edge terminal identifies idle robots in the robot cluster and assigns at least some of the sub-tasks to them. Idle robots utilize their own hardware resources to process the sub-tasks, fully leveraging the computing potential of each robot in the cluster and achieving efficient resource utilization. After completing the sub-tasks, the idle robots send the results to the edge terminal, which receives these results and uses them to complete the task, thereby determining the robot task and the target robot to execute it. The entire solution, through a dynamic task allocation mechanism, flexibly adjusts the task processing method according to the edge terminal's computing load, avoiding processing delays or task failures caused by excessive edge terminal computing load, thus improving system stability and reliability. At the same time, it makes full use of the computing resources of idle robots in the robot cluster, improves the overall task processing efficiency, and enables the robot cluster to respond to environmental changes and execute corresponding tasks more quickly and accurately, thereby enhancing the system's adaptability and collaborative work capabilities in complex environments.

[0153] In one embodiment, step S103, where the edge terminal determines the robot task based on the dynamic information of the dynamic target and determines the target robot in the robot cluster to perform the robot task, includes:

[0154] Step S301: When there is only one robot task, the edge terminal obtains the physical performance parameters, sensor and cognitive ability parameters, and historical task execution success rates of each robot in the robot cluster; performs a weighted calculation on the physical performance parameters, the sensor and cognitive ability parameters, and the historical task execution success rates to obtain the comprehensive capability value of each robot; and determines the robot with the highest comprehensive capability value as the target robot to perform the robot task.

[0155] Physical performance parameters are indicators that describe the performance of robot hardware, such as the robot's moving speed, maximum load capacity, and endurance. These parameters reflect the robot's ability to perform tasks at the physical level.

[0156] Sensory cognitive ability parameters are indicators that reflect a robot's ability to perceive and understand its environment. These parameters include the accuracy of sensors, the range of perception, and the ability to identify different environmental features. They determine the accuracy and comprehensiveness of the environmental information acquired by the robot.

[0157] Historical task execution success rate is the ratio of the number of times a robot has successfully completed a task in the past to the total number of tasks performed. It reflects the reliability and stability of the robot in actual task execution.

[0158] The overall capability score is a comprehensive index calculated by weighting the robot's physical performance parameters, sensory and cognitive ability parameters, and historical task execution success rate. It is used to measure the robot's overall ability to perform tasks.

[0159] Step S302: When there are multiple robot tasks, the edge terminal acquires the task parameters of each robot task and the capability parameters, current load, and location information of each robot in the robot cluster; wherein, the task parameters include task requirements, task priority, and task execution time limit; based on the task parameters of the robot tasks, the capability parameters of each robot, the current load, and the location information, multiple sets of task schemes and the execution efficiency corresponding to each set of task schemes are obtained; wherein, the task scheme indicates each robot task and the robot corresponding to each robot task; from the multiple sets of task schemes, the task scheme that satisfies the task priority, execution time limit, and has the highest execution efficiency is determined as the optimal task scheme; based on the optimal task scheme, each target robot corresponding to each robot task is determined.

[0160] Among them, the task requirements specify the specific requirements that need to be met to complete the task; the task priority indicates the importance and urgency of the task; and the task execution deadline indicates when the task must be completed.

[0161] Capability parameters are indicators that comprehensively reflect a robot's ability to perform various tasks. They cover multiple aspects such as physical performance and sensory and cognitive abilities, and are used to evaluate whether a robot has the ability to complete a specific task.

[0162] Current load refers to the number or complexity of tasks that the robot is currently performing, reflecting the robot's current workload.

[0163] Location information refers to the robot's specific coordinates in the working environment, which is of great significance for rationally scheduling the robot's tasks and considering factors such as task execution paths and time.

[0164] A task plan is a combination of methods for allocating and arranging robot tasks and the robots that perform those tasks, specifying which robots will perform each task.

[0165] Execution efficiency is an indicator for measuring the quality of a task plan. It comprehensively considers factors such as task completion time, resource consumption, and task completion quality, and reflects the effectiveness of the task plan in achieving the task objectives.

[0166] The optimal task scheme is the one that meets the requirements of task priority, execution time limit and execution efficiency among multiple task schemes. It is the best allocation strategy selected by the edge terminal to efficiently complete multiple robot tasks.

[0167] In this embodiment, the edge terminal employs different target robot determination strategies based on the number of robot tasks. When there is only one robot task, the edge terminal obtains the physical performance parameters, sensory and cognitive ability parameters, and historical task execution success rates of each robot in the robot cluster, performs weighted calculations to obtain a comprehensive capability value, and then determines the robot with the highest comprehensive capability value as the target robot. This approach comprehensively considers multiple aspects of the robot, such as hardware performance, perception capabilities, and task execution reliability, and can accurately select the robot most suitable for performing the single task, ensuring that the task is completed with high quality and efficiency. When there are multiple robot tasks, the edge terminal obtains the task parameters of each robot task, as well as the robot's capability parameters, current load, and location information. Based on this rich information, it generates multiple sets of task schemes and calculates the execution efficiency of each set of schemes. Finally, it determines the optimal task scheme that satisfies the task priority, execution time limit, and highest execution efficiency, and determines the target robot corresponding to each task accordingly. This strategy fully considers the priority relationships, time constraints, and actual capabilities and working status of multiple tasks. Through comprehensive evaluation and optimized allocation, it achieves efficient collaborative execution of multiple tasks in the robot cluster, avoids task conflicts and resource waste, improves the task processing capability and work efficiency of the entire robot cluster, and enables the system to better cope with complex and ever-changing task scenarios and environmental requirements.

[0168] In one embodiment, step S302, where the edge terminal obtains multiple task schemes and the corresponding execution efficiency of each task scheme based on the task parameters of the robot task, the capability parameters of each robot, the current load, and the location information; and determines the optimal task scheme from the multiple task schemes as the one that satisfies the task priority, execution time limit, and highest execution efficiency, includes:

[0169] Step S3021: The edge terminal generates several initial task schemes based on the task parameters of the robot task, the capability parameters of each robot, the current load, and the location information; wherein each initial task scheme includes a mapping relationship between each robot task and each robot; the mapping relationship satisfies that the total task volume of each robot does not exceed its current load and the capability parameters meet the task requirements of the corresponding robot task.

[0170] In this step, each initial task plan clarifies the mapping relationship between each robot task and each robot, that is, it determines which robot will perform each task. This mapping relationship must meet two key conditions: first, the total number of tasks for each robot cannot exceed its current load, to avoid the robot being unable to perform effectively due to too many tasks; second, the robot's capability parameters must meet the task requirements of the corresponding robot task, for example, a robot performing a high-precision detection task needs to have sensors with the corresponding precision.

[0171] In step S3022, the edge terminal calculates the comprehensive cost of each of the initial task schemes; wherein the comprehensive cost is based on the execution time of all robot tasks in the initial task scheme, the energy consumption of the corresponding robot, the task execution reliability score, and the robot load congestion penalty value.

[0172] For each generated initial task plan, the edge terminal calculates its overall cost. The overall cost is a comprehensive metric based on the execution time of all robot tasks in the initial task plan, the corresponding robot's energy consumption, task execution reliability score, and robot load congestion penalty value. Specifically, the calculation can be achieved by assigning different weights to each metric based on their importance, and then summing the weighted averages of all metrics.

[0173] Step S3023: The edge terminal performs the following iterative optimization steps based on the several initial task schemes: Selecting the initial task schemes with lower overall costs from the several initial task schemes as parent schemes; cross-recombining the mapping relationship between robot tasks and robots in the parent schemes to generate child task schemes; randomly adjusting the mapping relationship between robot tasks and robots in some of the child task schemes; calculating the overall cost of each adjusted child task scheme; repeating the above iterative optimization steps until the difference in overall cost between two consecutive iterations is less than a preset threshold, or the number of iterations reaches a preset maximum number, and determining the child task scheme with the lowest overall cost at this point as the optimal task scheme.

[0174] This step selects initial task schemes with lower overall costs from several initial task schemes as parent schemes. The selection ratio can be set according to actual conditions, for example, selecting the top 30% of schemes in terms of overall cost as parent schemes. The mapping relationship between robot tasks and robots in the parent schemes is then cross-recombined. Specifically, two parent schemes can be randomly selected, and then some of their robot task-robot mapping relationships can be swapped according to certain rules to generate new child task schemes. For example, a task can be randomly selected, and the corresponding robots in the two parent schemes can be swapped to obtain two new child task schemes. The cross-recombination operation allows for extensive searching and exploration in the solution space, combining the advantages of different parent schemes, and potentially generating child task schemes that are superior to the parent schemes. This operation can break the limitations of the original schemes and increase the possibility of finding the global optimum. Furthermore, the mapping relationship between robot tasks and robots in some child task schemes is randomly adjusted. For example, a task in a child task scheme can be randomly selected, and then a robot that meets the conditions (i.e., the robot's capability parameters meet the task requirements and the current load allows it) can be randomly assigned to it. The random adjustment operation further increases the randomness and diversity of the search, helping to avoid the optimization process getting trapped in local optima. By randomly changing some mapping relationships, some under-explored regions in the solution space can be explored, potentially discovering better task allocation schemes. Finally, for each child task scheme after random adjustment, its comprehensive cost is calculated using the same method as in step S3022. The iterative optimization steps of selecting parent schemes, cross-recombination, random adjustment, and calculating comprehensive costs are repeated until one of the following two conditions is met: first, the difference in comprehensive cost between two consecutive iterations is less than a preset threshold, indicating that the optimization process has stabilized and further iterations have no significant effect on reducing comprehensive costs; second, the number of iterations reaches a preset maximum number, which is to avoid the optimization process from continuing indefinitely and to ensure that results are obtained within a reasonable time. When the termination condition is met, the child task scheme with the lowest comprehensive cost at this point is determined as the optimal task scheme.

[0175] In this embodiment, the edge terminal employs an efficient iterative optimization method to determine the optimal task scheme from multiple initial task schemes. First, by generating initial task schemes that meet certain conditions, the initial rationality of task allocation is ensured. This means that the total task load of each robot does not exceed its current load, and its capability parameters meet the task requirements of the corresponding robot task, avoiding problems such as task inability to execute or low execution efficiency due to unreasonable task allocation. Second, a comprehensive cost metric is introduced to measure the merits of the task scheme. This metric comprehensively considers multiple key factors such as task execution time, energy consumption, task execution reliability, and robot load congestion penalty value, comprehensively and accurately reflecting the cost and benefits of the task scheme in actual execution, providing a reliable basis for subsequent optimization. During the iterative optimization process, by selecting parent schemes with lower comprehensive costs and performing cross-recombination and random adjustments to generate child task schemes, optimization ideas such as genetic algorithms are fully utilized. This allows for searching and exploration in a wider solution space, avoiding getting trapped in local optima. Meanwhile, by setting the iteration termination condition as the difference between the comprehensive costs of two consecutive iterations being less than a preset threshold or the number of iterations reaching a preset maximum, the rationality and efficiency of the optimization process are ensured. This allows the system to find the optimal task solution that satisfies task priority, execution time limit, and highest execution efficiency within a limited time. This iterative optimization method can dynamically adjust the task allocation scheme according to actual task requirements and robot status, improving the task execution efficiency and resource utilization of the robot cluster, and enabling the system to better adapt to complex and ever-changing task scenarios and environmental changes.

[0176] In one specific embodiment, the initial task scheme is generated through population initialization using a genetic algorithm, specifically as follows:

[0177] Generate the initial population It contains M random assignment schemes (chromosomes):

[0178] in, This represents the i-th allocation scheme. Let U(a,b) represent the robot ID to which task j is assigned, and let U(a,b) be a uniformly distributed random integer. Let M be the total number of robots and M be the population size.

[0179] The overall cost is calculated using the following formula:

[0180]

[0181] in, Let k be the execution time of task k; Energy consumption for task k; For the reliability of task k; As a task priority factor; These are the weighting coefficients; This refers to the congestion penalty weighting coefficient;

[0182] Where Ccong is the resource congestion penalty term, calculated using the following formula:

[0183]

[0184] in, The current load of robot r, This represents the maximum load capacity of robot r.

[0185] The core genetic operations of the above iterative optimization include:

[0186] Selection: A roulette wheel selection method is used, with selection probability... ,in To select pressure parameters;

[0187] Crossover: Two-point crossover method is used, crossover probability ;

[0188] Mutation: Adaptive mutation is used, with mutation probability... , where t is the current iteration number and T is the maximum iteration number;

[0189] The optimization process converges when the maximum number of iterations or cost is reached. The process will end at that time.

[0190] In step S104, the edge terminal generates task execution information for the target robot based on the robot task and sends the task execution information to the target robot.

[0191] Among them, task execution information is detailed instruction information generated for the target robot, which may include the specific content of the task, execution steps, action route, speed requirements, etc., with the aim of guiding the target robot to complete the robot task accurately and efficiently.

[0192] In this step, the edge terminal generates detailed task execution information for the target robot based on the determined robot task, combined with the global environment model and the current state of the target robot. Then, the edge terminal sends the task execution information to the target robot via wireless communication to ensure that the target robot can receive instructions in a timely manner.

[0193] In one embodiment, before step S104, the edge terminal generates task execution information for the target robot based on the robot task; and before sending the task execution information to the target robot, the method further includes:

[0194] Step S1041: The edge terminal identifies whether the robot task is a first type of task or a second type of task;

[0195] The first type of task is a specific type of robot task. This type of task allows the target robot to perform autonomous calculations based on the local environmental data it collects and to execute the task according to the calculation results. This means that the target robot has a relatively high degree of autonomous decision-making power when handling this type of task.

[0196] The second type of task is another type of robot task in contrast to the first type of task. For this type of task, the edge terminal will generate second task execution information containing task planning information. The target robot needs to execute the task according to the task planning information provided by the edge terminal, and the space for autonomous decision-making is relatively small.

[0197] Step S1042: When the robot task is a first type of task, the edge terminal generates first task execution information for the target robot based on the robot task; and sends the first task execution information to the target robot; wherein, the first task execution information instructs the target robot to autonomously calculate and execute based on the local environment data;

[0198] The first task execution information is the information generated by the edge terminal based on the task when the robot task is the first type of task. Its core function is to instruct the target robot to perform autonomous calculations using the local environmental data it collects, and to execute the corresponding task based on the calculation results, focusing on giving the target robot the ability to autonomously process tasks.

[0199] Step S1043: When the robot task is a second type of task, the edge terminal generates second task execution information for the target robot based on the robot task; and sends the second task execution information to the target robot; wherein, the second task execution information includes task planning information of the robot task, which is used to instruct the target robot to execute the robot task according to the task planning information.

[0200] The second task execution information is the information generated by the edge terminal when the robot task is a second type of task. It includes the task planning information of the robot task. Its main function is to provide clear execution guidance for the target robot, so that the target robot can complete the task according to the predetermined plan.

[0201] In this embodiment, before generating and sending the target robot's task execution information, the edge terminal first identifies the type of the robot task, determining whether it is a first-type task or a second-type task. For first-type tasks, the edge terminal generates first-type task execution information instructing the target robot to autonomously calculate and execute based on local environmental data, fully leveraging the target robot's local computing power and autonomous decision-making capabilities. Because the target robot can perform calculations and decisions in real time based on changes in its surrounding environment, it can respond more quickly to unexpected situations in the local environment, improving the flexibility and timeliness of task execution, especially suitable for task scenarios with high real-time requirements and frequent environmental changes. For second-type tasks, the edge terminal generates second-type task execution information containing task planning information, enabling the target robot to execute the task according to pre-planned steps and paths. This approach ensures the accuracy and consistency of task execution, avoiding deviations that may arise from the target robot's autonomous decision-making, and is suitable for scenarios with high requirements for task execution accuracy and standardization, such as complex assembly tasks or precise material handling tasks. By adopting different task execution information generation and transmission strategies according to different types of tasks, this solution can give full play to the global coordination capabilities of the edge terminal and the local processing capabilities of the target robot, achieving complementary advantages between the two, thereby improving the task execution efficiency and adaptability of the entire robot cluster, enabling it to better cope with various complex and ever-changing working environments and task requirements.

[0202] In one embodiment, after step S1043, where the edge terminal sends the second task execution information to the target robot, the method further includes:

[0203] Step S1044: The target robot sends the execution parameters obtained when performing the robot task to the edge terminal;

[0204] Step S1045: The edge terminal receives the execution parameters of the target robot, generates a task control instruction based on the execution parameters, and sends the task control instruction to the target robot.

[0205] In step S1046, the target robot responds to the task control command, adjusts the robot task, and executes the adjusted robot task.

[0206] Among them, execution parameters are a series of data generated by the target robot during the execution of robot tasks. These data can reflect the actual state and effect of task execution, such as the robot's moving speed, position coordinates, task progress, and information on obstacles encountered.

[0207] Task control instructions are generated by the edge terminal after analyzing and processing the execution parameters of the target robot received. Their function is to guide the target robot to adjust the robot task it is currently executing in order to optimize the task execution effect or deal with problems that occur during the task execution process.

[0208] In this embodiment, after the edge terminal sends the second task execution information to the target robot, the target robot does not execute the task in isolation. The target robot feeds back the execution parameters obtained during the task execution process to the edge terminal in real time. This feedback mechanism allows the edge terminal to understand the actual situation of the target robot at the task execution site in a timely manner. After receiving these execution parameters, the edge terminal generates task control instructions based on these parameters and sends them to the target robot. This dynamic interaction and control method has significant technical advantages. On the one hand, it enhances the adaptability and flexibility of the system. Since the actual task execution environment may contain various uncertainties, such as sudden obstacles and environmental changes, the execution parameters fed back by the target robot allow the edge terminal to grasp these changes in a timely manner and guide the target robot to adjust its task execution strategy through task control instructions, thereby better adapting to the complex and ever-changing environment and avoiding task execution failure or inefficiency due to environmental changes. On the other hand, it improves the quality and efficiency of task execution. The edge terminal can monitor and evaluate the task execution process in real time based on the execution parameters. If it finds that the task execution deviates or fails to achieve the expected results, it can quickly generate corresponding control instructions to guide the target robot to correct in a timely manner, ensuring that the task is executed in the optimal way and reducing unnecessary resource waste and time consumption. Furthermore, this two-way information interaction and dynamic control mechanism enhances the stability and reliability of the system, enabling the entire robot cluster to complete various tasks more efficiently and stably under the overall coordination of the edge terminal.

[0209] In step S105, the target robot performs the robot task based on the task execution information.

[0210] After receiving the task execution information sent by the edge terminal, the target robot's internal control system parses and processes the task execution information, and then controls the robot's various actuators to perform corresponding actions according to the requirements of the task execution information, thereby completing the robot's task.

[0211] In one embodiment, step S105, whereby the target robot performs the robot task based on the task execution information, includes:

[0212] Step S1051: When the communication between the target robot and the edge terminal is interrupted, the target robot performs robot tasks based on the latest received task execution information;

[0213] Step S1052: When the target robot is unable to continue performing robot tasks based on the latest received task execution information, the target robot invokes the pre-stored emergency handling rules and executes the emergency handling rules.

[0214] Among them, the emergency handling rules are a set of processing schemes and strategies pre-stored inside the target robot. They are used to guide the target robot to handle special situations such as being unable to continue the task based on the latest task execution information, so as to ensure the continuation of the task as much as possible or avoid serious consequences.

[0215] This embodiment addresses the process of a target robot executing tasks, considering two possible special situations: communication interruption with the edge terminal and inability to continue task execution based on the latest task execution information. Corresponding strategies are developed for each situation. When communication between the target robot and the edge terminal is interrupted, the target robot can continue its task based on the latest received task execution information. This design ensures that the target robot does not stagnate after real-time communication with the edge terminal, but can continue to advance the task using existing valid information, guaranteeing the continuity of task execution, reducing task interruptions and delays caused by communication problems, and improving the stability and reliability of the system in complex communication environments. Conversely, when the target robot cannot continue its task based on the latest received task execution information, pre-stored emergency handling rules are invoked to perform emergency processing. This provides the target robot with the ability to autonomously solve problems when encountering unexpected difficulties or abnormal situations. The pre-setting of emergency handling rules allows the target robot to react reasonably according to preset strategies even without real-time guidance from the edge terminal, ensuring the continuation of the task as much as possible or avoiding damage to the surrounding environment and itself, further enhancing the robustness and adaptability of the system. Overall, this solution, through these two coping strategies, fully considers various adverse situations that the target robot may encounter during task execution, effectively improving the target robot's task execution capability in complex environments and the overall stability of the system, enabling the robot swarm to complete various tasks more reliably.

[0216] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and this application also intends to include these modifications and variations.

Claims

1. A robot swarm control method, characterized in that, Includes the following steps: Several robots in a robot swarm collect several local environmental data and send the several local environmental data to an edge terminal; The edge terminal acquires global environmental data and several local environmental data. The edge terminal generates a global environment model based on the global environment data and the several local environment data; identifies dynamic targets and their dynamic information based on the global environment model; determines the robot task based on the dynamic information of the dynamic targets; and identifies the target robot in the robot cluster that will perform the robot task. The edge terminal generates task execution information for the target robot based on the robot task; and sends the task execution information to the target robot. The target robot performs the robot task based on the task execution information; The step of collecting several local environmental data points from several robots in the robot cluster and sending the several local environmental data points to the edge terminal includes: A swarm of robots collects local environmental data from several robots. When the plurality of local environmental data are identified to have preset characteristics, the plurality of robots send the plurality of local environmental data to the edge terminal; When the identified local environmental data does not have preset features, the robots determine the task to be executed based on the local environmental data and execute the task to be executed. The edge terminal generates a global environment model based on the global environment data and the plurality of local environment data; identifies dynamic targets and their dynamic information based on the global environment model; determines robot tasks based on the dynamic information of the dynamic targets, and determines the target robot in the robot cluster to execute the robot tasks, further comprising the following steps: The edge terminal monitors its own computing load; When the computational load exceeds a preset threshold, the edge terminal splits the task to be computed into several sub-tasks; wherein, the task to be computed refers to generating a global environment model based on the global environment data and the several local environment data; identifying dynamic targets and their dynamic information based on the global environment model; determining robot tasks based on the dynamic information of the dynamic targets; and determining the computational tasks of the target robot in the robot cluster that executes the robot tasks. The edge terminal identifies the idle robots in the robot cluster and assigns at least a portion of the sub-computation tasks to the idle robots. The idle robot processes the sub-computation task to obtain the calculation result, and sends the calculation result to the edge terminal; The edge terminal receives the calculation result, completes the task to be calculated based on the calculation result, determines the robot task, and selects the target robot in the robot cluster to execute the robot task.

2. The robot swarm control method according to claim 1, characterized in that, The steps of several robots in the robot swarm collecting several local environmental data include: Several robots in the robot swarm each collect corresponding local environmental data through a first hyper-converged module; the first hyper-converged module integrates at least two of the following: a visual sensor, a radar sensor, and an inertial measurement unit; The steps for the edge terminal to acquire global environmental data include: The edge terminal collects global environmental data through a second hyperconverged module; wherein the second hyperconverged module integrates at least two of the following: a visual sensor, a radar sensor, and an inertial measurement unit.

3. The robot swarm control method according to claim 2, characterized in that, The steps of several robots in the robot cluster collecting corresponding local environmental data through their onboard first hyper-converged module include: The robots respectively acquire several first local image data, several first local point cloud data, and several first local motion data through the vision sensor, radar sensor, and inertial measurement unit of the first hyper-fusion module they are equipped with; and obtain several local environmental data based on the several first local image data, the several first local point cloud data, and the several first local motion data. The steps for the edge terminal to collect global environmental data through the second hyperconverged module include: The edge terminal acquires first image data, first point cloud data, and first motion data through the visual sensor, radar sensor, and inertial measurement unit of the second hyper-fusion module, respectively; and obtains global environment data based on the first image data, the first point cloud data, and the first motion data.

4. The robot swarm control method according to claim 1, characterized in that, The edge terminal generates task execution information for the target robot based on the robot task; before sending the task execution information to the target robot, the method further includes: The edge terminal identifies whether the robot task is a first type of task or a second type of task; When the robot task is a first type of task, the edge terminal generates first task execution information for the target robot based on the robot task; and sends the first task execution information to the target robot; wherein, the first task execution information instructs the target robot to autonomously calculate and execute based on the local environmental data; When the robot task is a second type of task, the edge terminal generates second task execution information for the target robot based on the robot task; and sends the second task execution information to the target robot; wherein, the second task execution information includes task planning information of the robot task, which is used to instruct the target robot to execute the robot task according to the task planning information.

5. The robot swarm control method according to claim 4, characterized in that, After the edge terminal sends the second task execution information to the target robot, the method further includes: The target robot sends the execution parameters obtained when performing the robot task to the edge terminal; The edge terminal receives the execution parameters of the target robot, generates task control instructions based on the execution parameters, and sends the task control instructions to the target robot. The target robot responds to the task control command, adjusts the robot task, and executes the adjusted robot task.

6. The robot swarm control method according to claim 1, characterized in that, The steps by which the target robot performs the robot task based on the task execution information include: When communication between the target robot and the edge terminal is interrupted, the target robot performs robot tasks based on the latest received task execution information; When the target robot is unable to continue performing its task based on the latest received task execution information, the target robot invokes and executes the pre-stored emergency handling rules.

7. The robot swarm control method according to claim 1, characterized in that, The steps of determining the robot task based on the dynamic information of the dynamic target and determining the target robot in the robot cluster to execute the robot task include: When there is only one robot task, the edge terminal obtains the physical performance parameters, sensor and cognitive ability parameters, and historical task execution success rates of each robot in the robot cluster; it performs a weighted calculation on the physical performance parameters, the sensor and cognitive ability parameters, and the historical task execution success rates to obtain the comprehensive capability value of each robot; and it determines the robot with the highest comprehensive capability value as the target robot to perform the robot task. When there are multiple robot tasks, the edge terminal acquires the task parameters of each robot task and the capability parameters, current load, and location information of each robot in the robot cluster. The task parameters include task requirements, task priority, and task execution time limit. Based on the task parameters of the robot tasks, the capability parameters of each robot, the current load, and the location information, multiple task schemes and their corresponding execution efficiencies are obtained. Each task scheme indicates a robot task and the corresponding robot. From the multiple task schemes, the task scheme that satisfies the task priority, execution time limit, and has the highest execution efficiency is determined as the optimal task scheme. Based on the optimal task scheme, each target robot corresponding to each robot task is determined.

8. The robot swarm control method according to claim 7, characterized in that, The edge terminal obtains multiple task schemes and the corresponding execution efficiency of each task scheme based on the task parameters of the robot task, the capability parameters of each robot, the current load, and the location information. The steps for determining the optimal task plan from the multiple task plans, which satisfies the task priority, execution time limit, and highest execution efficiency, include: The edge terminal generates several initial task schemes based on the task parameters of the robot task, the capability parameters of each robot, the current load, and the location information; wherein each initial task scheme contains a mapping relationship between each robot task and each robot; the mapping relationship satisfies that the total task volume of each robot does not exceed its current load and the capability parameters meet the task requirements of the corresponding robot task; The edge terminal calculates the comprehensive cost of each of the initial task schemes; wherein the comprehensive cost is based on the execution time of all robot tasks in the initial task scheme, the energy consumption of the corresponding robot, the task execution reliability score, and the robot load congestion penalty value; The edge terminal performs the following iterative optimization steps based on the several initial task schemes: selecting the initial task schemes with lower overall costs from the several initial task schemes as parent schemes; cross-recombining the mapping relationship between robot tasks and robots in the parent schemes to generate child task schemes; randomly adjusting the mapping relationship between robot tasks and robots in some of the child task schemes; calculating the overall cost of each adjusted child task scheme; repeating the above iterative optimization steps until the difference in overall cost between two consecutive iterations is less than a preset threshold, or the number of iterations reaches a preset maximum number, and determining the child task scheme with the lowest overall cost at this time as the optimal task scheme.

Images