Unmanned aerial vehicle cooperation system based on intelligent unmanned aerial vehicle and multi-vehicle cooperation control method

Abstract

The invention discloses an intelligent unmanned aerial vehicle cooperation system and a multi-machine cooperation control method, and the system comprises an instruction collection unit, an instruction coding unit, an environment sensing unit, an environment coding unit, a fusion coding unit, an action decoding unit, and a self-adaptive control unit. According to the units, instruction information and unmanned aerial vehicle sensing information can be efficiently fused, complex multi-mode environment information and complex cooperation commands in a natural language form can be understood, an additional system does not need to carry out calibration of a control command or an environment, and therefore the cooperative operation requirement of the unmanned aerial vehicle under the complex environment condition is met, and the cooperative operation efficiency of the unmanned aerial vehicle is improved. And the use scene of the unmanned aerial vehicle is expanded. According to the multi-unmanned-aerial-vehicle cooperative control method, multiple unmanned aerial vehicles are allowed to access a cooperative system through a self-adaptive control algorithm so as to meet the cooperative work requirements of different types of aircrafts.

Description

Technical Field

[0001] The present application relates to the field of drone control technology, and specifically to a drone collaboration system based on embodied intelligence and a multi-machine collaboration control method. Background Art

[0002] Against the backdrop of the rapid development of artificial intelligence technology, drones, as an important intelligent tool, have shown great application potential in various fields. At present, drone collaboration has become a research hotspot, but the existing drone collaboration system still has many shortcomings: for example, the motion state and behavior strategy of ordinary drone collaboration systems are relatively fixed, and cannot be dynamically adjusted with environmental changes and task requirements, lacking adaptability; the ability to understand the environment is poor, relying on complex algorithms to analyze environmental characteristics, and unable to achieve end-to-end understanding of complex environmental spaces; collaborative work relies on location communication between drones; execution instructions are relatively fixed, and cannot understand more efficient natural language instructions.

[0003] Based on the technological development trend and market demand of UAV collaboration, there is an urgent need to provide an advanced and efficient solution. Summary of the invention

[0004] In order to solve the functional limitation problem caused by the existing UAV cooperation system, the present application provides a UAV cooperation system based on embodied intelligence and a multi-machine cooperation control method.

[0005] According to the first aspect of the present application, the present application provides a UAV cooperation system based on embodied intelligence, including: an instruction acquisition unit, used to acquire voice instructions and text instructions; an instruction encoding unit, connected to the instruction acquisition unit by signal, used to preprocess the voice instructions and the text instructions, and obtain first encoding information through instruction encoding; an environment perception unit, used to acquire multi-environment sensor data of the UAV; an environment encoding unit, connected to the environment perception unit by signal, used to process the multi-environment sensor data, and obtain second encoding information through data encoding; a fusion encoding unit, connected to the instruction encoding unit and the environment encoding unit by signal, used to fuse the first encoding information from the instruction encoding unit with the second encoding information from the environment encoding unit, and generate fused encoding information through fusion; an action decoding unit, connected to the fusion encoding unit by signal, used to decode the fused encoding information and generate an action control instruction for the UAV; an adaptive control unit, connected to the action decoding unit by signal, used to predict the motion state of the UAV based on the action control instruction to obtain a state vector, and perform motion control on the UAV according to the action control instruction and the state vector.

[0006] Furthermore, the instruction collection unit includes a voice instruction module and a text instruction module, and the voice instruction module and the text instruction module are integrated in a terminal controller for the drone. The voice instruction module is used to generate the voice instruction by recognizing voice, and the text instruction module is used to recognize text to generate the text instruction.

[0007] Furthermore, the instruction encoding unit includes an instruction preprocessing module and a first LLM encoding module integrated on the terminal controller for the drone; the instruction preprocessing module is used to convert the voice instructions and the text instructions into a word vector matrix, and the first LLM encoding module is used to perform instruction encoding on the word vector matrix and output the first encoding information.

[0008] Furthermore, the environmental perception unit includes a camera module, a laser point cloud module, and a posture sensing module; the camera module, the laser point cloud module, and the posture sensing module are integrated in the drone, the camera module is used to generate RGB image data by photographing the environment in which the drone is located, the laser point cloud module is used to generate laser point cloud data by scanning the environment in which the drone is located by laser, and the posture sensing module is used to collect the altitude, inclination, speed, and position of the drone itself to generate flight posture data; the RGB image data, the laser point cloud data, and the flight posture data are integrated to form the multi-element environmental sensing data.

[0009] Furthermore, the environmental coding unit includes an environmental preprocessing module and a second LLM coding module integrated on the terminal controller for the UAV; the environmental preprocessing module is used to convert the multi-environmental sensor data into a sensor data matrix, and the second LLM coding module is used to perform data encoding on the sensor data matrix and output the second coding information; the second coding information has a streaming characteristic and can retain the dynamic characteristics between consecutive frames.

[0010] Furthermore, the fusion coding unit is integrated in the terminal controller for the UAV, and the fused coding information generated by fusing the first coding information and the second coding information adopts mixed coding Tokens; the action decoding unit includes a multi-layer decoding module and a coding feedforward module integrated on the terminal controller for the UAV, the multi-layer decoding module is used to decode the fused coding information into action features, and the coding feedforward module is used to convert the action features into executable action control instructions.

[0011] Furthermore, the adaptive control unit includes an end-to-end control module and a state prediction module; the state prediction module and the end-to-end control module are integrated in the UAV, the state prediction module is used to predict the motion state of the UAV based on the action control instruction to obtain a state vector, and the end-to-end control module performs motion control on the UAV according to the flight attitude data of the UAV and the state vector.

[0012] Furthermore, the drone is regarded as an intelligent agent, and the terminal controller for the drone is regarded as a decision-making core; the intelligent agent can accept the environmental influence of the working environment and adapt to the working environment through actions executed by decision-making; the decision-making core can communicate with the intelligent agent, receive collected information from the intelligent agent and work instructions from the user, and push action control instructions to the intelligent agent.

[0013] According to the second aspect of the present application, a multi-machine cooperative control method based on embodied intelligence is provided, including: checking the communication connection status of all drones and terminal controllers for the drones, recording each successfully connected drone to obtain the number of vehicles; when the number of vehicles changes, adjusting the size of the tensor for receiving input information and buffering output data according to the change in the number of vehicles; obtaining flight status information and sensor status information of each successfully connected drone, and updating the unit state vector according to the flight status information and sensor status information of each drone; obtaining corresponding multi-environment sensor data from drones in normal state based on the unit state vector; obtaining voice commands and text commands; and The voice command and the text command are encoded to obtain first encoding information, and the multi-environment sensor data are encoded to obtain second encoding information; the first encoding information and the second encoding information are fused to generate fused encoding information; the fused encoding information is decoded to generate action control instructions for one or more of the drones; the corresponding action output tensors are calculated according to the action control instructions for one or more of the drones, and it is determined whether each of the action output tensors exceeds a preset threshold; when an action output tensor exceeds the threshold, the corresponding action control instruction is set to zero, otherwise the corresponding action control instruction is sent to the corresponding drone.

[0014] The beneficial effects of this application are:

[0015] According to the above-mentioned UAV cooperation system based on embodied intelligence and multi-machine cooperation control method, the UAV cooperation system includes a command acquisition unit, a command encoding unit, an environment perception unit, an environment encoding unit, a fusion encoding unit, an action decoding unit and an adaptive control unit. According to these units, the command information and the UAV sensor information can be efficiently integrated, and the complex multi-modal environmental information and the complex cooperation commands in the form of natural language can be understood, and no additional system is required for the calibration of control commands or environment, thereby meeting the cooperative operation requirements of UAVs under complex environmental conditions and expanding the use scenarios of UAVs. The multi-machine cooperation control method allows multiple UAVs to access the cooperation system through an adaptive control algorithm to meet the needs of cooperative work of different types of aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A system structure diagram of a drone cooperation system based on embodied intelligence for this application;

[0017] Figure 2 An information exchange diagram of a drone cooperation system based on embodied intelligence is provided for this application;

[0018] Figure 3 This application provides a flow chart of a multi-machine collaborative control method based on embodied intelligence. DETAILED DESCRIPTION

[0019] The present application is further described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0020] This embodiment discloses a drone cooperation system based on embodied intelligence technology. Figure 1 It mainly includes an instruction acquisition unit A1, an instruction encoding unit A2, an environment perception unit A3, an environment encoding unit A4, a fusion encoding unit A5, an action decoding unit A6 and an adaptive control unit A7. Each unit is introduced below.

[0021] The instruction acquisition unit A1 is used to obtain voice instructions and text instructions.

[0022] The instruction encoding unit A2 is signal-connected to the instruction acquisition unit A1, and is used for pre-processing the voice instructions and text instructions from the instruction acquisition unit A1, and obtaining first encoding information through instruction encoding.

[0023] The environmental sensing unit A3 is used to obtain the multi-environment sensing data of the drone.

[0024] The environment encoding unit A4 is signal-connected to the environment sensing unit A3, and is used for processing the multi-environment sensing data from the environment sensing unit A3, and obtaining the second coding information through data coding.

[0025] The fusion coding unit A5 is signal-connected to the instruction coding unit A2 and the environment coding unit A4, and is used to fuse the first coding information from the instruction coding unit A2 and the second coding information from the environment coding unit A4, and generate fused coding information through fusion.

[0026] The action decoding unit A6 is signal-connected to the fusion coding unit A5, and is used to decode the fusion coding information from the fusion coding unit A5 to generate action control instructions for the drone.

[0027] The adaptive control unit A7 is connected to the motion decoding unit signal A6, and is used to predict the motion state of the drone based on the motion control instruction generated by the motion decoding unit A6 to obtain a state vector, and to control the motion of the drone according to the motion control instruction and the state vector.

[0028] In a specific embodiment, the instruction collection unit A1 includes a voice instruction module 11 and a text instruction module 12. The voice instruction module 11 and the text instruction module 12 are integrated in the terminal controller for the drone. The voice instruction module 11 is used to generate voice instructions by recognizing voice, and the text instruction module 12 is used to recognize text to generate text instructions. It can be understood that the voice instruction module 11 and the text instruction module 12 can be processor components that process voice and text, and can collect work instructions in voice or text form, and send the instruction information to the instruction preprocessing module 18.

[0029] In a specific embodiment, the instruction encoding unit A2 includes an instruction preprocessing module 18 and a first LLM encoding module 19 integrated on the terminal controller for the drone; the instruction preprocessing module 18 is used to convert voice instructions and text instructions into a word vector matrix, and the first LLM encoding module 19 is used to perform instruction encoding on the word vector matrix and output first encoding information. It can be understood that the instruction preprocessing module 18 here processes the voice and text signals from the instruction acquisition unit A1 into a word vector matrix, and then the first LLM encoding module 19 is responsible for feature encoding of the word vector matrix.

[0030] In a specific embodiment, the environment perception unit A3 includes a camera module 13, a laser point cloud module 14, and a posture sensing module 15. The camera module 13, the laser point cloud module 14, and the posture sensing module 15 are integrated in the drone to collect corresponding sensor information on the drone. For example, the camera module 13 is used to generate RGB image data by photographing the environment in which the drone is located, the laser point cloud module 14 is used to generate laser point cloud data by scanning the environment in which the drone is located by laser, and the posture sensing module 15 is used to collect the height, inclination, speed, and position of the drone itself to generate flight posture data. It can be understood that the camera module 13, the laser point cloud module 14, and the posture sensing module 15 are used to obtain RGB image data, laser point cloud data, and flight posture data, respectively, where the RGB image data, laser point cloud data, and flight posture data are integrated into multi-element environment sensing data for transmission to the environment preprocessing module 112.

[0031] In a specific embodiment, the environmental coding unit A4 includes an environmental preprocessing module 112 and a second LLM coding module 111 integrated on the terminal controller for the drone. The environmental preprocessing module 112 is used to convert the multi-environment sensor data into a sensor data matrix, and the second LLM coding module 111 is used to encode the sensor data matrix and output the second coding information. The second coding information here has a streaming characteristic and can retain the dynamic characteristics between consecutive frames. It can be understood that the environmental coding unit A4 can transmit data with the environmental perception unit A3 through a wireless communication component, and the included environmental preprocessing module 112 converts the sensor information from the environmental perception unit A3 into a sensor data matrix, and the second LLM coding module 111 is responsible for feature encoding of the sensor data.

[0032] In a specific embodiment, the fusion coding unit A5 is integrated into the terminal controller for the drone, and the fusion coding information generated by fusing the first coding information and the second coding information adopts hybrid coding Tokens. It can be understood that the fusion coding module A5 can fuse the coding features from the instruction coding unit A2 and the environment coding unit A4, and generate new hybrid coding Tokens, and send the generated new code to the multi-layer decoding module 113.

[0033] In a specific embodiment, the action decoding unit A6 includes a multi-layer decoding module 113 and a coding feedforward module 114 integrated on the terminal controller for the drone, the multi-layer decoding module 113 is used to decode the fused coding information into action features, and the coding feedforward module 114 is used to convert the action features into executable action control instructions. It can be understood that the multi-layer decoding module 113 and the coding feedforward module 114 here can be responsible for respectively decoding the mixed feature encoding from the fusion coding module A5 into action features and converting the action features into executable action instructions.

[0034] In a specific embodiment, the adaptive control unit A7 includes an end-to-end control module 16 and a state prediction module 17; wherein the state prediction module 17 and the end-to-end control module 16 are integrated in the drone, the state prediction module 17 is used to predict the motion state of the drone based on the action control instruction to obtain the state vector, and the end-to-end control module 16 performs motion control on the drone according to the flight attitude data and state vector of the drone. It can be understood that the state prediction module 17 here is used to calculate the state vector of the drone, the end-to-end control module 16 can receive the attitude data and flight state vector of the drone to control the movement of the drone, and the specific action instructions of the drone come from the encoding feedforward module 114, which can automatically adapt to the characteristics of different drones and adjust the control strategy in real time to maintain the stability of the drone. Moreover, the adaptive control unit A7 allows aircraft with various different parameters to directly join the collaborative system, and stable control of the aircraft can be achieved without cumbersome parameter adjustment in advance.

[0035] It should be noted that the instruction encoding unit A2, environment encoding unit A4, fusion encoding unit A5 and action decoding unit A6 are mainly integrated on the terminal controller for the UAV, and can be considered as the decision-making layer, responsible for processing various information or data and distributing control instructions. Since the environment perception unit A3 and adaptive control unit A7 are mainly integrated on the UAV, they can be considered as the cluster layer, responsible for the sensing collection and state control of the UAV.

[0036] Of course, the drone is regarded as an intelligent agent 22, and the terminal controller for the drone is regarded as the decision core 23, see Figure 2 . Among them, the intelligent agent 22 can accept the environmental influence of the working environment in which it is located, and adapt to the working environment in which it is located through the actions executed by decision-making. For example, the drone collects information about the working environment through sensors to reflect environmental changes, and then adjusts its own flight state after receiving control instructions. It can also be understood that the decision core 23 can communicate with the intelligent agent 22, such as long-distance wireless network communication, receive the collected information from the intelligent agent 22 and the work instructions from the user, and push action control instructions to the intelligent agent. It can be understood that information exchange can be carried out between the intelligent agent 22 and the decision core 23. The intelligent agent 22 accepts the environmental influence of the working environment and affects the working environment through decision execution. The decision core 23 receives the sensor information from the intelligent agent 22 and the work instructions from the user, and pushes the state vector and action instructions to the intelligent agent 22. The action instructions can control the spatial displacement and movement speed of the drone.

[0037] It should be noted that this application is equivalent to introducing the control logic of natural instructions in traditional drone control, which requires the introduction of a large language model so that drones can obtain rich environmental information and task knowledge, so that they can perform better in complex scenarios. As a powerful deep learning model, the language large model has excellent representation and generalization capabilities. With the development of spatial intelligence technology, the language large model has the ability to analyze and understand environmental characteristics. In the fusion coding processing of various information, it can be achieved with the help of a large language model, which can dynamically adjust the motion state and behavior strategy of the drone according to the immediate environmental changes and task requirements, and achieve more intelligent control. The speech large model is integrated with multiple drones, and the environment and the relationship between intelligent agents are analyzed through the embodied intelligent decision-making core, and the tasks of the intelligent agents are reasonably allocated. As a result, the intelligent crew can efficiently and accurately complete complex drone collaborative tasks according to the requirements of natural language instructions, opening up new possibilities for the wide application of drones. In addition, the language large model receives multimodal environmental data and control signals, such as images, lidar depth and posture data, etc., to infer and understand spatial information, environmental changes and relative relationships between drones. Combined with the task requirements obtained by natural language command reasoning, corresponding action instructions are assigned to all vehicles in the system, making it easy to achieve efficient collaborative work among multiple drones.

[0038] Understandably, based on Figure 1 The units in the system can efficiently integrate command information and UAV sensor information, understand complex multimodal environmental information and complex collaborative commands in natural language, and do not require additional systems to calibrate control commands or the environment. This meets the needs of UAV collaborative operations in complex environmental conditions and expands the use scenarios of UAVs.

[0039] In one embodiment, based on the aforementioned UAV cooperation system, the present application also provides a multi-machine cooperation control method based on embodied intelligence, which can be executed on a terminal controller (or decision core) for one or more UAVs, with specific reference to Figure 3 .

[0040] The algorithm starts with step S31, which is the entry point for algorithm execution.

[0041] Step S32 of the communication status check checks the communication connection status between all drones and the terminal controller (decision-making core) for the drones, and records the successfully connected drones to obtain the number of vehicles. The connection status is divided into two types: successful connection and abnormal connection, and the number of vehicles with successfully connected status is recorded.

[0042] The step S33 of judging the change of the number of connections determines whether the number of successfully connected vehicles has changed. If so, the step S34 of IO tensor scaling is executed; otherwise, the step S35 of updating the unit status is executed.

[0043] In the step S34 of IO tensor scaling, when the number of vehicles changes, the size of the tensor for receiving input information and buffering output data is adjusted according to the change in the number of vehicles; it can be understood that the size of the tensor for the decision core cache input information and output data is adjusted according to the change in the number of connections, and then the unit status update step S35 is executed.

[0044] In step S35 of updating the crew status, the flight status information and sensor status information of each successfully connected UAV are obtained, and the crew state vector is updated according to the flight status information and sensor status information of each UAV. It can be understood that the crew state vector is updated according to the flight status and sensor status of the aircraft. The crew state vector is a one-dimensional array with a length of the number of successful connections. The values ​​in the array represent the status of the corresponding aircraft, which is divided into aircraft health, sensor abnormality, controller abnormality, and flight status abnormality.

[0045] In step S36 of updating the sensor information, corresponding multi-environment sensor data is obtained from the drone in normal state based on the crew state vector; it can be understood that the sensor data is read from the aircraft whose crew state vector indicates that the sensor is healthy, and the sensor data can be RGB image data and three-dimensional point cloud data.

[0046] In step S37 of updating the control signal, voice instructions and text instructions are obtained; it can be understood that the natural language control signal is read from the voice instruction module and the text instruction module.

[0047] In step S38 of input tensor fusion, the voice command and the text command are encoded to obtain the first encoded information, and the multi-environment sensor data are encoded to obtain the second encoded information, and the first encoded information and the second encoded information are fused to generate fused encoded information; it can be understood that the input tensor is encoded by the environment encoding unit and the instruction encoding unit, and the environment and instruction features are fused by the fusion encoding module.

[0048] In step S39 of controlling output calculation, the fused coding information is decoded to generate action control instructions for one or more drones. It can be understood that the action output tensor is calculated by the action decoding unit, and the action output can be a three-dimensional displacement and movement speed.

[0049] In the step S310 of judging the legal output, the corresponding action output tensors are calculated according to the action control instructions for one or more drones, and it is judged whether each action output tensor exceeds a preset threshold. If so, the control signal distribution step S311 is entered, otherwise, the control signal is set to zero step S312. It can be understood that this is equivalent to judging whether the size of the output action is within a preset range.

[0050] In step S311 of controlling signal distribution, when an action output tensor does not exceed a threshold value, the corresponding action control instruction is sent to the corresponding UAV; it can be understood that here the output action tensor is split and the corresponding action is distributed to each UAV.

[0051] In step S312 of setting the control signal to zero, when an action output tensor exceeds a threshold, the corresponding action control instruction is set to zero; it can be understood that here the illegal output action is set to zero, and the corresponding drone will temporarily stop the action.

[0052] Understandably, Figure 3 In the multi-machine cooperative control method shown, multiple drones are allowed to access the cooperative system through an adaptive control algorithm to meet the needs of cooperative work of different types of aircraft. Each drone can be equipped with a set of parameter-free adaptive controllers to achieve stable control of different types of drones. Any drone can be connected to the multi-machine cooperative work system without additional parameter adjustment, and the efficiency and accuracy of drone cooperation can be improved, while reducing the requirements for operator skills and experience.

[0053] The above contents are further detailed descriptions of the present application in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. For ordinary technicians in the technical field to which the present application belongs, several simple deductions or substitutions can be made without departing from the inventive concept of the present application.

Claims

1. A drone collaboration system based on embodied intelligence, characterized in that: include: A command acquisition unit, used to obtain voice commands and text commands; An instruction encoding unit, connected to the instruction acquisition unit by signal, for preprocessing the voice instruction and the text instruction, and obtaining first encoding information by instruction encoding; Environmental perception unit, used to obtain multi-environment sensing data of the drone; An environment encoding unit, connected to the environment sensing unit signal, for processing the multi-environment sensing data and obtaining second encoding information through data encoding; a fusion encoding unit, connected to the instruction encoding unit and the environment encoding unit by signal, and configured to fuse the first encoding information from the instruction encoding unit with the second encoding information from the environment encoding unit, and generate fused encoding information by fusion; An action decoding unit, connected to the fusion encoding unit signal, for decoding the fusion encoding information to generate an action control instruction for the UAV; An adaptive control unit is connected to the motion decoding unit signal, and is used to predict the motion state of the drone based on the motion control instruction to obtain a state vector, and to perform motion control on the drone according to the motion control instruction and the state vector.

2. The drone cooperation system based on embodied intelligence as claimed in claim 1, characterized in that: The instruction collection unit includes a voice instruction module and a text instruction module, and the voice instruction module and the text instruction module are integrated in a terminal controller for the drone. The voice instruction module is used to generate the voice instruction by recognizing voice, and the text instruction module is used to recognize text to generate the text instruction.

3. The drone cooperation system based on embodied intelligence as claimed in claim 2, characterized in that: The instruction encoding unit includes an instruction preprocessing module and a first LLM encoding module integrated on the terminal controller for the drone; the instruction preprocessing module is used to convert the voice instruction and the text instruction into a word vector matrix, and the first LLM encoding module is used to perform instruction encoding on the word vector matrix and output the first encoding information.

4. The drone collaboration system based on embodied intelligence as claimed in claim 1, characterized in that: The environment perception unit includes a camera module, a laser point cloud module, and a posture sensing module; the camera module, the laser point cloud module, and the posture sensing module are integrated in the drone, the camera module is used to generate RGB image data by photographing the environment in which the drone is located, the laser point cloud module is used to generate laser point cloud data by scanning the environment in which the drone is located by laser, and the posture sensing module is used to collect the altitude, inclination, speed, and position of the drone itself to generate flight posture data; The RGB image data, the laser point cloud data and the flight attitude data are integrated to form the multi-environment sensing data.

5. The drone cooperation system based on embodied intelligence as claimed in claim 4, characterized in that: The environment encoding unit includes an environment preprocessing module integrated on a terminal controller for the UAV and a second LLM encoding module; The environment preprocessing module is used to convert the multivariate environment sensing data into a sensing data matrix, and the second LLM encoding module is used to perform data encoding on the sensing data matrix and output the second encoding information; The second encoding information has a streaming characteristic and can retain dynamic features between consecutive frames.

6. The drone cooperation system based on embodied intelligence as claimed in claim 1, characterized in that: The fusion coding unit is integrated in a terminal controller for the UAV, and the fusion coding information generated by fusing the first coding information and the second coding information adopts hybrid coding Tokens; The action decoding unit includes a multi-layer decoding module and a coding feedforward module integrated on the terminal controller for the drone, the multi-layer decoding module is used to decode the fused coding information into action features, and the coding feedforward module is used to convert the action features into executable action control instructions.

7. The drone cooperation system based on embodied intelligence as claimed in claim 6, characterized in that: The adaptive control unit includes an end-to-end control module and a state prediction module; the state prediction module and the end-to-end control module are integrated in the drone, the state prediction module is used to predict the motion state of the drone based on the action control instruction to obtain a state vector, and the end-to-end control module performs motion control on the drone according to the flight attitude data of the drone and the state vector.

8. The drone collaboration system based on embodied intelligence according to any one of claims 1 to 7, characterized in that: The drone is regarded as an intelligent agent, and the terminal controller for the drone is regarded as a decision-making core; the intelligent agent can accept the environmental influence of the working environment and adapt to the working environment through actions executed by decision-making; the decision-making core can communicate with the intelligent agent, receive collected information from the intelligent agent and work instructions from the user, and push action control instructions to the intelligent agent.

9. A multi-machine collaborative control method based on embodied intelligence, characterized in that: include: Check the communication connection status of all drones with the terminal controller for the drones, and record the successfully connected drones to obtain the number of vehicles; When the number of the vehicles changes, the size of the tensor for receiving input information and buffering output data is adjusted according to the change in the number of the vehicles; Obtain the flight status information and sensor status information of each successfully connected UAV, and update the crew state vector according to the flight status information and sensor status information of each UAV; Acquire corresponding multi-element environmental sensor data from a UAV in a normal state based on the unit state vector; Get voice and text commands; Performing instruction encoding on the voice instruction and the text instruction to obtain first encoding information, and performing data encoding on the multi-environment sensing data to obtain second encoding information; Fusion the first coding information and the second coding information to generate fused coding information; Decoding the fused coding information to generate action control instructions for one or more of the drones; Calculating corresponding action output tensors respectively according to the action control instructions for one or more of the drones, and determining whether each of the action output tensors exceeds a preset threshold; When an action output tensor exceeds the threshold, the corresponding action control instruction is set to zero, otherwise the corresponding action control instruction is sent to the corresponding drone.

Images