Unmanned aerial vehicle group global view angle data generation method and device, electronic equipment and medium

Abstract

The invention provides an unmanned aerial vehicle group global view angle data generation method and device, electronic equipment and a medium, and relates to the technical field of semantic communication. The method comprises the following steps: acquiring state data of each unmanned aerial vehicle; determining original image data and original point cloud data in data acquired by each unmanned aerial vehicle according to the state data of all the unmanned aerial vehicles; encoding the original image data and the original point cloud data through a preset semantic encoding model to obtain semantic encoding data; the semantic coding data and the state data of each unmanned aerial vehicle are sent to a ground central control unit, and the ground central control unit is used for decoding the semantic coding data to obtain decoded image data and decoded point cloud data; and generating global view data according to the state data of each unmanned aerial vehicle, the decoded image data and the decoded point cloud data. According to the invention, data redundancy is reduced, and collaborative operation performance and task execution efficiency of the unmanned aerial vehicle cluster in a complex environment are enhanced.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of communication, in particular to the technical field of semantic communication, and more particularly to a method and device for generating global perspective data of a drone swarm, an electronic device, and a medium. BACKGROUND

[0002] With the rapid development of wireless communication technology, especially under the background of rapid evolution of 6G wireless communication systems, the application of drone swarms in specific fields is becoming more and more widespread. One of the core tasks of drone swarms is to transmit the high-dimensional data (such as images, videos, point clouds, etc.) collected in real time to the ground control station, so that the operator can obtain real-time situational awareness information or perform specific tasks.

[0003] However, the existing technology has many problems in the transmission and processing of drone swarm data. Traditional communication methods directly transmit complete bit-level data, resulting in high bandwidth resource occupation, low transmission efficiency, poor transmission stability in complex electromagnetic environments, and easy loss of critical information. In addition, the data redundancy is high in drone swarm operations, and there is a lack of intelligent information extraction and selection mechanism, which cannot fully utilize the limited communication resources. The existing technology also has deficiencies in integrating multi-source data to generate global perspective, which is difficult to provide comprehensive and accurate situational awareness information, affecting the accuracy and timeliness of decision-making. Therefore, there is an urgent need for a high-efficiency, intelligent, and complex-environment-adaptive technical solution to improve the collaborative operation capability and task execution efficiency of drone swarms. SUMMARY

[0004] The present disclosure provides a method and device for generating global perspective data of a drone swarm, an electronic device, and a medium.

[0005] According to an aspect of the present disclosure, a method for generating global perspective data of a drone swarm is provided, which is applied to a drone swarm control device, and the method comprises:

[0006] Obtaining the position information and attitude information of each drone to obtain the state data of each drone;

[0007] Determining the to-be-transmitted data in the data collected by each drone according to the state data of all drones, wherein the to-be-transmitted data includes original image data and original point cloud data;

[0008] Encoding the original image data and the original point cloud data through a preset semantic encoding model respectively to obtain semantic encoding data;

[0009] The semantic coding data and the state data of each unmanned aerial vehicle are sent to a ground central control unit, the ground central control unit is configured to decode the semantic coding data to obtain decoded image data and decoded point cloud data, and generate global perspective data according to the state data of each unmanned aerial vehicle, the decoded image data and the decoded point cloud data.

[0010] According to another aspect of the present disclosure, a method for generating global perspective data of a group of unmanned aerial vehicles is provided, which is applied to a ground central control unit, and the method comprises:

[0011] Receiving semantic coding data sent by a group of unmanned aerial vehicle control devices, wherein the group of unmanned aerial vehicle control devices are configured to obtain position information and attitude information of each unmanned aerial vehicle to obtain state data of each unmanned aerial vehicle; determining to-be-transmitted data in the data collected by each unmanned aerial vehicle according to the state data of all unmanned aerial vehicles, wherein the to-be-transmitted data comprises original image data and original point cloud data; encoding the original image data and the original point cloud data through a preset semantic coding model respectively to obtain semantic coding data;

[0012] Receiving state data of each unmanned aerial vehicle sent by the group of unmanned aerial vehicle control devices;

[0013] Decoding the semantic coding data to obtain decoded image data and decoded point cloud data;

[0014] Generating global perspective data according to the state data of each unmanned aerial vehicle, the decoded image data and the decoded point cloud data.

[0015] According to a third aspect of the present disclosure, a method for generating global perspective data of a group of unmanned aerial vehicles is provided, and the method comprises:

[0016] A group of unmanned aerial vehicle control devices obtain position information and attitude information of each unmanned aerial vehicle to obtain state data of each unmanned aerial vehicle; determine to-be-transmitted data in the data collected by each unmanned aerial vehicle according to the state data of all unmanned aerial vehicles, wherein the to-be-transmitted data comprises original image data and original point cloud data; encode the original image data and the original point cloud data through a preset semantic coding model respectively to obtain semantic coding data; and send the semantic coding data and the state data of each unmanned aerial vehicle to a ground central control unit;

[0017] The ground central control unit decodes the semantic coding data to obtain decoded image data and decoded point cloud data, and generates global perspective data according to the state data of each unmanned aerial vehicle, the decoded image data and the decoded point cloud data.

[0018] According to a fourth aspect of the present disclosure, a UAV group global perspective data generation apparatus is provided, applied to a UAV group control device, comprising:

[0019] An acquisition module is configured to acquire position information and attitude information of each UAV to obtain state data of each UAV;

[0020] A determination module is configured to determine, according to the state data of all UAVs, to-be-transmitted data in the data collected by each UAV, wherein the to-be-transmitted data comprises original image data and original point cloud data;

[0021] An encoding module is configured to encode the original image data and the original point cloud data through a preset semantic encoding model respectively to obtain semantic encoding data;

[0022] A sending module is configured to send the semantic encoding data and the state data of each UAV to a ground central control unit, wherein the ground central control unit is configured to decode the semantic encoding data to obtain decoded image data and decoded point cloud data, and generate global perspective data according to the state data of each UAV, the decoded image data and the decoded point cloud data.

[0023] According to a fifth aspect of the present disclosure, a UAV group global perspective data generation apparatus is provided, applied to a ground central control unit, comprising:

[0024] A first receiving module is configured to receive semantic encoding data sent from a UAV group control device, wherein the UAV group control device is configured to acquire position information and attitude information of each UAV to obtain state data of each UAV; determine, according to the state data of all UAVs, to-be-transmitted data in the data collected by each UAV, wherein the to-be-transmitted data comprises original image data and original point cloud data; and encode the original image data and the original point cloud data through a preset semantic encoding model respectively to obtain semantic encoding data;

[0025] A second receiving module is configured to receive state data of each UAV sent from the UAV group control device;

[0026] A decoding module is configured to decode the semantic encoding data to obtain decoded image data and decoded point cloud data;

[0027] A generation module is configured to generate global perspective data according to the state data of each UAV, the decoded image data and the decoded point cloud data.

[0028] According to a sixth aspect of the present disclosure, an electronic device is provided, comprising:

[0029] at least one processor; and

[0030] a memory in communication with the at least one processor; wherein

[0031] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of the above technical solutions.

[0032] According to a seventh aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of any one of the above technical solutions.

[0033] According to an eighth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of any one of the above technical solutions.

[0034] The present disclosure provides a UAV swarm global perspective data generation method, device, equipment and storage medium. The present disclosure determines the state data of each UAV by obtaining the position and attitude information of each UAV, and accurately filters the to-be-transmitted data in the data collected by each UAV based on the state data, wherein the to-be-transmitted data includes original image data and original point cloud data. This process effectively reduces data redundancy, avoids the transmission of duplicate data, and significantly improves communication efficiency. Subsequently, the data is efficiently compressed using a preset semantic encoding model to generate semantic encoding data. Semantic encoding technology not only further reduces the amount of data, but also enhances the transmission robustness of data in complex channel environments, reducing the risk of transmission errors and data loss. Finally, the semantic encoding data and UAV state data are sent to a ground central control unit. The ground central control unit decodes the received semantic encoding data and generates global perspective data in combination with the UAV state data. This process not only ensures the integrity and accuracy of the data, but also provides comprehensive and real-time site situation information for operators through the generation of global perspective data, significantly improving the accuracy and timeliness of decision-making. In addition, by optimizing the data transmission and processing process, the cooperative operation capability of the UAV cluster in complex environments is enhanced, and the overall performance and task execution efficiency of the system are improved, providing strong support for efficient cooperative operation of the UAV cluster.

[0035] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings are used to better understand the present scheme and do not constitute a limitation on the present disclosure. Among them:

[0037] Figure 1 is a step schematic diagram of a UAV group global perspective data generation method in an embodiment of the present disclosure;

[0038] Figure 2 is a step schematic diagram of a UAV group global perspective data generation method in another embodiment of the present disclosure;

[0039] Figure 3 is a system architecture diagram corresponding to the UAV group global perspective data generation method in an embodiment of the present disclosure;

[0040] Figure 4 is a whole flow schematic diagram of the UAV group global perspective data generation method in an embodiment of the present disclosure;

[0041] Figure 5 is a principle block diagram of a UAV group global perspective data generation device in an embodiment of the present disclosure;

[0042] Figure 6 is a principle block diagram of a UAV group global perspective data generation device in another embodiment of the present disclosure;

[0043] Figure 7 is a block diagram of an electronic device for implementing the UAV group global perspective data generation method in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] The exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0045] The present disclosure provides a UAV group global perspective data generation method, referring to Figure 1 shown, Figure 1 is a step schematic diagram of a UAV group global perspective data generation method in an embodiment of the present disclosure, which is applied to a UAV group control device, and the method comprises:

[0046] Step S101, obtaining the position information and attitude information of each UAV to obtain the state data of each UAV.

[0047] Specifically, the position information refers to the specific coordinates of the UAV in three-dimensional space, usually obtained through the Global Positioning System (GPS), including longitude, latitude, and altitude. The attitude information refers to the orientation and tilt state of the UAV, usually obtained through the Inertial Measurement Unit (IMU), including the pitch angle (Pitch), yaw angle (Yaw), and roll angle (Roll). These position and attitude information together constitute the state data of each UAV, which describes the real-time position and orientation of the UAV during flight.

[0048] The specific implementation process of this scheme includes that each UAV is equipped with a GPS module and an IMU sensor. The GPS module obtains the accurate position information of the UAV in real time, while the IMU sensor monitors the attitude changes of the UAV in real time. These sensors send the collected data to the control system of the UAV, which processes and integrates these data to form the state data of each UAV. The state data is not only used for autonomous flight control of the UAV to ensure it flies according to the preset path and attitude, but also used for subsequent data collection and processing. For example, by analyzing the position and attitude information of the UAV, the coverage area of its sensors can be determined, thereby optimizing the data collection strategy and avoiding repeated collection of the same area of data by other UAVs. This process provides precise navigation and positioning support for the cooperative work of the UAV cluster, ensuring efficient operation of the entire system and smooth execution of the task.

[0049] Step S102, according to the state data of all UAVs, determine the to-be-transmitted data in the data collected by each UAV, the to-be-transmitted data including original image data and original point cloud data.

[0050] Specifically, the state data refers to the position information (such as GPS coordinates) and attitude information (such as pitch angle, yaw angle, and roll angle) of each UAV, which is obtained in real time through sensors on the UAV. The original image data refers to high-resolution image information collected by the camera of the UAV, while the original point cloud data refers to three-dimensional space point information collected by sensors such as LiDAR, used to construct a three-dimensional model of the environment.

[0051] The specific implementation process of this scheme includes first collecting the state data of all UAVs, including their position and attitude information. These data are used to calculate the coverage area of each UAV, i.e. the spatial range of its sensor data collection. By comparing the coverage areas of each UAV, the overlapping areas between them are determined. Based on these overlapping areas, further to-be-transmitted data is determined from the data collected by each UAV. This process not only reduces data redundancy, but also optimizes the use of communication resources, improving the efficiency of data transmission.

[0052] Step S103, the original image data and the original point cloud data are respectively encoded by a preset semantic encoding model to obtain semantic encoding data.

[0053] Specifically, the original image data refers to the original image information collected by the UAV group control device, which usually contains high-resolution visual content. The original point cloud data is three-dimensional spatial point information collected by sensors such as LiDAR, which is used to construct a three-dimensional model of the environment. The semantic encoding model is a pre-trained deep learning model that can extract the core semantic information of the data and remove redundant content, thereby achieving efficient compression of the data.

[0054] The specific implementation process of this scheme includes: after the UAV group control device collects the original image and point cloud data, it processes these data using a preset semantic encoding model. The semantic encoding model extracts key features and semantic information from the data through deep learning algorithms, removes unnecessary redundant data, and generates semantic encoding data. This process not only significantly reduces the data volume, but also improves the transmission stability of the data in harsh channel environments. For example, for image data, the semantic encoding model can extract key features such as the outline and texture of the target object; for point cloud data, it can extract key points and structural information in three-dimensional space. In this way, the UAV group control device can efficiently transmit the compressed semantic encoding data to the ground control unit without losing key information. This feature not only improves communication efficiency, but also enhances the robustness and adaptability of the entire system, providing strong support for efficient collaborative work of the UAV cluster in complex environments.

[0055] Step S104, the semantic encoding data and the state data of each UAV are sent to the ground central control unit, and the ground central control unit is used to decode the semantic encoding data to obtain decoded image data and decoded point cloud data, and generate global perspective data according to the state data of each UAV, the decoded image data and the decoded point cloud data.

[0056] Specifically, the semantic encoding data refers to the compressed data processed by the pre-trained semantic encoding model, which extracts the core semantic information of the original image and point cloud data, significantly reduces the data volume, and enhances the robustness of transmission. The state data refers to the position information (such as GPS coordinates) and attitude information (such as pitch angle, yaw angle, and roll angle) of each UAV, which is obtained in real time by sensors on the UAV and used to describe the specific position and orientation of the UAV in three-dimensional space.

[0057] In the specific implementation process of the scheme, the original image data and the original point cloud data are encoded by a preset semantic encoding model to generate semantic encoding data. Meanwhile, the state data of each unmanned aerial vehicle is sent to the ground central control unit. After receiving the data, the ground central control unit decodes the semantic encoding data by using a semantic decoding model matched with the unmanned aerial vehicle end to restore the image data and the point cloud data. Subsequently, the ground central control unit combines the state data of each unmanned aerial vehicle to perform spatial alignment and splicing on the decoded image data and point cloud data to generate global perspective data. This process not only ensures the integrity and accuracy of the data in the transmission process, but also provides the operator with a complete three-dimensional field situation view through the generation of the global perspective data, significantly improving the accuracy and real-time performance of the decision.

[0058] The present disclosure provides a method and device for generating global perspective data of a group of unmanned aerial vehicles, an apparatus, and a storage medium. The present disclosure determines the state data of each unmanned aerial vehicle by obtaining its position and attitude information, and accurately filters the to-be-transmitted data in the data collected by each unmanned aerial vehicle based on the state data, wherein the to-be-transmitted data includes original image data and original point cloud data. This process effectively reduces data redundancy, avoids the transmission of duplicate data, and significantly improves communication efficiency. Subsequently, the data is efficiently compressed using a preset semantic encoding model to generate semantic encoding data. Semantic encoding technology not only further reduces the data volume, but also enhances the transmission robustness of the data in complex channel environments, reducing the risk of transmission errors and data loss. Finally, the semantic encoding data and the state data of the unmanned aerial vehicles are sent to the ground central control unit. The ground central control unit decodes the received semantic encoding data and generates global perspective data in combination with the state data of the unmanned aerial vehicles. This process not only ensures the integrity and accuracy of the data, but also provides the operator with comprehensive and real-time field situation information through the generation of global perspective data, significantly improving the accuracy and timeliness of the decision. In addition, by optimizing the data transmission and processing process, the collaborative operation capability of the unmanned aerial vehicle cluster in complex environments is enhanced, the overall performance and task execution efficiency of the system are improved, and strong support is provided for efficient collaborative operation of the unmanned aerial vehicle cluster.

[0059] In some optional embodiments, determining the to-be-transmitted data in the data collected by each unmanned aerial vehicle according to the state data of all unmanned aerial vehicles comprises:

[0060] selecting any one of all unmanned aerial vehicles as a current unmanned aerial vehicle;

[0061] calculating the coverage area of each unmanned aerial vehicle according to the position information and attitude information of the current unmanned aerial vehicle and other unmanned aerial vehicles;

[0062] comparing the coverage area of the current unmanned aerial vehicle with the coverage areas of other unmanned aerial vehicles to determine overlapping areas;

[0063] The data corresponding to the overlapping region is deleted from the data collected by the current UAV, to obtain the transmission data of each UAV.

[0064] Specifically, the position information refers to the specific coordinates of the UAV in three-dimensional space, usually obtained through a GPS module; the attitude information refers to the orientation and tilt state of the UAV, usually obtained through an IMU sensor. The coverage area refers to the spatial range in which the UAV's sensor can collect data, and its shape and size depend on the sensor's parameters (such as field of view, scanning range) and the UAV's position and attitude.

[0065] The specific implementation process of this scheme includes, first, selecting any one UAV from the UAV swarm as the current UAV. Then, according to the position information and attitude information of the current UAV and other UAVs, combined with the sensor parameters, the coverage area of each UAV is calculated. This calculation can be done through geometric methods, for example, for a camera, the coverage area can be regarded as a conical region centered on the UAV with the direction determined by the attitude angle; for a LiDAR, it can be regarded as a cylindrical region centered on the UAV. Subsequently, by comparing the coverage area of the current UAV with that of other UAVs, the overlapping part between these areas is determined. This comparison can be realized through computational geometry methods, for example, by detecting the intersection of two coverage areas to determine the overlapping region. Finally, the data corresponding to the overlapping region is deleted from the data collected by the current UAV, thereby obtaining the transmission data of each UAV. This process not only reduces data redundancy and avoids multiple UAVs repeatedly collecting the same information, but also optimizes the use of communication resources and improves the efficiency of data transmission.

[0066] In this way, by selecting any one UAV from the UAV swarm as the current UAV and calculating the coverage area of each UAV according to its position and attitude information with other UAVs, and then comparing and determining the overlapping region, the data corresponding to the overlapping region can be accurately deleted from the data collected by the current UAV, and then the transmission data of each UAV can be obtained. This process effectively reduces data redundancy, avoids multiple UAVs repeatedly collecting the same information, and significantly improves communication efficiency and data transmission accuracy. At the same time, by optimizing the data collection and transmission process, the cooperative working ability of the UAV swarm in complex environments is enhanced, and the overall performance and task execution efficiency of the system are improved, providing strong support for efficient cooperative work of the UAV swarm.

[0067] In some optional embodiments, the original image data and the original point cloud data are respectively encoded through a preset semantic encoding model to obtain semantic encoding data, including:

[0068] The original image data is encoded through an image semantic encoding model to obtain encoded image semantic data.

[0069] The original point cloud data is encoded through a point cloud semantic encoding model to obtain encoded point cloud semantic data.

[0070] The encoded image semantic data and the encoded point cloud semantic data are integrated as semantic encoding data.

[0071] Specifically, the original image data refers to high-resolution image information collected by unmanned aerial vehicles, which usually contains rich visual content. The original point cloud data is three-dimensional spatial point information collected by sensors such as LiDAR, which is used to construct a three-dimensional model of the environment. The image semantic encoding model and the point cloud semantic encoding model are pre-trained deep learning models that can extract core semantic information of the data and remove redundant content, thereby achieving efficient compression of the data.

[0072] The specific implementation process of this scheme includes: after the unmanned aerial vehicle group control device collects the original image and point cloud data, it first inputs these data into the pre-trained image semantic encoding model and point cloud semantic encoding model respectively. The image semantic encoding model extracts key features and semantic information in the image, such as the contours, textures, and categories of target objects, through deep learning algorithms, and generates encoded image semantic data. The point cloud semantic encoding model extracts key points and structural information in the point cloud data, such as the boundaries and shapes of objects in three-dimensional space, and generates encoded point cloud semantic data. These encoded data not only significantly reduces the data volume, but also retains key semantic information, improving the transmission stability of the data in harsh channel environments. Finally, the encoded image semantic data and point cloud semantic data are integrated as semantic encoding data for subsequent transmission and processing.

[0073] In this way, by encoding the original image data through the image semantic encoding model to obtain encoded image semantic data, and encoding the original point cloud data through the point cloud semantic encoding model to obtain encoded point cloud semantic data, and integrating the two as semantic encoding data, this process not only significantly reduces the data volume and reduces the demand for communication bandwidth, but also enhances the transmission stability of the data in complex channel environments, reducing the risk of data loss and transmission errors. At the same time, the core semantic information extracted by the semantic encoding model retains the key features of the data, ensuring the practicality and accuracy of the data, and providing a high-quality data basis for subsequent global situation awareness and decision support. In addition, this feature optimizes the resource utilization of the entire system, enabling the unmanned aerial vehicle cluster to complete tasks more efficiently under limited communication resources, improving the overall collaborative operation capability and intelligent level.

[0074] In some optional embodiments, the method further comprises:

[0075] Obtain other load data in the original data except the original image data and the original point cloud data;

[0076] Send other payload data to the ground central control unit for forwarding to the payload operation end through the ground central control unit.

[0077] Specifically, raw data refers to all types of data collected by drones, including imagery, point cloud data, and other sensor data. Other payload data refers to sensor data other than imagery and point cloud data, such as meteorological data, electromagnetic spectrum data, and infrared thermal imaging data. This data is valuable for specific missions and operations. The payload operator is the terminal responsible for processing and analyzing this other payload data, typically monitored and operated by a professional operator.

[0078] The specific implementation process of this solution involves the following: While performing a mission, drones in the swarm control system not only collect image and point cloud data, but also various other payload data through other sensors. This data is collected and stored as part of the raw data via the drone's communication system. The drone then transmits this additional payload data to the ground control unit via an ad hoc network communication link. Upon receiving this data, the ground control unit forwards it to the appropriate payload operator based on the pre-defined communication protocol and mission requirements. This process ensures that all relevant data is delivered to the required location in a timely and accurate manner, enabling operators to fully understand the mission environment and make more accurate decisions. For example, meteorological data can help operators assess flight conditions, electromagnetic spectrum data can be used for electronic warfare and signals intelligence, and infrared thermal imaging data can be used for target detection at night or in complex environments. In this way, the drone swarm not only efficiently completes the acquisition and processing of image and point cloud data, but also fully supports a variety of mission requirements, enhancing the versatility and adaptability of the entire system.

[0079] In this way, by acquiring other payload data in addition to the original image data and original point cloud data from the original data, and sending this other payload data to the ground central control unit, which is then forwarded by the ground control unit to the payload operation terminal, the comprehensive sharing and efficient utilization of the data collected by the drone cluster is achieved. This process not only ensures that the operator can obtain rich multi-source information, thereby gaining a more comprehensive understanding of the mission environment, but also improves the flexibility and adaptability of mission execution. In addition, through the centralized management and forwarding of the ground control unit, the use of communication resources is optimized, the burden on the communication link is reduced, the timely transmission of key data is ensured, and the collaborative operation capability and mission execution efficiency of the entire system are improved.

[0080] The present disclosure provides a method for generating global perspective data of a UAV swarm, referring to Figure 2 , Figure 2 is a schematic diagram of the steps of a method for generating global perspective data of a UAV swarm in another embodiment of the present disclosure, which is applied to a ground central control unit. The method comprises:

[0081] In step S201, semantic encoding data sent from a UAV swarm control device is received. The UAV swarm control device is used to obtain position information and attitude information of each UAV, and obtain state data of each UAV. According to the state data of all UAVs, determine the to-be-transmitted data in the data collected by each UAV, which includes original image data and original point cloud data. The original image data and the original point cloud data are respectively encoded through a preset semantic encoding model to obtain semantic encoding data.

[0082] Specifically, the feature of "receiving semantic encoding data sent from a UAV swarm control device" describes the process of the ground control unit receiving the processed data sent by the UAV swarm. The UAV swarm control device is a device responsible for managing and coordinating the UAV swarm, which can obtain the position information and attitude information of each UAV, and thus obtain the state data of each UAV. The position information refers to the specific coordinates of the UAV in the three-dimensional space, which is usually obtained through the GPS module; the attitude information refers to the orientation and inclination state of the UAV, which is usually obtained through the IMU sensor. These information together constitute the state data of the UAV, which is used to describe the real-time position and orientation of the UAV during flight.

[0083] The specific implementation process of this scheme includes that the UAV swarm control device first obtains the position and attitude information of each UAV to determine its state data. Then, according to the state data of all UAVs, determine the to-be-transmitted data in the data collected by each UAV, which includes original image data and original point cloud data. The original image data refers to the high-resolution image information collected by the camera of the UAV, while the original point cloud data refers to the three-dimensional space point information collected by sensors such as LiDAR, which is used to construct a three-dimensional model of the environment.

[0084] To further optimize data transmission, the original image data and the original point cloud data are respectively encoded by a preset semantic encoding model to generate semantic encoding data. The semantic encoding model is a deep learning model that can extract the core semantic information of the data and remove redundant content, thereby achieving efficient compression of the data. In this way, the unmanned aerial vehicle group control device sends the compressed semantic encoding data to the ground control unit, reducing the data transmission volume, improving the transmission efficiency, and enhancing the transmission stability of the data in complex channel environments. The implementation of this feature not only improves the data processing and transmission efficiency of the unmanned aerial vehicle group, but also enhances the overall performance and adaptability of the system, providing strong support for efficient cooperative operation of the unmanned aerial vehicle group.

[0085] Step S202, receiving the state data of each unmanned aerial vehicle sent by the unmanned aerial vehicle group control device.

[0086] Specifically, the state data refers to the position information (such as GPS coordinates) and attitude information (such as pitch angle, yaw angle, and roll angle) of each unmanned aerial vehicle, which are obtained in real time by sensors (such as GPS modules and IMUs) on the unmanned aerial vehicle, and are used to describe the specific position and orientation of the unmanned aerial vehicle in three-dimensional space.

[0087] The specific implementation process of this scheme includes that each unmanned aerial vehicle is equipped with a GPS module and an IMU sensor, which can collect the position and attitude information of the unmanned aerial vehicle in real time. These information are integrated into state data and sent to the unmanned aerial vehicle group control device through the ad hoc network communication link. The unmanned aerial vehicle group control device receives and processes these state data for monitoring the flight state of the unmanned aerial vehicle, optimizing task allocation, and coordinating cooperative work between unmanned aerial vehicles. For example, by analyzing the state data, the control device can determine the coverage area of each unmanned aerial vehicle, optimize the data acquisition strategy, avoid data redundancy, and ensure efficient cooperative work between unmanned aerial vehicles.

[0088] The implementation of this feature not only improves the management efficiency of the unmanned aerial vehicle cluster, but also enhances the overall performance and adaptability of the system, providing strong support for efficient cooperative operation of the unmanned aerial vehicle cluster in complex environments.

[0089] Step S203, decoding the semantic encoding data to obtain decoded image data and decoded point cloud data.

[0090] Specifically, semantic encoding data refers to compressed data processed by a pre-trained semantic encoding model, which extracts the core semantic information of the original image and point cloud data, significantly reducing the data volume while enhancing the robustness of transmission. Decoding refers to the process of restoring these compressed semantic encoding data to the original data containing key semantic information, while the decoded image data and decoded point cloud data are the recovered data after decoding, retaining the key features of the original data.

[0091] The specific implementation process of this scheme includes that after the ground control unit receives the semantic encoding data sent by the UAV group control device, it uses a semantic decoding model matched with the UAV end to decode these data. The decoding model is usually a deep learning model that can understand the information in the encoding data and restore it to a form closer to the original data. This process not only ensures the integrity and accuracy of the data during transmission, but also improves the efficiency of data transmission, reduces bandwidth demand, and enhances the adaptability of the system in complex channel environments. In this way, the ground control unit can obtain high-quality, low-latency key information, providing a solid data foundation for subsequent global situation awareness and decision support.

[0092] Step S204, generating global perspective data according to the state data of each UAV, decoded image data, and decoded point cloud data.

[0093] Specifically, decoded image data refers to recovered image information after decoding processing; decoded point cloud data refers to point cloud information after decoding processing. Global perspective data refers to a complete three-dimensional situation view generated by integrating all the information collected by the UAVs, providing operators with a comprehensive field situation awareness.

[0094] The specific implementation process of this scheme includes that the ground control unit first receives semantic encoding data and state data from the UAV group. Through the decoding model, the semantic encoding data is restored to decoded image data and point cloud data. Subsequently, the ground control unit uses the state data of each UAV to spatially align and stitch the decoded image data and point cloud data. The position and attitude information in the state data is used to determine the spatial position and direction of each UAV's data collection, ensuring that the data of different UAVs can be accurately matched in three-dimensional space. Through alignment and stitching, a seamless global three-dimensional point cloud model, i.e., global perspective data, is generated. This process not only ensures the integrity and accuracy of the data, but also provides operators with a complete three-dimensional field situation view through the generation of global perspective data, significantly improving the accuracy and real-time of decision-making.

[0095] In this way, by receiving the semantic encoding data sent by the UAV group control device and the state data of each UAV, an efficient process from data acquisition to global situation awareness is realized. The UAV group control device obtains the position and attitude information of each UAV, determines its state data, and based on these state data, filters out the to-be-transmitted data in the data collected by each UAV, wherein the to-be-transmitted data includes original image data and original point cloud data. These data are compressed by a preset semantic encoding model to generate semantic encoding data, which significantly reduces the data volume and enhances the robustness of transmission. After the ground control unit receives these data, the semantic encoding data is decoded to restore the image and point cloud data. Finally, combined with the state data of each UAV and the decoded data, global perspective data is generated. This process not only optimizes the data transmission efficiency and reduces redundancy, but also provides comprehensive and real-time situation awareness for the operator through the generation of global perspective data, significantly improves the cooperative operation ability and task execution efficiency of the UAV cluster in complex environments, and enhances the overall performance and adaptability of the system.

[0096] In some optional embodiments, the semantic encoding data is decoded to obtain decoded image data and decoded point cloud data, including:

[0097] The image semantic data in the semantic encoding data is decoded by using an image semantic decoding model to obtain decoded image data;

[0098] The point cloud semantic data in the semantic encoding data is decoded by using a point cloud semantic decoding model to obtain decoded point cloud data.

[0099] Specifically, the semantic encoding data refers to compressed data processed by a pre-trained semantic encoding model, which extracts the core information of the original image and point cloud data, significantly reduces the data volume, and enhances the robustness of transmission. The image semantic decoding model and the point cloud semantic decoding model are deep learning models matched with the encoding model, which are used to restore the compressed semantic encoding data to the original data containing key information.

[0100] The specific implementation process of the scheme includes: after the ground control unit receives the semantic encoding data sent by the unmanned aerial vehicle group control device, first input the image semantic data into the image semantic decoding model for decoding. The image semantic decoding model restores the compressed image semantic data to image data close to the original image, i.e. the decoded image data, through a deep learning algorithm. Then, input the point cloud semantic data into the point cloud semantic decoding model for decoding. The point cloud semantic decoding model restores the compressed point cloud semantic data to point cloud data close to the original point cloud, i.e. the decoded point cloud data. Through the accurate decoding process, the integrity and accuracy of the data in the transmission process are ensured, the efficiency of data transmission is improved, the bandwidth requirement is reduced, and the adaptability of the system in complex channel environment is enhanced. In this way, the ground control unit can obtain high-quality and low-delay key information, which provides a solid data foundation for subsequent global situation awareness and decision support.

[0101] In this way, by using the image semantic decoding model to decode the image semantic data in the semantic encoding data and the point cloud semantic decoding model to decode the point cloud semantic data, this process not only ensures the integrity and accuracy of the data in the transmission process, but also significantly improves the efficiency of data transmission, reduces the bandwidth requirement, and enhances the adaptability of the system in complex channel environment.

[0102] In some optional embodiments, the global perspective data is generated according to the state data of each unmanned aerial vehicle, the decoded image data, and the decoded point cloud data, including:

[0103] The decoded point cloud data is fine-tuned according to the decoded image data to obtain an adjusted three-dimensional point cloud model;

[0104] The adjusted three-dimensional point cloud model is spatially aligned and spliced according to the state data of each unmanned aerial vehicle to obtain the global perspective data.

[0105] Specifically, the decoded image data refers to the recovered image information after decoding processing; the decoded point cloud data refers to the recovered three-dimensional space point information after decoding processing. Fine-tuning refers to optimizing the point cloud data using the detailed information in the image data to improve the accuracy and detail performance of the three-dimensional model. The global perspective data refers to a complete three-dimensional situation view generated by integrating all the information collected by the unmanned aerial vehicles, which provides a comprehensive field situation awareness for the operator.

[0106] The specific implementation process of the scheme includes, first fine-tuning the decoded point cloud data using the decoded image data. This process maps the detailed information in the image to the point cloud model through an algorithm, thereby optimizing the details of the point cloud model, improving its accuracy and integrity. After fine-tuning, an adjusted three-dimensional point cloud model is obtained.

[0107] Then, according to the state data of each UAV (including position and attitude information), the adjusted three-dimensional point cloud model is spatially aligned and spliced. This process uses the position and attitude information of the UAV to accurately align the point cloud data collected by different UAVs in three-dimensional space, ensuring the consistency and accuracy of the data. Through alignment and splicing, a seamless global three-dimensional point cloud model, i.e. global perspective data, is generated. Global perspective data provides a complete three-dimensional field situation view, integrating all information collected by UAVs, providing operators with comprehensive field situation awareness.

[0108] In this way, by fine-tuning the decoded point cloud data using the decoded image data, the accuracy and detail performance of the three-dimensional point cloud model can be significantly improved, enhancing the model's ability to describe the environment and targets. Further, with the state data of each UAV, the adjusted three-dimensional point cloud model is spatially aligned and spliced to generate seamless global perspective data. This process not only ensures the accurate alignment of data collected by different UAVs in three-dimensional space, but also provides a complete three-dimensional situation view, providing operators with comprehensive, real-time field situation awareness. This high-precision data processing and global perspective generation significantly improves the collaborative work capability of UAV clusters in complex environments, enhancing the overall performance and task execution efficiency of the system, providing decision-makers with more accurate and comprehensive field information, thereby improving the intelligence level and adaptability of the entire system.

[0109] In some optional embodiments, fine-tuning the decoded point cloud data according to the decoded image data to obtain an adjusted three-dimensional point cloud model includes:

[0110] Matching the decoded image data with the decoded point cloud data to map the image color and texture information in the decoded image data onto the decoded point cloud data to obtain an initial three-dimensional point cloud model;

[0111] Comparing the target position in the decoded image data with the position of the corresponding target in the initial three-dimensional point cloud model, adjusting the position of the target in the initial three-dimensional point cloud model until it is consistent with the target position in the decoded image data, to obtain the adjusted three-dimensional point cloud model.

[0112] Specifically, decoded image data refers to image data restored using the image semantic decoding model. Decoded point cloud data refers to point cloud data restored using the point cloud semantic decoding model. Matching refers to aligning the color and texture information in the image data with the spatial position information in the point cloud data to enhance the detail of the point cloud model. The initial 3D point cloud model refers to the 3D point cloud model after preliminary matching, while the adjusted 3D point cloud model refers to the final 3D point cloud model after position adjustment.

[0113] The implementation of this solution involves first matching the decoded image data with the decoded point cloud data. This process uses an algorithm to map the color and texture information in the image onto the point cloud model, adding rich visual detail to the point cloud model. For example, color information in the image can be used to distinguish different objects from the background, while texture information can enhance the details of the object's surface. This mapping generates an initial 3D point cloud model that contains both color and texture information.

[0114] Next, the target position in the decoded image data is compared with the position of the corresponding target in the initial 3D point cloud model. If a deviation is found between the target position in the point cloud model and the target position in the image data, the position of the target in the point cloud model is adjusted to be consistent with the target position in the image data. This adjustment process ensures the accuracy and consistency of the 3D point cloud model and improves the overall quality of the model. The implementation of this feature not only enhances the detail representation of the 3D point cloud model, but also improves the precision and accuracy of the model. By mapping the color and texture information in the image semantic data to the point cloud model and accurately adjusting the target position, the generated 3D point cloud model can more realistically reflect the actual conditions of the environment and the target.

[0115] In this way, by matching the decoded image data with the decoded point cloud data and mapping the color and texture information in the image to the point cloud model, an initial 3D point cloud model containing rich visual details is generated. Furthermore, by comparing the target positions in the image semantic data with the positions of the corresponding targets in the initial point cloud model and making adjustments, the accuracy and consistency of the 3D point cloud model are ensured. This process not only enhances the detailed representation of the 3D point cloud model and improves the accuracy of the model, but also provides a high-quality data foundation for subsequent global perspective generation and situational awareness. This significantly improves the collaborative operation capabilities and task execution efficiency of drone clusters in complex environments, provides decision makers with more accurate and comprehensive site information, and thus enhances the intelligence level and adaptability of the entire system.

[0116] In some optional embodiments, the method further includes:

[0117] The global view data is sent to the payload operation terminal and the control terminal respectively.

[0118] Specifically, global perspective data refers to a complete three-dimensional view of the site situation generated by integrating and processing data collected by the drone swarm's control equipment. It includes information such as the three-dimensional structure of the environment and the location, shape, and texture of target objects. The payload operation terminal is the operating terminal responsible for processing and analyzing specific payload data, typically monitored and operated by a professional operator. The control terminal is responsible for drone flight control and mission management, typically used by the command department.

[0119] The specific implementation process of this solution involves the ground control unit generating global view data and sending it to the payload operator and control terminal via pre-set communication links. The payload operator receives the global view data and uses it to perform mission-related analysis and decision-making, such as target identification and environmental monitoring. The control terminal, in turn, uses the global view data to plan flight paths and adjust missions, ensuring safe flight and efficient mission execution. This process is implemented through layered transmission channels, ensuring that different types of data are efficiently transmitted to the corresponding terminals, avoiding data redundancy and overload.

[0120] In this way, by sending global-viewpoint data to the payload operator and control terminal respectively, efficient data sharing and collaborative decision-making within the drone swarm are achieved. The payload operator can use this global-viewpoint data for task-related analysis and decision-making, such as target identification and environmental monitoring, thereby improving the accuracy and efficiency of task execution. Simultaneously, the control terminal uses this global-viewpoint data for flight path planning and task adjustment, ensuring safe flight and efficient mission execution. This layered transmission mechanism not only improves the efficiency of data sharing but also enhances the system's collaborative operation capabilities, significantly enhancing the efficiency and adaptability of drone swarms in complex environments and providing strong support for efficient collaborative operations within drone swarms.

[0121] In some optional embodiments, the method further includes:

[0122] Obtaining other payload data collected by the drone swarm control device, where the other payload data is other data in the original data collected by the drone swarm control device except the original image data and the original point cloud data;

[0123] Transfer other payload data to the payload operation terminal.

[0124] Specifically, other payload data refers to data collected by the UAVs other than image and point cloud data during task execution, such as weather data, electromagnetic spectrum data, infrared thermal imaging data, etc. These data have important value for specific tasks and operations.

[0125] The specific implementation process of this scheme includes: when the UAVs are executing tasks, they will not only collect image and point cloud data, but also collect various other payload data through other sensors. These data are collected and stored as part of the raw data through the communication system of the UAVs. Subsequently, the UAVs transmit these other payload data to the ground control unit through the ad hoc network communication link. After receiving these data, the ground control unit forwards the other payload data to the corresponding payload operation end according to the preset communication protocol and task requirements. This process ensures that all relevant data can be timely and accurately delivered to the place where it is needed, so that the operator can have a comprehensive understanding of the task environment and make more accurate decisions. The implementation of this feature not only improves the efficiency of data sharing, but also enhances the versatility and adaptability of the system. By transmitting other payload data to the payload operation end, the operator can use these data for various task-related analysis and decision-making, such as weather data to help the operator assess flight conditions, electromagnetic spectrum data for electronic warfare and signal intelligence tasks, and infrared thermal imaging data for target detection in night or complex environments. This comprehensive data sharing mechanism significantly improves the efficiency and adaptability of the UAV swarm in complex environments, providing strong support for efficient collaborative work of the UAV swarm.

[0126] In this way, by obtaining the other payload data collected by the UAV swarm control device and transmitting these data to the payload operation end, comprehensive sharing of UAV swarm data and multi-task support are achieved. This process not only ensures that the operator can obtain rich multi-source information and thus have a more comprehensive understanding of the task environment, but also improves the flexibility and adaptability of task execution. Through this comprehensive data sharing mechanism, the UAV swarm can more efficiently complete various tasks in complex environments, significantly improving the versatility and task execution efficiency of the system, providing strong support for efficient collaborative work of the UAV swarm.

[0127] In some optional embodiments, after receiving the status data of each UAV sent from the UAV swarm control device, the method further includes:

[0128] Sending the status data of each UAV to the flight operation end.

[0129] Specifically, the state data of each drone refers to the real-time state information of the drone during flight, mainly including the position (such as GPS coordinates), attitude (such as pitch angle, yaw angle, roll angle), speed, remaining power and other key parameters of the drone. These data are collected in real time by sensors on the drone (such as GPS module, IMU, speed sensor, etc.). The flight operation terminal refers to the terminal responsible for the flight control and task management of the drone, usually used by the flight operator to monitor the flight state of the drone and make necessary operations and adjustments.

[0130] The specific implementation process of this scheme includes: the drone collects the state data of the drone in real time through its built-in sensor system during flight. These data are not only used for autonomous flight control of the drone, but also sent to the ground control unit through the ad hoc network communication link. After receiving these data, the ground control unit further forwards the aircraft state data to the flight operation terminal. The flight operator can view the flight state of the drone in real time through the interface of the flight operation terminal, including position, attitude, speed and other information, so as to make timely adjustments and decisions to ensure the safe flight of the drone and the accuracy of task execution.

[0131] In this way, by sending the state data of each drone to the flight operation terminal, real-time monitoring and accurate control of the flight state of the drone are realized. The flight operator can obtain the position, attitude, speed and other key information of the drone in real time, so as to make timely adjustments and decisions to ensure the safe flight of the drone and the accuracy of task execution. This process not only improves the efficiency and accuracy of flight control, but also enhances the overall performance and safety of the system, significantly improves the collaborative work capability and task execution efficiency of the drone swarm in complex environment, and provides strong support for efficient collaborative work of the drone swarm.

[0132] The above embodiments respectively describe the processes of the unmanned aerial vehicle group control device and the ground central control unit in detail. In order to facilitate the overall understanding of the technical scheme of the present application, the communication process of the scheme is described in the embodiment of the present disclosure. The method comprises: the unmanned aerial vehicle group control device obtains the position information and attitude information of each unmanned aerial vehicle to obtain the state data of each unmanned aerial vehicle; determining the to-be-transmitted data in the data collected by each unmanned aerial vehicle according to the state data of all unmanned aerial vehicles, the to-be-transmitted data including original image data and original point cloud data; encoding the original image data and the original point cloud data through the preset semantic encoding model respectively to obtain semantic encoding data; and sending the semantic encoding data and the state data of each unmanned aerial vehicle to the ground central control unit;

[0133] The ground central control unit decodes the semantic coding data to obtain decoded image data and decoded point cloud data, and generates global perspective data according to the state data of each unmanned aerial vehicle, the decoded image data and the decoded point cloud data.

[0134] In this way, the position and attitude information of each unmanned aerial vehicle is obtained by the unmanned aerial vehicle group control device, the state data thereof is determined, and the to-be-transmitted data in the data collected by each unmanned aerial vehicle is screened based on the state data, wherein the to-be-transmitted data includes original image data and original point cloud data. These data are compressed by a preset semantic coding model to generate semantic coding data, which significantly reduces the data amount and enhances the robustness of transmission. Subsequently, the semantic coding data and the state data of each unmanned aerial vehicle are sent to the ground central control unit. The ground central control unit decodes the received semantic coding data to restore the image and point cloud data, and generates global perspective data in combination with the state data of each unmanned aerial vehicle. This process not only optimizes the data transmission efficiency and reduces redundancy, but also provides comprehensive and real-time situational awareness for the operator through the generation of global perspective data, significantly improves the cooperative operation capability and task execution efficiency of the unmanned aerial vehicle cluster in complex environments, and enhances the overall performance and adaptability of the system.

[0135] In order to facilitate the overall understanding of the technical solutions of the present application, the communication process of the scheme is described from the whole by the embodiments of the present disclosure. Referring to Figure 3 , Figure 3 is the system architecture diagram corresponding to the unmanned aerial vehicle group global perspective data generation method in an embodiment of the present disclosure. The diagram shows a system architecture in which a unmanned aerial vehicle group control device and a ground control station work cooperatively, wherein the unmanned aerial vehicle group control device is responsible for collecting enemy information and sharing data through the unmanned aerial vehicle group control device cooperative communication link; the ground control station is composed of a ground central control unit, a flight operation terminal and a payload operation terminal, the ground central control unit processes data from the unmanned aerial vehicle, including semantic coding data and aircraft state data, the flight operation terminal monitors and controls the flight state of the unmanned aerial vehicle through a flight control link, and the payload operation terminal is responsible for decoding and analyzing reconnaissance data; the command center (also referred to as a control terminal) receives the processed global perspective data through the ground control station and command center communication link, which is used for high-level tactical analysis and decision-making. The whole system realizes efficient cooperation of information collection, data processing, flight control and task load operation of the unmanned aerial vehicle group control device through these links to support remote command and control.

[0136] Referring to Figure 4 , Figure 4is a schematic diagram of the overall process of the method for generating global perspective data of a UAV group in an embodiment of the present disclosure. It should be noted that this diagram only shows the communication process between the various parts, and does not show the process of determining the data to be transmitted in the data collected by each UAV, which has been described in the above embodiments and will not be repeated here. In the scheme of this embodiment, it is assumed that the data to be transmitted in the data collected by each UAV has been determined. First, the point cloud data and image data in the data to be transmitted are combined and semantically encoded. The UAV group control device transmits the semantically encoded data to the ground central control unit through a data link, and transmits the state data of each UAV to the ground central control unit through a control link. The ground central control unit transmits other payload data to the payload operation end, and forwards the state data of each UAV to the flight operation end. The ground central control unit decodes the received semantically encoded data, restores the decoded image data and decoded point cloud data, and generates global perspective data in combination with the state data of each UAV. Finally, the generated global perspective data is forwarded to the payload operation end and the command center. This process not only greatly improves the efficiency and accuracy of data transmission, ensures the reliability of data in bandwidth-limited and complex communication environments, but also provides a visual, accurate and information-rich field situation awareness view for ground control personnel through the generated global perspective data, thereby significantly improving the quality and response speed of command and decision-making, and enhancing the operational efficiency and coordination capability of the UAV group control device when performing diversified tasks.

[0137] The device embodiment of the present application is introduced below, which can be used to execute the method for generating global perspective data of a UAV group in the above embodiments of the present application. For details not disclosed in the device embodiment of the present application, please refer to the above embodiments of the method for generating global perspective data of a UAV group.

[0138] The present disclosure also provides a device 500 for generating global perspective data of a UAV group, as shown in Figure 5 applied to a UAV group control device, comprising:

[0139] The acquisition module 501 is configured to acquire the position information and attitude information of each UAV to obtain the state data of each UAV.

[0140] The determination module 502 is configured to determine the data to be transmitted in the data collected by each UAV according to the state data of all UAVs, wherein the data to be transmitted includes original image data and original point cloud data.

[0141] The encoding module 503 is configured to encode the original image data and the original point cloud data through a preset semantic encoding model respectively to obtain semantic encoding data.

[0142] The sending module 504 is configured to send the semantic coding data and the state data of each unmanned aerial vehicle to a ground central control unit, and the ground central control unit is configured to decode the semantic coding data to obtain decoded image data and decoded point cloud data, and generate global perspective data according to the state data of each unmanned aerial vehicle, the decoded image data and the decoded point cloud data.

[0143] In some optional embodiments, the determining module 502 determines the data to be transmitted in the data collected by each unmanned aerial vehicle according to the state data of all unmanned aerial vehicles, including:

[0144] selecting any one of all unmanned aerial vehicles as a current unmanned aerial vehicle;

[0145] calculating the coverage area of each unmanned aerial vehicle according to the position information and the attitude information of the current unmanned aerial vehicle and other unmanned aerial vehicles;

[0146] comparing the coverage area of the current unmanned aerial vehicle with the coverage area of other unmanned aerial vehicles to determine an overlapping area;

[0147] deleting the data corresponding to the overlapping area from the data collected by the current unmanned aerial vehicle to obtain the transmission data of each unmanned aerial vehicle.

[0148] In some optional embodiments, the encoding module 503 encodes the original image data and the original point cloud data through a preset semantic coding model respectively to obtain the semantic coding data, including:

[0149] encoding the original image data through an image semantic coding model to obtain encoded image semantic data;

[0150] encoding the original point cloud data through a point cloud semantic coding model to obtain encoded point cloud semantic data;

[0151] taking the encoded image semantic data and the encoded point cloud semantic data as the semantic coding data.

[0152] In some optional embodiments, the determining module 502 is further configured to:

[0153] obtain other payload data in the original data except the original image data and the original point cloud data;

[0154] send the other payload data to the ground central control unit to be forwarded to a payload operation end by the ground central control unit.

[0155] The present disclosure also provides an unmanned aerial vehicle group global perspective data generation device 600, as shown in Figure 6 applied to a ground central control unit, including:

[0156] The first receiving module 601 is configured to receive semantic coding data sent by a UAV group control device, wherein the UAV group control device is configured to obtain position information and attitude information of each UAV to obtain state data of each UAV; determine to-be-transmitted data in data collected by each UAV according to the state data of all the UAVs, the to-be-transmitted data including original image data and original point cloud data; and encode the original image data and the original point cloud data through preset semantic coding models respectively to obtain the semantic coding data.

[0157] The second receiving module 602 is configured to receive state data of each UAV sent by the UAV group control device.

[0158] The decoding module 603 is configured to decode the semantic coding data to obtain decoded image data and decoded point cloud data.

[0159] The generating module 604 is configured to generate global perspective data according to the state data of each UAV, the decoded image data, and the decoded point cloud data.

[0160] In some optional embodiments, the decoding module 603 decodes the semantic coding data to obtain the decoded image data and the decoded point cloud data, including:

[0161] The image semantic data in the semantic coding data is decoded by using an image semantic decoding model to obtain the decoded image data.

[0162] The point cloud semantic data in the semantic coding data is decoded by using a point cloud semantic decoding model to obtain the decoded point cloud data.

[0163] In some optional embodiments, the generating module 604 generates global perspective data according to the state data of each UAV, the decoded image data, and the decoded point cloud data, including:

[0164] The decoded point cloud data is fine-tuned according to the decoded image data to obtain an adjusted three-dimensional point cloud model.

[0165] The adjusted three-dimensional point cloud model is spatially aligned and spliced according to the state data of each UAV to obtain the global perspective data.

[0166] In some optional embodiments, the generating module 604 fine-tunes the decoded point cloud data according to the decoded image data to obtain an adjusted three-dimensional point cloud model, including:

[0167] The decoded image data and the decoded point cloud data are matched to map image color and texture information in the decoded image data onto the decoded point cloud data to obtain an initial three-dimensional point cloud model.

[0168] Comparing the target position in the decoded image data with the position of the corresponding target in the initial three-dimensional point cloud model, adjusting the position of the target in the initial three-dimensional point cloud model until the position of the target in the decoded image data is consistent, to obtain an adjusted three-dimensional point cloud model.

[0169] In some optional embodiments, the generating module 604 is further configured to:

[0170] The global perspective data is sent to the payload operation terminal and the control terminal respectively.

[0171] In some optional embodiments, the first receiving module 601 is further configured to:

[0172] Obtaining other payload data collected by the UAV group control device, the other payload data being data other than the original image data and the original point cloud data in the original data collected by the UAV group control device;

[0173] The other payload data is transmitted to the payload operation terminal.

[0174] In some optional embodiments, after receiving the state data of each UAV sent by the UAV group control device, the second receiving module 602 is further configured to:

[0175] The state data of each UAV is sent to the flight operation terminal.

[0176] In the technical solution of the present disclosure, the acquisition, storage and application of user personal information involved comply with relevant laws and regulations and do not violate public order and good customs.

[0177] According to embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0178] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present disclosure described and / or claimed in this document.

[0179] As Figure 7As shown, electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of device 700 can also be stored in RAM 703. Computing unit 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to bus 704.

[0180] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0181] The computing unit 701 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the method for generating global perspective data for a swarm of drones. For example, in some embodiments, the method for generating global perspective data for a swarm of drones can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the applet distribution described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the method for generating global perspective data for a swarm of drones by any other suitable means (e.g., via firmware).

[0182] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0183] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0184] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0186] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0187] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server is generally established by computer programs running on the respective computers and having a client-server relationship to each other. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0188] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without departing from the desired results of the technical solutions of the present disclosure, and are not limited herein.

[0189] The specific embodiments described above are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Any further modifications, equivalent replacements, improvements, and the like made within the spirit and principles of the present disclosure should be included within the scope of the present disclosure.

Claims

1. A method for generating global perspective data of a drone swarm, applied to a drone swarm control device, wherein: The method comprises: Obtain the location information and attitude information of each drone and obtain the status data of each drone; Determine the data to be transmitted in the data collected by each drone based on the status data of all drones, wherein the data to be transmitted includes original image data and original point cloud data; Encoding the original image data and the original point cloud data respectively through a preset semantic coding model to obtain semantically coded data; The semantically coded data and the status data of each UAV are sent to a ground central control unit, which is used to decode the semantically coded data to obtain decoded image data and decoded point cloud data, and generate global perspective data based on the status data of each UAV, the decoded image data and the decoded point cloud data.

2. The method according to claim 1, wherein The step of determining the data to be transmitted from the data collected by each drone based on the status data of all drones includes: Select any one from all drones as the current drone; Calculate the coverage area of ​​each drone based on the position information and posture information of the current drone and other drones; Comparing the coverage area of ​​the current drone with the coverage areas of the other drones to determine an overlapping area; The data corresponding to the overlapping area is deleted from the data collected by the current drone to obtain the transmission data of each drone.

3. The method according to claim 1, wherein The encoding of the original image data and the original point cloud data by using a preset semantic coding model to obtain semantically coded data includes: Encoding the original image data through an image semantic coding model to obtain encoded image semantic data; Encoding the original point cloud data through a point cloud semantic coding model to obtain encoded point cloud semantic data; The encoded image semantic data and the encoded point cloud semantic data are used as the semantic encoding data.

4. The method according to any one of claims 1 to 3, wherein: The method further comprises: Acquiring other load data in the original data except the original image data and the original point cloud data; The other payload data is sent to a ground central control unit, so as to be forwarded to a payload operation terminal via the ground central control unit.

5. A method for generating global perspective data of a drone swarm, applied to a ground central control unit, wherein: The method comprises: Receiving semantically coded data sent from a drone swarm control device, wherein the drone swarm control device is used to obtain position information and posture information of each drone to obtain status data of each drone; determining data to be transmitted from data collected by each drone based on the status data of all drones, wherein the data to be transmitted includes original image data and original point cloud data; encoding the original image data and the original point cloud data respectively using a preset semantic coding model to obtain semantically coded data; Receiving status data of each drone sent from the drone swarm control device; Decoding the semantically coded data to obtain decoded image data and decoded point cloud data; Global perspective data is generated according to the status data of each drone, the decoded image data, and the decoded point cloud data.

6. The method according to claim 5, wherein: The decoding of the semantically coded data to obtain decoded image data and decoded point cloud data includes: Decoding the image semantic data in the semantically coded data using an image semantic decoding model to obtain decoded image data; The point cloud semantic data in the semantically coded data is decoded using a point cloud semantic decoding model to obtain decoded point cloud data.

7. The method according to claim 6, wherein: Generating global perspective data according to the status data of each drone, the decoded image data, and the decoded point cloud data includes: Fine-tuning the decoded point cloud data according to the decoded image data to obtain an adjusted three-dimensional point cloud model; According to the status data of each drone, the adjusted three-dimensional point cloud model is spatially aligned and spliced ​​to obtain global perspective data.

8. The method according to claim 7, wherein: The step of fine-tuning the decoded point cloud data according to the decoded image data to obtain an adjusted three-dimensional point cloud model includes: Matching the decoded image data with the decoded point cloud data to map image color and texture information in the decoded image data to the decoded point cloud data to obtain an initial three-dimensional point cloud model; The target position in the decoded image data is compared with the position of the corresponding target in the initial three-dimensional point cloud model, and the position of the target in the initial three-dimensional point cloud model is adjusted until it is consistent with the target position in the decoded image data to obtain an adjusted three-dimensional point cloud model.

9. The method according to any one of claims 5 to 8, wherein: The method further comprises: The global view data is sent to the payload operation terminal and the control terminal respectively.

10. The method according to any one of claims 5 to 8, wherein: The method further comprises: Acquire other payload data collected by the drone swarm control device, where the other payload data is other data in the original data collected by the drone swarm control device except the original image data and the original point cloud data; The other payload data is transmitted to the payload operation terminal.

11. The method according to any one of claims 5 to 8, wherein: After receiving the status data of each drone sent from the drone swarm control device, the method further includes: The status data of each drone is sent to the flight operation terminal.

12. A method for generating global perspective data of a drone swarm, wherein: The method comprises: The drone swarm control device obtains the position information and attitude information of each drone to obtain the status data of each drone; determines the data to be transmitted in the data collected by each drone based on the status data of all drones, wherein the data to be transmitted includes original image data and original point cloud data; encodes the original image data and the original point cloud data using a preset semantic coding model to obtain semantically coded data; and transmits the semantically coded data and the status data of each drone to the ground central control unit; The ground central control unit decodes the semantically coded data to obtain decoded image data and decoded point cloud data, and generates global perspective data based on the status data of each drone, the decoded image data, and the decoded point cloud data.

13. A device for generating global perspective data of a drone swarm, applied to a drone swarm control device, wherein: include: The acquisition module is used to obtain the position information and attitude information of each drone and obtain the status data of each drone; A determination module is used to determine the data to be transmitted in the data collected by each drone based on the status data of all drones, wherein the data to be transmitted includes original image data and original point cloud data; An encoding module, configured to encode the original image data and the original point cloud data respectively through a preset semantic encoding model to obtain semantically encoded data; A sending module is used to send the semantically coded data and the status data of each UAV to a ground central control unit, and the ground central control unit is used to decode the semantically coded data to obtain decoded image data and decoded point cloud data, and generate global perspective data based on the status data of each UAV, the decoded image data and the decoded point cloud data.

14. A device for generating global perspective data of a drone swarm, applied to a ground central control unit, wherein: include: a first receiving module, configured to receive semantically coded data transmitted from a drone swarm control device, wherein the drone swarm control device is configured to obtain position information and attitude information of each drone to obtain status data of each drone; determine data to be transmitted from data collected by each drone based on the status data of all drones, wherein the data to be transmitted includes original image data and original point cloud data; and encode the original image data and the original point cloud data using a preset semantic coding model to obtain semantically coded data; A second receiving module is used to receive status data of each drone sent from the drone group control device; A decoding module, configured to decode the semantically coded data to obtain decoded image data and decoded point cloud data; A generation module is used to generate global perspective data based on the status data of each drone, the decoded image data and the decoded point cloud data.

15. An electronic device, wherein: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-12.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images