A multi-sensor fusion-based unmanned aerial vehicle cluster control method and system

Abstract

The application provides a kind of unmanned aerial vehicle cluster control method and system based on multi-sensor fusion, belongs to unmanned aerial vehicle and data transmission technical field.The method comprises: obtaining the current sensing state value of the first unmanned aerial vehicle in the unmanned aerial vehicle cluster;When the current sensing state value is less than the standard state value, the control center sends a dispatch signal to other unmanned aerial vehicles in the unmanned aerial vehicle cluster, and the dispatch signal instructs other unmanned aerial vehicles to establish a data forwarding link with the first unmanned aerial vehicle;The first unmanned aerial vehicle performs data transmission with the control center through the data forwarding link.In the initial state, the control center opens N processes to interact with N unmanned aerial vehicles respectively;And through the continuous process expansion operation, the current sensing state value ST i of the i-th unmanned aerial vehicle is monitored.The application can accurately identify the blocking period of multi-process communication between the unmanned aerial vehicle cluster and the control center and take chain transfer measures, thereby ensuring the smoothness of multi-sensor fusion data transmission of the unmanned aerial vehicle cluster.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of unmanned aerial vehicle cluster control and data transmission, and particularly relates to an unmanned aerial vehicle cluster control method and system based on multi-sensor fusion, a computer readable storage medium / computer program product implementing the method, an electronic device, and a related unmanned aerial vehicle cluster. BACKGROUND

[0002] Unmanned aerial vehicle cluster control refers to the coordinated management and control of multiple unmanned aerial vehicles, enabling them to complete complex tasks. Unmanned aerial vehicle cluster control technology has high flexibility, efficiency, and collaboration, and has a wide range of application scenarios in various fields. For example, in the power grid monitoring scenario, when the power grid fails, the unmanned aerial vehicle cluster can quickly reach the fault area and carry out search and investigation. They can target the line segment that may have problems according to the fault characteristics and early warning information, quickly locate the fault point, and save valuable time for maintenance personnel; they can carry meteorological sensors to monitor the meteorological conditions along the power grid in real time, including wind speed, wind direction, temperature, humidity, lightning, etc. These meteorological data are of great significance for assessing the operation risk of the power grid and preparing for severe weather in advance. For example, before strong winds arrive, according to the wind speed information monitored by the unmanned aerial vehicle, the power grid equipment can be reinforced or other protective measures can be taken in advance.

[0003] Unmanned aerial vehicle cluster control technology is mainly divided into centralized, distributed, and hybrid. Under centralized control, all decision-making and control instructions of the unmanned aerial vehicles are generated by a central control unit. The central control unit obtains the state information of the entire cluster, then calculates the specific flight tasks and action instructions for each unmanned aerial vehicle according to the task target and environmental information, and sends them to each unmanned aerial vehicle through the communication link. Each unmanned aerial vehicle then transmits data to the control unit based on the multi-process communication mechanism. Under distributed control, the system has no absolute central control unit, and the unmanned aerial vehicles are connected to each other through a communication network, and each makes autonomous decisions and controls based on local information and information obtained from other unmanned aerial vehicles. Distributed control has high flexibility and robustness, and even if some unmanned aerial vehicles fail, other unmanned aerial vehicles can still complete the task. However, it has the disadvantage of being difficult to coordinate, and effective distributed algorithms need to be designed to ensure the collaborative work of unmanned aerial vehicles. Hybrid control combines the advantages of centralized and distributed control, allowing for global planning and coordination by the central control unit, while allowing for local autonomous decision-making and control by the unmanned aerial vehicles.

[0004] In the related art, the authorized invention patent CN113867382B proposes a topology control method of a UAV cluster network. Through the distributed adaptive topology control of the UAVs in the UAV cluster network, the UAV cluster network can adapt to the sensing task requirements of the sensing targets in different scenarios, which helps to reduce the overall energy consumption of the UAV cluster network and improve the overall performance of the UAV cluster network, and avoids unnecessary losses caused by sudden failure of the UAV nodes.

[0005] In the power grid monitoring scenario, the purpose of scheduling UAV cluster operations is mainly to collect various sensing data by multiple UAVs and send them to the central control unit (also known as the control center, control platform, ground server, cloud server, etc.) in a timely manner, i.e., using centralized control. Since there is only one central control unit and data transmission needs to be performed with multiple UAVs, some UAVs may experience communication congestion and data loss. SUMMARY

[0006] To solve the above technical problems, the present application proposes a UAV cluster control method and system based on multi-sensor fusion, a computer readable storage medium / computer program product implementing the method, an electronic device, and a related UAV cluster.

[0007] In the first aspect of the present application, a UAV cluster control method based on multi-sensor fusion is proposed, which includes the following steps:

[0008] Obtain the current sensing state value of the first UAV in the UAV cluster, which represents the data transmission performance between the first UAV and the control center;

[0009] Under normal circumstances, each UAV collects data based on its multiple combined sensors, fuses the data, obtains multi-sensor fusion data to be transmitted, and sends it to the control center directly through the data transmission link with the control center;

[0010] When the current sensing state value of the first UAV is less than the standard state value, the control center sends a scheduling signal to other UAVs in the UAV cluster, which instructs other UAVs to establish a data forwarding link with the first UAV;

[0011] The first UAV performs data transmission with the control center through the data forwarding link;

[0012] The current sensing state value of the second UAV that establishes a data forwarding link with the first UAV is greater than the standard state value.

[0013] The UAV cluster includes N UAVs, and N is an integer greater than 2;

[0014] Each of the UAVs in the UAV cluster is configured with a position sensor and a proximity sensor;

[0015] The position sensor is used to detect the own position, and the proximity sensor is used to calculate the proximity trend of the own UAV and other UAVs, the proximity trend including the approaching trend and the moving away trend.

[0016] In the initial state, the control center opens N processes to interact with the N UAVs respectively; the ith process corresponds to the ith UAV; the control center obtains the current sensing state value STi of the ith UAV through the ith process; i = 1, 2, …, N.

[0017] The current sensing state value of the first UAV in the UAV cluster is obtained, specifically including:

[0018] S1: taking the first process as the scheduling process of the first UAV;

[0019] S2: the control center obtains the current sensing state value ST1 of the first UAV through the scheduling process;

[0020] S3: determining whether the current sensing state value ST1 obtained by all scheduling processes is less than the standard state value STd;

[0021] If yes, performing a process expansion operation for each scheduling process to obtain a plurality of scheduling processes; entering step S4;

[0022] If no, it is determined that the current sensing state value of the first UAV is greater than the standard state value;

[0023] S4: determining whether the number of current scheduling processes is greater than N, if yes, it is determined that the current sensing state value of the first UAV is less than the standard state value;

[0024] If no, returning to step S2.

[0025] When it is determined that the current sensing state value of the first UAV is greater than the standard state value, release part of the scheduling processes;

[0026] When it is determined that the current sensing state value of the first UAV is less than the standard state value, release all scheduling processes.

[0027] After the first UAV and the second UAV establish the data forwarding link, the current sensing state value of the first UAV is continuously detected;

[0028] When the sensing state value is greater than the standard state value, the first UAV disconnects the data forwarding link with the second UAV.

[0029] Obviously, in the method, each of the UAVs in the UAV cluster preferentially performs data transmission with the control center directly (as long as the condition is met, i.e. the current sensing state value is greater than the standard state value); only when the condition is not met (the current sensing state value is less than the standard state value) can the UAV communicate (transmit data) with the control center through the forwarding link; but even in the process of communicating (transmitting data) with the control center through the forwarding link, the current sensing state value of the UAV is still continuously detected; when the sensing state value is greater than the standard state value, the data forwarding link with the second UAV is disconnected, and the UAV directly performs data transmission with the control center directly.

[0030] The control center sends a scheduling signal to other UAVs in the UAV cluster, instructing the other UAVs to establish a data forwarding link with the first UAV, specifically including:

[0031] The control center obtains the current sensing state value of the other UAVs in the UAV cluster;

[0032] Obtain the median and mean of the current sensing state values of all other UAVs;

[0033] If the current sensing state value of a certain other UAV is greater than the median and the mean at the same time, the other UAV is selected as a candidate other UAV;

[0034] A target other UAV is determined from the candidate other UAVs;

[0035] The scheduling signal instructs the target other UAV to establish a data forwarding link with the first UAV.

[0036] Determining a target other UAV from the candidate other UAVs further includes:

[0037] When the candidate other UAVs are multiple, the control center obtains the current position information of each of the candidate other UAVs and the first UAV;

[0038] Based on the current position information, a target other UAV is determined from the candidate other UAVs.

[0039] After obtaining the median and mean of the current sensing state values of all other UAVs, if there is no other UAV whose current sensing state value is greater than the median and the mean, the control center recalls at least part of the UAV cluster.

[0040] In a second aspect of the present application, a UAV cluster is also provided, the UAV cluster comprising N UAVs, N>2; each UAV in the UAV cluster is configured with a position sensor and a proximity sensor; the position sensor is used to detect its own position, and the proximity sensor is used to calculate its own proximity trend with other UAVs, the proximity trend comprising a proximity trend and an away trend;

[0041] The UAV cluster communicates with the control center to perform the multi-sensor fusion-based UAV cluster control method of the first aspect.

[0042] In a third aspect of the present application, a multi-sensor fusion-based UAV cluster control system is also provided, the system comprising a control center, the control center communicating with the UAV cluster through multi-processes to implement the multi-sensor fusion-based UAV cluster control method of the first aspect.

[0043] The present application can accurately identify the blocking period of multi-process communication between the UAV cluster and the control center and take chain transfer measures, thereby ensuring smooth transmission of multi-sensor fusion data of the UAV cluster, and the specific advantages and implementation principles will be further embodied in detail in the specific embodiment part combined with the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0045] Figure 1 is a main flow diagram of a multi-sensor fusion-based UAV cluster control method according to an embodiment of the present application;

[0046] Figure 2 is Figure 1 a further preferred embodiment of the multi-sensor fusion-based UAV cluster control method;

[0047] Figure 3 is Figure 1 a principle diagram of determining target candidate other UAVs in the method;

[0048] Figure 4 is a multi-process communication scenario diagram of a multi-sensor fusion-based UAV cluster control system according to an embodiment of the present application;

[0049] Figure 5 is a principle diagram of monitoring the current sensor state value of the UAV through continuous process expansion operations. Detailed Implementation

[0050] In the specific embodiments of this application, if the embodiments of the relevant technical solutions involve user-related data, then when the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0051] The technical solution of this invention mainly improves the operation control process of unmanned aerial vehicle (UAV) swarms in power grid monitoring scenarios. Therefore, relevant background technical knowledge is introduced first.

[0052] In power grid monitoring scenarios, the purpose of scheduling drone swarm operations is primarily to collect various sensor data from multiple drones and promptly transmit it to the central control unit (also known as the control center or control platform), i.e., to employ centralized control. Under normal circumstances, each drone collects data from its multiple combined sensors, fuses the collected data to obtain multi-sensor fused data to be transmitted, and then sends it directly to the control center via the data transmission link.

[0053] Specifically, drone swarms equipped with high-definition cameras, infrared thermal imagers, and lidar can conduct detailed inspections of power transmission line components such as towers, insulators, and conductors. For example, infrared thermal imaging technology can detect hot spots in the line and promptly identify potential faults; lidar can create 3D models of the line, accurately measure the distance between the line and surrounding objects, and obtain the corresponding data to be transmitted.

[0054] Drone swarms can observe fault points from different angles, obtaining comprehensive fault information. For example, some drones can capture details of the fault point from the front, while others can take pictures from the side or above and transmit them to the control center, allowing maintenance personnel to more accurately determine the severity and scope of the fault and formulate appropriate maintenance plans.

[0055] Multiple drones in a drone swarm can monitor the growth of trees around power lines, potential geological hazards (such as landslides and mudslides), and the dynamics of construction sites in real time. Once it is detected that trees are growing close to the power lines, there are signs of geological hazards, or construction sites may endanger power grid safety, the data can be promptly transmitted to the control center to issue an early warning so that relevant departments can take appropriate measures.

[0056] The multiple UAVs in the UAV cluster can also carry meteorological sensors to monitor the meteorological conditions along the power grid in real time, including wind speed, wind direction, temperature, humidity, lightning, etc. These meteorological data are of great significance for assessing the operation risk of the power grid and preparing in advance for severe weather. For example, before strong wind weather arrives, the power grid equipment can be reinforced or other protective measures can be taken according to the wind speed information monitored by the UAVs.

[0057] For the control center, the UAV cluster can collect a large amount of data during the monitoring process, including images, videos, temperatures, distances, and various types of data. After fusion, multi-sensor fusion data can be obtained, which can be transmitted to the ground control center or cloud server through the wireless communication network, providing rich data resources for the operation and management of the power grid. The ground control center or cloud server processes and analyzes the collected data using big data analysis, artificial intelligence, etc., which can realize intelligent evaluation and fault prediction of the state of the power grid equipment. For example, by analyzing the images of insulators, it can be determined whether the insulators have damage, aging, etc.; by long-term analysis of line temperature data, it can be predicted that the line may be overloaded, and load adjustment can be made in advance to ensure the safe and stable operation of the power grid.

[0058] Generally, the UAV cluster includes N UAVs, and N is an integer greater than 2. For this cluster, the N UAVs are equal in status, and there is no priority in importance or level. The N UAVs independently collect on-site sensing data in different regions, different angles, or even different time periods, and send them to the control center through the transmission link with the control center. The control center itself can be a ground control center or a cloud server.

[0059] When the performance of the control center is strong enough, for example, it is sufficient to receive concurrent data transmission of N UAVs (i.e., the concurrency performance is not less than N), N UAVs can transmit data to the control center at the same time; or in other words, each of the N UAVs can transmit data to the control center "on demand and at any time";

[0060] When the performance of the control center is not strong enough, i.e., it is not sufficient to receive concurrent data transmission of N UAVs (i.e., the concurrency performance is less than N), through the pre-set communication protocol and transmission strategy, such as the time slice strategy of multi-process time sharing, under the condition of ensuring that the data transmission is not lost, within a certain delay tolerance threshold, each of the N UAVs can also send the sensing data collected by itself to the control center completely.

[0061] However, in actual application, no matter which case, there may be serious communication congestion and delay, so that the integrity and accuracy of the sensing data cannot be guaranteed. For example, in the first case (concurrent performance is not less than N), the performance of individual drones (data acquisition capability, data transmission capability, etc.) is different, and some drones will have low transmission performance when the distance between the drone and the control center exceeds the threshold; some drones may have long-term access to more process resources due to their excellent performance, resulting in transmission process blocking of the drones that are preempted; in the second case, if the communication protocol and transmission strategy are unreasonable, or the performance of the control center is low, it is more likely that the transmission channel will be blocked, resulting in data loss.

[0062] Traditional technologies usually improve the above problems by setting message priority, node transmission priority and other strategies. However, as mentioned above, for the cluster, the status of the N drones is equal, and there is no priority of importance or level. The node transmission priority of each drone itself and the message priority to be sent by each drone should be the same, and should not be treated differently. Therefore, the related strategies designed by traditional technologies are not suitable for the operation control process of the drone cluster in the power grid monitoring scene.

[0063] Therefore, the technical solutions of the present application are proposed.

[0064] Referring to Figure 1 , Figure 1 is a main flowchart of a multi-sensor fusion-based drone cluster control method according to an embodiment of the present application.

[0065] Figure 1 The method comprises the following steps (for the convenience of description, each step is assigned a number, but the related numbers are omitted in each figure) :

[0066] S100: obtaining a current sensing state value of a first drone in the drone cluster, the current sensing state value representing the data transmission performance of the first drone and a control center;

[0067] S200: when the current sensing state value of the first drone is less than a standard state value, the control center sends a scheduling signal to other drones in the drone cluster, and the scheduling signal instructs other drones to establish a data forwarding link with the first drone;

[0068] S300: the first drone performs data transmission with the control center through the data forwarding link;

[0069] Among them, the current sensing state value of the second drone which establishes the data forwarding link with the first drone is greater than the standard state value.

[0070] Next, in combination with Figures 2-3 , the various steps of the above method embodiment are further preferred embodiment.

[0071] In a specific application, the UAV cluster includes N UAVs, N is an integer greater than 2;

[0072] In the initial state, the control center opens N processes to interact with the N UAVs respectively; the ith process corresponds to the ith UAV; the control center obtains the current sensing state value STi of the ith UAV through the ith process; i = 1, 2, …, N.

[0073] The following embodiments take i = 1 as an example, that is, taking the first UAV as an example. It can be understood that the "first UAV" here can actually be any one of the "N UAVs", and the case of "i = 1" here can be generalized to the case of "i = 2, …, N".

[0074] Step S100 obtains the current sensing state value of the first UAV, which represents the data transmission performance of the first UAV and the control center.

[0075] As mentioned earlier, the N UAVs are equal in status, and there is no priority in importance or level. The N UAVs independently collect on-site sensing data of different regions, different angles, and even different time periods, and send them to the control center through the transmission link with the control center.

[0076] In theory, each of the N UAVs should equally share the data transmission link resources with the control center, whether it is "real-time equality" (the control center concurrent performance is not less than N) or "time-sharing equality" (the control center concurrent performance is not less than N), that is, in theory, the current sensing state value of each UAV should be equal, that is, the data transmission performance value of each UAV and the control center should be the same (for example, not too different).

[0077] However, in actual application, due to the different individual performance of each UAV, the different positions of each UAV, the different data collection strategies of each UAV, and other factors, the data transmission performance value of each UAV and the control center will gradually differ. That is, the current sensing state value of each UAV in the UAV cluster will gradually change and cause large differences.

[0078] For example, for a certain UAV, if its data collection capability is strong, but the data transmission capability is weak, it can result in low data transmission performance with the control center, i.e. the current sensing state value is low; for another UAV, since its distance from the control center exceeds the preset threshold, its communication with the control center can occur sporadic interruption or intermittent instability, i.e. the data transmission performance with the control center is low, i.e. the current sensing state value is low; in another scenario, UAV A and UAV B are in the same collection area, but the comprehensive performance of UAV B (such as uplink connection capability + data transmission capability) exceeds that of UAV A, which can result in that the transmission link resource of UAV A with the control center is forced by UAV B, i.e. the data transmission performance of UAV A with the control center is low, i.e. the current sensing state value of UAV A is low, etc.

[0079] In actual application, the current sensing state value of the first UAV in the UAV cluster can be measured by various indicators (or combinations of indicators) as above, such as the amount of data successfully transmitted by the first UAV to the control center in unit time, the amount of feedback signals received from the control center in unit time, the current uplink transmission rate of the first UAV, etc.

[0080] Specifically, the control center obtains the current state value of each UAV through a corresponding data transmission process, and the specific means include heartbeat detection mechanism, packet loss monitoring mechanism, data transmission amount, data transmission rate, data transmission delay, etc. The present application does not make specific limitation.

[0081] When the current sensing state value of the first UAV is less than the standard state value, i.e. step S200 is entered: the control center sends a scheduling signal to other UAVs in the UAV cluster, and the scheduling signal instructs other UAVs to establish a data forwarding link with the first UAV;

[0082] The standard state value here can be a sensing state value that can meet the basic data transmission performance, which can be set according to historical statistical value or expert experience value, and can be determined based on the transmission performance of the UAV and the transmission bandwidth resource of the current control center, etc.

[0083] In most scenarios, when the current sensing state value of the first UAV is less than the standard state value, it means that the data transmission performance of the first UAV with the control center is already poor, and cannot meet the basic data transmission performance requirement, and it is likely to occur data packet loss;

[0084] Correspondingly, when the current sensing state value of the first UAV is less than the standard state value, it means that the transmission link resource of the first UAV is likely to be occupied by other UAVs (because the total amount of transmission link resources is certain), that is, there is likely to be another UAV whose transmission performance exceeds the standard state value, for example, other UAVs (for example, a second UAV) whose current sensing state value is greater than the standard state value.

[0085] At this time, based on the scheduling signal of step S200, instructing other UAVs (for example, a second UAV) to establish a data forwarding link with the first UAV;

[0086] Then step S300 is performed: the first UAV performs data transmission with the control center through the data forwarding link; and the current sensing state value of the second UAV which establishes a data forwarding link with the first UAV is greater than the standard state value.

[0087] That is, when the current sensing state value of the first UAV is low, that is, the data transmission performance of the first UAV with the control center is not good, the first UAV can perform data transmission with the control center through the data forwarding link, thereby avoiding data packet loss;

[0088] However, it should be understood that the UAVs in the UAV cluster are generally the most stable to use the direct communication link with the control center for data transmission, and only when the direct communication link of the control center is not good, data transmission with the control center through the data forwarding link is performed to prevent data packet loss.

[0089] Therefore, as a further preferred embodiment, in Figure 1 addition, referring to Figure 2 , Figure 2 is Figure 1 a further preferred embodiment of the multi-sensor fusion-based UAV cluster control method.

[0090] Figure 2 The embodiment of Figure 1 after the step S300 (“the first UAV performs data transmission with the control center through the data forwarding link”), further comprising: continuously detecting the current sensing state value of the first UAV;

[0091] When the sensing state value is greater than the standard state value, the first UAV disconnects the data forwarding link with the second UAV.

[0092] Therefore, Figure 2 the preferred embodiment of

[0093] S110: Obtain a current sensor state value of a first UAV in the UAV cluster;

[0094] S210: When the current sensor state value of the first UAV is less than a standard state value, determine a second UAV that can establish a data forwarding link with the first UAV;

[0095] S310: The second UAV establishes a data forwarding link with the first UAV;

[0096] S410: The first UAV performs data transmission with the control center through the data forwarding link;

[0097] S510: Continuously detect the current sensor state value of the first UAV;

[0098] S610: When the sensor state value is greater than the standard state value, the first UAV disconnects the data forwarding link with the second UAV and directly performs data transmission with the control center.

[0099] That is, after the first UAV establishes the data forwarding link with the second UAV, the current sensor state value of the first UAV is continuously detected;

[0100] When the sensor state value is greater than the standard state value, the first UAV disconnects the data forwarding link with the second UAV.

[0101] Next, refer to Figure 3 , Figure 3 Further provided is Figure 1 The principle diagram of determining the target candidate other UAV in the method.

[0102] In the foregoing embodiment, it is limited that the current sensor state value of the second UAV that establishes the data forwarding link with the first UAV is greater than the standard state value.

[0103] In actual application, there can be multiple second UAVs.

[0104] Therefore, Figure 3 The method further includes the following implementation:

[0105] The control center obtains a current sensor state value of other UAVs in the UAV cluster;

[0106] Obtain the median and average of the current sensor state values of all other UAVs;

[0107] If the current sensor state value of a certain other UAV is greater than the median and average, the other UAV is taken as a candidate other UAV;

[0108] determining a target other unmanned aerial vehicle from the candidate other unmanned aerial vehicles;

[0109] The scheduling signal instructs the target other unmanned aerial vehicle to establish a data forwarding link with the first unmanned aerial vehicle.

[0110] Unlike the way of only adopting the mean value in the conventional technology, the embodiments of the present application further combine the mean value and the median value (considered at the same time), so that the current "second unmanned aerial vehicle" (set) that is more suitable as a forwarding link can be more significantly found out.

[0111] The median and the mean value are two commonly used statistics to describe the trend in the data set. The median has the following advantages compared with the mean value:

[0112] Not affected by extreme values: the mean value is significantly affected by extreme values (maximum value or minimum value) in the data, while the median is relatively stable. For example, in a set of data {1, 2, 3, 4, 100}, the mean value is 22, which is greatly affected by the extreme value 100 and cannot well represent the general level of the data; while the median is 3, which better reflects the intermediate position level of the data and is not disturbed by the extreme value 100.

[0113] Better representative for skew distribution data: for skew distribution data, the mean value may deviate to the long tail direction and cannot accurately reflect the central tendency of the data.

[0114] More robust when the data distribution is not clear: when the distribution of the data is not clear, or the data has missing values or abnormal values, the calculation of the median is relatively simple and the result is more robust. It only needs to determine the middle position of the data and does not depend on the specific values of all the data, so in some complex data situations, it can provide more reliable information.

[0115] In the unmanned aerial vehicle cluster, due to the different flight lengths of different unmanned aerial vehicles (resulting in the difference in performance indicators), the flight positions are different (especially in the autonomous control network), and other factors, the current sensing state value takes into account the extreme value, skew distribution, and distribution situation not clear, missing, and other characteristic points. Therefore, in view of the above problems, the technical scheme of the present application creatively combines the median and the mean value, which has better effect in practical application.

[0116] Preferably, determining a target other unmanned aerial vehicle from the candidate other unmanned aerial vehicles further comprises:

[0117] When the candidate other unmanned aerial vehicles are multiple, the control center acquires the current position information of each other unmanned aerial vehicle of the candidate other unmanned aerial vehicles and the first unmanned aerial vehicle;

[0118] determine a target other unmanned aerial vehicle from the candidate other unmanned aerial vehicles based on the current position information.

[0119] When the current sensing state value is greater than the median and the average value of the other unmanned aerial vehicles, there are more than two, then one of the best other unmanned aerial vehicles needs to be selected from them.

[0120] In practical applications, each unmanned aerial vehicle in the unmanned aerial vehicle cluster is configured with a position sensor and a proximity sensor;

[0121] The position sensor is used to detect the position of itself, and the proximity sensor is used to calculate the proximity trend of itself and other unmanned aerial vehicles, which includes the approaching trend and the moving away trend.

[0122] At this time, when the candidate other unmanned aerial vehicles are multiple, the control center obtains the current position information of each other unmanned aerial vehicle of the candidate other unmanned aerial vehicles and the first unmanned aerial vehicle, determines a target other unmanned aerial vehicle from the candidate other unmanned aerial vehicles based on the current position information, and specifically includes:

[0123] Select an other unmanned aerial vehicle with a moving away trend as the target other unmanned aerial vehicle from the candidate other unmanned aerial vehicles.

[0124] That is, if there are multiple candidate other unmanned aerial vehicles, select a second unmanned aerial vehicle with a moving away trend from the first unmanned aerial vehicle as the target other unmanned aerial vehicle.

[0125] Because, generally, the reason why the current sensing state value of unmanned aerial vehicle A is less than the standard state value is that unmanned aerial vehicle A and unmanned aerial vehicle B are in the same collection area, but because the comprehensive performance (such as uplink connection ability + data sending ability) of unmanned aerial vehicle B exceeds that of unmanned aerial vehicle A, it may cause the transmission link resource of unmanned aerial vehicle A and the control center to be forced by unmanned aerial vehicle B, that is, the data transmission performance of unmanned aerial vehicle A and the control center is low, that is, the current sensing state value of unmanned aerial vehicle A is low.

[0126] That is, if multiple other candidate unmanned aerial vehicles and the first unmanned aerial vehicle are located in the same collection area, causing the current sensing state value of the first unmanned aerial vehicle to be low, at this time, the other candidate unmanned aerial vehicle with a moving away trend from the first unmanned aerial vehicle should be preferred as the target other unmanned aerial vehicle, so that after the first unmanned aerial vehicle establishes a forwarding link with the target other unmanned aerial vehicle, the risk of resource occupation is reduced as much as possible, thereby restoring the direct communication transmission link with the control center as much as possible.

[0127] This further emphasizes that the focus of the technical solution of the present application is that the overall unmanned aerial vehicle in the unmanned aerial vehicle cluster is preferentially to adopt the direct communication link with the control center for data transmission, and only when the direct communication link of the control center is not good, in order to prevent data packet loss, the data forwarding link is used to perform data transmission with the control center, but it is also necessary to restore to the direct communication transmission link with the control center as soon as possible.

[0128] In another aspect, after the first unmanned aerial vehicle establishes the data forwarding link with the second unmanned aerial vehicle, the current sensing state value of the second unmanned aerial vehicle is continuously detected;

[0129] When the current sensing state value of the second unmanned aerial vehicle is less than the standard state value, the first unmanned aerial vehicle disconnects the data forwarding link with the second unmanned aerial vehicle.

[0130] At this time, since the second unmanned aerial vehicle also does not meet the link forwarding condition, the first unmanned aerial vehicle disconnects the data forwarding link with the second unmanned aerial vehicle, and then reselects other candidate unmanned aerial vehicles that meet the link forwarding condition, that is, repeats the steps of the method, or if the first unmanned aerial vehicle has restored the normal communication state with the control center.

[0131] In an extreme case, the candidate other unmanned aerial vehicles are 0, that is, after obtaining the median and average of the current sensing state values of all other unmanned aerial vehicles, there is no other unmanned aerial vehicle whose current sensing state value is greater than the median and average.

[0132] Such a situation indicates that when there is a first unmanned aerial vehicle whose current sensing state value is less than the standard state value, and there is no other unmanned aerial vehicle whose current sensing state value is greater than the median and average, at this time, it means that the current transmission state of all unmanned aerial vehicles in the unmanned aerial vehicle cluster is not good, which is extremely rare and can be regarded as an abnormal situation, at this time, the preferred processing manner is to recall part of the unmanned aerial vehicles to perform abnormal situation processing.

[0133] In actual application, the control center obtains the current sensing state value STi of the i-th unmanned aerial vehicle through the i-th process.

[0134] However, if only the current sensing state value STi of the i-th unmanned aerial vehicle obtained at a single time is used as a judgment standard, since fluctuations and abnormalities cannot be avoided in actual transmission process, and the frequent establishment of the forwarding link will also waste system resources, for this, the present application proposes a further improved implementation as follows:

[0135] The obtaining of the current sensing state value of the first unmanned aerial vehicle in the unmanned aerial vehicle cluster specifically comprises:

[0136] S1: taking the first process as the scheduling process of the first unmanned aerial vehicle;

[0137] S2: The control center obtains the current sensing state value ST1 of the first unmanned aerial vehicle through a scheduling process;

[0138] S3: Determine whether the current sensing state value ST1 obtained by all scheduling processes is less than the standard state value STd;

[0139] If yes, perform a process expansion operation for each scheduling process to obtain a plurality of scheduling processes; go to step S4;

[0140] If no, it is determined that the current sensing state value of the first unmanned aerial vehicle is greater than the standard state value;

[0141] S4: Determine whether the number of current scheduling processes is greater than N, if yes, determine that the current sensing state value of the first unmanned aerial vehicle is less than the standard state value;

[0142] If no, return to step S2.

[0143] It can be seen that in order to ensure that "the unmanned aerial vehicles in the unmanned aerial vehicle cluster are generally preferentially adopt direct communication link with the control center for data transmission", when obtaining the current sensing state value of the first unmanned aerial vehicle, the application does not only take the current sensing state value STi of the ith unmanned aerial vehicle obtained at a single time as the judgment standard, but performs a process expansion operation, and only when all processes detect that the current sensing state value ST1 is less than the standard state value STd after multiple process expansions, it is determined that the current sensing state value of the first unmanned aerial vehicle is less than the standard state value; otherwise, as long as one process detects that the current sensing state value ST1 is greater than the standard state value STd, it is determined that the current sensing state value of the first unmanned aerial vehicle is greater than the standard state value, thereby avoiding frequent forwarding link establishment and ensuring that "the unmanned aerial vehicles in the unmanned aerial vehicle cluster are generally preferentially adopt direct communication link with the control center for data transmission"; at the same time, the process expansion is not wireless, but based on the number and size of the current unmanned aerial vehicle cluster, that is, a certain unmanned aerial vehicle occupies at most all N process equivalent resource processes for monitoring the current sensing state value.

[0144] On the basis of the above method embodiment, refer to Figure 4 , Figure 4 is a multi-process communication scene schematic diagram of a multi-sensor fusion based unmanned aerial vehicle cluster control system according to an embodiment of the application.

[0145] Generally, the unmanned aerial vehicle cluster includes N unmanned aerial vehicles, N>2; each unmanned aerial vehicle in the unmanned aerial vehicle cluster is configured with a position sensor and a proximity sensor; the position sensor is used to detect the position of itself, and the proximity sensor is used to calculate the proximity trend of itself and other unmanned aerial vehicles, the proximity trend including the approaching trend and the moving away trend; the unmanned aerial vehicle cluster communicates with the control center to perform the unmanned aerial vehicle cluster control method based on multi-sensor fusion.

[0146] Figure 4 The case of N=4 is shown, that is, the first-fourth unmanned aerial vehicles independently perform data transmission and instruction interaction with the control center through the first-fourth processes respectively.

[0147] As a summary, in the initial state, the control center opens N processes to respectively interact with the N unmanned aerial vehicles; the ith process corresponds to the ith unmanned aerial vehicle; the control center obtains the current sensing state value STi of the ith unmanned aerial vehicle through the ith process; i=1, 2, …, N.

[0148] Taking the first process corresponding to the first unmanned aerial vehicle as an example, referring to Figure 5 , Figure 5 The principle diagram for monitoring the current sensing state value of the unmanned aerial vehicle through the continuous process expansion operation is shown.

[0149] At this time, the current sensing state value of the first unmanned aerial vehicle in the unmanned aerial vehicle cluster is obtained, specifically including:

[0150] S1: taking the first process as the scheduling process of the first unmanned aerial vehicle;

[0151] S2: the control center obtains the current sensing state value ST1 of the first unmanned aerial vehicle through the scheduling process;

[0152] S3: judging whether the current sensing state values ST1 obtained by all scheduling processes are all less than the standard state value STd;

[0153] If yes, the process expansion operation is performed for each scheduling process to obtain a plurality of scheduling processes; step S4 is entered;

[0154] If no, it is determined that the current sensing state value of the first unmanned aerial vehicle is greater than the standard state value;

[0155] S4: judging whether the number of current scheduling processes is greater than N, if yes, it is determined that the current sensing state value of the first unmanned aerial vehicle is less than the standard state value;

[0156] If no, return to step S2.

[0157] In Figure 5In the embodiment, assuming that the first process is taken as the scheduling process of the first UAV, if the current sensing state value ST1 of the first UAV obtained by the control center through the scheduling process (i.e., the first process) is less than the standard state value STd, process expansion operation is needed to expand the first process into the 11th process and the 12th process;

[0158] At this time, the number of scheduling processes is 2, and since 2<4, the control center continues to obtain the current sensing state value (two ST1) of the first UAV through the scheduling processes (i.e., the 11th process and the 12th process); if both of the two ST1 are less than the standard state value STd, process expansion operation is needed to expand the 11th process into the 111th process and the 112th process, and to expand the 12th process into the 121th process and the 122th process;

[0159] At this time, the number of scheduling processes is 4, and since 4=4 (4>4 is not true), the control center continues to obtain the current sensing state value (four ST1) of the first UAV through the scheduling processes (i.e., the 111th process, the 112th process, the 121th process and the 122th process); if all the four ST1 are less than the standard state value STd, process expansion operation is needed to expand the 111th process into the 1111th process and the 1112th process, to expand the 112th process into the 1121th process and the 1122th process, to expand the 121th process into the 1211th process and the 1212th process, and to expand the 122th process into the 1221th process and the 1222th process;

[0160] At this time, the number of scheduling processes is 8, and since 8>4, it is determined that the current sensing state value of the first UAV is less than the standard state value.

[0161] In actual application, the process expansion operation can be process preemption, process fork, etc., which is not specifically limited in the present application.

[0162] To save process resources, when it is determined that the current sensing state value of the first UAV is greater than the standard state value, part of the scheduling processes are released, for example, only one process is reserved for the first UAV to communicate with the control center.

[0163] When it is determined that the current sensing state value of the first UAV is less than the standard state value, all the scheduling processes are released, at this time, since the first UAV needs to communicate through the forwarding link, it is temporarily unnecessary to reserve any process resource for the control center.

[0164] Although not shown in the drawings, preferably, the more product embodiments can also be a UAV cluster, the UAV cluster comprising N UAVs, N>2; each UAV in the UAV cluster is configured with a position sensor and a proximity sensor; the position sensor is used to detect its own position, and the proximity sensor is used to calculate its own proximity trend with other UAVs, the proximity trend including the approaching trend and the moving away trend; the UAV cluster communicates with the control center to perform the aforementioned multi-sensor fusion-based UAV cluster control method.

[0165] Although not shown in the drawings, preferably, the more product embodiments can also be an electronic device, the electronic device comprising: a memory and one or more processors. The memory stores one or more application programs, the one or more application programs being adapted to be executed by the one or more processors to implement the aforementioned multi-sensor fusion-based UAV cluster control method.

[0166] Although not shown in the drawings, the more embodiments also include a computer medium, the medium storing a computer program, when the computer program is executed, all or part of the steps of the aforementioned multi-sensor fusion-based UAV cluster control method are implemented.

[0167] It can be understood that the system, product, device, medium embodiments and method embodiments correspond to each other, and can be mutually referred to, and the principles are similar or the same, so they will not be repeated.

[0168] For the multi-sensor fusion data collection operation control process of the UAV cluster in the power grid monitoring scene, the application can accurately identify the blocking period of the multi-process communication between the UAV cluster and the control center and take chain transfer measures by continuously expanding the process to monitor the current sensor state value STi of the ith UAV, thereby ensuring the smooth transmission of the multi-sensor fusion data of the UAV cluster and preventing data packet loss.

[0169] Meanwhile, in the method, each of the UAVs in the UAV cluster preferentially performs data transmission with the control center directly (as long as the condition is met, i.e., the current sensing state value is greater than the standard state value); only when the condition is not met (the current sensing state value is less than the standard state value) can the data transmission with the control center be performed through the forwarding link; however, even in the process of the data transmission with the control center through the forwarding link, the current sensing state value of the UAV is continuously detected; when the sensing state value is greater than the standard state value, the data forwarding link with the second UAV is disconnected, and the data transmission with the control center is performed directly, thereby avoiding unnecessary process switching and link forwarding; when it is determined that the current sensing state value of the first UAV is greater than the standard state value, part of the scheduling processes are released, for example, only one process is reserved for the first UAV to communicate with the control center; when it is determined that the current sensing state value of the first UAV is less than the standard state value, all the scheduling processes are released, at this time, since the first UAV needs to communicate through the forwarding link, it is temporarily unnecessary to reserve any process resource for the communication with the control center. In summary, the above can significantly save the process resource.

[0170] Other technologies, principles, algorithms or models not described in detail in the present application can be referred to the prior art.

[0171] The method embodiments and system of the present application have been shown and described, but it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirit of the present application, the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for controlling a swarm of unmanned aerial vehicles (UAVs) based on multi-sensor fusion, characterized in that, The method includes the following steps: Obtain the current sensor status value of the first drone in the drone cluster, wherein the current sensor status value characterizes the data transmission performance between the first drone and the control center; When the current sensor status value of the first UAV is less than the standard status value, the control center sends a scheduling signal to the other UAVs in the UAV cluster. The scheduling signal instructs the other UAVs to establish a data forwarding link with the first UAV. The first UAV transmits data to the control center via the data forwarding link; The current sensor status value of the second UAV that has established a data forwarding link with the first UAV is greater than the standard status value; The drone cluster consists of N drones, where N is an integer greater than 2; In the initial state, the control center opens N processes to interact with the N drones using data commands. The i-th process corresponds to the i-th UAV; the control center obtains the current sensor status value STi of the i-th UAV through the i-th process; i = 1, 2, ..., N; The step of obtaining the current sensor status value of the first drone in the drone cluster specifically includes: S1: Use process 1 as the scheduling process for the first UAV; S2: The control center obtains the current sensor status value ST1 of the first UAV through the scheduling process; S3: Determine whether the current sensor status value ST1 obtained by all scheduling processes is less than the standard status value STd; If so, perform a process expansion operation for each scheduling process to obtain multiple scheduling processes; proceed to step S4; If not, then determine that the current sensing state value of the first UAV is greater than the standard state value; S4: Determine whether the number of currently scheduled processes is greater than N. If so, determine that the current sensor status value of the first UAV is less than the standard status value. If not, return to step S2.

2. The UAV swarm control method based on multi-sensor fusion as described in claim 1, characterized in that: After the first UAV establishes the data forwarding link with the second UAV, the current sensor status value of the first UAV is continuously monitored. When the current sensor status value of the first UAV is greater than the standard status value, the first UAV disconnects the data forwarding link with the second UAV.

3. The UAV swarm control method based on multi-sensor fusion as described in claim 1, characterized in that: The control center sends a scheduling signal to other drones in the drone cluster, instructing the other drones to establish a data forwarding link with the first drone, specifically including: The control center obtains the current sensor status values ​​of other drones in the drone cluster; Obtain the median and average of the current sensor status values ​​of all other drones; If the current sensor status value of another drone is greater than the median and the average value, then that other drone is considered as a candidate other drone. Identify a target other drone from the candidate other drones; The scheduling signal instructs other target drones to establish a data forwarding link with the first drone.

4. The UAV swarm control method based on multi-sensor fusion as described in claim 3, characterized in that: Determining a target other drone from the candidate other drones further includes: When there are multiple candidate other drones, the control center obtains the current location information of each of the candidate other drones and the first drone; Based on the current location information, a target other drone is determined from the candidate other drones.

5. The UAV swarm control method based on multi-sensor fusion as described in claim 3, characterized in that: After obtaining the median and average of the current sensor status values ​​of all other drones, if there are no other drones with current sensor status values ​​greater than the median and average, the control center recalls a portion of the drones in the drone cluster.

6. The UAV swarm control method based on multi-sensor fusion as described in claim 1, characterized in that: Each drone in the drone swarm is equipped with a position sensor and a proximity sensor; The position sensor is used to detect its own position, and the proximity sensor is used to calculate its own approach trend with other drones, the approach trend including approach trend and distance trend.

7. A drone swarm, the drone swarm comprising N drones, where N>2; each drone in the drone swarm is equipped with a position sensor and a proximity sensor; The position sensor is used to detect its own position, and the proximity sensor is used to calculate its approach trend to other drones, including approach trend and distance trend. The drone swarm communicates with the control center to execute the drone swarm control method based on multi-sensor fusion as described in any one of claims 1-6.

8. A multi-sensor fusion-based unmanned aerial vehicle (UAV) swarm control system, the system comprising a control center, the control center communicating with the UAV swarm through multiple processes to implement the multi-sensor fusion-based UAV swarm control method according to any one of claims 1-6.

Images