Self-adaptive gesture control unmanned aerial vehicle cluster system and method

Abstract

The invention discloses a self-adaptive gesture control unmanned aerial vehicle cluster system and method, and aims to improve the gesture recognition precision and execution efficiency of an unmanned aerial vehicle formation in a complex environment. The system comprises an unmanned aerial vehicle formation unit and a cooperative work communication unit, wherein each unmanned aerial vehicle is equipped with an environment parameter acquisition module, a gesture recognition module, a data fusion module and a flight control module. The system collects multi-dimensional environment data in real time, automatically selects and starts a sensor with the strongest adaptability to collect gesture data, and calculates the comprehensive recognition confidence coefficient based on the sensor adaptability and the recognition confidence coefficient. And the data fusion module is used for generating an optimal control instruction by fusing the data of the multiple unmanned aerial vehicles, and adjusting the flight path, formation, spacing and speed of the unmanned aerial vehicles in the formation according to the optimal instruction. According to the system, the adaptive capacity and the cooperative work efficiency of the unmanned aerial vehicle formation in a complex environment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle control, and particularly to an adaptive gesture control unmanned aerial vehicle cluster system and solution. Background Art

[0002] With the rapid development of unmanned aerial vehicle technology, unmanned aerial vehicle formations have become an increasingly important tool when performing various tasks, especially in scenarios such as rescue, monitoring, and environmental detection. Traditional unmanned aerial vehicle formation control systems usually rely on fixed flight paths and task presets, lacking the ability to interact with the operator in real time, resulting in limited system flexibility and emergency response capabilities. In certain specific scenarios, the operator cannot control the unmanned aerial vehicle in a timely and efficient manner through a remote controller or other traditional input devices, which requires a more intuitive, fast, and flexible control method. Gesture control thus becomes an ideal choice.

[0003] For example, in emergency scenarios such as disaster rescue, fire fighting, and environmental monitoring, the operator often needs to issue instructions quickly and accurately, while traditional control methods (such as remote controllers or voice instructions) may not be able to respond in a timely manner due to complex environments, inconvenient operations, or communication limitations. Especially in complex environments such as thick smoke, low light, and harsh climates, the operator often cannot directly interact with the system effectively. In these scenarios, the operator's hands may already be occupied with other tasks or in a situation where it is inconvenient to use traditional devices, and the introduction of a gesture control system is particularly necessary, enabling the operator to control the unmanned aerial vehicle only through simple and intuitive gesture actions, improving the task response speed.

[0004] However, gesture control systems face challenges in complex environments and diverse operation requirements in practical applications. In these environments, gesture recognition systems must have high precision, strong adaptability, and high robustness to ensure that the operator's instructions can still be effectively recognized under various complex conditions, and timely adjust flight control parameters such as the flight path and formation of the unmanned aerial vehicle. Therefore, how to design an unmanned aerial vehicle formation system that can cope with environmental changes and adjust control instructions in a timely manner according to the operator's gestures is an important topic in current technical research. Summary of the Invention

[0005] The purpose of the present invention is to provide an adaptive gesture control unmanned aerial vehicle cluster system, which has the advantages of efficiently and reliably completing gesture recognition in complex environments.

[0006] The above technical purpose of the present invention is achieved through the following technical solutions:

[0007] An adaptive gesture control unmanned aerial vehicle cluster system, comprising: an unmanned aerial vehicle formation unit and a collaborative working communication unit,

[0008] The UAV formation unit is a formation composed of multiple UAVs with independent flight control capabilities. Each UAV includes: an environmental parameter acquisition module, a gesture recognition module, a data fusion module, and a flight control module.

[0009] The environmental parameter acquisition module includes multiple sensors for real-time acquisition of multi-dimensional environmental data of the environment where the UAV is located, including but not limited to: smoke concentration, temperature, humidity, and light intensity.

[0010] The gesture recognition module works in cooperation with the environmental parameter acquisition module. Based on the real-time acquired multi-dimensional environmental data, it selects and enables sensors to collect gesture data for gesture instructions, and obtains a gesture recognition result according to the gesture data.

[0011] The data fusion module is used to calculate the sensing recognition confidence of each UAV under the multi-dimensional environmental data at its location, and perform multi-UAV data fusion based on the gesture recognition result and sensing recognition confidence of each UAV to obtain an optimal control instruction.

[0012] The flight control module adjusts the flight path, formation, spacing, and speed parameters of each UAV in the formation based on the optimal control instruction generated by the data fusion module to execute the gesture instruction.

[0013] The cooperative working communication unit is used for wireless communication and data transmission between multiple UAVs to ensure that the UAVs in the UAV formation unit share multi-dimensional environmental parameters, gesture data, and optimal control instructions.

[0014] Further setting: The step of selecting and enabling sensors to collect gesture data for gesture instructions based on the real-time acquired multi-dimensional environmental data specifically includes the following steps:

[0015] Calculate the adaptation functions of the light intensity, smoke concentration, temperature, and humidity in the multi-dimensional environmental data respectively.

[0016] Calculate the adaptation weights corresponding to each sensor according to the adaptation functions of the multi-dimensional environmental data.

[0017] Select the sensor with the highest adaptation weight as the enabled sensor to collect gesture data for gesture instructions according to the adaptation weights of each sensor.

[0018] Further setting: The data fusion module is used to calculate the sensing recognition confidence of each UAV under the multi-dimensional environmental data at its location specifically including:

[0019] Obtain the working condition data of the sensor according to the enabled sensor of the UAV.

[0020] Calculate the sensing recognition confidence of the UAV at its current location according to the sensor adaptation function and the operating conditions data of the sensor.

[0021] Further set: The multi-aircraft data fusion based on the gesture recognition results and sensing recognition confidence of each UAV to obtain the optimal control instruction specifically includes:

[0022] Each UAV calculates the gesture recognition confidence according to the enabled sensors and the obtained gesture recognition results;

[0023] Calculate the comprehensive recognition confidence of the UAV according to the gesture recognition confidence and sensing recognition confidence, and the comprehensive recognition confidence reflects the reliability of the gesture recognition of each UAV;

[0024] Perform data fusion on the comprehensive recognition confidence of each UAV to obtain the final recognition confidence;

[0025] Combine the final recognition confidence of all UAVs with the corresponding gesture recognition results, and select the gesture recognition result with the highest final recognition confidence as the optimal control instruction.

[0026] Further set: It also includes a dynamic adaptation module. The dynamic adaptation module obtains the gesture recognition result of the optimal control instruction and its corresponding final recognition confidence, judges whether the final recognition confidence is lower than the preset threshold. If it is less than the preset threshold, generate an adjustment instruction to the flight control module to adjust the position of the UAV to re-collect the gesture data, and perform fusion on the re-collected gesture data to obtain a new final recognition confidence and obtain the optimal control instruction according to the new final recognition confidence; if it is greater than or equal to the preset threshold, execute the optimal control instruction.

[0027] Further set: The generation of an adjustment instruction to the flight control module to adjust the position of the UAV to re-collect the gesture data specifically includes the following steps:

[0028] Calculate the supplementary acquisition area according to the positions of all UAVs in the current UAV formation unit, the position of the operator, and the comprehensive recognition confidence of the UAV;

[0029] Screen out several UAVs with the highest comprehensive confidence among all UAVs;

[0030] Generate an adjustment instruction according to the selected several UAVs and the supplementary acquisition area to control the UAV to the supplementary acquisition area to re-collect the gesture data through the gesture recognition module and the environmental parameter acquisition module.

[0031] Further set: The calculation of the supplementary acquisition area according to the positions of all UAVs in the current UAV formation unit, the position of the operator, and the comprehensive recognition confidence of the UAV specifically includes:

[0032] Calculate whether there is an uncovered area based on the positions of all drones and the position of the operator. The uncovered area represents the area that the sensor detection range of the drone fails to cover relative to the operator.

[0033] If there is an uncovered area, use the uncovered area as a supplementary acquisition area.

[0034] If there is no uncovered area, screen according to the comprehensive recognition confidence of the drones, and select the area covered by the drone with the lowest comprehensive recognition confidence as the supplementary acquisition area.

[0035] Another object of the present invention is to provide an adaptive gesture control method for a drone swarm, which has the advantages of efficiently and reliably completing gesture recognition in a complex environment.

[0036] The above technical object of the present invention is achieved through the following technical solutions:

[0037] An adaptive gesture control method for a drone swarm, which is applied to an adaptive gesture control drone swarm system described above, specifically includes the following steps:

[0038] Obtain multi-dimensional environmental data at the position of each drone. The multi-dimensional environmental data includes but is not limited to: smoke concentration, temperature, humidity, light intensity.

[0039] Based on the multi-dimensional environmental data, calculate the adaptation function of each sensor, and calculate the adaptation weight of each sensor according to the adaptation function.

[0040] Select the sensor with the highest adaptation weight as the enabled sensor according to the adaptation weight, collect gesture data and identify the gesture recognition result.

[0041] Calculate the comprehensive recognition confidence of each drone based on the gesture recognition result of each drone and its corresponding sensing recognition confidence.

[0042] Perform data fusion on the comprehensive recognition confidences of all drones to obtain the final recognition confidence.

[0043] Combine the final recognition confidences of all drones with the corresponding gesture recognition results, and select the gesture recognition result with the highest final recognition confidence as the optimal control instruction.

[0044] Judge whether the final recognition confidence is lower than a preset threshold. If it is lower than the preset threshold, generate an adjustment instruction to the flight control module to adjust the position of the drone for supplementary acquisition until the final recognition confidence reaches the preset threshold.

[0045] If the final recognition confidence reaches or exceeds the preset threshold, the optimal control instruction is executed to adjust parameters such as the flight path, formation, spacing, and speed of the UAV formation.

[0046] In summary, the present invention has the following beneficial effects:

[0047] This system selects and enables the most suitable sensors based on multi-dimensional environmental data collected in real time (such as light intensity, smoke concentration, temperature, humidity, etc.). In this way, the system can automatically select the best sensors according to environmental changes, improving the accuracy and reliability of gesture recognition. This adaptive sensor selection mechanism effectively solves the problem in traditional systems where the types of sensors are fixed and cannot cope with complex environmental changes, ensuring gesture recognition effects under various environmental conditions.

[0048] By fusing the gesture recognition results and sensing recognition confidence of each UAV, this system can achieve higher recognition accuracy under the condition of multi-UAV cooperation. The recognition results of each UAV are weighted according to their confidence weights to ensure the accuracy and effectiveness of the final instruction. The multi-UAV data fusion method improves the overall robustness of the system. Even when the recognition performance of some UAVs is low, the data of other UAVs with high confidence can be used for supplementation to ensure the successful execution of the gesture recognition task.

[0049] When the recognition confidence is lower than the preset threshold, the system can automatically allocate the UAV with the lowest recognition confidence to fly to the uncovered area for supplementary collection of gesture data. This adaptive adjustment mechanism can effectively solve the problem of insufficient recognition accuracy caused by local environmental factors or sensor problems, ensuring the smooth progress of the task. The dynamic task allocation and supplementary data collection functions greatly improve the adaptability and flexibility of the system in complex scenarios, and are especially suitable for complex rescue, monitoring and other tasks, ensuring the reliability and accuracy of operations.

[0050] In the selection of the final gesture recognition result, the system selects according to the fused final recognition confidence of each UAV to ensure the generation of the optimal control instruction. This can avoid task failures caused by incorrect recognition or errors of a single UAV, improve the control accuracy, and through weighted fusion of gesture recognition confidence and sensor adaptability, the final gesture instruction can accurately guide the UAV to perform complex cooperative flight tasks. Brief Description of the Drawings

[0051] Figure 1 is the overall structural block diagram of the embodiment. Detailed Embodiment

[0052] The present invention will be further described in detail below with reference to the accompanying drawings.

[0053] Embodiment:

[0054] As shown Figure 1 in the figure, an adaptive gesture control UAV cluster system includes: a UAV formation unit and a collaborative communication unit,

[0055] The UAV formation unit is a formation composed of multiple UAVs with independent flight control capabilities. Each UAV includes: an environmental parameter acquisition module, a gesture recognition module, a data fusion module, and a flight control module.

[0056] The environmental parameter acquisition module includes multiple sensors for real-time acquisition of multi-dimensional environmental data of the environment where the UAV is located, including but not limited to: smoke concentration, temperature, humidity, light intensity, atmospheric pressure, and noise;

[0057] The gesture recognition module works in cooperation with the environmental parameter acquisition module. Based on the real-time acquired multi-dimensional environmental data, it selects and enables sensors to collect gesture data for gesture commands;

[0058] The data fusion module is used to calculate the recognition confidence of each UAV under the multi-dimensional environmental data at its location, and perform multi-UAV data fusion based on the gesture data and recognition confidence of each UAV to obtain an optimal control command;

[0059] The flight control module adjusts the flight path, formation, spacing, and speed parameters of each UAV in the formation based on the optimal control command generated by the data fusion module to execute the gesture command;

[0060] The collaborative communication unit is used for wireless communication and data transmission between multiple UAVs to ensure that the UAVs in the UAV formation unit share multi-dimensional environmental parameters, gesture data, and optimal control commands.

[0061] In practical applications, different environmental parameters (such as light intensity, smoke concentration, temperature, and humidity, etc.) will affect the working efficiency of each sensor. The system needs to calculate the adaptation weight of each sensor according to these real-time environmental parameters and select the appropriate sensor based on the adaptation weight. Therefore, it specifically includes the following steps:

[0062] Calculate the adaptation functions of light intensity, smoke concentration, temperature, and humidity in the multi-dimensional environmental data respectively;

[0063] Calculate the adaptation weight corresponding to each sensor according to the adaptation functions of the multi-dimensional environmental data;

[0064] Select the sensor with the highest adaptation weight as the enabled sensor to collect gesture data for gesture commands according to the adaptation weight of each sensor.

[0065] The following is the specific process:

[0066] First, the system collects the current environmental data through the environmental parameter acquisition module, including the following key environmental parameters:

[0067] Light intensity: Measures the lighting conditions of the environment and affects the effectiveness of the image sensor; Smoke concentration: Measures the concentration of smoke in the air and affects the effectiveness of the image and infrared sensors; Temperature: The environmental temperature, which affects the performance of the infrared sensor; Humidity: The environmental humidity, which affects the adaptability of the sensor.

[0068] For each sensor, the system calculates its adaptability weight based on the environmental parameters. The adaptability weight value is used to measure the effectiveness of the sensor in the current environment. The specific calculation formula is as follows:

[0069] W sensor = α 1 · f light (I light ) + α 2 · f smoke (C smoke ) + α 3 · f temp (T env ) + α 4 · f humidity (H env )

[0070] Where W sensor is the adaptability weight of each sensor, and f() is the function for calculating the adaptability weight. To make the calculation of the adaptability weight flexible, α 1 , α 2 , α 3 , α 4 are the weight coefficients used to adjust the influence of different environmental parameters on the adaptability weight.

[0071] To improve the flexibility of the solution, the adaptability function of each sensor should be defined as follows according to different environmental conditions:

[0072] Light sensor adaptability function: This function can be a linear or non-linear function, reflecting the relationship between the light intensity and the adaptability of the image sensor. For example, the stronger the light intensity, the better the adaptability of the image sensor.

[0073] f ligh t (I ligh t ) = sigmoid(I light - I threshold )

[0074] Where I threshold is the threshold of light, and sigmoidO is the standard Sigmoid function that maps the light intensity to the adaptability value.

[0075] Smoke concentration sensor adaptation function: For high smoke concentrations, the adaptability of image sensors and infrared sensors decreases, while millimeter-wave radar sensors are not affected by smoke. Therefore, their adaptability increases as the smoke concentration increases.

[0076]

[0077] Among them, f smoke (C smoke ) is the influence function of smoke concentration on sensor adaptability, k smoke is the sensitivity coefficient of smoke concentration, C threshold is the threshold of smoke concentration, C smoke is the smoke concentration.

[0078] Temperature sensor adaptation function: When the temperature is high, the effectiveness of the infrared sensor is enhanced. Therefore, this function can be expressed as the adaptability of the infrared sensor gradually increasing as the temperature increases.

[0079]

[0080] Among them, f temp (T env ) is the influence function of temperature on sensor fitness, T env is the ambient temperature, T threshold is the reference value of temperature, which is used to adjust the adaptability of the sensor to temperature changes.

[0081] Humidity sensor adaptation function: When the humidity is high, the adaptability of the millimeter-wave radar is good. Therefore, this function increases as the humidity increases.

[0082]

[0083] Among them, f humidity (H env ) is the influence function of humidity on sensor fitness, H env is the ambient humidity, H threshold is the threshold of humidity, k humidity is the humidity sensitivity coefficient, which is used to control the influence of humidity changes on sensitivity.

[0084] Finally, the system calculates the adaptation weights of each sensor and selects the sensor with the highest adaptability to collect gesture data for gesture commands.

[0085] The data fusion module is used to calculate the sensing recognition confidence of each drone under multi-dimensional environmental data at its location, specifically including:

[0086] Obtain the working condition data of the sensor according to the sensors enabled by the drone;

[0087] Calculate the sensing recognition confidence of the UAV at its current location according to the sensor adaptation function and the operating condition data of the sensor.

[0088] The operating condition data reflects the state and working conditions of the activated sensor when collecting gesture data. By combining the operating condition data and the sensor adaptation function with the multi-dimensional environmental data, the sensing recognition confidence of the UAV at the current location can be calculated. The sensing recognition confidence characterizes the reliability of the activated sensor of the UAV when working at the current location.

[0089] The multi-aircraft data fusion based on the gesture recognition results and the sensing recognition confidence of each UAV to obtain the optimal control instruction specifically includes:

[0090] Each UAV calculates the gesture recognition confidence according to the activated sensor and the obtained gesture recognition result.

[0091] Calculate the comprehensive recognition confidence of the UAV according to the gesture recognition confidence and the sensing recognition confidence. The comprehensive recognition confidence reflects the reliability of the gesture recognition of each UAV.

[0092] Perform data fusion on the comprehensive recognition confidence of each UAV to obtain the final recognition confidence.

[0093] Combine the final recognition confidence of all UAVs with the corresponding gesture recognition results, and select the gesture recognition result with the highest final recognition confidence as the optimal control instruction.

[0094] When each UAV performs a gesture recognition task, it will use its activated sensor to obtain environmental data and recognize gestures. Each sensor calculates a gesture recognition confidence according to its recognition result. For example, the image sensor may output the confidence of the "wave" gesture based on the image recognition algorithm; the infrared sensor may output the confidence of "fist clenching" based on the heat source contour; the millimeter-wave radar may output a confidence related to the "open hand" gesture through point cloud data. Since different sensors have different recognition accuracies for gestures in different postures, such as the infrared sensor has a higher accuracy in recognizing "fist clenching" than in recognizing a more delicate gesture like "open three fingers", the gesture recognition confidence characterizes the reliability of the gesture recognition result recognized by the UAV under the current activated sensor conditions. The data fusion module will calculate the comprehensive recognition confidence of each UAV, which is a weighted average based on the sensor adaptation and the confidence of the gesture recognition result of the UAV.

[0095] Next, the data fusion module weights and fuses the recognition confidence levels of all the UAVs participating in the formation to generate a global final recognition confidence level, which is used to determine the optimal gesture command. The fused final recognition confidence level reflects the reliability of gesture recognition for each UAV. The data fusion module combines the final recognition confidence levels of all UAVs with the corresponding gesture classification results and finally selects an optimal gesture command. After calculating the fusion of the recognition confidence level of each UAV and the gesture recognition result, the system selects the gesture with the highest recognition confidence level as the finally executed gesture. Once the optimal gesture command is determined, the system transmits this gesture command as a control command to all UAVs to ensure that the UAVs complete the task according to this gesture.

[0096] In addition to the UAV formation unit and the cooperative work communication unit, this system also includes a dynamic adaptation module. The dynamic adaptation module obtains the gesture recognition result of the optimal control command and its corresponding final recognition confidence level, and judges whether the final recognition confidence level is lower than a preset threshold. If it is less than the preset threshold, it generates an adjustment command to the flight control module to adjust the positions of the UAVs to re-collect the gesture data, and performs fusion based on the re-collected gesture data to obtain a new final recognition confidence level and obtains an optimal control command based on the new final recognition confidence level; if it is greater than or equal to the preset threshold, it executes this optimal control command.

[0097] The step of generating an adjustment command to the flight control module to adjust the positions of the UAVs to re-collect the gesture data specifically includes the following steps:

[0098] Calculate a supplementary acquisition area based on the positions of all UAVs in the current UAV formation unit, the position of the operator, and the comprehensive recognition confidence level of the UAVs;

[0099] Select several UAVs with the highest comprehensive confidence levels among all UAVs;

[0100] Generate an adjustment command based on the selected several UAVs and the supplementary acquisition area to control the UAVs to the supplementary acquisition area to re-collect the gesture data through the gesture recognition module and the environmental parameter acquisition module.

[0101] The step of calculating a supplementary acquisition area based on the positions of all UAVs in the current UAV formation unit, the position of the operator, and the comprehensive recognition confidence level of the UAVs specifically includes:

[0102] Calculate whether there is an uncovered area based on the positions of all UAVs and the position of the operator. The uncovered area represents the area that the sensor detection range of the UAVs fails to cover with respect to the operator;

[0103] If there is an uncovered area, use the uncovered area as the supplementary acquisition area;

[0104] If there is no uncovered area, screening is performed according to the comprehensive recognition confidence of the drones, and the area covered by the drone with the lowest comprehensive recognition confidence is selected as the supplementary acquisition area.

[0105] When the drone formation is performing tasks, due to the complexity of the environment or the limitations of some drone sensors, gesture recognition in some areas may result in a recognition confidence lower than the preset threshold. At this time, the system needs to intelligently dispatch other drones for data collection and supplementation until the gesture data in all areas is fully collected and the final recognition confidence reaches the preset requirements.

[0106] The system first determines the uncovered areas, which are usually determined by the positions of each drone and the operator. Due to the limited coverage range of the sensors and the limited number of drones in the drone formation unit, it is impossible to ensure coverage in all 360-degree directions of the operator when the operator executes gesture commands. Therefore, first calculate which areas around the operator are not covered. For example, a drone formation in an arc array in front of the operator can cover approximately 180 degrees in front of the operator. Therefore, the area behind the operator belongs to the uncovered area, and this part of the area is used as the supplementary acquisition area.

[0107] If the entire 360-degree range around the operator is within the coverage of the drones, but the final recognition confidence obtained is still lower than the preset threshold, then it is necessary to screen out the area range recognized by the drone with the lowest comprehensive recognition confidence, discard the data recognized by this drone at this position, swap the position of the drone with the highest comprehensive confidence in the drones with this drone, and let the drone with the highest comprehensive confidence perform recognition again to obtain gesture data, recognize the gesture recognition result, and participate in data fusion to obtain a new final recognition confidence so as to determine the optimal control command.

[0108] An adaptive gesture control method for a drone swarm, applied to an adaptive gesture control drone swarm system described above, specifically includes the following steps:

[0109] Obtain multi-dimensional environmental data at the position of each drone, and the multi-dimensional environmental data includes but is not limited to: smoke concentration, temperature, humidity, light intensity;

[0110] Calculate the adaptation function of each sensor based on the multi-dimensional environmental data, and calculate the adaptation weight of each sensor according to the adaptation function;

[0111] Select the sensor with the highest adaptation weight as the enabled sensor according to the adaptation weight, collect gesture data and recognize the gesture recognition result;

[0112] Calculate the gesture recognition result and its corresponding sensing recognition confidence for each drone to obtain the comprehensive recognition confidence for each drone;

[0113] Perform data fusion on the comprehensive recognition confidences of all drones to obtain the final recognition confidence;

[0114] Combine the final recognition confidence of all drones with the corresponding gesture recognition results, and select the gesture recognition result with the highest final recognition confidence as the optimal control instruction;

[0115] Judge whether the final recognition confidence is lower than the preset threshold. If it is lower than the preset threshold, generate an adjustment instruction to the flight control module to adjust the position of the drone for supplementary acquisition until the final recognition confidence reaches the preset threshold;

[0116] If the final recognition confidence reaches or is greater than the preset threshold, execute the optimal control instruction to adjust parameters such as the flight path, formation, spacing, and speed of the drone formation.

[0117] In summary, the present invention has the following beneficial effects:

[0118] This system selects and enables the most suitable sensors through multi-dimensional environmental data (such as light intensity, smoke concentration, temperature, humidity, etc.) collected in real time. In this way, the system can automatically select the best sensors according to environmental changes, improving the accuracy and reliability of gesture recognition. This adaptive sensor selection mechanism effectively solves the problem in traditional systems where the types of sensors are fixed and cannot cope with complex environmental changes, ensuring the gesture recognition effect under various environmental conditions.

[0119] By fusing the gesture recognition results and sensing recognition confidences of each drone, this system can achieve higher recognition accuracy under the condition of multi-aircraft collaboration. The recognition results of each drone are weighted according to their confidence weights to ensure the accuracy and effectiveness of the final instruction. The multi-aircraft data fusion method improves the robustness of the overall system. Even when the recognition performance of some drones is low, it can be supplemented by the data of other high-confidence drones to ensure the successful execution of the gesture recognition task.

[0120] When the recognition confidence is lower than the preset threshold, the system can automatically allocate the drone with the lowest recognition confidence to go to the uncovered area for supplementary acquisition of gesture data. This adaptive adjustment mechanism can effectively solve the problem of insufficient recognition accuracy caused by local environmental factors or sensor problems, ensuring the smooth progress of the task. The dynamic task allocation and supplementary data acquisition functions greatly improve the adaptability and flexibility of the system in complex scenarios, especially suitable for complex rescue, monitoring and other tasks, ensuring the reliability and accuracy of operations.

[0121] In the selection of the final gesture recognition result, the system makes a choice based on the final recognition confidence after fusion for each drone to ensure the generation of optimal control instructions. This can avoid mission failures caused by incorrect recognition or errors of individual drones, improve the control accuracy, and through weighted fusion of gesture recognition confidence and sensor adaptability, the final gesture instructions can accurately guide the drones to perform complex cooperative flight tasks.

[0122] The embodiments described above do not constitute a limitation on the protection scope of the technical solution. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the above embodiments shall be included within the protection scope of the technical solution.

Claims

1. An adaptive gesture-controlled drone cluster system, characterized in that: include: UAV formation unit and collaborative communication unit, The UAV formation unit is a formation composed of multiple UAVs with independent flight control capabilities, wherein each UAV includes: an environmental parameter acquisition module, a gesture recognition module, a data fusion module, and a flight control module. The environmental parameter acquisition module includes multiple sensors for real-time acquisition of multi-dimensional environmental data of the environment in which the drone is located, including but not limited to: smoke concentration, temperature, humidity, and light intensity; The gesture recognition module works in coordination with the environmental parameter acquisition module, selects and enables sensors to collect data of gesture instructions based on multi-dimensional environmental data acquired in real time to obtain gesture data, and obtains gesture recognition results based on gesture data recognition; The data fusion module is used to calculate the sensor recognition confidence of each drone under the multi-dimensional environmental data of its location, and to perform multi-machine data fusion according to the gesture recognition results of each drone and its sensor recognition confidence to obtain the optimal control instruction; The flight control module adjusts the flight path, formation, spacing, and speed parameters of each UAV in the formation based on the optimal control instructions generated by the data fusion module to execute gesture instructions; The collaborative communication unit is used for wireless communication and data transmission between multiple drones, ensuring that the drones in the drone formation unit share multi-dimensional environmental parameters, gesture data, and optimal control instructions.

2. The adaptive gesture-controlled drone cluster system according to claim 1, characterized in that: The method of selecting and enabling sensors to collect data of gesture instructions based on multi-dimensional environmental data collected in real time to obtain gesture data specifically includes the following steps: The adaptability function is calculated for the light intensity, smoke concentration, temperature and humidity in the multi-dimensional environmental data respectively; Calculate the adaptability weight corresponding to each sensor according to the adaptability function of multi-dimensional environmental data; According to the adaptability weight of each sensor, a sensor with the highest adaptability weight is selected as an enabled sensor to collect data of gesture instructions to obtain gesture data.

3. The adaptive gesture-controlled drone cluster system according to claim 2, characterized in that: The data fusion module is used to calculate the sensor recognition confidence of each UAV under the multi-dimensional environmental data at its location, specifically including: According to the sensors enabled by the drone, the working condition data of the sensors are obtained; The sensor recognition confidence of the UAV at its location is calculated based on the sensor adaptability function and the sensor's working condition data.

4. The adaptive gesture-controlled drone cluster system according to claim 3, characterized in that: The method of performing multi-machine data fusion according to the gesture recognition result of each drone and its sensor recognition confidence to obtain the optimal control instruction specifically includes: Each drone calculates the gesture recognition confidence based on the enabled sensors and the gesture recognition results obtained; The comprehensive recognition confidence of the UAV is calculated based on the gesture recognition confidence and the sensor recognition confidence, and the comprehensive recognition confidence reflects the reliability of gesture recognition of each UAV; Perform data fusion on the comprehensive recognition confidence of each UAV to obtain the final recognition confidence; The final recognition confidences of all drones are combined with the corresponding gesture recognition results, and the gesture recognition result with the highest final recognition confidence is selected as the optimal control instruction.

5. The adaptive gesture-controlled drone cluster system according to claim 4, characterized in that: It also includes a dynamic adaptation module, which obtains the gesture recognition result of the optimal control instruction and its corresponding final recognition confidence, and determines whether the final recognition confidence is lower than a preset threshold. If it is lower than the preset threshold, an adjustment instruction is generated to the flight control module to adjust the position of the drone to re-collect the gesture data, and a new final recognition confidence is obtained by fusing the re-collected gesture data and obtaining the optimal control instruction according to the new final recognition confidence; if it is greater than or equal to the preset threshold, the optimal control instruction is executed.

6. The adaptive gesture-controlled drone cluster system according to claim 5, characterized in that: The generating of the adjustment instruction to the flight control module to adjust the position of the drone to re-collect the gesture data specifically includes the following steps: The supplementary collection area is calculated based on the positions of all drones in the current drone formation unit, the position of the operator and the comprehensive recognition confidence of the drones; Filter out several drones with the highest comprehensive confidence among all drones; According to the selected UAVs and the supplementary collection area, an adjustment instruction is generated to control the UAV to the supplementary collection area, and the gesture data is supplementarily collected through the gesture recognition module and the environmental parameter collection module.

7. The adaptive gesture-controlled drone cluster system according to claim 6, characterized in that: The method of calculating the supplementary acquisition area according to the positions of all drones in the current drone formation unit, the position of the operator and the comprehensive recognition confidence of the drones specifically includes: Calculate whether there is an uncovered area based on the positions of all drones and the position of the operator, where the uncovered area is the area that is not covered by the sensor detection range of the drone relative to the operator; If there is an uncovered area, the uncovered area will be used as a supplementary collection area; If there is no uncovered area, it will be screened according to the comprehensive recognition confidence of the drone, and the area covered by the drone with the lowest comprehensive recognition confidence will be selected as the supplementary collection area.

8. An adaptive gesture-controlled drone cluster method, applied to an adaptive gesture-controlled drone cluster system according to any one of claims 1 to 7, characterized in that: The specific steps include: Obtain multi-dimensional environmental data of each drone's location, the multi-dimensional environmental data including but not limited to: smoke concentration, temperature, humidity, and light intensity; Calculating the adaptability function of each sensor based on the multi-dimensional environmental data, and calculating the adaptability weight of each sensor according to the adaptability function; According to the adaptability weight, a sensor with the highest adaptability weight is selected as an enabled sensor to collect gesture data and identify gesture recognition results; The gesture recognition results of each UAV and its corresponding sensor recognition confidence are calculated to obtain the comprehensive recognition confidence of each UAV; Perform data fusion on the comprehensive recognition confidence of all UAVs to obtain the final recognition confidence; According to the final recognition confidence of all drones and the corresponding gesture recognition results, the gesture recognition result with the highest final recognition confidence is selected as the optimal control instruction; Determine whether the final recognition confidence is lower than a preset threshold. If it is lower than the preset threshold, generate an adjustment instruction to the flight control module to adjust the position of the drone for additional collection until the final recognition confidence reaches the preset threshold; If the final recognition confidence reaches or exceeds the preset threshold, the optimal control instructions are executed to adjust the flight path, formation, spacing, speed and other parameters of the UAV formation.

Images