Unmanned aerial vehicle cooperative control method and system based on intelligent perception

Abstract

The invention discloses an unmanned aerial vehicle cooperative control method and system based on intelligent perception, and belongs to the technical field of cooperative control. The system comprises an intelligent sensing module, an object analysis module, a collaborative planning module and a driving execution module. The intelligent sensing module collects image and position data through each unmanned aerial vehicle, and receives instruction parameters of the command center; the object analysis module dynamically selects and matches reference objects for the offline unmanned aerial vehicles according to the instruction parameters, establishes queues, establishes a following relationship through multicolor light flicker signals, and dynamically adjusts the number of the unmanned aerial vehicles in each queue and the following relationship in the flight process; the collaborative planning module is used for analyzing the obstacle distribution condition of the target direction space and planning a plurality of independent flight channels; calculating matching indexes between the queues and the independent flight channels, and planning distributed collaborative airways for the queues according to the matching indexes; and the driving execution module is used for driving the unmanned aerial vehicle queues to enter a planned air route in order to fly, and the unmanned aerial vehicles cooperatively execute inspection tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cooperative control, in particular to a method and system for cooperative control of unmanned aerial vehicles based on intelligent sensing. BACKGROUND

[0002] In recent years, with the rapid development of unmanned aerial vehicle technology, the application demand of unmanned aerial vehicles in closed or complex indoor environments has significantly increased, especially in the field of cooperative inspection tasks of unmanned aerial vehicle clusters. Such tasks require unmanned aerial vehicle clusters to efficiently cooperatively complete autonomous path planning, state sensing capability construction, and environmental monitoring tasks in spaces with dense obstacle distribution, GPS signal absence or limitation, while ensuring the safety and reliability of the operation process.

[0003] Existing unmanned aerial vehicle cluster cooperative control technologies for indoor scenarios still have significant deficiencies. First, the system highly depends on continuous and stable communication connection with the command center. Once a single machine is disconnected from the command center, the unmanned aerial vehicle not only cannot perceive the real-time state of surrounding unmanned aerial vehicles, but also loses effective decision-making basis, which easily leads to loss of control or task interruption, seriously affecting the overall coordination and task robustness. Second, in the disconnected state, existing technologies lack reliable mechanisms for unmanned aerial vehicles to autonomously find suitable neighboring unmanned aerial vehicles and efficiently form cooperative queues, and there is also a lack of real-time interaction means for dynamic management of queues through non-radio frequency communication. Finally, in terms of path planning, traditional methods are difficult to efficiently convert three-dimensional obstacle distribution information of the target space into multiple independent obstacle-free safe channels suitable for ordered flight of queues, and to match different queues with suitable channels, resulting in uneven resource allocation, increased collision risk, and low overall efficiency. These defects collectively limit the effectiveness and reliability of unmanned aerial vehicle clusters in cooperative operation in complex indoor environments. SUMMARY

[0004] The purpose of the present application is to provide a method and system for cooperative control of unmanned aerial vehicles based on intelligent sensing to solve the problems raised in the background.

[0005] To solve the above technical problems, the present application provides a method for cooperative control of unmanned aerial vehicles based on intelligent sensing, comprising:

[0006] S100, when the unmanned aerial vehicle cluster is flying and operating indoors, each unmanned aerial vehicle collects image and position data through an onboard sensor and receives instruction parameters from the command center, and executes the inspection task according to the instruction parameters.

[0007] The image refers to the real scene image collected by the unmanned aerial vehicle through the onboard depth camera. The position data includes the real-time spatial position of the unmanned aerial vehicle itself.

[0008] The instruction parameters include control instructions, communication states and spatial positions of each unmanned aerial vehicle in the unmanned aerial vehicle group. The communication state refers to the link state between the unmanned aerial vehicle and the command center, and specifically includes a networking state and a network interruption state.

[0009] Each unmanned aerial vehicle reports its spatial position to the command center in real time, and the command center aggregates data and then transmits the control instructions of each unmanned aerial vehicle, the communication state and the spatial position of the unmanned aerial vehicle group to each unmanned aerial vehicle again, so as to realize real-time sensing of the communication and position between the unmanned aerial vehicles.

[0010] Each unmanned aerial vehicle monitors the communication state change of the unmanned aerial vehicle group and the spatial position distribution of other unmanned aerial vehicles in real time according to the instruction parameters. The command center is used to send instruction parameters to each unmanned aerial vehicle, so as to control the unmanned aerial vehicle group to perform the inspection task.

[0011] Through real-time data sharing and state sensing, the fault tolerance and cooperative reliability of the unmanned aerial vehicle group in a complex indoor environment are enhanced, the risk of single point failure caused by network interruption is reduced, and the overall execution efficiency and coordination ability of the inspection task are improved.

[0012] S200, dynamically selecting a reference object for the network interruption unmanned aerial vehicle according to the instruction parameters and establishing a queue, establishing a following relationship through multi-color light flashing signals, and dynamically adjusting the number of unmanned aerial vehicles in each queue and the following relationship in the flight process. Specifically, it includes:

[0013] S201, when the unmanned aerial vehicle UAV1 is interrupted, the latest communication state and spatial position of each unmanned aerial vehicle in the instruction parameters before the network interruption are obtained, and other unmanned aerial vehicles whose spatial position is less than a threshold distance from the spatial position of the unmanned aerial vehicle UAV1 are marked.

[0014] When the unmanned aerial vehicle is just interrupted, the latest data in the instruction parameters is used as the reference.

[0015] When the network interruption time of the unmanned aerial vehicle exceeds a set threshold, the communication state and spatial position change of other unmanned aerial vehicles in the visual range are analyzed in real time through an image recognition algorithm, and the latest image analysis result is used as the reference.

[0016] The communication state can be visually displayed through an indicator light on the unmanned aerial vehicle. Generally, a red light indicates a network interruption state, and a green light indicates a networking state.

[0017] S202, flight trajectories of each marked unmanned aerial vehicle are drawn through historical spatial positions of the marked unmanned aerial vehicles, a coordination coefficient is calculated according to the latest communication state of each marked unmanned aerial vehicle and the flight trajectory, and a marked unmanned aerial vehicle with the maximum coordination coefficient is selected as a reference object. Specifically, it includes:

[0018] S2021, the historical instruction parameters saved by the unmanned aerial vehicle UAV1 are obtained, the latest communication state of each marked unmanned aerial vehicle in the last time T recentThe spatial position of the inner UAV UAV1 and each marker UAV changes, thereby drawing a flight trajectory.

[0019] S2022, Q time points are evenly set on each flight trajectory, and the spatial distance between the UAV UAV1 and each marker UAV UAV is calculated. mark The Euclidean distance at the same time point and the standard deviation of the Euclidean distance at all time points are used as the difference coefficient.

[0020] S2023, after analyzing the latest communication state of each marker UAV and excluding the marker UAVs that do not belong to any team and are in a network outage state, the number of UAVs in the team to which each marker UAV in a network outage state belongs is obtained.

[0021] S2024, a state index N is set and takes a or b, N=a when the marker UAV is in a networked state, and N=b when the marker UAV is in a network outage state. Wherein, a>b. The coordination coefficient of each marker UAV is calculated by substituting the formula:

[0022]

[0023] In the formula, CC i is the coordination coefficient of the ith marker UAV, d i is the spatial distance between the ith marker UAV and the UAV UAV1, d max is the maximum spatial distance between all marker UAVs and the UAV UAV1.

[0024] γ and τ are constants, M i is the difference coefficient of the ith marker UAV, S i is the number of UAVs in the team to which the ith marker UAV belongs, S max is the maximum number of UAVs in the team to which all marker UAVs belong.

[0025] The coordination coefficient is used to quantify the cooperation suitability between the network outage UAV and the surrounding candidate reference object, and the optimal reference object is selected by dynamically calculating the coordination coefficient.

[0026] The formula integrates the spatial distance weight, flight trajectory consistency, communication state of the candidate object, and team size.

[0027] Finally, the most stable connection object is selected by maximizing the coordination coefficient, thereby ensuring the reliability and cooperation efficiency of team establishment in a network outage scenario.

[0028] S203, the UAV establishes a team with the reference object, or joins the team in which the reference object is located. The following relationship is established through the interaction of multi-color light flashing signals, and the external relationship and internal relationship of the team are dynamically adjusted during flight.

[0029] When the reference object is in a networked state, the UAV establishes a queue with the reference object. The UAV interacts with the reference object through multi-color light flashing signals to establish a following relationship, and flies along the path of the reference object through real-time image analysis.

[0030] When the reference object is in a networked state, the UAV joins the queue in which the reference object is located. The UAV interacts with the reference object through multi-color light flashing signals to establish a following relationship, and the UAVs in the queue interact in turn through multi-color light flashing signals to inform the new member of the joining.

[0031] The UAV flies along the path of the reference object through real-time image analysis, and the UAVs in the queue follow the path in turn according to the following relationship.

[0032] Dynamic adjustment of external relationship refers to:

[0033] When any UAV in the queue encounters a new marked UAV during flight and the new marked UAV is calculated as a reference object through a coordination coefficient, the corresponding UAV automatically leaves the original queue and establishes a new queue with the reference object or joins the queue in which the reference object is located.

[0034] Dynamic adjustment of internal relationship refers to:

[0035] Each UAV in the queue analyzes images in real time during flight, draws flight trajectories of other UAVs in the team, analyzes flight trajectories of two UAVs with a following relationship, and calculates a difference coefficient of the following UAV.

[0036] After data aggregation, the average value of the difference coefficient of each following UAV is calculated and used as a difference index. The following relationship of each UAV in the queue is dynamically adjusted in order from small to large according to the difference index.

[0037] Each UAV in the queue usually flies in a straight line, and each UAV only has a following relationship with one UAV in the queue. The following UAV analyzes the flight trajectory of the reference object in real time through image analysis, and flies synchronously at different times according to the same flight trajectory.

[0038] Each UAV calculates the difference coefficient of all following UAVs in front, and the number of following UAVs that can be calculated by the UAV in front is less. Data aggregation is visually transmitted through light signal coding in order according to the following relationship.

[0039] The autonomous cooperation ability of the UAV in the case of network disconnection is strengthened, the formation maintenance and redundancy mechanism are optimized through visual signals and dynamic queue management, the failure rate of the inspection task is reduced, and the overall stability is improved, especially in the link loss environment, the system adaptability and flexibility are enhanced.

[0040] S300, analyze the obstacle distribution of the target direction space, and plan multiple independent flight channels. Calculate the matching index between the queue and the independent flight channel, and plan the distributed cooperative route for each queue according to the matching index. Specifically, it includes:

[0041] S301, obtain the obstacle distribution image of the target direction space collected by the networked unmanned aerial vehicle in the unmanned aerial vehicle group, and perform feature extraction and timestamp synchronization processing on each view image.

[0042] S302, perform stereo matching calculation on the cross-view feature points through epipolar constraint, jointly solve the spatial three-dimensional coordinate values of the homonymous points based on the matching results, and generate a global dense point cloud model by fusing all view solving results.

[0043] S303, the edge detection algorithm identifies the contour boundary of the vertical obstacle, thereby dividing the target direction space into several parallel subspaces. The connectivity analysis algorithm is used to extract the continuous subspaces that meet the passability standard as independent flight channels.

[0044] S304, calculate the passing index of each independent flight channel and the priority index of each queue in the unmanned aerial vehicle group, multiply the priority index and the passing index to obtain the matching index between the queue and the independent flight channel.

[0045] Divide the independent flight channel into h cross sections according to the vertical flight direction.

[0046] Analyze the length of the longest and shortest line segments from the center point of each cross section to the edge line, and calculate the area standard deviation V sum of all cross sections, and substitute it into the formula to calculate the passing index TX:

[0047]

[0048] In the formula, AR e is the area of the e-th cross section, λ is a constant, and are the lengths of the longest and shortest line segments from the center point of the e-th cross section to the edge line.

[0049] Quantify the passability and safety of the independent flight channel through geometric and statistical features to provide channel quality indicators for queue matching. The core design includes three parts:

[0050] Cross section shape analysis: calculate the longest / shortest line segment ratio of each cross section center point to the edge, which reflects the width change of the channel. The larger the ratio, the stronger the penalty.

[0051] Area stability: the area standard deviation V sum of all cross sections measures the overall consistency of the channel. The greater the fluctuation, the lower the score.

[0052] Cross-section weight: taking each cross-section area AR e The weighted value is calculated in combination with the aforementioned proportional coefficient, taking each cross-section area AR

[0053] The priority index YZ is calculated according to the following formula:

[0054]

[0055] In the formula, p is the number of UAVs in the queue, CZ u is the difference index of the u-th UAV.

[0056] The overall priority index of the queue is calculated by accumulating the difference indexes of all UAVs in the queue, which is used for priority ranking when allocating flight channels.

[0057] The design logic is to quantify the stability of the queue as the difference index, which reflects the degree of trajectory deviation of each UAV tracking the reference object, so the smaller the sum YZ, the better the overall coordination of the queue members.

[0058] When allocating channels, the queue with a higher priority index has the priority to choose, ensuring that the low-stability queue has priority to pass through the optimal channel, thereby improving the efficiency of system resource allocation and reducing the probability of collision.

[0059] S305, select the independent flight channel with the largest matching index for the queue, and select the independent flight channel for each queue in order from large to small according to the priority index, thereby planning a distributed cooperative route.

[0060] There is no spatial overlap between different independent flight channels, and the route of the UAV is usually planned at the middle point of the flight channel.

[0061] Efficient and safe route planning is achieved, collision risk is reduced through quantitative obstacle avoidance and optimized resource allocation, task parallelism and response speed are improved, energy consumption is reduced, and high-precision cooperation of the UAV group in a limited space is supported.

[0062] S400, drive each UAV queue to enter the planned route in order, and each UAV cooperates to perform the inspection task.

[0063] The orderliness and safety of route execution are ensured, the task reliability and overall efficiency are improved through real-time sensing and dynamic adjustment mechanisms, the need for manual intervention is reduced, and the autonomous execution capability of the UAV group in a dynamic environment is enhanced.

[0064] The application also provides a UAV cooperative control system based on intelligent sensing, which includes an intelligent sensing module, an object analysis module, a cooperative planning module and a driving execution module.

[0065] The intelligent perception module collects image and position data through each UAV and receives instruction parameters from the command center.

[0066] The real-time real scene image and spatial position data are collected through the onboard depth camera of each UAV, and the instruction parameters sent by the command center are received, including control instructions, communication status of each UAV, and spatial position data of each UAV in the group.

[0067] The UAVs report these data to the command center in real time, and the command center aggregates the data and returns updated instruction parameters, so that each UAV can continuously monitor the communication status and position distribution changes of each other, thereby dynamically adjusting its own behavior.

[0068] Real-time state perception and data sharing of the UAV group are realized, and the reliability and fault tolerance capability of the system in complex indoor environments are enhanced.

[0069] By continuously updating the communication and position information, it is ensured that the UAV can respond to the task requirements even if a single point is disconnected, and the overall coordination and execution efficiency of the inspection task are improved.

[0070] The object analysis module dynamically selects reference objects for disconnected UAVs according to instruction parameters and establishes queues, establishes a following relationship through multi-color light flashing signals, and dynamically adjusts the number of UAVs in each queue and the following relationship during flight.

[0071] Based on historical instruction parameters or real-time image recognition, dynamic selection of reference objects is performed, and the flight trajectory of the marked UAV is drawn and the difference coefficient and coordination coefficient are calculated.

[0072] After selecting the optimal reference object, multi-color light flashing signals are used to interact to establish or join the queue, and the queue size and following relationship are dynamically adjusted through real-time image analysis during flight.

[0073] The autonomous collaboration capability of the UAV in the disconnected state is improved, and through visual signals and real-time trajectory analysis, dynamic maintenance and optimization of the queue are realized, enhancing the flexibility and redundancy of the system, thereby reducing the failure risk of the overall inspection task and improving the team stability.

[0074] The cooperative planning module is used to analyze the obstacle distribution of the target direction space and plan multiple independent flight channels. The matching index between the queue and the independent flight channel is calculated, and the distributed cooperative route is planned for each queue according to the matching index.

[0075] After collecting the target direction space image provided by the networked UAV, performing feature extraction and timestamp synchronization, and using the epipolar constraint stereo matching algorithm to generate a global dense point cloud model, the space is segmented into multiple obstacle-free independent flight channels through edge detection and connectivity analysis.

[0076] The passage index and the queue priority index are calculated, the matching index is combined through multiplication, the queue is dynamically distributed to the best channel, and the distributed cooperative route is planned to ensure that the channels are not overlapped.

[0077] The efficient and safe route planning is realized, the obstacle avoidance logic and the queue priority distribution are optimized, and the collision risk is reduced. Through the quantitative index matching, the resource utilization efficiency and the task parallelism are improved, so that the unmanned aerial vehicle is more rapid in response and has lower collision risk when cooperatively performing the inspection task in the limited space.

[0078] The driving execution module is used for driving each unmanned aerial vehicle queue to enter the planned route and fly, and each unmanned aerial vehicle cooperatively performs the inspection task.

[0079] Each unmanned aerial vehicle queue is driven to enter the independent flight channel in an orderly manner, and each unmanned aerial vehicle analyzes real-time images and adopts the flight trajectory of a reference object.

[0080] The orderliness and safety of route execution are ensured, real-time sensing adjustment is combined, the reliability and overall efficiency of the inspection task are improved, and through the accurate driving mechanism, the demand for human intervention is reduced, and the autonomous ability of the unmanned aerial vehicle in a complex environment is enhanced.

[0081] Compared with the prior art, the beneficial effects achieved by the present application are:

[0082] Cross-network state sensing and adaptive cooperation capability: through real-time sharing of instruction parameters (including communication state and spatial position) and data return mechanism, the unmanned aerial vehicle group can dynamically monitor the network interruption state (such as visualized through light), so that any single machine can still make autonomous decisions based on the latest data or image recognition when the network is interrupted; meanwhile, multi-color light flashing signals are used to establish / join the queue, realizing self-organizing cooperation without network dependence, and significantly improving the system robustness and task continuity in a complex indoor environment.

[0083] Queue dynamic optimization and following relationship self-adjustment mechanism: based on flight trajectory analysis and difference index calculation, the system can optimize the queue size and following relationship in real time: the external relationship supports adaptive separation and recombination of the queue when a new reference object is encountered, and the internal relationship optimizes the formation through continuous rearrangement of the following order (the smaller the difference index, the closer to the front), and cooperates with the "single-order asynchronous flight" mechanism to ensure the stability of the queue and the accuracy of path tracking in the network interruption scenario.

[0084] Obstacle sensing and channel resource quantitative adaptation: a global dense point cloud model is constructed by using the images of networked unmanned aerial vehicles, independent flight channels are segmented through edge detection and connectivity analysis, a matching index is constructed by combining the passage index (measuring the unobstructedness of the channel) and the queue priority index (measuring the cooperativeness of the queue), and the optimal channel resource is dynamically distributed to eliminate route conflicts and improve the efficiency and safety of the cluster passage. BRIEF DESCRIPTION OF DRAWINGS

[0085] The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the application, and are intended to explain the application without limiting the application. In the drawings:

[0086] Figure 1 is a flow diagram of the intelligent perception-based UAV cooperative control method of the application;

[0087] Figure 2 is a structural diagram of the intelligent perception-based UAV cooperative control system of the application. DETAILED DESCRIPTION

[0088] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the application.

[0089] Please refer to Figure 1 The application provides an intelligent perception-based UAV cooperative control method, comprising:

[0090] S100, when the UAV group is flying and working indoors, each UAV collects image and position data through an onboard sensor and receives instruction parameters from a command center, and performs a patrol task according to the instruction parameters.

[0091] The image refers to a real scene image collected by the UAV through an onboard depth camera. The position data include a real-time spatial position of the UAV itself.

[0092] The instruction parameters include a control instruction, a communication state and a spatial position of each UAV in the UAV group. The communication state refers to a link state between the UAV and the command center, and specifically includes a networking state and a network outage state.

[0093] Each UAV reports the spatial position of itself to the command center in real time, and the command center collects data and then transmits the control instruction of each UAV, and the communication state and the spatial position of the UAV group to each UAV again, so as to realize real-time perception of the communication state and the spatial position of the UAV group.

[0094] Each UAV monitors the communication state change of the UAV group and the spatial position distribution of other UAVs in real time according to the instruction parameters. The command center is used to send the instruction parameters to each UAV, so as to control the UAV group to perform the patrol task.

[0095] Through real-time data sharing and state awareness, the fault tolerance and cooperative reliability of the UAV group in a complex indoor environment are enhanced, the risk of single-point failure caused by network interruption is reduced, and the overall execution efficiency and coordination ability of the inspection task are improved.

[0096] S200, dynamically selecting a reference object for the disconnected UAV according to the instruction parameters and establishing a queue, establishing a following relationship through multi-color light flashing signals, and dynamically adjusting the number of UAVs in each queue and the following relationship during flight. Specifically, it includes:

[0097] S201, when the UAV UAV1 is disconnected, obtaining the latest communication status and spatial position of each UAV in the instruction parameters before disconnection, and marking other UAVs whose spatial position is less than the threshold distance from the spatial position of the UAV UAV1.

[0098] When the UAV is just disconnected, the latest data in the instruction parameters is used as the reference.

[0099] When the disconnection time of the UAV exceeds the set threshold, the communication status and spatial position changes of other UAVs within the visible range are analyzed in real time through image recognition algorithm, and the latest image analysis result is used as the reference.

[0100] The communication status can be visualized through the indicator light on the UAV, usually red indicating disconnection and green indicating connection.

[0101] S202, draw the flight trajectory by the historical spatial position of each marked UAV, calculate the coordination coefficient according to the latest communication status of each marked UAV and the flight trajectory, and select the marked UAV with the largest coordination coefficient as the reference object. Specifically, it includes:

[0102] S2021, obtain the historical instruction parameters saved by the UAV UAV1, analyze the spatial position changes of the UAV UAV1 and each marked UAV in the recent time T recent , and draw the flight trajectory.

[0103] S2022, evenly set Q time points on each flight trajectory, calculate the Euclidean distance between the UAV UAV1 and the marked UAV UAV mark at the same time point, and the standard deviation of the Euclidean distance at all time points as the difference coefficient.

[0104] S2023, analyze the latest communication status of each marked UAV, exclude the marked UAVs that do not belong to any queue and are in the disconnected state, and obtain the number of UAVs in the queue to which each disconnected state marked UAV belongs.

[0105] S2024, set state index N and take value a or b, when the marker drone is in network state N=a, when the marker drone is in network outage state N=b. Wherein, a>b. Substituting formula to calculate the coordination coefficient of each marker drone:

[0106]

[0107] In the formula, CC i is the coordination coefficient of the ith marker drone, d i is the spatial distance between the ith marker drone and the drone UAV1, d max is the maximum spatial distance between all marker drones and the drone UAV1.

[0108] γ and τ are constants, M i is the difference coefficient of the ith marker drone, S i is the number of drones in the team to which the ith marker drone belongs, S max is the maximum number of drones in all teams to which the marker drones belong.

[0109] The coordination coefficient is used to quantify the cooperation suitability between the network outage drone (such as UAV1) and the surrounding candidate reference object. By dynamically calculating the coordination coefficient CC i The optimal reference object is selected.

[0110] The formula combines the spatial distance weight (d i ÷d max is inversely proportional to the coordination coefficient), flight trajectory consistency (difference coefficient M i ), communication state of the candidate object (networking state N=a gives higher weight) and team size (S i ÷S max reduces the possibility of large-scale team candidates).

[0111] Finally, the most stable connection object is selected by maximizing the coordination coefficient, so as to ensure the reliability and cooperation efficiency of the team establishment in the network outage scenario.

[0112] S203, the drone establishes a team with the reference object, or joins the team in which the reference object is located. The following relationship is established through multi-color light flashing signal interaction, and the external relationship and internal relationship of the team are dynamically adjusted during flight.

[0113] When the reference object is in the networking state, the drone establishes a team with the reference object. The drone interacts with the reference object through multi-color light flashing signal to establish the following relationship, and flies along the path of the reference object through real-time image analysis.

[0114] When the reference object is in a disconnected state, the UAV joins the queue in which the reference object is located. The UAV interacts with the reference object through multi-color light flashing signals to establish a following relationship. The UAVs in the queue interact in turn through multi-color light flashing signals according to the following relationship to inform the new member of the joining.

[0115] The UAV flies along the path followed by the reference object through real-time image analysis. The UAVs in the queue follow the path in turn according to the following relationship.

[0116] Dynamic adjustment of external relationships refers to:

[0117] When any UAV in the queue encounters a new marked UAV during flight and the new marked UAV is calculated as a reference object through a coordination coefficient, the corresponding UAV automatically leaves the original queue and re-establishes a queue with the reference object or joins the queue in which the reference object is located.

[0118] Dynamic adjustment of internal relationships refers to:

[0119] Each UAV in the queue analyzes images in real time during flight, draws flight trajectories of other UAVs in the team, analyzes flight trajectories of two UAVs with a following relationship, and calculates a difference coefficient of the following UAV.

[0120] After data aggregation, the average value of the difference coefficient of each following UAV is calculated and used as a difference index. The following relationship of each UAV in the queue is dynamically adjusted in order from small to large according to the difference index.

[0121] Each UAV in the queue usually flies in a straight line, and each UAV only has a following relationship with one UAV in the queue. The following UAV analyzes the flight trajectory of the reference object in real time through images and flies synchronously at different times according to the same flight trajectory.

[0122] Each UAV calculates the difference coefficient of all following UAVs in front. The number of following UAVs that can be calculated by the UAV in front is smaller. Data aggregation is visually transmitted through light signal coding in order according to the following relationship.

[0123] The autonomous cooperation ability of the UAV in a disconnected state is strengthened. The formation maintenance and redundancy mechanism are optimized through visual signals and dynamic queue management, the failure rate of the inspection task is reduced, and the overall stability is improved. The system adaptability and flexibility are enhanced, especially in a link loss environment.

[0124] S300, analyze the obstacle distribution of the target direction space, and plan multiple independent flight channels. Calculate the matching index between the queue and the independent flight channel, and plan a distributed cooperative route for each queue according to the matching index. Specifically, it includes:

[0125] S301, obtain the obstacle distribution image of the target direction space collected by the networking state unmanned aerial vehicle in the unmanned aerial vehicle group, and perform feature extraction and timestamp synchronization processing on each view image.

[0126] S302, perform stereo matching calculation on the cross-view feature points by epipolar constraint, jointly solve the spatial three-dimensional coordinate values of the homonymic points based on the matching results, and generate a global dense point cloud model by fusing all the view solving results.

[0127] S303, the edge detection algorithm identifies the contour boundary of the vertical obstacle, so as to divide the target direction space into several parallel subspaces. The connectivity analysis algorithm is used to extract the continuous subspaces that meet the passing standard as independent flight channels.

[0128] S304, the passing index of each independent flight channel and the priority index of each queue in the unmanned aerial vehicle group are calculated respectively, and the matching index between the queue and the independent flight channel is obtained by multiplying the priority index and the passing index.

[0129] The independent flight channel is divided into h cross sections according to the vertical flight direction.

[0130] The length of the longest line segment and the shortest line segment from each cross section center point to the edge line is analyzed, and the area standard deviation V of all cross sections is calculated. sum , the passing index TX is calculated by substituting the formula:

[0131]

[0132] In the formula, AR e is the area of the e-th cross section, λ is a constant, and are the length of the longest line segment and the shortest line segment from the e-th cross section center point to the edge line, respectively.

[0133] The passability and safety of the independent flight channel are quantified by geometric and statistical characteristics, and the channel quality index is provided for queue matching. The core design includes three parts:

[0134] Cross section shape analysis: calculate the longest / shortest line segment ratio of each cross section center point to the edge Reflect the change of channel width, the larger the ratio (set the weight size by λ) the stronger the punishment.

[0135] Area stability: all cross section area standard deviation V sum (the denominator) measures the overall consistency of the channel, the greater the fluctuation, the lower the score.

[0136] Cross section weight: take the area AR eAs a benchmark, the weighted value is calculated in combination with the aforementioned proportion coefficient. The higher the comprehensive result pass index TX, the more uniform the channel, indicating that the queue is suitable for safe passage.

[0137] The priority index YZ is calculated by the following formula:

[0138]

[0139] In the formula, p is the number of UAVs in the queue, CZ u is the difference index of the u-th UAV.

[0140] The overall priority index YZ of the queue is calculated by accumulating the difference indexes CZ u of all UAVs in the queue, which is used for priority ranking when allocating flight channels.

[0141] The design logic is to quantify the stability of the queue as the difference index, which reflects the degree of trajectory deviation of each UAV tracking the reference object (such as CZ u The smaller the value, the higher the flight consistency. Therefore, the smaller the total YZ, the better the overall coordination of the queue members.

[0142] When allocating channels, the queue with a higher priority index (YZ value) has the priority to choose, ensuring that the low-stability queue has priority to pass through the optimal channel, thereby improving the efficiency of system resource allocation and reducing the probability of collision.

[0143] S305, select the independent flight channel with the largest matching index for the queue, and select the independent flight channel for each queue in order of priority index from large to small, thereby planning a distributed collaborative route.

[0144] There is no spatial overlap between different independent flight channels, and the route of the UAV is usually planned at the middle point of the flight channel.

[0145] Efficient and safe route planning is achieved, collision risk is reduced through quantitative obstacle avoidance and optimized resource allocation, task parallelism and response speed are improved, energy consumption is reduced, and high-precision cooperation of the UAV swarm in a limited space is supported.

[0146] S400, drive each UAV queue to enter the planned route in order and fly, and each UAV cooperates to perform the inspection task.

[0147] The orderliness and safety of route execution are ensured, the task reliability and overall efficiency are improved through real-time sensing and dynamic adjustment mechanisms, the need for manual intervention is reduced, and the autonomous execution capability of the UAV swarm in a dynamic environment is enhanced.

[0148] Please refer to Figure 2The application also provides an intelligent perception-based unmanned aerial vehicle cooperative control system, comprising an intelligent perception module, an object analysis module, a cooperative planning module, and a driving execution module.

[0149] The intelligent perception module collects image and position data through each unmanned aerial vehicle and receives instruction parameters from the command center.

[0150] Real-time real scene images and spatial position data (such as geographic coordinates) are collected by the on-board depth camera of each unmanned aerial vehicle, and instruction parameters sent by the command center are received, including control instructions, communication states of each unmanned aerial vehicle (networked or disconnected), and spatial position data of each unmanned aerial vehicle in the group.

[0151] The unmanned aerial vehicles report these data to the command center in real time, the command center aggregates the data and returns updated instruction parameters, so that each unmanned aerial vehicle can continuously monitor the communication state (or visualized display through indicator lights) and position distribution change (or spatial position change through image analysis) of each other, thereby dynamically adjusting its own behavior.

[0152] Real-time state perception and data sharing of the unmanned aerial vehicle group are realized, and the reliability and fault tolerance capability of the system in a complex indoor environment are enhanced.

[0153] By continuously updating the communication and position information, it is ensured that the unmanned aerial vehicle can respond to the task requirements even if a single point is disconnected, and the overall coordination and execution efficiency of the inspection task are improved.

[0154] The object analysis module dynamically selects reference objects for the disconnected unmanned aerial vehicle according to the instruction parameters and establishes a queue, establishes a following relationship through multi-color light flashing signals, and dynamically adjusts the number of unmanned aerial vehicles in each queue and the following relationship during flight.

[0155] Based on historical instruction parameters or real-time image recognition (analysis of visual indicator lights such as red lights indicating disconnection), reference objects are dynamically selected, flight trajectories of marked unmanned aerial vehicles are drawn, and difference coefficients (such as standard deviation of Euclidean distance) and cooperation coefficients (formula for non-dimensional calculation based on communication state, spatial distance, and queue size) are calculated.

[0156] After selecting the optimal reference object, multi-color light flashing signals are used to interact to establish or join a queue (for example, a networked object establishes a new queue, and a disconnected object joins an existing queue), and the queue size (such as external relationship processing of new marked unmanned aerial vehicles joining) and following relationship (internal relationship through difference index rearrangement) are dynamically adjusted through real-time image analysis during flight.

[0157] The autonomous collaboration capability of the unmanned aerial vehicle in the disconnected state is improved, the queue is dynamically maintained and optimized through visual signals and real-time trajectory analysis, the flexibility and redundancy of the system are enhanced, thereby reducing the failure risk of the overall inspection task and improving the team stability.

[0158] The cooperative planning module is used to analyze the obstacle distribution of the target direction space and plan multiple independent flight channels. The matching index between the queue and the independent flight channel is calculated, and the distributed cooperative flight path is planned for each queue according to the matching index.

[0159] After collecting the target direction space image provided by the networked unmanned aerial vehicle, performing feature extraction and timestamp synchronization, and using the epipolar constraint stereo matching algorithm to generate a global dense point cloud model, the space is segmented into multiple obstacle-free independent flight channels through edge detection and connectivity analysis.

[0160] The passage index (based on the area standard deviation of the channel cross section and the ratio formula of the long and short line segments of the center point) and the queue priority index (based on the sum of the difference index) are calculated. After the matching index is combined by multiplication, the queue is dynamically distributed to the best channel (such as the channel with the largest matching index), and the distributed cooperative flight path is planned to ensure that the channels do not overlap.

[0161] The efficient and safe flight path planning optimizes the obstacle avoidance logic and queue priority allocation, reducing the risk of collision. Through quantitative index matching, the resource utilization efficiency and task parallelism are improved, making the unmanned aerial vehicle respond more quickly and have lower collision risk when cooperatively performing the inspection task in a limited space.

[0162] The driving execution module is used to drive each unmanned aerial vehicle queue to enter the planned flight path and cooperatively perform the inspection task.

[0163] Each unmanned aerial vehicle queue is driven to enter the independent flight channel in order (the path is planned based on the channel center point), and each unmanned aerial vehicle analyzes the real-time image and uses the flight trajectory of the reference object (such as one-character sequential asynchronous flight).

[0164] The orderliness and safety of the flight path execution are ensured, and the real-time perception adjustment is combined to improve the reliability and overall efficiency of the inspection task. Through the precise driving mechanism, the need for human intervention is reduced, and the autonomous ability of unmanned aerial vehicles in complex environments is enhanced.

[0165] Embodiment one:

[0166] Suppose there are A1, A2, A3, and A4, a total of 4 unmanned aerial vehicles in the queue, and the difference index is 2.3, 1.5, 3.2, and 2.6 respectively. The priority index is calculated by substituting the formula:

[0167] 2.3 + 1.5 + 3.2 + 2.6 = 9.6

[0168] Then the priority index of the queue is 9.6.

[0169] It is to be noted that, in the present text, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0170] Finally, it should be noted that the above-mentioned only constitutes preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications, equivalent replacements, improvements and the like of the technical solutions described in the foregoing embodiments can still be made. Any modifications, equivalent replacements, improvements and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for intelligent perception-based cooperative control of unmanned aerial vehicles, characterized in that: The method comprises: S100, when the UAV group is flying indoors, each UAV collects image and position data through an on-board sensor and receives instruction parameters from a command center, and executes a patrol task according to the instruction parameters; S200, reference objects are dynamically selected for the disconnected UAVs according to the instruction parameters, and queues are established, a following relationship is established through multi-color light flashing signals, and the number of UAVs in each queue and the following relationship are dynamically adjusted during flight; S300, the distribution of obstacles in the target direction space is analyzed, a plurality of independent flight channels are planned, a matching index between the queues and the independent flight channels is calculated, and a distributed cooperative route is planned for each queue according to the matching index; S400, each UAV queue is driven to enter the planned route in order and fly, and each UAV cooperatively executes the patrol task. 2.The smart perception based UAV cooperative control method of claim 1, wherein: In S100, the image refers to a real scene image collected by the UAV through an on-board depth camera; the position data includes the real-time spatial position of the UAV itself; The instruction parameters include control instructions, communication states and spatial positions of each UAV in the UAV group; the communication state refers to the link state between the UAV and the command center, and specifically includes a networked state and a disconnected state; Each UAV monitors the communication state changes of the UAV group and the spatial position distribution of other UAVs in real time according to the instruction parameters; the command center is used to send instruction parameters to each UAV, so as to control the UAV group to execute the patrol task. 3.The smart perception based UAV cooperative control method of claim 2, wherein: S200 comprises: S201, when the UAV UAV1 is disconnected, the latest communication state and spatial position of each UAV in the instruction parameters before disconnection are obtained, and other UAVs whose spatial position is less than a threshold distance from the spatial position of the UAV UAV1 are marked; S202, flight trajectories of each marked UAV are drawn through the historical spatial positions of the marked UAVs, a cooperation coefficient is calculated according to the latest communication state of each marked UAV and the flight trajectory, and a marked UAV with the maximum cooperation coefficient is selected as a reference object; S203, the UAV establishes a queue with the reference object, or joins the queue in which the reference object is located; a following relationship is established through multi-color light flashing signals, and the external relationship and the internal relationship of the queue are dynamically adjusted during flight. 4.The smart perception based UAV cooperative control method of claim 3, wherein: S202 comprises: S2021、Obtain the historical instruction parameters stored by the unmanned aerial vehicle UAV1, analyze the recent time length T recent The spatial position changes of the unmanned aerial vehicle UAV1 and each marker unmanned aerial vehicle, thereby drawing a flight trajectory; S2022、Each flight trajectory is evenly set with Q time points, and the distance between the unmanned aerial vehicle UAV1 and the marker unmanned aerial vehicle UAV mark The Euclidean distance at the same time point, and the standard deviation of the Euclidean distance at all time points as the difference coefficient; S2023, the latest communication state of each marked UAV is analyzed, and after excluding the marked UAVs that do not belong to any queue and are in a disconnected state, the number of UAVs in the queue to which each disconnected state marked UAV belongs is obtained; S2024, a state index N is set and takes a or b, N=a when the marked UAV is in a networked state, and N=b when the marked UAV is in a disconnected state; wherein a>b; the cooperation coefficient of each marked UAV is calculated by substituting the formula: In the formula, CC i is the cooperative coefficient of the ith marked UAV, d i is the spatial distance between the ith marked UAV and the UAV UAV1, d max is the maximum value of the spatial distance between all marked UAVs and the UAV UAV1; γ and τ are constants, M i is the difference coefficient for the i-th labeled drone, S i is the number of drones in the fleet to which the i-th labeled drone belongs, S max is the maximum number of drones in the fleet to which all labeled drones belong. 5.The smart perception based UAV cooperative control method of claim 4, wherein: In S203, when the reference object is in a networked state, the UAV establishes a queue with the reference object; the UAV interacts with the reference object through multi-color light flashing signals to establish a following relationship, and flies along the path of the reference object through real-time image analysis; When the reference object is in a disconnected state, the UAV joins the queue in which the reference object is located; the UAV interacts with the reference object through multi-color light flashing signals to establish a following relationship, and the UAVs in the queue interact in turn through multi-color light flashing signals according to the following relationship to inform the new member to join; The UAVs fly along the path followed by the reference object through real-time image analysis, and the UAVs in the queue follow the path in turn according to the following relationship. 6.The smart perception based UAV cooperative control method of claim 5, wherein: In S203, the dynamic adjustment of the external relationship refers to: When any UAV in the queue encounters a new marked UAV in the flight process and the new marked UAV is calculated as a reference object through the coordination coefficient, the corresponding UAV automatically leaves the original queue, re-establishes a queue with the reference object, or joins the queue in which the reference object is located; The dynamic adjustment of the internal relationship refers to: Each UAV in the queue analyzes images in real time during the flight process, draws flight trajectories of other UAVs in the team, analyzes flight trajectories of two UAVs with a following relationship, and calculates a difference coefficient of the following UAV; After data collection, the average value of the difference coefficient of each following UAV is calculated, and is used as a difference index; the following relationship of each UAV in the queue is dynamically adjusted in order from small to large according to the difference index. 7.The smart perception based UAV cooperative control method of claim 6, wherein: S300 includes: S301, acquiring an obstacle distribution image of a target direction space collected by a networked UAV in the UAV group, performing feature extraction and time stamp synchronization processing on each view image; S302, performing stereo matching calculation on the cross-view feature points through epipolar constraint, jointly solving spatial three-dimensional coordinate values of the corresponding points based on the matching result, and generating a global dense point cloud model by fusing all view solving results; S303, an edge detection algorithm identifies the contour boundary of the vertical obstacle, so as to divide the target direction space into a plurality of parallel subspaces; a connectivity analysis algorithm is used to extract a continuous subspace that meets the continuity requirement as an independent flight channel; S304, the passing index of each independent flight channel and the priority index of each queue in the UAV group are calculated, and the priority index and the passing index are multiplied to obtain a matching index between the queue and the independent flight channel; S305, selecting an independent flight channel with the largest matching index for the queue, and selecting independent flight channels for each queue in order from large to small according to the priority index, so as to plan a distributed cooperative route. 8.The smart perception based UAV cooperative control method of claim 7, wherein: The passing index calculation includes: dividing the independent flight channel into h cross sections according to the vertical flight direction; Analyze the length of the longest and shortest line segments from each cross-section center point to the edge line, and calculate the area standard deviation V of all cross-sections sum Substitute the formula to calculate the traffic index TX: where AR e is the area of the e-th cross section, λ is a constant, and are the lengths of the longest and shortest line segments from the e-th cross section center point to the edge line, respectively. 9.The smart perception based UAV cooperative control method of claim 7, wherein: The priority index YZ calculation formula is: In the formula, p is the number of UAVs in the queue, CZ u is the difference index of the u-th UAV.

10. An intelligent perception based UAV cooperative control system characterized by: The system includes an intelligent perception module, an object analysis module, a cooperative planning module, and a driving execution module; The intelligent perception module collects images and position data through each UAV, and receives instruction parameters from the command center; The object analysis module dynamically selects a reference object for a disconnected UAV and establishes a queue according to the instruction parameters, establishes a following relationship through multi-color light flashing signals, and dynamically adjusts the number of UAVs in each queue and the following relationship during flight; The cooperative planning module is used to analyze the obstacle distribution of the target direction space, plan multiple independent flight channels, calculate the matching index between the queue and the independent flight channel, and plan a distributed cooperative route for each queue according to the matching index; The driving execution module is configured to drive each unmanned aerial vehicle queue to fly along the planned flight path and cooperatively perform the inspection task.

Images