Purely visual distributed swarm flight emergence method and system for crowded environments

Abstract

The present invention discloses a purely visual distributed swarm flight emergence method and system for crowded environments, belonging to the field of unmanned aerial vehicle (UAV) swarm flight technology. The method comprises the following steps: S1, obtaining an observed image; S2, parsing the image to obtain an environmental depth map and RGB map; S3, analyzing the RGB map to obtain the position and speed information of neighboring UAVs; S4, generating a raster map and an ESDF map from the environmental depth map; S5, an onboard intelligent terminal integrates the mission waypoints with the information in S3 and S4, uses a cluster model to make a cluster decision, and obtains a speed decision variable; S6, the onboard intelligent terminal sends the speed decision variable to the UAV autopilot; S7, feeds back flight status information to the onboard intelligent terminal. The present invention aims to address the problems of UAV swarm systems that rely too much on inter-machine communication, are unable to cope with complex crowded environments, and have low adaptability of individual UAVs to complex crowded environments.

Description

Technical Field

[0001] The present invention relates to the field of UAV swarm flight technology, and in particular to a purely visual distributed swarm flight emergence method and system for crowded environments. Background Art

[0002] As drone technology shines in battlefields, aerial photography, search and rescue, surveying and mapping and other fields, distributed drone clusters can handle more complex tasks and have higher fault tolerance than single drones, and have therefore received widespread attention in both civilian and military fields.

[0003] For fully autonomous drone swarms to achieve higher mission performance, collaborative decision-making is essential. To date, research on collaborative decision-making and control technologies has yielded significant results. From the perspective of core decision-making and control technologies, current research can be broadly categorized into approaches based on bionic behavioral rules, artificial potential fields, and collaborative planning.

[0004] However, most current research still has the following problems: 1) It is assumed that there is efficient communication between drones, which makes the cluster system unable to be effectively applied in complex environments with limited communication; 2) The default usage scenario is relatively open, or there are only a few obstacles, which cannot cope with crowded and complex indoor environments; 3) Although the collaborative planning-based method can avoid obstacles and perform cluster flight under effective communication conditions, the nature of trajectory planning increases the probability of it being unable to be solved in complex environments, and cluster individuals are prone to fall into local optimality, which in turn affects the entire cluster. Summary of the Invention

[0005] The purpose of this invention is to provide a purely visual distributed swarm flight emergence method and system for crowded environments. In this method, individual drones can use visual sensors to obtain information about environmental obstacles and the locations of their neighbors. This addresses the issues of drone swarm systems that rely too heavily on inter-machine communication, are unable to cope with complex and crowded environments, and have low adaptability to these environments for individual drones.

[0006] To achieve the above objectives, the present invention provides a purely visual distributed cluster flight emergence method for crowded environments, comprising the following steps:

[0007] S1, the rotorcraft obtains the observed visible light image through the onboard depth camera;

[0008] S2, obtains the environment depth map and RGB map by analyzing the image;

[0009] S3, analyzes the RGB image, performs Kalman filtering on the location information of neighboring drones, and estimates the location and speed information of neighboring drones;

[0010] S4, generating a raster map and ESDF map from the environment depth map;

[0011] In step S5, the airborne intelligent terminal weights the cluster mission waypoints of the cluster ground station and the position and speed information of the neighboring UAVs. After weighting, it uses the cluster model to make cluster decisions based on the information of the grid map and ESDF map to obtain the speed decision value.

[0012] S6: The onboard intelligent terminal sends the speed decision value to the UAV autopilot, and the UAV autopilot controls the rotorcraft UAVs to perform swarm flight based on the speed decision value.

[0013] S7, the rotorcraft drone feeds back flight status information to the onboard intelligent terminal.

[0014] Preferably, in S2, the specific process of parsing the image is: first, identify the friendly drone in the image, obtain the identification frame coordinates and size of the friendly drone, generate an RGB image, and then remove the neighbor individual image coordinates identified in the RGB image from the image to obtain an environmental depth map.

[0015] Preferably, the specific steps of S3 are:

[0016] S31, use yolov8 to identify individuals, and then perform data association on the identification results to distinguish different neighboring individuals in the field of view;

[0017] S32, obtain the pixel coordinates of the center point of the observed neighbor drone identification frame (u c ,v c ), and identification box parameters (s b ,r b ), where s b is the detection box area, r b is the aspect ratio of the detection frame;

[0018] S33, project the pixel coordinates and the identification frame parameters into the image to obtain the distance information d of the observed neighboring drone. The local coordinates of the neighboring drone in the inertial coordinate system are

[0019] M1M2[x,y,z,1] T =d[u,v,1] T ;

[0020] Where M1 is the camera internal parameter, and M2 is the rotation matrix of the camera relative to the world system;

[0021] S34, performing Kalman filtering on the observed neighbor drone position information to estimate the neighbor drone position and speed information.

[0022] Preferably, the specific process of S4 is to open source the environment depth map using an open source method to generate a raster map and an ESDF map.

[0023] Preferably, the specific steps of S5 are:

[0024] S51, establish cluster model;

[0025] S51, using the ESDF map, obtains obstacle position information and obtains obstacle avoidance behavior as the first speed decision variable;

[0026] S52, using the neighboring drone position and speed information in S3, obtains the distance keeping behavior as the second speed decision quantity;

[0027] S53, using the cluster mission waypoints in the cluster ground station, obtains the mission behavior as the speed decision quantity three;

[0028] S54, weighting the speed decision quantity 2 and the speed decision quantity 3, with the weight being 1, to obtain the speed decision quantity 4;

[0029] S55, combining the speed decision quantity 4 with the speed decision quantity 1 to perform cluster decision making and obtain the speed decision quantity.

[0030] Preferably, the specific process of establishing the cluster model is:

[0031] Using an undirected graph Represents a drone cluster with N individuals, where Assemble for drones, is the interaction edge between UAVs, where R max is the maximum sensing range of the UAV, d ij is the displacement vector of UAV i relative to j, and the detected neighbor set of UAV i is defined as Each drone i in the swarm model follows the following motion model:

[0032]

[0033] where p i and v i is the position and velocity of UAV i, k is a control parameter greater than 0, v d,i It is the input of the UAV speed controller and also the speed decision variable, which satisfies the maximum speed limit of UAV i.

[0034] Preferably, the obstacle avoidance behavior is obtained as follows:

[0035] Obtain obstacle information from ESDF maps Where l is the number of obstacles obtained, αk is the confidence of obstacle information, d obs,k is the shortest vector pointing from UAV i to the surface of obstacle k, k∈l;

[0036] The position coordinates of drone i are p i (x i ,y i ,z i ), the obstacle position coordinates are p g (x g ,y g ,z g ), the relative position relationship between UAV i and the obstacle is With the current speed v of the drone i i Predict n time steps t later step Position p i,pred (t setp ), the relative position relationship between each predicted position and the nearest obstacle point is d i,pred (t setp );

[0037] Introducing the predicted obstacle avoidance parameter R safe , that is, UAV i predicts that there will be obstacles less than R in a certain step length according to the current speed. safe , then the obstacle avoidance rule needs to be executed, R reach R is the threshold for the UAV to determine whether it has reached the target area. avoid The distance threshold at which the drone determines it needs to start avoiding obstacles, where R avoid =R0+kv, R0 is set to 3 meters, k is 1; the following conditions are met

[0038]

[0039] The expected velocity vector generated by the obstacle avoidance rule is:

[0040]

[0041] Preferably, the distance keeping behavior is obtained in the following manner:

[0042] Assume there are N individuals in the cluster, and the position coordinates of drone i are p i (x i ,y i ,z i ), and its relative position relationship with the j-th UAV j is d i,j =p i -p j , where the position of UAV j is obtained by visual observation, and its credibility is α nei,j , and 0≤α nei,j≤1, according to the distance keeping rule, the expected velocity component generated by UAV j on UAV i is:

[0043]

[0044] where R align Represents the safe distance between drones. The distance between drone j and drone i is less than the safe distance R between drones. align When , the desired velocity components moving away from each other are generated, and vice versa, the desired velocity components moving towards each other are generated;

[0045] The way to obtain task behavior is:

[0046] The position coordinates of drone i are p i (x i ,y i ,z i ), the target position coordinate is p g (x g ,y g ,z g ), the relative position relationship between UAV i and the target is d goal,i =p g -p i , according to the mission behavior rules, the expected velocity vector generated by UAV i is:

[0047]

[0048] in is a unit vector; when the distance d from UAV i to the target goal,i Less than R mission As the distance to the target gets closer, the expected speed gradually decreases. When the distance is greater than R mission When the desired speed is kept at V mmax , where R mission The size of is related to the number of individuals in the cluster N. k is the coefficient;

[0049] The speed decision quantity is:

[0050] v cmd =k1v obs +k2v align +k3v mission ;

[0051] Where k1, k2, and k3 represent the weight of speed decision quantity one, the weight of speed decision quantity two, and the weight of speed decision quantity three, respectively, and the sum of k2 and k3 is 1.

[0052] Preferably, the optimization process of cluster flight in S6 is as follows:

[0053] Setting Φobs represents the proportion of individuals in the cluster colliding with obstacles,

[0054]

[0055] where Θ(·) is the Heaviside step function, For p i (t) in symbolic form, when p i (t) When in an obstacle, Take positive value, otherwise take negative value;

[0056] Setting Φ col represents the proportion of collisions between individuals in the cluster,

[0057]

[0058] Setting Φ mis Indicates the completion of the cluster task, the time it takes for the centroid to reach the target point, and the number of surviving individuals.

[0059]

[0060] According to the above measurement indicators, a single-objective optimization function is constructed.

[0061]

[0062] Use open source tools to solve the algorithm and obtain the optimal control parameters.

[0063] The present invention also provides a purely visual distributed cluster flight emergence system for crowded environments, which adopts the above-mentioned purely visual distributed cluster flight emergence method for crowded environments, including:

[0064] A visual perception module for obtaining visible light images;

[0065] Cluster ground station, used to issue mission instructions;

[0066] Cluster decision module, used to process and extract information from visible light images and task instructions to obtain speed decision variables;

[0067] The unmanned driving module is used to receive and execute speed decision quantities to control the movement of the UAV; and the UAV body.

[0068] Therefore, the present invention adopts the above-mentioned structure for the pure visual distributed cluster flight emergence method and system for crowded environments, which has the following advantages:

[0069] 1. While most existing swarm systems rely on inter-machine communication, this invention, thanks to the design of the visual perception module, relies solely on the onboard depth camera module to obtain information about obstacles and individual neighbors in the environment. This invention uses visible light to classify visual information into information about individual neighbors and environmental obstacles, and further uses depth information to accurately estimate the position of individual drones regarding their neighbors and obstacles in the environment.

[0070] 2. Existing research on behavioral rule-based swarm systems often assumes a relatively empty flight environment or only a few obstacles. This invention uses perceived neighbor and obstacle information as input, considers perception errors and reliability, and designs reliable obstacle avoidance, distance maintenance, and mission rules to drive individuals to behave rationally. This results in a behavioral rule-based distributed drone swarm capable of navigating crowded, obstacle-filled environments.

[0071] 3. In order to solve the problem of difficulty in selecting behavioral weights and balancing multiple indicators,

[0072] This paper quantifies the task execution capability of the cluster system and designs the target optimization function.

[0073] The optimal weight parameters are obtained through genetic algorithm.

[0074] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 A block diagram of individual drones in a cluster according to an embodiment of the present invention's purely visual distributed cluster flight emergence method and system for crowded environments;

[0076] Figure 2 ESDF map of an embodiment of a purely visual distributed cluster flight emergence method and system for crowded environments according to the present invention;

[0077] Figure 3 A schematic diagram of obstacle avoidance rules for an embodiment of a purely visual distributed swarm flight emergence method and system for crowded environments according to the present invention;

[0078] Figure 4 This is a schematic diagram of distance maintenance in an embodiment of a purely visual distributed cluster flight emergence method and system for crowded environments according to the present invention;

[0079] Figure 5 A schematic diagram of the task behavior of an embodiment of a purely visual distributed swarm flight emergence method and system for crowded environments according to the present invention;

[0080] Figure 6This is the cluster flight simulation effect of a typical crowded scene in the embodiment of the pure visual distributed cluster flight emergence method and system for crowded environments of the present invention;

[0081] Figure 7 This is the second cluster flight simulation effect in a typical crowded scenario of the pure visual distributed cluster flight emergence method and system embodiment of the present invention for crowded environments. DETAILED DESCRIPTION

[0082] Example

[0083] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0084] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0085] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0086] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they should not be understood as limiting the present invention.

[0087] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, the terms "disposed," "installed," and "connected" should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0088] The following embodiments of the present invention are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0089] In the description of the present invention, it should be noted that the terms "up", "down", "left", "right", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the present invention.

[0090] The specific model specifications need to be selected and determined based on the actual specifications of the device. The specific selection calculation method adopts the existing technology in this field, so it will not be described in detail.

[0091] like Figure 1 As shown, a purely visual distributed cluster flight emergence method for crowded environments includes the following steps:

[0092] S1, the rotorcraft obtains the observed visible light image through the onboard depth camera;

[0093] S2, obtains the environment depth map and RGB map by analyzing the image;

[0094] In S2, the specific process of image parsing is as follows: first, the friendly drone in the image is identified, the coordinates and size of the friendly drone's identification box are obtained, and an RGB image is generated. Then, the coordinates of the neighboring individual images identified in the RGB image are removed from the image to obtain the environment depth map.

[0095] S3, analyzes the RGB image, performs Kalman filtering on the location information of neighboring drones, and estimates the location and speed information of neighboring drones;

[0096] The specific steps of S3 are:

[0097] S31, use yolov8 to identify individuals, and then perform data association on the identification results to distinguish different neighboring individuals in the field of view;

[0098] Taking a drone individual in the field of view as an example, the location information of the drone is further estimated. S32, the pixel coordinates of the center point of the observed neighbor drone identification frame (u c ,v c ), and identification box parameters (s b ,r b ), where s b is the detection box area, r b is the aspect ratio of the detection frame;

[0099] S33, project the pixel coordinates and the identification frame parameters into the image to obtain the distance information d of the observed neighboring drone. The local coordinates of the neighboring drone in the inertial coordinate system are

[0100] M1M2[x,y,z,1] T =d[u,v,1] T ;

[0101] Where M1 is the camera intrinsic parameter, and M2 is the rotation matrix of the camera relative to the world system

[0102] S34, performing Kalman filtering on the observed neighbor drone position information to estimate the neighbor drone position and speed information.

[0103] S4, generating a raster map and ESDF map from the environment depth map;

[0104] like Figure 2 As shown in Figure 1, the specific process of S4 is to obtain the environmental depth map from the depth camera, remove the coordinates of the neighboring individual pixels identified in the RGB image from the depth map, and then build a map based only on the environmental information in the depth map. Open-source methods such as grid_map and orb_slam3 are used to generate a grid map and an ESDF map. Subsequent obstacle avoidance actions directly use the ESDF map to obtain obstacle location information.

[0105] In step S5, the airborne intelligent terminal weights the cluster mission waypoints of the cluster ground station and the position and speed information of the neighboring UAVs. After weighting, it uses the cluster model to make cluster decisions based on the information of the grid map and ESDF map to obtain the speed decision value.

[0106] The S5 process includes:

[0107] S51, establish cluster model;

[0108] The specific process of establishing the cluster model is as follows:

[0109] Using an undirected graph Represents a drone cluster with N individuals, where Assemble for drones, is the interaction edge between UAVs, where R max is the maximum sensing range of the UAV, d ij is the displacement vector of UAV i relative to j.

[0110] All drones use mature commercial quadrotors and autopilots that can provide speed controller functions. The neighbor set detected by drone i is defined as Each drone i in the swarm model follows the following motion model:

[0111]

[0112] where p i and v i is the position and velocity of UAV i, k is a control parameter greater than 0, v d,i It is the input of the UAV speed controller and also the speed decision variable, which satisfies the maximum speed limit of UAV i.

[0113] S51, using the ESDF map, obtains obstacle position information and obtains obstacle avoidance behavior as the first speed decision variable;

[0114] Obstacle avoidance behavior means that when each individual in the rotor cluster approaches an obstacle, it obtains the distance and direction of the nearest obstacle in the environment through perception methods under the action of the obstacle avoidance rules, and obtains the expected velocity vector required for obstacle avoidance.

[0115] Obstacle avoidance behavior is obtained as follows:

[0116] Obtain obstacle information from ESDF maps Where l is the number of obstacles obtained, α k is the confidence of obstacle information, d obs,k is the shortest vector pointing from UAV i to the surface of obstacle k, k∈l;

[0117] The position coordinates of drone i are p i (x i ,y i ,z i ), the obstacle position coordinates are p g (x g ,y g ,z g ), the relative position relationship between UAV i and the obstacle is With the current speed v of the drone i i Predict n time steps t later step Position p i,pred (t setp ), the relative position relationship between each predicted position and the nearest obstacle point is d i,pred (t setp );

[0118] like Figure 3 As shown, the predicted obstacle avoidance parameter R is introduced safe , that is, UAV i predicts that there will be obstacles less than R in a certain step length according to the current speed. safe , then the obstacle avoidance rule needs to be executed, R reach The threshold for the drone to determine whether it has reached the target area, that is, when the distance between the drone and the target center is Less than R reach When , it is considered that the target has been reached. avoid The distance threshold at which the drone determines that it needs to start avoiding obstacles, that is, when the distance d between the drone and the obstacle is obs,k Less than R avoid When the obstacle avoidance rule starts to work. avoid =R0+kv, R0 is set to 3 meters, k is 1; the following conditions are met

[0119]

[0120] The expected velocity vector generated by the obstacle avoidance rule is:

[0121]

[0122] If the above conditions are not met, the obstacle avoidance decision amount is 0. Under the action of this rule, the UAV will generate both the decision amount of moving away from the obstacle and the decision amount of moving parallel to the obstacle surface for the obstacle k, driving the individual to perform avoidance and exploration maneuvers respectively.

[0123] S52, using the neighboring drone position and speed information in S3, obtains the distance keeping behavior as the second speed decision quantity;

[0124] Distance-keeping behavior means that each individual in the rotor cluster obtains the position information of other individuals in the cluster through inter-machine communication under the action of the distance-keeping rule to maintain the clustered state, calculates the relative position vector with other individuals as input, and obtains the desired velocity vector to maintain the clustered state.

[0125] The way to obtain the distance behavior is:

[0126] like Figure 4 As shown, there are N individuals in the cluster, and the position coordinates of drone i are p i (x i ,y i ,z i ), and its relative position relationship with the j-th UAV j is d i,j =p i -p j , where the position of UAV j is obtained by visual observation, and its credibility is α nei,j , and 0≤α nei,j ≤1, according to the distance keeping rule, the expected velocity component generated by UAV j on UAV i is:

[0127]

[0128] where R align Represents the safe distance between drones. The distance between drone j and drone i is less than the safe distance R between drones.align When , the desired velocity components moving away from each other are generated, and vice versa, the desired velocity components moving towards each other are generated;

[0129] S53, using the cluster mission waypoints in the cluster ground station, obtains the mission behavior as the speed decision quantity three;

[0130] Mission behavior means that each individual in the rotor cluster obtains the target point position information released by the ground station as input under the action of the mission behavior rules to fly to the target waypoint, and obtains the expected speed vector to fly to the target waypoint.

[0131] The way to obtain task behavior is:

[0132] like Figure 5 As shown, the position coordinates of UAV i are p i (x i ,y i ,z i ), the target position coordinate is p g (x g ,y g ,z g ), the relative position relationship between UAV i and the target is d goal,i =p g -p i , according to the mission behavior rules, the expected velocity vector generated by UAV i is:

[0133]

[0134] in is a unit vector. When the distance d between UAV i and the target goal,i Less than R mission As the distance to the target gets closer, the expected speed gradually decreases. When the distance is greater than R mission When the desired speed is kept at V mmax , where R mission The size of is related to the number of individuals in the cluster N. k is the coefficient;

[0135] S54, weighting the speed decision quantity 2 and the speed decision quantity 3, with the weight being 1, to obtain the speed decision quantity 4;

[0136] S55, combining the speed decision quantity 4 with the speed decision quantity 1 to perform cluster decision making and obtain the speed decision quantity.

[0137] Therefore, the speed decision quantity is

[0138] v cmd =k1v obs +k2v align +k3v mission ;

[0139] Where k1, k2, and k3 represent the weight of speed decision quantity one, the weight of speed decision quantity two, and the weight of speed decision quantity three, respectively, and the sum of k2 and k3 is 1.

[0140] S6: The onboard intelligent terminal sends the speed decision value to the UAV autopilot, and the UAV autopilot controls the rotorcraft UAVs to perform swarm flight based on the speed decision value.

[0141] The optimization process for cluster flight in S6 is as follows:

[0142] Setting Φ obs represents the proportion of individuals in the cluster colliding with obstacles,

[0143]

[0144] where Θ(·) is the Heaviside step function, For p i (t) in symbolic form, when p i (t) When in an obstacle, Take positive value, otherwise take negative value;

[0145] Setting Φ col represents the proportion of collisions between individuals in the cluster,

[0146]

[0147] Setting Φ mis Indicates the completion of the cluster task, that is, the time it takes for the centroid to reach the target point and the number of surviving individuals.

[0148]

[0149] According to the above measurement indicators, a single-objective optimization function is constructed.

[0150]

[0151] Use open source tools such as MATLAB's GA tool to solve the algorithm and obtain the optimal control parameters.

[0152] S7, the rotorcraft drone feeds back flight status information to the onboard intelligent terminal.

[0153] Flight status information is fed back to the onboard intelligent terminal and serves as an important input to the swarm algorithm for swarm decision-making: the aircraft's current position is used in obstacle avoidance rules to calculate the two components parallel to and perpendicular to the obstacle; in distance keeping behavior, the aircraft's current position and estimated neighbor position information are used to calculate the decision value of the distance keeping item.

[0154] During the flight, the speed decision quantity of the cluster decision is sent to the UAV autopilot in real time at a frequency of 20hz. The autopilot calculates the motor speed control quantity based on the speed decision quantity and controls the UAV to perform cluster flight. Figure 6 Figure 7 shown.

[0155] The present invention also provides a purely visual distributed cluster flight emergence system for crowded environments, which adopts the above-mentioned purely visual distributed cluster flight emergence method for crowded environments, including:

[0156] A visual perception module for obtaining visible light images;

[0157] Cluster ground station, used to issue mission instructions;

[0158] Cluster decision module, used to process and extract information from visible light images and task instructions to obtain speed decision variables;

[0159] The unmanned driving module is used to receive and execute speed decision quantities to control the movement of the UAV; and the UAV body.

[0160] Therefore, the present invention employs the aforementioned purely visual distributed swarm flight emergence method and system for crowded environments, in which individual drones can acquire information about environmental obstacles and the locations of their neighbors through visual sensors. This addresses the issues of drone swarm systems that rely too heavily on inter-machine communication, are unable to cope with complex and crowded environments, and have low adaptability to these environments for individual drones.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A purely visual distributed swarm flight emergence method for crowded environments, characterized by: The following steps are involved: S1, the rotorcraft obtains the observed visible light image through the onboard depth camera; S2, obtains the environment depth map and RGB map by analyzing the image; S3, analyzes the RGB image, performs Kalman filtering on the location information of neighboring drones, and estimates the location and speed information of neighboring drones; S4, generating a raster map and ESDF map from the environment depth map; In step S5, the airborne intelligent terminal weights the cluster mission waypoints of the cluster ground station and the position and speed information of the neighboring UAVs. After weighting, it uses the cluster model to make cluster decisions based on the information of the grid map and the ESDF map to obtain the speed decision value. S6: The onboard intelligent terminal sends the speed decision value to the UAV autopilot, and the UAV autopilot controls the rotorcraft UAVs to perform swarm flight based on the speed decision value. S7, the rotorcraft feeds back flight status information to the onboard intelligent terminal; The specific steps of S5 are: S51, establish cluster model; S51, using the ESDF map, obtains obstacle position information and obtains obstacle avoidance behavior as the first speed decision variable; S52, using the neighboring drone position and speed information in S3, obtains the distance keeping behavior as the second speed decision quantity; S53, using the cluster mission waypoints in the cluster ground station, obtains the mission behavior as the speed decision quantity three; S54, weighting the speed decision quantity 2 and the speed decision quantity 3, with the weight being 1, to obtain the speed decision quantity 4; S55, combining the speed decision quantity 4 with the speed decision quantity 1 to perform cluster decision making and obtain the speed decision quantity; The specific process of establishing the cluster model is as follows: Using an undirected graph Indicated by the number of individuals A swarm of drones, including Assemble for drones, is the interaction edge between UAVs, where is the maximum sensing range of the drone, For drones Relative to The displacement vector of the drone The detected neighbor set of , each drone in the swarm model All follow the following motion model: ； in and For drones The position and velocity of is a control parameter greater than 0, It is the input quantity of the UAV speed controller and also the speed decision quantity, which satisfies the UAV Maximum speed limit; Obstacle avoidance behavior is obtained as follows: Obtain obstacle information from ESDF maps ,in is the number of obstacles obtained, , is the confidence of obstacle information, For drones Pointing to obstacles The shortest vector on the surface, ; drones The position coordinates are , the location coordinates of the obstacle are , drones The relative position relationship with the obstacle is , and drones Current speed After the prediction time steps Location , the relative position relationship between each predicted position and the nearest obstacle point is ; Introducing predicted obstacle avoidance parameters , drones According to the current speed, it is predicted that there will be obstacles within a certain distance in the future. , then the obstacle avoidance rules need to be implemented, The threshold for the drone to determine whether it has reached the target area. The distance threshold at which the drone determines it needs to start avoiding obstacles, where , Set to 3 meters, is 1; The following conditions are met: ； The expected velocity vector generated by the obstacle avoidance rule is: 。 2. The purely visual distributed cluster flight emergence method for crowded environments according to claim 1 is characterized by: In S2, the specific process of image parsing is as follows: first, the friendly drone in the image is identified, the coordinates and size of the friendly drone's identification box are obtained, and an RGB image is generated. Then, the coordinates of the neighboring individual images identified in the RGB image are removed from the image to obtain the environment depth map.

3. The purely visual distributed cluster flight emergence method for crowded environments according to claim 2 is characterized by: The specific steps of S3 are: S31, use yolov8 to identify individuals, and then perform data association on the identification results to distinguish different neighboring individuals in the field of view; S32, obtain the pixel coordinates of the center point of the observed neighbor drone identification frame , and recognition box parameters ,in is the detection box area, is the aspect ratio of the detection frame; S33, project the pixel coordinates and recognition frame parameters into the image to obtain the distance information of the observed neighboring drone , the local coordinates of the neighbor UAV in the inertial coordinate system are ； in is the camera internal parameter, is the rotation matrix of the camera relative to the world system; S34, performing Kalman filtering on the observed neighbor drone position information to estimate the neighbor drone position and speed information.

4. The purely visual distributed cluster flight emergence method for crowded environments according to claim 3 is characterized by: The specific process of S4 is: open source the environment depth map using open source methods to generate raster maps and ESDF maps.

5. The purely visual distributed cluster flight emergence method for crowded environments according to claim 1 is characterized by: The way to obtain the distance behavior is: Assume that there are Individuals, The position coordinates are , and the drones The relative position relationship is ,in The position is obtained by visual observation, and its reliability is ,and , according to the distance keeping rule, right The resulting desired velocity component is: ； in Indicates the safe distance between drones. arrive The distance is less than the safe distance between machines , produces the desired velocity components moving away from each other, whereas the opposite produces the desired velocity components moving toward each other; The way to obtain task behavior is: The position coordinates are , the target position coordinates are , The relative position relationship with the target is , according to the task behavior rules, The resulting desired velocity vector is: ； in is a unit vector; when Distance to target Less than , the closer to the target, the lower the expected speed. When the distance is greater than When the expected speed remains ,in The size of the cluster and the number of individuals related, , is the coefficient; The speed decision quantity is: ； in They represent the weight of speed decision quantity 1, the weight of speed decision quantity 2 and the weight of speed decision quantity 3 respectively, where and Hewei .

6. The purely visual distributed cluster flight emergence method for crowded environments according to claim 5 is characterized by: The optimization process for cluster flight in S6 is as follows: set up represents the proportion of individuals in the cluster colliding with obstacles, ； in is the Heaviside step function, for The symbolic form of When in an obstacle, Take positive value, otherwise take negative value; set up represents the proportion of collisions between individuals in the cluster, ； set up Indicates the completion of the cluster task, the time it takes for the centroid to reach the target point, and the number of surviving individuals. ； According to the above measurement indicators, a single-objective optimization function is constructed. ； Use open source tools to solve the algorithm and obtain the optimal control parameters.

7. A purely visual distributed swarm flight emergence system for crowded environments, characterized by: The pure visual distributed cluster flight emergence method for crowded environments according to any one of claims 1 to 6 is adopted. include, A visual perception module for obtaining visible light images; Cluster ground station, used to issue mission instructions; Cluster decision module, used to process and extract information from visible light images and task instructions to obtain speed decision variables; The unmanned driving module is used to receive and execute speed decision quantities to control the movement of the UAV; and the UAV body.

Images