Large-scale unmanned aerial vehicle marshalling method for emergency fire rescue task

Abstract

The invention discloses a large-scale unmanned aerial vehicle marshalling method for an emergency fire rescue task, and the method comprises the steps: carrying out the clustering analysis of task points of a rescue region through an FCM algorithm, determining the membership degree of a flight task, and generating a task region; on the basis of real-time environment change of a fire scene, a dynamic membership updating task allocation state is realized through a visual detection algorithm of fusing a convolutional neural network with an FCM; meanwhile, through 3DT route planning optimization based on the laser radar technology, a 3D model recombination map model is established, linear optimization is carried out by introducing time parameters, and optimal path planning is provided for the unmanned aerial vehicle in real time. In the rescue process, data are automatically transmitted in a segmented mode through the slicing technology of the 5G network, an autonomous Mesh centerless network topology structure is implemented when signals are intense, real-time signal optimization is conducted through a repeater of the high-altitude unmanned aerial vehicle, and high efficiency and flexibility of rescue are ensured.

Description

Technical Field

[0001] The present invention relates to the fields of emergency rescue and intelligent control, and in particular to a large-scale unmanned aerial vehicle grouping method for emergency fire rescue tasks. Background Art

[0002] Emergency fire rescue drones are intelligent drones designed for fire rescue. With the acceleration of urbanization and the improvement of industrialization, the frequency and scale of fire accidents are also increasing year by year. Traditional fire rescue mainly relies on the coordinated operations of ground firefighting teams and equipment, but this method has problems such as slow response speed, low rescue efficiency, and high personnel risk. In recent years, the rapid development of drone technology has provided new innovative rescue ideas for fire rescue. Drones have gradually emerged in emergency rescue due to their advantages such as flexibility, wide field of view, rapid response, and high safety. However, the capabilities of a single drone are limited and it is difficult to cope with fire rescue tasks in large-scale and complex environments. Therefore, it is particularly important to efficiently allocate drone rescue tasks and reliably optimize the range, as well as adaptive real-time dynamic adjustment control systems.

[0003] At present, although the existing large-scale drone grouping technology can achieve fast response speed, its application in fire rescue still faces the problems of real-time control and rapid response. How to organize and manage large-scale drone groups through scientific and reasonable grouping methods and efficient control strategies has become a technical problem that needs to be solved urgently. First, the task allocation and path planning algorithms of drone grouping are complex, and it is necessary to ensure real-time and safety while ensuring efficiency. Secondly, the collaborative operation of multiple drones requires an efficient and stable communication mechanism to achieve real-time information sharing and task coordination, and automatically update the communication transmission speed when disturbed in a complex fire scene environment. Furthermore, at a dynamically changing fire scene, the tasks and paths of drone grouping need to be continuously adjusted according to real-time data to ensure the efficiency and flexibility of rescue. Summary of the invention

[0004] The purpose of the present invention is to solve the problems that UAVs are unable to effectively identify mission targets in complex fire environments, optimize tracking tasks for real-time response of rescue mission allocation, and the UAVs' information transmission efficiency response speed and real-time communication interference, thereby proposing a large-scale UAV formation method for emergency fire rescue missions.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A large-scale UAV formation method for emergency fire rescue missions includes the following steps:

[0007] Step 1: The visual monitoring sensor on the rescue drone collects multi-directional images of the fire scene around the rescue drone;

[0008] Step 2: The image is transmitted to the signal server of the relay high-altitude UAV. The signal server performs image preprocessing and information transmission, and uses the adaptive fusion fuzzy filtering method to denoise and enhance the image features, obtain the horizontal and vertical gradients of the enhanced image, and transmit it to the ground command station.

[0009] Step 3: The ground command station uses the remote control video terminal to enhance the fire feature information in the image and apply the FCM algorithm to cluster the fire disaster types in the fire scene. The adaptive fuzzy index is introduced to dynamically adjust the fuzziness in the clustering process of the FCM algorithm to accurately identify the fire type in the fire scene. At the same time, the task membership is determined, and the rescue task allocation of the rescue drone is determined based on the CNN feature extraction;

[0010] Step 4. Transmit the rescue mission assignment information to the rescue drone through the signal server of the relay high-altitude drone; at the same time, the signal server of the relay high-altitude drone transmits the established 3DT trajectory optimization path planning information to the corresponding rescue drone. After receiving the path planning information, the rescue drone quickly transmits it to all communicating rescue drones in the surrounding network. The relay high-altitude drone and the ground command station continuously perform on-site real-time information correction and feedback. The ground command station uses real-time data to fuse three-dimensional map construction data, historical training rescue drone historical grouping rescue planning, rescue time, and rescue speed data for comparison, and dynamically controls the group of rescue drones to perform rescue missions through the relay controller in the signal server.

[0011] Furthermore, information is transmitted at high speed between the signal server of the relay high-altitude drone and the controller of the rescue drone, the ground command station, the visual monitoring sensor, the relay controller, and the rescue drone based on the 5G-Mesh distributed collaborative network communication framework.

[0012] Furthermore, in step 2, image preprocessing and information transmission include the following steps:

[0013] S21. Feature fusion is performed on the video sensor data w1 and the thermal imaging sensor data w2 acquired by the visual monitoring sensor, and the eigenvalue matrix Q is obtained based on the eigenvalues. By comparing the features of the images, the dimensional feature means μ1 and μ2 of the horizontal axis and the vertical axis of the image and the thermal imaging feature mean μ3 are obtained;

[0014]

[0015] Among them, y is the total number of pixels in the vertical direction, x is the total number of pixels in the horizontal direction, and q 1k is the gray value of the kth pixel in the horizontal direction, q2k is the gray value of the kth pixel in the vertical direction, q 3k is the temperature value of the kth pixel in the vertical direction, q 4k is the temperature value of the kth pixel in the horizontal direction;

[0016] S22. Calculate the covariance COT of image i through the eigenvalue matrix i , obtain the k×1 eigenvalue matrix of the horizontal axis and the k×1 eigenvalue matrix of the vertical axis based on the image eigenvalue, calculate the covariance matrix COT1 in the horizontal direction and the covariance matrix COT2 in the vertical direction; calculate the similarity value SIM of image i based on the Mahalanobis distance of the image i :

[0017]

[0018] Among them, f i+1 is the feature vector of the previous frame image, f i is the feature vector of the subsequent frame image;

[0019] S23. Similarity value SIM of adjacent time frame images i Constantly changing, based on the real-time similarity value SIM i , and calculate the mean square error with the true value:

[0020]

[0021] Among them, y0 represents the average image similarity, and n represents the total number of frames during calculation;

[0022] Through the mean square error, using the gradient descent method, the feature and denoising coefficient gradient descent formula is obtained;

[0023]

[0024] Among them, β is the image learning rate, α is the denoising coefficient;

[0025] The feature set Q and the denoising coefficient α are continuously iterated. When the mean square error E is less than the iteration threshold, the updated feature Q is ω and the updated denoising coefficient α ω The central processor in the server that relays the high-altitude drone signal updates the image enhancement parameters;

[0026] S24. Update the feature Q ω and the updated denoising coefficient α ω After being transmitted to the central processor, the gradient descent method is used to continuously iterate and adaptively adjust the real-time image learning rate β until the image learning rate β is maximized, and the final image enhancement and denoising are performed to obtain the preprocessed enhanced image.

[0027] Furthermore, the rescue drone also includes a roadbed detector and an anti-collision radar; based on the images collected by the visual monitoring sensor and the roadbed detector, the image processor uses a 3D model reconstruction algorithm to construct a three-dimensional map of the fire scene.

[0028] Furthermore, in step 3, the fire feature information in the enhanced image is used to cluster the fire damage types in the fire scene using the FCM algorithm, including the following steps:

[0029] S31. Based on the enhanced video data, the captured key image is segmented by the FCM algorithm to obtain h cluster center point sets N:

[0030] N={N1,N2,N3,...,N h};

[0031] S32. For each pixel point in the segmented image grid, its membership value L is calculated, and the cluster center point is continuously updated according to the membership value L. The cluster fuzzy coefficient Ω is dynamically adjusted through the texture features in the key frame:

[0032]

[0033] Among them, L mo is the membership degree of the oth pixel, c o is the coordinate of the oth pixel point, p o is the coordinate of the pixel point of the oth cluster center, c k is the kth pixel cluster coordinate, p k is the k-th center pixel cluster coordinate, k is the number of clusters, d(c o ,p o ) represents the pixel coordinate c o and the cluster center pixel coordinates p o The distance, d(c k ,p k ) is the pixel cluster coordinate c k and the cluster center pixel cluster coordinates p k distance;

[0034] Continue to iterate until convergence to each cluster center threshold;

[0035] S33. Based on the horizontal and vertical grayscale values ​​of the pixels of the segmented image, the corresponding convolution operator g is calculated x , g y , obtain the horizontal and vertical gradient information of the image, and use g x , g y Construct the directional gradient tensor matrix T(i,j) of each pixel point for the local image block;

[0036] U x (i,j)=g x ·U

[0037] U y (i,j)=g y ·U

[0038]

[0039] Among them, U x (i, j) is the horizontal gradient value, U y (i, j) is the vertical gradient value, U is the grayscale value of the image pixel at position (i, j); the local gradient direction information and texture features are obtained through the tensor matrix T(i, j);

[0040] S34. Based on the obtained local gradient direction information and texture features, through convolution image feature recognition, the FCM algorithm is continuously applied for iteration, and finally the real-time image recognition object results are obtained through convolution image features.

[0041] Furthermore, in step 4, the 3DT trajectory optimization path planning information includes the following establishment steps:

[0042] S41. Obtain the gradient value change between the three-dimensional coordinate system of the rescue drone and the three-dimensional coordinate system of the visual monitoring sensor, and obtain the rotation matrix R of the yaw angle ψ based on the gradient value change. ψ , the rotation matrix R of the pitch angle θ θ , the rotation matrix R of the roll angle φ φ

[0043]

[0044] Among them, ψ is the yaw angle of the UAV, θ is the pitch angle of the UAV, and φ is the roll angle of the UAV;

[0045] The coordinate rotation matrix R of the rescue drone is calculated:

[0046] R=R ψ ·R θ ·R φ ;

[0047] S42. Based on the rotation matrix R, the coordinate system parameter changes are sampled to establish a three-dimensional map of the rescue drone flight, and after obtaining the three-dimensional map information and the rescue drone rotation matrix R in real time through the ground command station, the three-dimensional map information is sent to the signal server of the relay high-altitude drone through the 5G-Mesh distributed collaborative network communication framework;

[0048] S43. Fusing the three-dimensional map and the real-time map collected by the visual monitoring sensor to generate a local fusion map, looping the above steps of forming the local fusion map to form a global fusion map for the rescue drone to perform rescue work;

[0049] S44. The central processor in the server that relays the signal of the high-altitude drone collects the power parameters and speed v of all the rescue drones p involved in the rescue work 1p , acceleration a p , Weight G p , rotor rotation speed v 2p , get the initial constraint matrix Y

[0050]

[0051] Establish the initial cost matrix DJ

[0052]

[0053] YS=x pq (DJ+ρ*max(0,t pq ))

[0054] Among them, λ1, λ2, λ3, λ4 are variable matrix coefficients that are dynamically adjusted according to the gradient value, YS is the power constraint matrix of the rescue drone, and x pq is the membership degree of the pth rescue drone to the qth rescue mission, DJ is the initial cost matrix of the rescue drone, ρ is the time constraint coefficient of the rescue mission, t pq is the rescue flight time of the qth rescue mission of the pth rescue drone;

[0055] Time constraints are introduced to calculate and obtain different applicable rescue flight plans for different UAVs.

[0056] Furthermore, the steps for establishing the 5G-Mesh distributed collaborative network communication framework are as follows:

[0057] S51. The WIFI6 module in the signal server of the high-altitude drone receives the 5G signal source from the nearest ground base station, amplifies the 5G signal through the signal amplifier, and the anti-interference multi-antenna automatically deploys to resist the interference signal source signal;

[0058] S52. The rescue drone receives the expanded 5G signal transmitted by the signal server that relays the high-altitude drone. At the same time, using the 5G slicing technology, the 5G signals of adjacent rescue drones in each rescue drone group composed of rescue drones are in different 5G communication segments;

[0059] S53. The communication signal node of the rescue drone automatically links to the surrounding adjacent signal nodes to form an autonomous decentralized network topology structure;

[0060] S54. The signal receiving node working in the rescue drone senses the signal interference and transmission speed in real time. When the signal interference and transmission speed are lower than the set threshold, multi-hop communication is automatically formed between multiple working rescue drones.

[0061] Furthermore, in step 3, the rescue mission allocation of the rescue drone is determined, including the following steps:

[0062] S61. The signal server relaying the high-altitude UAV obtains the image feature recognition information of the convolutional network based on FCM, and obtains the historical data in real time for the identified fire type;

[0063] S62. Associating the membership degree to perform task allocation, according to the real-time identified fire type, using the FCM algorithm to obtain the membership degree of the current fire data, and calculating the applicable task cost matrix of the rescue drone:

[0064] FP = x pq (DJ+ρ*max(0,t pq )+γ*(1-L mq ));

[0065] Among them, FP is the rescue mission cost function, x pq is the membership degree of the pth rescue drone to the qth rescue mission, DJ is the initial cost matrix of the rescue drone, ρ is the time constraint coefficient of the rescue mission, t pq is the rescue flight time of the qth rescue mission of the pth rescue drone, γ is the membership adjustment coefficient, L mq is the membership degree of rescue mission q;

[0066] S63. Use the gradient optimization planning method to optimize task allocation, the goal is to minimize the total cost,

[0067]

[0068] Where b is the number of rescue drones, a is the number of rescue missions, and FP pq is the cost function value of the qth rescue mission of p rescue drones.

[0069] Furthermore, the rescue drone is a low-altitude multi-rotor drone.

[0070] Beneficial effects:

[0071] The present invention is a large-scale UAV formation method for emergency fire rescue tasks. It uses low-altitude lightweight UAVs to collect real-time images and videos of the fire scene, recognizes and analyzes the gradient features and texture features of the images, strengthens the image and video features and calculates the clustering center. Based on fuzzy mean clustering, the image features are recognized by the final clustering center gradient features and texture features, and the recognition results are evaluated to effectively identify the fire type. In addition, the task allocation is dynamically adjusted according to the cluster membership. At the same time, based on the relay server of the relay high-altitude UAV, the 5G communication of the rescue UAV is sliced ​​and optimized and monitored in real time, and the update information of the ground command station is responded to in real time to ensure the efficiency and flexibility of the rescue. DETAILED DESCRIPTION

[0072] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0073] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside" and "outside" are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the body or component referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention.

[0074] In order to make the technical means for realizing the present invention, the creative features, the objectives and effects to be achieved easily understood, specific embodiments are described and illustrated below.

[0075] Example 1

[0076] This embodiment discloses a large-scale UAV formation method for emergency fire rescue tasks, including the following steps:

[0077] Step 1: The visual monitoring sensor installed on the rescue drone collects multi-directional images of the fire in the high-rise residential area around the rescue drone. The rescue drone uses a low-altitude multi-rotor drone to rescue the fire scene.

[0078] Step 2: The image is transmitted to the signal server of the relay high-altitude UAV through the 5G-Mesh distributed collaborative network communication module. The signal server of the relay high-altitude UAV performs image preprocessing and information transmission, and performs image feature denoising and enhancement processing through the adaptive fusion fuzzy filtering method. The obtained enhanced image has a similarity of more than 90% with the source image, and the horizontal and vertical gradients of the enhanced image are obtained and transmitted to the ground command station.

[0079] Step 3: The ground command station uses the remote control video terminal to enhance the fire feature information in the image and apply the FCM algorithm to cluster the fire disaster types in the fire scene, that is, cluster the pixel grayscale values ​​of the real-time scene image and video, and introduce an adaptive fuzzy index, which gradually increases from the fuzziness level 1. The fuzziness level in the clustering process of the FCM algorithm is dynamically adjusted according to the cluster center point threshold credibility, and the fire type of the fire scene is accurately identified. At the same time, the task membership is determined, and the most relevant high-rise residential fire identification parameters are extracted based on CNN feature extraction. At this time, the feature value similarity is greater than 70%, and the confidence is greater than 90%. According to the high-rise residential fire type matching of the feature value, the precise rescue task allocation of the rescue drone is realized.

[0080] Step 4. The rescue mission allocation information is transmitted to the rescue drone through the signal server of the relay high-altitude drone. At the same time, the signal server of the relay high-altitude drone transmits the established 3DT trajectory optimization path planning information to the rescue drone corresponding to the rescue mission, and dynamically adjusts the trajectory of the rescue drone according to the degree of affiliation. After receiving the path planning information, the rescue drone will quickly transmit it with all communication rescue drones in the surrounding 5G-Mesh network. The relay high-altitude drone and the ground command station continuously perform on-site real-time information correction and feedback. The ground command station will use real-time data to fuse the three-dimensional map construction data and the historical training of the rescue drone to compare the historical grouping rescue planning, rescue time, and rescue speed data. The relay controller in the signal server dynamically controls the group of rescue drones to perform efficient and rapid rescue mission grouping tasks.

[0081] The relay high-altitude drone signal server and the controller that controls the rescue drone, the ground command station, the visual monitoring sensor, the relay controller, and the rescue drone transmit information at high speed based on the 5G-Mesh distributed collaborative network communication framework.

[0082] Visual monitoring sensors are used to provide real-time image observation information of low-altitude lightweight drones.

[0083] In addition to visual monitoring sensors, low-altitude multi-rotor drones also include roadbed detectors and anti-collision radars. Based on the visual monitoring sensors, roadbed detectors, and image processors, a 3D model reconstruction algorithm is used to construct a three-dimensional rescue map of the fire scene where the drone is working.

[0084] Among them, image preprocessing and information transmission during high-rise fire rescue include the following steps:

[0085] S21. Perform feature fusion on the video sensor data w1 and the thermal imaging sensor data w2, map all feature combinations to the eigenvalue matrix Q, and obtain the dimensional feature mean μ1 and μ2 of the horizontal and vertical axes of the image and the thermal imaging feature mean μ3 by comparing the features of the image;

[0086]

[0087] Among them, y is the total number of pixels in the vertical direction, x is the total number of pixels in the horizontal direction, and q 1k is the gray value of the kth pixel in the horizontal direction, q 2k is the gray value of the kth pixel in the vertical direction, q 3k is the temperature value of the kth pixel in the vertical direction, q 4k is the temperature value of the kth pixel in the horizontal direction.

[0088] The visual monitoring sensor used in this embodiment has both video transmission function and thermal imaging function, and can collect video sensor data w1 and thermal imaging sensor data w2 at the same time.

[0089] S22. Eigenvalue matrix obtained based on eigenvalues

[0090] r represents the number of counts;

[0091] Calculate the covariance matrix COT of image i through the eigenvalue matrix i ;

[0092] Obtain a k×1 eigenvalue matrix of the horizontal axis and a k×1 eigenvalue matrix of the vertical axis based on the image eigenvalues, and calculate the covariance matrix COT1 in the horizontal direction and the covariance matrix COT2 in the vertical direction;

[0093]

[0094] The Mahalanobis distance correlation of the image is calculated by the eigenvalues ​​of different dimensions to obtain the similarity value SIM of image i i :

[0095]

[0096] Among them, i is the image category, f i+1 is the feature vector of the previous frame image i, f i is the feature vector of the subsequent frame image i.

[0097] S23. Similarity value SIM of adjacent time frame images i i Constantly changing, based on the real-time similarity value SIM i , and calculate the mean square error with the true value;

[0098]

[0099] Where E represents the mean square error between the actual value and the average value, y0 represents the average image similarity, and n represents the total number of frames during the calculation time;

[0100] Through the mean square error, using the gradient descent method, we get the characteristic parameter gradient descent formula Q ω , α ω , and continuously optimize the results based on gradient descent.

[0101] S24. Update the feature Q ω and the updated denoising coefficient α ω After being transmitted to the central processor, the gradient descent method is used to continuously iterate and adaptively adjust the real-time image learning rate β until the image learning rate β is maximized, and the final image enhancement and denoising are performed to obtain the preprocessed enhanced image.

[0102] Among them, the FCM fusion calculation method used in the central processor module of the relay high-altitude drone signal server to cluster the fire disaster types includes the following steps:

[0103] S31. Based on the enhanced video data, obtain h cluster center point sets N: N = {N1, N2, N3, ..., N h}, different cluster centers are a category denoted by N h ;

[0104] S32. For each pixel point in the segmented image grid, calculate its membership value, continuously update the cluster center point according to the membership value, introduce the dynamic adjustment clustering fuzzy coefficient Ω through the texture features in the key frames around the fire source point, set the number of cycles to 600, and cyclically calculate and optimize the clustering results:

[0105]

[0106] Among them, L mo is the membership degree of the oth pixel, c o is the coordinate of the oth pixel point, p o is the coordinate of the pixel point of the oth cluster center, c k is the kth pixel cluster coordinate, p k is the k-th center pixel cluster coordinate, k is the number of clusters, d(c o ,p o ) represents the pixel coordinate c o and the cluster center pixel coordinates p o The distance, d(c k ,p k ) is the pixel cluster coordinate c k and the cluster center pixel cluster coordinates p k distance.

[0107] Iterate continuously until it converges to the cluster center threshold of high-rise residential fire type images, with a deviation of no more than 15%; classify the segmented images based on the cluster center threshold, and transmit them directly to the image processor in different transmission network segments through 5G slicing technology;

[0108] S33. Based on the horizontal and vertical grayscale values ​​of the pixels of the segmented image around the fire source, and by calculating the convolution operator g x , g y , obtain the horizontal and vertical gradient information of the image, and construct the directional gradient tensor matrix T(i,j) for the local image block;

[0109] U x (i,j)=g x ·U

[0110] U y (i,j)=g y ·U

[0111]

[0112] U x (i, j) is the horizontal gradient value, U y (i, j) is the vertical gradient value, U is the grayscale value of the image pixel at position (i, j); the local gradient direction information and texture features are obtained through the tensor matrix T(i, j);

[0113] S34. Based on the obtained local gradient direction information and texture features, through convolution image feature recognition, the FCM algorithm is continuously applied for iteration, and finally the real-time image recognition object results are obtained through convolution image features.

[0114] Finally, the high-rise residential feature data with a confidence level higher than 90% is passed into the convolution layer.

[0115] The establishment of a three-dimensional map of high-rise residential rescue work and the 3DT trajectory optimization path planning based on this map include the following steps:

[0116] S41. Obtain the gradient value changes tb1, tb2, tb3 between the coordinate system parameters of the rescue drone's three-dimensional coordinate system and the three-dimensional coordinate system of the visual monitoring sensor, and obtain the rotation matrix of the yaw angle based on the gradient value changes Rotation matrix of pitch angle Rotation matrix of roll angle The rotation matrix of the rescue drone is calculated to be R = R ψ ·R θ ·R φ ;

[0117] Among them, ψ is the yaw angle of the UAV, θ is the pitch angle of the UAV, and φ is the roll angle of the UAV;

[0118] S42. Based on the rotation matrix R, the changes in the coordinate system parameters are sampled, and a three-dimensional map of the rescue drone flight based on lidar is established. After the three-dimensional map information and the rescue drone rotation matrix R are obtained in real time through the ground command station, the three-dimensional map information is sent to the relay high-altitude drone signal server through the 5G-Mesh distributed collaborative network communication framework; the lidar is a built-in configuration for the rescue drone.

[0119] S43. The three-dimensional map and the real-time map collected by the visual monitoring sensor are fused in the relay high-altitude drone signal server to generate a local fusion map, and the above local fusion map formation steps are repeated to form a global fusion map for the rescue drone to perform rescue work;

[0120] S44. The central processor in the high-altitude drone signal relay server will collect the power parameters MD of all rescue drones p involved in the rescue work, including the speed v 1p , acceleration a p , Weight G p , rotor rotation speed v 2p , get the initial constraint matrix Y:

[0121]

[0122] Establish the initial cost matrix DJ, dynamically adjust the variable matrix coefficients λ1, λ2, λ3, λ4 according to the gradient value changes on different floors, and select 2 times the initial value coefficient for high-rise residential buildings to ensure the rescue capability of drones;

[0123]

[0124] YS=x pq (2*DJ+ρ*max(0,t pq ))

[0125] Among them, YS is the power constraint matrix of the rescue UAV, x pq is the membership degree of the pth rescue drone to the qth rescue mission, DJ is the initial cost matrix of the rescue drone, ρ is the time constraint coefficient of the rescue mission, t pq is the rescue flight time of the qth rescue mission of the pth rescue drone. The cost matrix DJ is obtained according to the power parameters of the drone. According to the cost matrix, the constraint matrix YS is calculated by the fire type membership, and has a time constraint part, which is equivalent to one constraint for power, one constraint for time, and one constraint for membership. In order to avoid invalid parameters, the rescue mission time constraint ρ is introduced. Finally, different applicable rescue flight plans for different drones are calculated.

[0126] The steps to establish the 5G-Mesh distributed collaborative network communication framework are as follows:

[0127] S51. The WIFI6 module in the high-altitude drone signal server receives the 5G signal source from the nearest ground base station, and amplifies the 5G signal through the signal amplifier of the repeater. At the same time, the anti-interference multi-antenna is automatically deployed to resist the interference signal source signal;

[0128] S52. The rescue drone receives the expanded 5G signal transmitted by the relay high-altitude drone signal server. At the same time, using the 5G slicing technology, the 5G signals of adjacent rescue drones in each rescue drone group composed of rescue drones are in different 5G communication segments;

[0129] S53. The communication signal node of the rescue drone automatically links to the surrounding adjacent signal nodes to form an autonomous decentralized Mesh network topology structure;

[0130] S54. The signal receiving nodes working in the rescue drones sense the signal interference and transmission speed in real time. When the signal interference and transmission speed are lower than the set threshold, such as when the signal interference speed is greater than 50ms, the upload transmission speed is lower than 7Mbps, and the download transmission speed is lower than 3Mbps, multi-hop communication is automatically formed between multiple working rescue drones to avoid interference in signal transmission.

[0131] Among them, the realization of accurate rescue mission allocation and dynamic mission update system of drones includes the following steps:

[0132] S61. Based on the power parameters MD and flight planning parameters YS classified by the relay high-altitude UAV signal server, obtain the convolutional network image feature recognition information based on FCM, and obtain historical data in real time to assist in the allocation of rescue tasks for the identified fire type;

[0133] S62. According to the real-time identification of the fire type, the FCM algorithm is used to obtain the membership of the current fire data, and the applicable mission cost matrix of the UAV is calculated, including power parameters, flight planning parameters and membership

[0134] FP = x pq (DJ+ρ*max(0,t pq )+γ*(1-L mq )), at this time, the membership around the fire source takes the maximum value, and the gradient optimization planning method is used Optimize task allocation.

[0135] Among them, FP is the rescue mission cost function, x pqis the membership degree of the pth rescue drone to the qth rescue mission, DJ is the initial cost matrix of the rescue drone, ρ is the time constraint coefficient of the rescue mission, t pq is the rescue flight time of the qth rescue mission of the pth rescue drone, γ is the membership adjustment coefficient, L mq is the membership degree of rescue mission q, b is the number of rescue drones, a is the number of rescue missions, FP pq is the cost function value of the qth rescue mission of p rescue drones.

[0136] Example 2

[0137] Image preprocessing and information transmission are performed for fire rescue in a building of a hazardous materials distribution point similar to a chemical plant area. The specific steps are as follows:

[0138] S221. The global image data of the fire scene captured by the visual monitoring sensor of the low-altitude multi-rotor drone is transmitted to the signal server of the relay high-altitude drone through the 5G-Mesh distributed collaborative network communication module. The global image data includes the data w1 of the video sensor and the data w2 of the thermal imaging sensor;

[0139] S222. The thermal imaging sensor data w2 cluster points are preferentially transmitted to the eigenvalue matrix Q for feature fusion, and the mean values ​​of multiple thermal imaging features are obtained by comparing the features of the image;

[0140]

[0141] Get the image similarity value of thermal imaging

[0142] S223. The data w1 of the video sensor is transferred into the eigenvalue matrix Q, and the features of the image at the fire source point are quickly locked and compared, including the dimensional feature mean μ1 and μ2 of the horizontal and vertical axes of the image, and the image similarity value SIM is updated. i At this time, the chemical type of the fire source is verified by gradient descent through the mean square error;

[0143] Among them, based on the verified fire source chemical types, the central processor uses the FCM fusion calculation method to cluster the fire disaster types, including the following steps:

[0144] S231. Obtain the preliminary chemical type after fusion calculation, and on this basis, perform key image recognition on the key image through the FCM algorithm, intercept the key image for image segmentation, and obtain further image texture features through the segmented grid map;

[0145] S232. For each pixel point in the segmented grid map, select the grid map that meets the characteristics of the verified chemical and calculate its membership value The cluster center point is continuously updated according to the membership value, and the iteration is continued until it converges to the cluster center threshold of the fire type image;

[0146] S233. Update the clustering results of chemical feature values ​​through FCM clustering, verify them using CNN convolutional image features, and finally retain the chemical feature data with a confidence level higher than 90%, and output the final clustering results.

[0147] The steps of establishing a three-dimensional map for rescue work and optimizing the path planning of 3DT trajectories based on this map are as follows:

[0148] S241. Gradient value change between the coordinate system parameters (x, y, z) of the UAV flight three-dimensional coordinate system and the coordinate system parameters (s, r, h) of the visual monitoring sensor three-dimensional coordinate system Get the rotation matrix of the yaw angle Rotation matrix of pitch angle Rotation matrix of roll angle The rotation matrix of the rescue drone is calculated to be R = R ψ ·R θ ·R φ ; Among them, ψ is the yaw angle of the UAV, θ is the pitch angle of the UAV, and φ is the roll angle of the UAV;

[0149] S242. Based on the rotation matrix R of the rescue drone, the change of the coordinate system parameters is sampled, and a three-dimensional map of the rescue drone flight based on the lidar is established. After obtaining the information in real time through the ground command station, the three-dimensional map information is sent to the relay high-altitude drone signal server through the 5G-Mesh distributed collaborative network communication framework;

[0150] S243. At the same time, the central processor module in the high-altitude drone signal relay server obtains the parameter speed v of all rescue drones involved in the rescue work 1p , acceleration a p , Weight G p , rotor rotation speed v 2p , get the initial constraint matrix

[0151]

[0152] Establishing the initial cost matrix of the rescue drone

[0153]

[0154] Among them, λ1, λ2, λ3, λ4 are variable matrix coefficients that are dynamically adjusted according to the gradient value;

[0155] Selecting 2 times the initial value coefficient and introducing the time constraint ρ, we can get the rescue flight planning for different rescue drones:

[0156] YS=x pq (2*DJ+ρ*max(0,t pq )),

[0157] Among them, YS is the power constraint matrix of the rescue UAV, x pq is the membership degree of the pth rescue drone to the qth rescue mission, ρ is the time constraint coefficient of the rescue mission, t pq is the rescue flight time of the qth rescue mission of the pth rescue drone.

[0158] At this time, for the rescue work of key chemicals, the cost matrix DJ parameters λ1, λ2, λ3, and λ4 are maximized and rescue is given priority.

[0159] The image transmission of the 5G-Mesh distributed collaborative network communication module includes the following steps:

[0160] S251. The high-altitude drone signal relay server uses 5G slicing technology. In each rescue drone group, the 5G communication signals of adjacent rescue drones are in different 5G communication segments;

[0161] S252. The communication signal nodes of the drones involved in the rescue work will automatically link to the surrounding adjacent signal nodes to form an autonomous decentralized network topology structure and a small Mesh network group;

[0162] S253. When the signal interference speed is greater than 30ms, the upload transmission speed is lower than 8Mbps, and the download transmission speed is lower than 4Mbps, multi-hop communication is automatically formed between multiple working drones to avoid interference in signal transmission.

[0163] The following steps are involved in achieving accurate rescue mission allocation and dynamic mission update for drones:

[0164] S261. After the high-altitude drone signal relay server receives the cost matrix DJ and the flight planning parameter information YS, it marks the maximum cost matrix parameters for the rescue work of key chemicals;

[0165] S262. Obtain image feature recognition information based on FCM convolutional network, and use low-altitude lightweight monitoring drones to obtain thermal imaging data in real time for the identified fire type, and assist rescue drones in determining mission completion;

[0166] S263. Associating the membership to perform task assignment, marking the task completion according to the real-time identified fire type, using the FCM algorithm to obtain the membership of the current fire data, and calculating the applicable task cost matrix of the UAV, including power parameters, flight planning parameters and membership. The task completion membership threshold is 0.1. If the task completion membership is greater than 0.1, then return to the rescue task assignment and continue the rescue work:

[0167] FP = x pq (DJ+ρ*max(0,t pq )+γ*(1-L mq ))

[0168] S264. Using Gradient Optimization Planning Method to Optimize Task Allocation

[0169] Among them, FP is the rescue mission cost function, γ is the membership adjustment coefficient, and L mq is the membership degree of rescue mission q, b is the number of rescue drones, a is the number of rescue missions, FP pq is the cost function value of the qth rescue mission of p rescue drones.

[0170] Assign tasks based on the optimization results, monitor the status of the drone and the progress of the mission in real time, and dynamically adjust the task allocation strategy to cope with environmental changes and mission requirements.

[0171] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Without departing from the spirit and essence of the present invention, technicians familiar with the field can make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A large-scale UAV formation method for emergency fire rescue missions, characterized in that: The steps include: Step 1: The visual monitoring sensor on the rescue drone collects multi-directional images of the fire scene around the rescue drone; Step 2: The image is transmitted to the signal server of the relay high-altitude UAV. The signal server performs image preprocessing and information transmission, and uses the adaptive fusion fuzzy filtering method to denoise and enhance the image features, obtain the horizontal and vertical gradients of the enhanced image, and transmit it to the ground command station. Step 3: The ground command station uses the remote control video terminal to enhance the fire feature information in the image and apply the FCM algorithm to cluster the fire disaster types in the fire scene. The adaptive fuzzy index is introduced to dynamically adjust the fuzziness in the clustering process of the FCM algorithm to accurately identify the fire type in the fire scene. At the same time, the task membership is determined, and the rescue task allocation of the rescue drone is determined based on the CNN feature extraction; Step 4. Transmit the rescue mission assignment information to the rescue drone through the signal server of the relay high-altitude drone; at the same time, the signal server of the relay high-altitude drone transmits the established 3DT trajectory optimization path planning information to the corresponding rescue drone. After receiving the path planning information, the rescue drone quickly transmits it to all communicating rescue drones in the surrounding network. The relay high-altitude drone and the ground command station continuously perform on-site real-time information correction and feedback. The ground command station uses real-time data to fuse three-dimensional map construction data, historical training rescue drone historical grouping rescue planning, rescue time, and rescue speed data for comparison, and dynamically controls the group of rescue drones to perform rescue missions through the relay controller in the signal server.

2. A large-scale UAV formation method for emergency fire rescue missions according to claim 1, characterized in that: The signal server of the relay high-altitude drone and the controller of the rescue drone, the ground command station, the visual monitoring sensor, the relay controller, and the rescue drone transmit information at high speed based on the 5G-Mesh distributed collaborative network communication framework.

3. A large-scale UAV formation method for emergency fire rescue missions according to claim 1, characterized in that: In step 2, image preprocessing and information transmission include the following steps: S21. Feature fusion is performed on the video sensor data w1 and the thermal imaging sensor data w2 acquired by the visual monitoring sensor, and the eigenvalue matrix Q is obtained based on the eigenvalues. By comparing the features of the images, the dimensional feature means μ1 and μ2 of the horizontal axis and the vertical axis of the image and the thermal imaging feature mean μ3 are obtained; Among them, y is the total number of pixels in the vertical direction, x is the total number of pixels in the horizontal direction, and q 1k is the gray value of the kth pixel in the horizontal direction, q 2k is the gray value of the kth pixel in the vertical direction, q 3k is the temperature value of the kth pixel in the vertical direction, q 4k is the temperature value of the kth pixel in the horizontal direction; S22. Calculate the covariance COT of image i through the eigenvalue matrix i , obtain the k×1 eigenvalue matrix of the horizontal axis and the k×1 eigenvalue matrix of the vertical axis based on the image eigenvalue, calculate the covariance matrix COT1 in the horizontal direction and the covariance matrix COT2 in the vertical direction; calculate the similarity value SIM of image i based on the Mahalanobis distance of the image i : Among them, f i+1 is the feature vector of the previous frame image, f i is the feature vector of the subsequent frame image; S23. Similarity value SIM of adjacent time frame images i Constantly changing, based on the real-time similarity value SIM i , and calculate the mean square error with the true value: Among them, y0 represents the average image similarity, and n represents the total number of frames during calculation; Through the mean square error, using the gradient descent method, the feature and denoising coefficient gradient descent formula is obtained; Among them, β is the image learning rate, α is the denoising coefficient; The feature set Q and the denoising coefficient α are continuously iterated. When the mean square error E is less than the iteration threshold, the updated feature Q is ω and the updated denoising coefficient α ω The central processor in the server that relays the high-altitude drone signal updates the image enhancement parameters; S24. Update the feature Q ω and the updated denoising coefficient α ω After being transmitted to the central processor, the gradient descent method is used to continuously iterate and adaptively adjust the real-time image learning rate β until the image learning rate β is maximized, and the final image enhancement and denoising are performed to obtain the preprocessed enhanced image.

4. A large-scale UAV formation method for emergency fire rescue missions according to claim 1, characterized in that: The rescue drone also includes a roadbed detector and an anti-collision radar; based on the images collected by the visual monitoring sensor and the roadbed detector, the image processor uses a 3D model reconstruction algorithm to construct a three-dimensional map of the fire scene.

5. The large-scale UAV formation method for emergency fire rescue missions according to claim 1 is characterized in that: In step 3, the fire feature information in the enhanced image is used to cluster the fire damage types in the fire scene using the FCM algorithm, which includes the following steps: S31. Based on the enhanced video data, the captured key image is segmented by the FCM algorithm to obtain h cluster center point sets N: <h2 style=";text-align:left;direction:ltr">N = {N1,N2,N3,...,N<h2 style=";text-align:left;direction:ltr"> h <h2 style=";text-align:left;direction:ltr">}; S32. For each pixel point in the segmented image grid, its membership value L is calculated, and the cluster center point is continuously updated according to the membership value L. The cluster fuzzy coefficient Ω is dynamically adjusted through the texture features in the key frame: Among them, L mo is the membership degree of the oth pixel, c o is the coordinate of the oth pixel point, p o is the coordinate of the pixel point of the oth cluster center, c k is the kth pixel cluster coordinate, p k is the k-th center pixel cluster coordinate, k is the number of clusters, d(c o ,p o ) represents the pixel coordinate c o and the cluster center pixel coordinates p o The distance, d(c k ,p k ) is the pixel cluster coordinate c k and the cluster center pixel cluster coordinates p k distance; Continue to iterate until convergence to each cluster center threshold; S33. Based on the horizontal and vertical grayscale values ​​of the pixels of the segmented image, the corresponding convolution operator g is calculated x , g y , obtain the horizontal and vertical gradient information of the image, and use g x , g y Construct the directional gradient tensor matrix T(i,j) of each pixel point for the local image block; IN x (i,j)=g x ·IN IN y (i,j)=g y ·IN Among them, U x (i, j) is the horizontal gradient value, U y (i, j) is the vertical gradient value, U is the grayscale value of the image pixel at position (i, j); the local gradient direction information and texture features are obtained through the tensor matrix T(i, j); S34. Based on the obtained local gradient direction information and texture features, through convolution image feature recognition, the FCM algorithm is continuously applied for iteration, and finally the real-time image recognition object results are obtained through convolution image features.

6. A large-scale UAV formation method for emergency fire rescue missions according to claim 1, characterized in that: In step 4, the 3DT trajectory optimization path planning information includes the following establishment steps: S41. Obtain the gradient value change between the three-dimensional coordinate system of the rescue drone and the three-dimensional coordinate system of the visual monitoring sensor, and obtain the rotation matrix R of the yaw angle ψ based on the gradient value change. ψ , the rotation matrix R of the pitch angle θ θ , the rotation matrix R of the roll angle φ φ Among them, ψ is the yaw angle of the UAV, θ is the pitch angle of the UAV, and φ is the roll angle of the UAV; The coordinate rotation matrix R of the rescue drone is calculated: R=R ψ ·R θ ·R φ ; S42. Based on the rotation matrix R, the coordinate system parameter changes are sampled to establish a three-dimensional map of the rescue drone flight, and after obtaining the three-dimensional map information and the rescue drone rotation matrix R in real time through the ground command station, the three-dimensional map information is sent to the signal server of the relay high-altitude drone through the 5G-Mesh distributed collaborative network communication framework; S43. Fusing the three-dimensional map and the real-time map collected by the visual monitoring sensor to generate a local fusion map, looping the above steps of forming the local fusion map to form a global fusion map for the rescue drone to perform rescue work; S44. The central processor in the server that relays the signal of the high-altitude drone collects the power parameters and speed v of all the rescue drones p involved in the rescue work 1p , acceleration a p , Weight G p , rotor rotation speed v 2p , get the initial constraint matrix Y Establish the initial cost matrix DJ Y-S=x pq (DJ+ρ*max(0,t pq )) Among them, λ1, λ2, λ3, λ4 are variable matrix coefficients that are dynamically adjusted according to the gradient value, YS is the power constraint matrix of the rescue drone, and x pq is the membership degree of the pth rescue drone to the qth rescue mission, DJ is the initial cost matrix of the rescue drone, ρ is the time constraint coefficient of the rescue mission, t pq is the rescue flight time of the qth rescue mission of the pth rescue drone; Time constraints are introduced to calculate and obtain different applicable rescue flight plans for different UAVs.

7. A large-scale UAV formation method for emergency fire rescue missions according to claim 6, characterized in that: The steps for establishing the 5G-Mesh distributed collaborative network communication framework are as follows: S51. The WIFI6 module in the signal server of the high-altitude drone receives the 5G signal source from the nearest ground base station, amplifies the 5G signal through the signal amplifier, and the anti-interference multi-antenna automatically deploys to resist the interference signal source signal; S52. The rescue drone receives the expanded 5G signal transmitted by the signal server that relays the high-altitude drone. At the same time, using the 5G slicing technology, the 5G signals of adjacent rescue drones in each rescue drone group composed of rescue drones are in different 5G communication segments; S53. The communication signal node of the rescue drone automatically links to the surrounding adjacent signal nodes to form an autonomous decentralized network topology structure; S54. The signal receiving node working in the rescue drone senses the signal interference and transmission speed in real time. When the signal interference and transmission speed are lower than the set threshold, multi-hop communication is automatically formed between multiple working rescue drones.

8. The large-scale UAV formation method for emergency fire rescue missions according to claim 1 is characterized in that: In step 3, the rescue mission allocation of the rescue drone is determined, which includes the following steps: S61. The signal server relaying the high-altitude UAV obtains the image feature recognition information of the convolutional network based on FCM, and obtains the historical data in real time for the identified fire type; S62. Associating the membership to perform task allocation, according to the real-time identified fire type, using the FCM algorithm to obtain the membership of the current fire data, and calculating the applicable task cost matrix of the rescue drone: FP=x pq (DJ+ρ*max(0,t pq )+γ*(1-L mq )); Among them, FP is the rescue mission cost function, x pq is the membership degree of the pth rescue drone to the qth rescue mission, DJ is the initial cost matrix of the rescue drone, ρ is the time constraint coefficient of the rescue mission, t pq is the rescue flight time of the qth rescue mission of the pth rescue drone, γ is the membership adjustment coefficient, L mq is the membership degree of rescue mission q; S63. Use the gradient optimization planning method to optimize task allocation, the goal is to minimize the total cost, Where b is the number of rescue drones, a is the number of rescue missions, and FP pq is the cost function value of the qth rescue mission of p rescue drones.

9. A large-scale UAV formation method for emergency fire rescue missions according to claim 1, characterized in that: The rescue drone is a low-altitude multi-rotor drone.

Images