A fully automatic low-altitude inspection and monitoring method and system based on drones
By establishing a three-dimensional grid space in urban planable space and using dandelion optimization algorithms, combining drone performance constraints and flight path costs, the failure detection and unreal-time update of shooting content in traditional drone inspection methods is solved. The problem has realized the fully automated low-altitude inspection and real-time update of surveillance information of drones.
Patent Information
- Application Number
- CN202411298462.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-18
AI Technical Summary
Traditional drone inspection methods have problems such as failure detection, unreal-time update of shooting content, and require a lot of manual intervention. They also lack the inspection range and emergency response capabilities during low-altitude flights, and the monitoring information is not updated in time.
By establishing a three-dimensional grid space in the urban planable space, combining drone performance constraints and flight path cost factors, using a dandelion optimization algorithm to plan patrol paths, and defog processing and object detection and recognition of urban aerial images, the city can achieve no Fully automatic low-altitude patrol and monitoring of human-machine machines.
It improves the accuracy of drone inspection path planning and detection and identification accuracy, realizes fully automated inspection and threat avoidance of drones, reduces manual intervention, and ensures real-time update of monitoring information.
Smart Images
Figure CN118857303B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of unmanned aerial vehicle control, and in particular to a fully automatic low-altitude inspection and monitoring method and system based on unmanned aerial vehicles. Background Art
[0002] Chinese patent CN116048119B discloses a drone cruise monitoring system and method based on artificial intelligence. The method specifically includes: obtaining terrain feature information of the drone cruise area, establishing a two-dimensional plane map to describe the terrain feature information, and then dividing the drone cruise area into uniform codes according to the two-dimensional plane map, wherein the drone cruise area is the external contour feature information of the drone cruise area; mapping the coding information of the drone cruise area according to the uniform coding division, obtaining a static drone cruise path set, calculating the safety factor and similarity of the static drone cruise path in the static drone cruise path set; placing a drone on the static drone cruise path in the static drone cruise path set for cruise, and when a drone fails, matching the static drone cruise path according to the similarity, finding the corresponding optimal static drone cruise path, and then dispatching the drone to continue to cruise the optimal static drone cruise path. However, this method cannot intelligently control drone cruise in real time, and does not further analyze and process the acquired content.
[0003] Traditional drone inspection methods are prone to problems such as untimely drone fault detection and failure to update drone footage in real time. They also require a lot of manual intervention, increasing the manpower burden. When drones fly at low altitudes, further requirements are placed on inspection range detection and emergency response capabilities. At the same time, drone monitoring information must also be updated to the supervision platform in real time. Summary of the invention
[0004] In response to the problems in the related technology, the present invention provides a fully automatic low-altitude inspection and monitoring method and system based on unmanned aerial vehicles to overcome the above-mentioned technical problems existing in the existing related technology.
[0005] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:
[0006] The present invention is a fully automatic low-altitude inspection and monitoring method based on an unmanned aerial vehicle, comprising the following steps:
[0007] S1. Based on the urban plannable space, a three-dimensional grid space is established using the three-dimensional grid method, the performance factors of the UAV on the flight path are calculated, the performance constraints of the UAV are completed, and the performance constraints of the UAV are obtained by combining the cost factors of the UAV on the flight path;
[0008] S2. Under the performance constraints of the drone, the drone flight path evaluation index is used as the fitness function, and the drone inspection path is planned based on the dandelion optimization algorithm. The optimal fitness function value is found by continuously iterating the fitness function. The optimal fitness function value corresponds to the individual position of the dandelion, that is, the optimal position of the drone. The optimal positions of all drones are connected to output the drone inspection path;
[0009] S3, the drone acquires the city aerial image on the drone inspection path, and when the city aerial image contains fog, locates the light source of the city aerial image, and then corrects the city aerial image according to the change of fog content to obtain the final city aerial image; and trains the YOLOv5 network to realize the detection and recognition of the city aerial objects;
[0010] S4. After the drone completes the detection and identification of the urban aerial photography objects, it will feedback the abnormal information to the drone monitoring platform according to the abnormal coefficient of the urban aerial photography objects, avoid threats and dynamically plan the inspection route.
[0011] The invention firstly realizes three-dimensional modeling of UAV flight by establishing a three-dimensional grid space in the urban plannable space, and obtains the UAV performance constraints according to the UAV flying under the constraints of maximum range, minimum turning radius, maximum yaw angle, maximum pitch angle and flight altitude; obtains the UAV flight path evaluation index by combining the UAV flight path length cost, UAV flight energy consumption cost, UAV flight threat cost and UAV flight altitude cost; secondly, establishes a UAV flight path landmark point matrix, regards the elements in the UAV flight path landmark point matrix as dandelion seed individuals, and plans the UAV inspection path based on the dandelion optimization algorithm; respectively, in the dandelion ascending stage, descending stage and landing stage, the UAV inspection path is calculated based on the dandelion optimization algorithm. In the land stage, the position of dandelion seeds is updated to obtain the landmark points of the optimal flight path of the drone; the global search is performed in the early stage, and the local development is performed in the later stage, so that the dandelion individuals can easily traverse more search areas during the iteration process and accurately converge to the global optimal solution. The algorithm has fast convergence speed, good stability and strong robustness; then the city aerial image is obtained, and when the city aerial image is foggy, the light source is judged and located, and then defogging is performed according to the change in the degree of fog to obtain the final city aerial image, effectively reducing the error of foggy image detection; and the YOLOv5 network is trained to realize the detection and recognition of urban aerial objects; finally, the drone feeds back abnormal information based on the abnormal coefficient of the detected and recognized object, and avoids threats.
[0012] Preferably, the S1 comprises the following steps:
[0013] S11. Treat the urban plannable space as a free grid and establish a three-dimensional rectangular coordinate system ,in O represents the coordinate midpoint, XRepresents the two-dimensional horizontal direction of the urban plannable space, Y Indicates the two-dimensional vertical direction of the urban plannable space, Z Represents the three-dimensional vertical direction of the urban plannable space, and uses the three-dimensional grid method to divide the urban plannable space into a×a×a A cube grid is obtained to obtain a three-dimensional grid space; the obstacles in the three-dimensional grid space are obstacle grids, and the non-obstacles are normal grids. The obstacle grids and normal grids are numbered in the three-dimensional grid space, and the grid coordinates of the midpoint of the coordinates are set to (0,0,0). The grid number set is , then the grid coordinate calculation formula in the three-dimensional grid space is as follows,
[0014] , , ,
[0015] in, Represents the grid coordinates in the three-dimensional grid space, mod represents the modulus operation, Represents the first Grid number;
[0016] S12, setting the grid coordinates of the drone in the three-dimensional grid space to , then the drone can fly in 26 directions and cannot fly in the obstacle grid. During the flight of the drone, the flight path of the drone is set. The sub-path, Indicates The length of the subpath, , UAV flight path , then the maximum range constraint of the UAV is ; Set the minimum turning radius of the drone in the drone flight path to , then the minimum turning radius constraint of the UAV is ;
[0017] Set the drone The horizontal plane projection vector of the strip path is , Indicates that the drone The transpose of the horizontal projection vector of the strip path, Indicates that the drone The magnitude of the horizontal projection vector of the strip path, Indicates that the drone The horizontal plane projection vector of the subpath is the modulus of the drone. The flight height of the strip path is , UAV The flight height of the strip path is , UAV maximum yaw angle and the maximum pitch angle of the drone The calculation formula is as follows,
[0018] , ;
[0019] The maximum yaw angle constraint of the UAV is , UAV maximum pitch angle constraint ; Set the maximum flight altitude of the drone to The minimum flight altitude of the drone is , then the UAV flight altitude constraint satisfy ; The UAV flies under the constraints of maximum range, minimum turning radius, maximum yaw angle, maximum pitch angle and flight altitude, completing the performance constraints of the UAV;
[0020] S13. Set the flight path of the drone to exist subpaths, the set of UAV flight subpaths is The lengths of the drone flight subpaths corresponding to the drone flight subpaths in the drone flight subpath set constitute the drone flight subpath length set ,in Indicates The length of the UAV flight sub-path corresponding to the UAV flight sub-path, the calculation formula of the UAV flight path length cost and the UAV flight energy consumption cost is as follows:
[0021] , ,
[0022] in, represents the cost of the UAV flight path length, represents the energy consumption cost of UAV flight, Indicates The length of the UAV flight sub-path corresponding to the UAV flight sub-path, and the UAV energy consumption proportional coefficient is , ;
[0023] Calculate the distance from the UAV flight subpath in the UAV flight subpath set to the obstacle grid to obtain an obstacle distance set, recorded as ,in Indicates The distance from the UAV flight subpath to the obstacle grid, the calculation formula of the UAV flight threat cost is as follows:
[0024] ,
[0025] in, Indicates the cost of drone flight threats, Indicates The distance from the UAV flight subpath to the obstacle grid, ;
[0026] Calculate the average flight height of the UAV in the UAV flight subpath and get the flight height set, recorded as ,in Indicates that the drone is The average flight height of the UAV flight sub-paths; according to the maximum flight height of the UAV is and the minimum flight altitude of the UAV is ,The calculation formula of the UAV flight altitude cost is as follows,
[0027] , ,
[0028] in, Indicates the cost of the drone's flight altitude, Indicates that the drone is The average flight altitude cost of the UAV flight subpaths, Indicates that the drone is The average flight height of the UAV flight subpaths, and represents a constant;
[0029] set up , , and Represents the weight, and combines the UAV flight path length cost, UAV flight energy consumption cost, UAV flight threat cost and UAV flight altitude cost to obtain the UAV flight path evaluation index .
[0030] The invention achieves three-dimensional modeling of the UAV flight environment by establishing a three-dimensional grid space in the urban plannable space, and obtains the UAV performance constraints based on the performance factors of the UAV on the flight path; combined with the cost factors of the UAV on the flight path, the UAV flight path evaluation index is obtained to improve the accuracy of planning the UAV inspection path.
[0031] Preferably, S2 comprises the following steps:
[0032] S21. Under the UAV performance constraint, the UAV flight path evaluation index is used as the fitness function, and landmark points are selected on the UAV flight path to obtain a set of UAV flight path landmark points, which is recorded as ,in Indicates the flight path of the drone i A set of landmark points is obtained by using three-dimensional coordinates to represent the positions of the landmark points in the set of landmark points in the flight path of the drone, and obtaining a set of landmark points in the flight path of the drone corresponding to the set of landmark points in the flight path of the drone. , set the location dimension of the landmark points on the UAV flight path to j , generate the drone flight path landmark point matrix e as follows,
[0033] ,
[0034] in, Indicates i The location dimension is j Waypoints of the drone’s flight path;
[0035] In the dandelion population, each dandelion seed corresponds to a UAV flight path landmark point in the UAV flight path landmark point matrix. e dandelion seeds, the first e dandelion seeds in the urban plannable space; set the dandelion population e The spatial upper bound of dandelion seeds is , the first in the dandelion population e The lower bound of the space of dandelion seeds is , represents a random number and , then the first e Initial position of dandelion seeds The calculation formula is as follows,
[0036] ;
[0037] At this time, the drone determines the starting position, which is the first e The initialization position of dandelion seeds;
[0038] S22, dandelion seeds are in the ascending stage, the drone starts at the starting position, and the current number of iterations is set to h , dandelion seeds in the dandelion population h The iteration position is , dandelion seeds in the dandelion population h +1 iteration position is , dandelion seeds in the dandelion population h +1 iteration random position is , represents the normal distribution function. The dandelion seeds in the dandelion population are affected by the vortex in the rising stage. The lift component coefficient of the dandelion seeds in the dandelion population is set to and , then the dandelion seed number one in the dandelion population h The calculation formula for the +1 iteration position is as follows:
[0039] ;
[0040] in, Represents the adaptive parameter for adjusting the search step size, setting the maximum number of iterations to h , represents a random number and , adjust the adaptive parameters of the search step size ,With the adaptive parameter adjustment of the search step length, the UAV continuously changes its position;
[0041] Dandelion seeds are searched locally on rainy days. The flight of the drone is constrained at this time. The adjustment parameters for rainy days are set as , the rainy day adjustment parameters converge with the optimal dandelion seeds as the number of iterations increases, and the dandelion seeds in the dandelion population are h +1 iteration position update is , the calculation formula is as follows,
[0042] ;
[0043] Dandelion seeds in the rising stage, set Represents a random number and obeys the normal distribution function ,when At this time, the dandelion seeds in the dandelion population h +1 iteration position satisfies , at this time the drone arrived Otherwise, the dandelion seeds in the dandelion population h +1 iteration position satisfies , at this time the drone arrived Department;
[0044] S23, dandelion seeds are in the descending stage. At this time, the fitness function value is continuously updated. The drone updates its own position again and sets the dandelion seed No. h The average position at the iteration is , represents a Brownian motion of random numbers that follows a normal distribution, represents a random number and , the dandelion seeds in the dandelion population h +1 iteration position update is , the calculation formula is as follows,
[0045] ;
[0046] The drone arrived At the point, the dandelion seed is regarded as the fitness function value, the best fitness function value is found, the non-optimal fitness function value is replaced by the best fitness function value, and the UAV changes its own position; during the landing phase, the dandelion seed adjusts the adaptive parameter of the search step length, and the dandelion seed corresponding to the best fitness function value is taken as the elite dandelion seed, and the position of the elite dandelion seed is recorded as , introduce Levy flight, and set the Levy flight function to , the linear increasing function is , represents a random number and , the dandelion seeds in the dandelion population h +1 iteration position update is , the calculation formula is as follows,
[0047] ;
[0048] The dandelion seeds in the dandelion population are h +1 iteration position Recorded as the first UAV flight path landmark point, the UAV flies to the first UAV flight path landmark point and iterates continuously. When the current number of iterations is equal to the maximum number of iterations, the iteration is stopped to obtain the best UAV flight path landmark point set, connect the UAV flight path landmark points in the best UAV flight path landmark point set, smooth the path, output the UAV inspection path, and transmit the UAV inspection path to the UAV monitoring platform.
[0049] The invention establishes a UAV flight path landmark point matrix, regards the elements in the human-machine flight path landmark point matrix as dandelion seed individuals, plans the UAV inspection path based on the dandelion optimization algorithm, continuously updates the dandelion seed positions according to the fitness function value, and obtains the UAV inspection path; in the algorithm, the dandelion individuals can easily traverse more search areas during the iteration process and converge accurately to the global optimal solution, with fast convergence speed, good stability and strong robustness.
[0050] Preferably, S3 comprises the following steps:
[0051] S31. The drone obtains an aerial image at time t on the drone inspection path, recorded as a city aerial image, converts the real-time city aerial image into a grayscale city aerial image, calculates the grayscale mean and grayscale variance of the grayscale city aerial image, and obtains a fog judgment formula based on the pixel values of the grayscale city aerial image, and sets the fog threshold to , when the result of the fog judgment formula is greater than When , the city aerial image does not contain fog, which is recorded as the final city aerial image; otherwise, the city aerial image contains fog, and a foggy city aerial image is obtained; the defogging processing steps for the foggy city aerial image are as follows:
[0052] S311, graying the aerial image of the foggy city to obtain a grayscale aerial image of the foggy city, and equally dividing the grayscale aerial image of the foggy city into four sub-areas, which are respectively recorded as the first sub-area, the second sub-area, the third sub-area and the fourth sub-area; respectively calculating the gradient value and the mean pixel value of the first sub-area, the second sub-area, the third sub-area and the fourth sub-area, and constructing a light source evaluation function, the calculation formula is as follows:
[0053] ,
[0054] in, F represents the light source evaluation function, represents the average pixel value of the first sub-region, the second sub-region, the third sub-region and the fourth sub-region, represents the maximum gradient value of the first sub-region, the second sub-region, the third sub-region and the fourth sub-region;
[0055] Set the light source judgment threshold to , according to the light source evaluation functions of the first sub-region, the second sub-region, the third sub-region and the fourth sub-region, the light source evaluation is calculated, and the sub-region corresponding to the maximum light source evaluation is selected and recorded as the area to be segmented, and S311 is repeated for the area to be segmented until the ratio of the gradient value to the mean pixel value is less than , and select the pixel position with the highest brightness as the light source position to obtain the aerial image of the foggy city with the located light source;
[0056] S312, calculating the minimum filter window and refractive index of the aerial image of the foggy city of the positioned light source, obtaining the aerial image at time t+1, recorded as the comparative aerial image of the city, performing light source positioning processing on the comparative aerial image of the city to obtain the comparative aerial image of the foggy city; setting the similarity threshold to , when the similarity between the aerial image of the foggy city taken by the positioning light source and the comparison aerial image of the foggy city is less than or equal to When the refractive index is used to perform defogging on the aerial image of the foggy city with the light source positioned; otherwise, the minimum filter window and refractive index of the aerial image of the foggy city with the light source positioned are updated to obtain a corrected aerial image of the city;
[0057] S313, converting the corrected urban aerial image into an HSV color image, performing equalization processing on the brightness of the HSV color image to obtain a balanced HSV color image, and then converting the balanced HSV color image into an RGB color image, which is recorded as the final urban aerial image;
[0058] S32, obtaining an image to be detected, performing defogging on the image to be detected, obtaining a processed image to be detected, forming an image sample set, and dividing the image sample set into an image training set and an image verification set; setting the initial learning rate of the YOLOv5 network to , the weight parameter is , the current number of iterations is k, the maximum number of iterations is K, the optimization method adopts the Adam optimization algorithm, the image training set is input into the YOLOv5 network, iterates continuously, and when the current number of iterations is equal to the maximum number of iterations, the iteration is stopped to obtain the trained YOLOv5 network; then the image verification set is input into the trained YOLOv5 network, and the accuracy threshold is set to , when the accuracy of the trained YOLOv5 network result is less than , and get the final YOLOv5 network; otherwise, adjust the weights until the trained YOLOv5 network result accuracy is less than ;
[0059] A city aerial image sample set is formed according to the final city aerial image, and the city aerial image sample set is input into the final YOLOv5 network to realize detection and recognition of city aerial objects.
[0060] The invention obtains city aerial images, determines whether the city aerial images contain fog, and performs defogging processing when the city aerial images contain fog to obtain the final city aerial images, which can effectively improve the accuracy of foggy image detection; and uses the images to be detected to train the YOLOv5 network to realize the detection and recognition of urban aerial objects. The neural network has high detection and recognition accuracy and strong ability to extract features of urban aerial objects.
[0061] Preferably, S4 comprises the following steps:
[0062] S41. After the drone completes the detection and identification of the urban aerial photography object, the detected and identified urban aerial photography object feeds back abnormal information to the drone, and the abnormal information includes the safety factor of the urban aerial photography object, the abnormality degree of the urban aerial photography object and the credibility of the urban aerial photography object, which are respectively recorded as , and ,in Indicates l Safety factor of aerial photography objects in each city, Indicates m The degree of abnormality of aerial objects in cities, Indicates n The credibility of the aerial photography objects in the city; the abnormal parameters are calculated according to the safety factor, abnormality degree and credibility corresponding to the aerial photography objects in the city, the abnormal parameters are arranged in descending order according to the size, and the aerial photography object images of the city corresponding to the abnormal parameters are transmitted to the UAV monitoring platform according to the size of the abnormal parameters;
[0063] S42. When the UAV encounters a threat on the UAV inspection path, when the threat occurs on the UAV inspection path, the UAV uses a distance before the threat as a local starting point and a distance after the threat as a local target point, and begins to replan the local inspection path to avoid the threat; when the threat does not occur on the UAV inspection path, the UAV inspection path is used as a static inspection route, the UAV flies along the static inspection route, and dynamically plans the inspection route in real time.
[0064] This embodiment also discloses a system for a fully automatic low-altitude inspection and monitoring method based on a drone, which specifically includes: a performance constraint and flight path evaluation index module, a drone inspection path optimization module, an urban aerial photography object detection and recognition module, and an abnormal threat avoidance module;
[0065] The performance constraint and flight path evaluation index module is used to establish the performance constraint of the UAV on the flight path and the cost factor of the UAV on the flight path;
[0066] The UAV inspection path optimization module is used to plan the UAV inspection path based on the Dandelion optimization algorithm;
[0067] The urban aerial photography object detection and recognition module is used to perform defogging on urban aerial photography images and then train the YOLOv5 network to detect and recognize urban aerial photography objects;
[0068] The abnormal threat avoidance module is used to combine the abnormal information feedback threat of the drone and dynamically plan a path to avoid the threat.
[0069] The present invention has the following beneficial effects:
[0070] 1. The invention realizes three-dimensional modeling of the UAV flight environment by establishing a three-dimensional grid space in the urban plannable space, and obtains the UAV performance constraints based on the performance factors of the UAV on the flight path; combined with the cost factors of the UAV on the flight path, the UAV flight path evaluation index is obtained to improve the accuracy of planning the UAV inspection path.
[0071] 2. The invention establishes a UAV flight path landmark point matrix, regards the elements in the UAV flight path landmark point matrix as dandelion seed individuals, plans the UAV inspection path based on the dandelion optimization algorithm, and continuously updates the dandelion seed position according to the fitness function value to obtain the UAV inspection path; the algorithm has fast convergence speed, good stability and strong robustness.
[0072] 3. The invention obtains aerial images of cities, performs defogging when the aerial images contain fog, and obtains the final aerial images of cities, which can effectively improve the detection accuracy of foggy images; and trains the YOLOv5 network to realize the detection and recognition of aerial objects in cities. The neural network has high detection and recognition accuracy and strong feature extraction capabilities for aerial objects in cities. Finally, the drone feeds back abnormal information based on the abnormal coefficient of the detected and recognized object, and avoids threats.
[0073] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying creative work.
[0075] Figure 1 The present invention provides a flow chart of fully automatic low-altitude inspection and monitoring by a fully automatic low-altitude inspection and monitoring system based on an unmanned aerial vehicle. DETAILED DESCRIPTION
[0076] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0077] In the description of the present invention, it is necessary to understand that the terms "opening", "upper", "lower", "top", "middle", "inside" and the like indicating orientation or positional relationship are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the invention.
[0078] This implementation discloses a fully automatic low-altitude inspection and monitoring method based on a drone, which specifically includes the following contents:
[0079] S1. Based on the urban plannable space, a three-dimensional grid space is established using the three-dimensional grid method to calculate the performance factors of the UAV on the flight path, complete the UAV performance constraints, and combine the cost factors of the UAV on the flight path to obtain the UAV flight path evaluation index;
[0080] The S1 comprises the following steps:
[0081] S11. Treat the urban plannable space as a free grid and establish a three-dimensional rectangular coordinate system ,in O represents the coordinate midpoint, X Represents the two-dimensional horizontal direction of the urban plannable space, Y Indicates the two-dimensional vertical direction of the urban plannable space, Z Represents the three-dimensional vertical direction of the urban plannable space, and uses the three-dimensional grid method to divide the urban plannable space into a×a×a A cube grid is obtained to obtain a three-dimensional grid space; the obstacles in the three-dimensional grid space are obstacle grids, and the non-obstacles are normal grids. The obstacle grids and normal grids are numbered in the three-dimensional grid space, and the grid coordinates of the midpoint of the coordinates are set to (0,0,0). The grid number set is , then the grid coordinate calculation formula in the three-dimensional grid space is as follows,
[0082] , , ,
[0083] in, Represents the grid coordinates in the three-dimensional grid space, mod represents the modulus operation, Represents the first Grid number;
[0084] S12, setting the grid coordinates of the drone in the three-dimensional grid space to , then the drone can fly in 26 directions and cannot fly in the obstacle grid. During the flight of the drone, the flight path of the drone is set. The sub-path, Indicates The length of the subpath, , UAV flight path , then the maximum range constraint of the UAV is ; Set the minimum turning radius of the drone in the drone flight path to , then the minimum turning radius constraint of the UAV is ;
[0085] Set the drone The horizontal plane projection vector of the strip path is , Indicates that the drone The transpose of the horizontal projection vector of the strip path, Indicates that the drone The magnitude of the horizontal projection vector of the strip path, Drone The horizontal plane projection vector of the subpath is the modulus of the drone. The flight height of the strip path is , UAV The flight height of the strip path is , UAV maximum yaw angle And the maximum pitch angle of the drone The calculation formula is as follows,
[0086] , ;
[0087] The maximum yaw angle constraint of the UAV is , UAV maximum pitch angle constraint ; Set the maximum flight altitude of the drone to The minimum flight altitude of the drone is , then the UAV flight altitude constraint satisfy ; The UAV flies under the constraints of maximum range, minimum turning radius, maximum yaw angle, maximum pitch angle and flight altitude, completing the performance constraints of the UAV;
[0088] S13. Set the flight path of the drone to exist subpaths, the set of UAV flight subpaths is The lengths of the drone flight subpaths corresponding to the drone flight subpaths in the drone flight subpath set constitute the drone flight subpath length set ,in Indicates The length of the UAV flight sub-path corresponding to the UAV flight sub-path, the calculation formula of the UAV flight path length cost and the UAV flight energy consumption cost is as follows:
[0089] , ,
[0090] in, represents the cost of the UAV flight path length, represents the energy consumption cost of UAV flight, Indicates The length of the UAV flight sub-path corresponding to the UAV flight sub-path, and the UAV energy consumption proportional coefficient is , ;
[0091] Calculate the distance from the UAV flight subpath in the UAV flight subpath set to the obstacle grid to obtain an obstacle distance set, recorded as ,in Indicates The distance from the UAV flight subpath to the obstacle grid, the calculation formula of the UAV flight threat cost is as follows: ,
[0092] in, Indicates the cost of drone flight threats, Indicates The distance from the UAV flight subpath to the obstacle grid, ;
[0093] Calculate the average flight height of the UAV in the UAV flight subpath and get the flight height set, recorded as ,in Indicates that the drone is The average flight height of the UAV flight sub-paths; according to the maximum flight height of the UAV is and the minimum flight altitude of the UAV is ,The calculation formula of the UAV flight altitude cost is as follows,
[0094] , ,
[0095] in, Indicates the cost of the drone's flight altitude, Indicates that the drone is The average flight altitude cost of the UAV flight subpaths, Indicates that the drone is The average flight height of the UAV flight subpaths, and represents a constant;
[0096] set up , , and Represents the weight, and combines the UAV flight path length cost, UAV flight energy consumption cost, UAV flight threat cost and UAV flight altitude cost to obtain the UAV flight path evaluation index ;
[0097] S2. Under the performance constraints of the drone, the drone flight path evaluation index is used as the fitness function, and the drone inspection path is planned based on the dandelion optimization algorithm. The optimal fitness function value is found by continuously iterating the fitness function. The optimal fitness function value corresponds to the individual position of the dandelion, that is, the optimal position of the drone. The optimal positions of all drones are connected to output the drone inspection path;
[0098] The S2 comprises the following steps:
[0099] S21. Under the UAV performance constraint, the UAV flight path evaluation index is used as the fitness function, and landmark points are selected on the UAV flight path to obtain a set of UAV flight path landmark points, which is recorded as ,in Indicates the flight path of the drone i A set of landmark points is obtained by using three-dimensional coordinates to represent the positions of the landmark points in the set of landmark points in the flight path of the drone, and obtaining a set of landmark points in the flight path of the drone corresponding to the set of landmark points in the flight path of the drone. , set the location dimension of the landmark points on the UAV flight path to j , generate the drone flight path landmark point matrix e as follows,
[0100] ,
[0101] in, Indicates i The location dimension is j Waypoints of the drone’s flight path;
[0102] In the dandelion population, each dandelion seed corresponds to a UAV flight path landmark point in the UAV flight path landmark point matrix. e dandelion seeds, the first e dandelion seeds in the urban plannable space; set the dandelion population e The spatial upper bound of dandelion seeds is , the first in the dandelion population e The lower bound of the space of dandelion seeds is , represents a random number and , then the first e Initial position of dandelion seeds The calculation formula is as follows,
[0103] ;
[0104] At this time, the drone determines the starting position, which is the firste The initialization position of dandelion seeds;
[0105] S22, dandelion seeds are in the ascending stage, the drone starts at the starting position, and the current number of iterations is set to h , dandelion seeds in the dandelion population h The iteration position is , dandelion seeds in the dandelion population h +1 iteration position is , dandelion seeds in the dandelion population h +1 iteration random position is , represents the normal distribution function. The dandelion seeds in the dandelion population are affected by the vortex in the rising stage. The lift component coefficient of the dandelion seeds in the dandelion population is set to and , then the dandelion seed number one in the dandelion population h The calculation formula for the +1 iteration position is as follows:
[0106] ;
[0107] in, Represents the adaptive parameter for adjusting the search step size, setting the maximum number of iterations to h , represents a random number and , adjust the adaptive parameters of the search step size ,With the adaptive parameter adjustment of the search step length, the UAV continuously changes its position;
[0108] Dandelion seeds are searched locally on rainy days. The flight of the drone is constrained at this time. The adjustment parameters for rainy days are set as , the rainy day adjustment parameters converge with the optimal dandelion seeds as the number of iterations increases, and the dandelion seeds in the dandelion population are h +1 iteration position update is , the calculation formula is as follows,
[0109] ;
[0110] Dandelion seeds in the rising stage, set Represents a random number and obeys the normal distribution function ,when At this time, the dandelion seeds in the dandelion population h +1 iteration position satisfies , at this time the drone arrived Otherwise, the dandelion seeds in the dandelion population h +1 iteration position satisfies , at this time the drone arrived Department;
[0111] S23, dandelion seeds are in the descending stage. At this time, the fitness function value is continuously updated. The drone updates its own position again and sets the dandelion seed No. h The average position at the iteration is , represents a Brownian motion of random numbers that follows a normal distribution, represents a random number and , the dandelion seeds in the dandelion population h +1 iteration position update is , the calculation formula is as follows,
[0112] ;
[0113] The drone arrived At the point, the dandelion seed is regarded as the fitness function value, the best fitness function value is found, the non-optimal fitness function value is replaced by the best fitness function value, and the UAV changes its own position; during the landing phase, the dandelion seed adjusts the adaptive parameter of the search step length, and the dandelion seed corresponding to the best fitness function value is taken as the elite dandelion seed, and the position of the elite dandelion seed is recorded as , introduce Levy flight, and set the Levy flight function to , the linear increasing function is , represents a random number and , the dandelion seeds in the dandelion population h +1 iteration position update is , the calculation formula is as follows,
[0114] ;
[0115] The dandelion seeds in the dandelion population are h +1 iteration position Recorded as the first UAV flight path landmark point, the UAV flies to the first UAV flight path landmark point, iterates continuously, and when the current number of iterations is equal to the maximum number of iterations, stops iterating, obtains the best UAV flight path landmark point set, connects the UAV flight path landmark points in the best UAV flight path landmark point set, smoothes the path, outputs the UAV inspection path, and transmits the UAV inspection path to the UAV monitoring platform;
[0116] S3, the drone acquires the city aerial image on the drone inspection path, and when the city aerial image contains fog, locates the light source of the city aerial image, and then corrects the city aerial image according to the change of fog content to obtain the final city aerial image; and trains the YOLOv5 network to realize the detection and recognition of the city aerial objects;
[0117] The S3 comprises the following steps:
[0118] S31. The drone obtains an aerial image at time t on the drone inspection path, recorded as a city aerial image, converts the real-time city aerial image into a grayscale city aerial image, calculates the grayscale mean and grayscale variance of the grayscale city aerial image, and obtains a fog judgment formula based on the pixel values of the grayscale city aerial image, and sets the fog threshold to , when the result of the fog judgment formula is greater than When , the city aerial image does not contain fog, which is recorded as the final city aerial image; otherwise, the city aerial image contains fog, and a foggy city aerial image is obtained; the defogging processing steps for the foggy city aerial image are as follows:
[0119] S311, graying the aerial image of the foggy city to obtain a grayscale aerial image of the foggy city, and equally dividing the grayscale aerial image of the foggy city into four sub-areas, which are respectively recorded as the first sub-area, the second sub-area, the third sub-area and the fourth sub-area; respectively calculating the gradient value and the mean pixel value of the first sub-area, the second sub-area, the third sub-area and the fourth sub-area, and constructing a light source evaluation function, the calculation formula is as follows:
[0120] ,
[0121] in, F represents the light source evaluation function, represents the average pixel value of the first sub-region, the second sub-region, the third sub-region and the fourth sub-region, represents the maximum gradient value of the first sub-region, the second sub-region, the third sub-region and the fourth sub-region;
[0122] Set the light source judgment threshold to , according to the light source evaluation functions of the first sub-region, the second sub-region, the third sub-region and the fourth sub-region, the light source evaluation is calculated, and the sub-region corresponding to the maximum light source evaluation is selected and recorded as the area to be segmented, and S311 is repeated for the area to be segmented until the ratio of the gradient value to the mean pixel value is less than , and select the pixel position with the highest brightness as the light source position to obtain the aerial image of the foggy city with the located light source;
[0123] S312, calculating the minimum filter window and refractive index of the aerial image of the foggy city of the positioned light source, obtaining the aerial image at time t+1, recorded as the comparative aerial image of the city, performing light source positioning processing on the comparative aerial image of the city to obtain the comparative aerial image of the foggy city; setting the similarity threshold to , when the similarity between the aerial image of the foggy city taken by the positioning light source and the comparison aerial image of the foggy city is less than or equal to When the refractive index is used to perform defogging on the aerial image of the foggy city with the light source positioned; otherwise, the minimum filter window and refractive index of the aerial image of the foggy city with the light source positioned are updated to obtain a corrected aerial image of the city;
[0124] S313, converting the corrected urban aerial image into an HSV color image, performing equalization processing on the brightness of the HSV color image to obtain a balanced HSV color image, and then converting the balanced HSV color image into an RGB color image, which is recorded as the final urban aerial image;
[0125] S32, obtaining an image to be detected, performing defogging on the image to be detected, obtaining a processed image to be detected, forming an image sample set, and dividing the image sample set into an image training set and an image verification set; setting the initial learning rate of the YOLOv5 network to , the weight parameter is , the current number of iterations is k, the maximum number of iterations is K, the optimization method adopts the Adam optimization algorithm, the image training set is input into the YOLOv5 network, iterates continuously, and when the current number of iterations is equal to the maximum number of iterations, the iteration is stopped to obtain the trained YOLOv5 network; then the image verification set is input into the trained YOLOv5 network, and the accuracy threshold is set to , when the accuracy of the trained YOLOv5 network result is less than , and get the final YOLOv5 network; otherwise, adjust the weights until the trained YOLOv5 network result accuracy is less than ;
[0126] According to the final urban aerial images, a sample set of urban aerial images is formed, and the sample set of urban aerial images is input into the final YOLOv5 network to realize detection and recognition of urban aerial objects;
[0127] S4. After the drone completes the detection and identification of the aerial photography objects in the city, it will feedback the abnormal information to the drone monitoring platform according to the abnormal coefficient of the aerial photography objects in the city, avoid threats and dynamically plan the inspection route;
[0128] The S4 comprises the following steps:
[0129] S41. After the drone completes the detection and identification of the urban aerial photography object, the detected and identified urban aerial photography object feeds back abnormal information to the drone, and the abnormal information includes the safety factor of the urban aerial photography object, the abnormality degree of the urban aerial photography object and the credibility of the urban aerial photography object, which are respectively recorded as , and ,in Indicates l Safety factor of aerial photography objects in each city, Indicates m The degree of abnormality of aerial objects in cities, Indicates n The credibility of the aerial photography objects in the city; the abnormal parameters are calculated according to the safety factor, abnormality degree and credibility corresponding to the aerial photography objects in the city, the abnormal parameters are arranged in descending order according to the size, and the aerial photography object images of the city corresponding to the abnormal parameters are transmitted to the UAV monitoring platform according to the size of the abnormal parameters;
[0130] S42. When the UAV encounters a threat on the UAV inspection path, when the threat occurs on the UAV inspection path, the UAV uses a distance before the threat as a local starting point and a distance after the threat as a local target point, and begins to replan the local inspection path to avoid the threat; when the threat does not occur on the UAV inspection path, the UAV inspection path is used as a static inspection route, the UAV flies along the static inspection route, and dynamically plans the inspection route in real time.
[0131] This embodiment also discloses a system for a fully automatic low-altitude inspection and monitoring method based on a drone, which specifically includes: a performance constraint and flight path evaluation index module, a drone inspection path optimization module, an urban aerial photography object detection and recognition module, and an abnormal threat avoidance module;
[0132] The performance constraint and flight path evaluation index module is used to establish the performance constraint of the UAV on the flight path and the cost factor of the UAV on the flight path;
[0133] The UAV inspection path optimization module is used to plan the UAV inspection path based on the Dandelion optimization algorithm;
[0134] The urban aerial photography object detection and recognition module is used to perform defogging on urban aerial photography images and then train the YOLOv5 network to detect and recognize urban aerial photography objects;
[0135] The abnormal threat avoidance module is used to combine the abnormal information feedback threat of the drone and dynamically plan a path to avoid the threat.
[0136] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.
[0137] The preferred embodiments of the invention disclosed above are only used to help explain the invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the invention, so that those skilled in the art can understand and use the invention well.
Claims
1. A fully automatic low-altitude inspection and monitoring method based on drones, characterized in that: The steps include: S1. Based on the urban plannable space, a three-dimensional grid space is established using the three-dimensional grid method to calculate the performance factors of the UAV on the flight path, complete the UAV performance constraints, and combine the cost factors of the UAV on the flight path to obtain the UAV flight path evaluation index; S2. Under the performance constraints of the drone, the drone flight path evaluation index is used as the fitness function, and the drone inspection path is planned. The optimal fitness function value is found by continuously iterating the fitness function. The optimal fitness function value corresponds to the individual position, that is, the optimal position of the drone. The optimal positions of all drones are connected to output the drone inspection path. S21, under the performance constraint of the drone, using the drone flight path evaluation index as the fitness function, setting landmark points on the drone flight path, and generating a drone flight path landmark point matrix; In the dandelion population, each dandelion seed corresponds to a UAV flight path landmark point in the UAV flight path landmark point matrix. e dandelion seeds, the first e Dandelion seeds in the city's plannable space; Set the dandelion population e The spatial upper bound of dandelion seeds is , the first in the dandelion population e The lower bound of the space of dandelion seeds is , represents a random number and , then the first e Initial position of dandelion seeds The calculation formula is as follows; ； At this time, the drone determines the starting position, which is the first e The initialization position of dandelion seeds; S22, dandelion seeds are in the ascending stage, the drone starts at the starting position, and the current number of iterations is set to h , dandelion seeds in the dandelion population h The iteration position is , dandelion seeds in the dandelion population h +1 iteration position is , dandelion seeds in the dandelion population h +1 iteration random position is , represents the normal distribution function, and the lift component coefficient of the dandelion seeds in the dandelion population under the vortex effect is set to and , then the dandelion seed number one in the dandelion population h The calculation formula for the +1 iteration position is as follows; ； in, Represents the adaptive parameter for adjusting the search step size, setting the maximum number of iterations to h , represents a random number and ,Adjust the adaptive parameters of the search step length.,As the adaptive parameters of the search step length are adjusted, the drone,constantly changes its position; Dandelion seeds are searched locally on rainy days. The flight of the drone is constrained at this time. The adjustment parameters for rainy days are set as , the rainy day adjustment parameters converge to the best dandelion seed as the number of iterations increases, and the dandelion seed No. h +1 iteration position update is , the calculation formula is as follows; ； S23, dandelion seeds are in the descending stage. At this time, the fitness function value is continuously updated. The drone updates its own position again and sets the dandelion seed No. h The average position at the iteration is , represents a Brownian motion of random numbers that follows a normal distribution, represents a random number and , the dandelion seeds in the dandelion population h +1 iteration position update is , the calculation formula is as follows; ； The drone arrived At the point, the dandelion seed is regarded as the fitness function value, and the best fitness function value is found; during the landing stage, the dandelion seed adjusts the adaptive parameter of the search step length, and the dandelion seed corresponding to the best fitness function value is taken as the elite dandelion seed, and the position of the elite dandelion seed is recorded as , introduce Levy flight, and set the Levy flight function to , the linear increasing function is , represents a random number and , the dandelion seeds in the dandelion population h +1 iteration position update is , the calculation formula is as follows; ； S3, the drone acquires the city aerial image on the drone inspection path, and when the city aerial image contains fog, locates the light source of the city aerial image, and then corrects the city aerial image according to the change of fog content to obtain the final city aerial image; and trains the YOLOv5 network to realize the detection and recognition of the city aerial objects; S4. After the drone completes the detection and identification of the urban aerial photography objects, it will feedback the abnormal information to the drone monitoring platform according to the abnormal coefficient of the urban aerial photography objects, avoid threats and dynamically plan the inspection route.
2. The fully automatic low-altitude inspection and monitoring method based on an unmanned aerial vehicle according to claim 1 is characterized in that: The S1 comprises the following steps: S11, establishing a three-dimensional rectangular coordinate system for the urban plannable space, dividing the urban plannable space using a three-dimensional grid method, and establishing a three-dimensional grid space; S12, the UAV flies under the constraints of maximum range, minimum turning radius, maximum yaw angle, maximum pitch angle and flight altitude, completing the performance constraints of the UAV; S13. Combining the UAV flight path length cost, the UAV flight energy consumption cost, the UAV flight threat cost and the UAV flight altitude cost, the UAV flight path evaluation index is obtained.
3. The fully automatic low-altitude inspection and monitoring method based on an unmanned aerial vehicle according to claim 2 is characterized in that: The dandelion seeds in the dandelion population are h +1 iteration position Record it as the first UAV flight path landmark point, and iterate continuously. When the current number of iterations is equal to the maximum number of iterations, stop iterating, and obtain the best UAV flight path landmark point set. Connect the UAV flight path landmark points in the best UAV flight path landmark point set, smooth the path, output the UAV inspection path, and transmit the UAV inspection path to the UAV monitoring platform.
4. The fully automatic low-altitude inspection and monitoring method based on an unmanned aerial vehicle according to claim 3 is characterized in that: The S3 comprises the following steps: S31, the drone acquires a city aerial image on the drone inspection path, and determines whether the city aerial image contains fog. If the city aerial image does not contain fog, it is recorded as the final city aerial image; otherwise, if the city aerial image contains fog, the foggy city aerial image is defogged to obtain the final city aerial image; S32, obtaining an image to be detected, performing defogging on the image to be detected, obtaining a processed image to be detected, forming an image sample set, and dividing the image sample set into an image training set and an image verification set; setting the initial learning rate of the YOLOv5 network to , the weight parameter is , the current number of iterations is k, the maximum number of iterations is K, the optimization method adopts the Adam optimization algorithm, the image training set is input into the YOLOv5 network, iterates continuously, and when the current number of iterations is equal to the maximum number of iterations, the iteration is stopped to obtain the trained YOLOv5 network; then the image verification set is input into the trained YOLOv5 network, and the accuracy threshold is set to , when the accuracy of the trained YOLOv5 network result is less than , and get the final YOLOv5 network; otherwise, adjust the weights until the trained YOLOv5 network result accuracy is less than , complete YOLOv5 network training.
5. The fully automatic low-altitude inspection and monitoring method based on an unmanned aerial vehicle according to claim 4 is characterized in that: A city aerial image sample set is formed according to the final city aerial image, and the city aerial image sample set is input into the final YOLOv5 network to realize detection and recognition of city aerial objects.
6. The fully automatic low-altitude inspection and monitoring method based on an unmanned aerial vehicle according to claim 5 is characterized in that: The S31 comprises the following steps: S311, determining the light source position of the aerial image of the foggy city, and obtaining the aerial image of the foggy city with the light source positioned; S312, updating the minimum filter window and refractive index of the aerial image of the foggy city with the locating light source, and then converting it into an RGB color image to obtain a final aerial image of the city.
7. The fully automatic low-altitude inspection and monitoring method based on an unmanned aerial vehicle according to claim 6 is characterized in that: The S4 comprises the following steps: S41, after the drone completes the detection and identification of the urban aerial photography object, the detected and identified urban aerial photography object feeds back abnormal information to the drone, calculates abnormal parameters, and feeds back abnormal information to the drone monitoring platform according to the abnormal parameters; S42. The drone encounters a threat on the drone inspection path, avoids the threat, and dynamically plans the inspection route in real time.
8. A system for implementing the fully automatic low-altitude inspection and monitoring method based on a drone as described in any one of claims 1 to 7, characterized in that: Specifically include: Performance constraint and flight path evaluation index module, UAV inspection path optimization module, urban aerial photography object detection and recognition module, and abnormal threat avoidance module; The performance constraint and flight path evaluation index module is used to establish the performance constraint of the UAV on the flight path and the cost factor of the UAV on the flight path; The UAV inspection path optimization module is used to plan the UAV inspection path based on the Dandelion optimization algorithm; The urban aerial photography object detection and recognition module is used to perform defogging on urban aerial photography images and then train the YOLOv5 network to detect and recognize urban aerial photography objects; The abnormal threat avoidance module is used to combine the abnormal information feedback threat of the drone and dynamically plan a path to avoid the threat.