Unmanned aerial vehicle identification and tracking method under complex city background based on image processing

Abstract

The invention relates to an unmanned aerial vehicle identification and tracking method under a complex city background based on image processing. Comprising the following steps: acquiring unmanned aerial vehicle image data, classifying unmanned aerial vehicle images according to a target background, performing quality evaluation, and establishing an unmanned aerial vehicle database; based on an unmanned aerial vehicle database, position coordinates and a working area of aiming equipment are determined; historical unmanned aerial vehicle tracking records are obtained, historical unmanned aerial vehicle object features are extracted, and historical unmanned aerial vehicle objects are divided into static unmanned aerial vehicle objects and moving unmanned aerial vehicle objects; based on the historical unmanned aerial vehicle object features, unmanned aerial vehicle object recognition is carried out in combination with the unmanned aerial vehicle image data, and position distribution analysis of the unmanned aerial vehicle object is carried out according to the recognition result of the unmanned aerial vehicle object; and evaluating whether to perform video monitoring on the unmanned aerial vehicle according to a position distribution analysis result, and exiting video monitoring when a video monitoring program is abnormal. According to the invention, the problems of misrecognition, missed recognition and the like are avoided, and the system stability and reliability are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition and tracking, and particularly relates to a method for identifying and tracking drones in a complex urban background based on image processing. Background Art

[0002] An intelligent aiming scheme that has gradually emerged in recent years with the rapid development of artificial intelligence and image recognition. This method takes image processing technology as the core and combines algorithms such as machine learning to achieve automatic recognition and tracking of drones through a sight, thereby improving the success rate of locking targets and realizing real-time and precise monitoring and management of "low, slow, and small" drone targets.

[0003] However, although the method for identifying and tracking drones in a complex urban background based on image processing technology has many advantages, it still faces some challenges and limitations in practical applications. For example, for drone targets in a complex urban background, due to various factors such as background object interference, line-of-sight occlusion during aiming, and a large amount of resources occupied by program operation, existing technologies often have difficulty accurately distinguishing and identifying drone targets in a complex urban background, which may lead to problems such as misidentification and missed identification, affecting the stability and reliability of the system. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for identifying and tracking drones in a complex urban background based on image processing.

[0005] The technical solution for achieving the purpose of the present invention is as follows: A method for identifying and tracking drones in a complex urban background based on image processing, including the following steps:

[0006] Step S100: Obtain drone image data from a database, classify the drone images according to the target background, evaluate the quality of the classified drone image data, and establish a drone database based on the quality evaluation results;

[0007] Step S200: Based on the drone database, determine the position coordinates of the aiming device and the working area of the aiming device;

[0008] Step S300: Obtain historical drone tracking records, extract historical drone object features from the historical drone tracking records, and classify the historical drone objects into two categories: stationary drone objects and moving drone objects according to the historical drone object features;

[0009] Step S400: Based on the historical drone object features, combine the drone image data to identify drone objects, and analyze the position distribution of the drone objects according to the identification results of the drone objects;

[0010] Step S500: Evaluate whether to conduct video surveillance on the UAV according to the analysis result of the UAV object's position distribution, and exit the video surveillance in a timely manner when an abnormality occurs in the video surveillance program.

[0011] Further, step S100 includes:

[0012] S101: Obtain the UAV image data of the selected target background from the database, and classify the UAV image data according to the target background. One set of UAV image data of a selected target background corresponds to one UAV database;

[0013] S102: Conduct quality evaluation on the UAV image data for each target background, thereby establishing a UAV image quality score A. The specific calculation formula for the UAV image quality score A is:

[0014]

[0015] where M and N represent the image size of M×N, and F(i,j) represents the pixel gray value of the image to be evaluated at the coordinate (i,j); the formula is the root mean square of the average pixel gray value and the standard deviation of the pixel gray value of the UAV image, comprehensively considering the average image brightness and the degree of dispersion of the UAV image pixel gray value relative to the mean value. The larger the calculation result of the formula, the higher the quality score A and the higher the UAV image quality;

[0016] S103: Randomly select some UAV images, obtain the quality score A of this part of UAV images, calculate the average value μ of the UAV image quality scores, compare the sizes of all UAV image quality scores A and the average value μ of the UAV image quality scores, use the UAV image data with A≥μ as the target image data, and delete the UAV image data with A<μ, thereby establishing a UAV database, and different UAVs under different target backgrounds correspond to different UAV databases.

[0017] Further, the calculation process of the average value μ of the UAV image quality scores in step S103 is as follows:

[0018] First, randomly select n scores from the quality scores A of the randomly selected part of UAV images, and denote them as A 1 , and the corresponding image is denoted as image 1; A 2 , and the corresponding image is denoted as image 2; A i , and the corresponding image is denoted as image i; …… An, and the corresponding image is denoted as image n; these are called the scoring benchmarks, and the corresponding n images are called the image benchmarks;

[0019] Except for the image benchmarks, the quality scores of the remaining randomly selected part of UAV images are respectively denoted as A i , where i represents the i-th randomly selected part of UAV images except for the randomly selected n scores;

[0020] Based on the shortest distance method for clustering analysis, except for the image benchmark, the distance D of the i-th randomly selected part of the UAV images i The calculation formula is:

[0021] D i = min{|A i - A 1 |, |A i - A 2 |, |A i - A 3 |}

[0022] If:

[0023] D i = |A i - A 1 |

[0024] Then, except for the image benchmark, the i-th randomly selected part of the UAV images is grouped into the same category as Image 1;

[0025] If:

[0026] D i = |A i - A 2 |

[0027] Then, except for the image benchmark, the i-th randomly selected part of the UAV images is grouped into the same category as Image 2;

[0028] If:

[0029] D i = |A i - A 3 |

[0030] Then, except for the image benchmark, the i-th randomly selected part of the UAV images is grouped into the same category as Image 3;

[0031] After clustering, 3 categories are obtained, denoted as category j (j = 1, 2, 3), the number of images in category j is denoted as N j , the mean value of the image quality score A in category j is denoted as μ j , the corrected mean value of the image quality score A in category j is denoted as v j , the number of randomly selected part of the UAV images is denoted as N;

[0032] The specific calculation formula for the corrected mean value v of the image quality score A in category j j is:

[0033]

[0034] The specific calculation formula for randomly extracting the average value μ of the UAV image quality scores is as follows:

[0035]

[0036] Furthermore, step S200 includes:

[0037] S201: According to the UAV database, establish a two-dimensional plane image of the aiming scope working area based on the target background, and establish a plane rectangular coordinate system on the two-dimensional plane image; mark the position when aiming at the UAV on the two-dimensional plane image of the aiming scope working area, so as to obtain the position coordinates of the aiming scope; number different position coordinates, and the UAV databases established by different position coordinates are different;

[0038] S202: Determine the position coordinates of the aiming device and the working area of the aiming device according to the average value A of the UAV image quality scores in the UAV databases established by different position coordinates.

[0039] Furthermore, "determine the position coordinates of the aiming device and the working area of the aiming device" in step S202 is specifically:

[0040] The average value A of the UAV image quality scores in the UAV databases established by different position coordinates is respectively denoted as μ s , and the arithmetic mean of all μ s is denoted as x; for μ s >x, the position coordinates are called easy-to-aim positions, and μ<x position coordinates are called difficult-to-aim positions;

[0041] Select the coordinates of the easy-to-aim positions as the position coordinates for the aiming device to work; if there are multiple adjacent easy-to-aim positions, then use the adjacent multiple easy-to-aim positions as the working area of the aiming device;

[0042] Aiming should be carried out at the position coordinates where the aiming device works or in the working area of the aiming device.

[0043] Furthermore, step S300 includes:

[0044] S301: Obtain the historical UAV tracking records from the UAV database, and read the relevant historical data according to the historical tracking records; the historical data includes the description and timestamp of the historical tracking records;

[0045] S302: Extract the corresponding UAV objects according to the description of the historical tracking records in the historical data, and analyze the characteristics corresponding to the UAV objects based on the images captured by the aiming scope, so as to obtain the historical characteristics of the UAV objects; the historical characteristics of the UAV objects include position characteristics and image blurring caused by movement; for each UAV object, extract the historical characteristics of the UAV object;

[0046] S303: Set the dynamic UAV target judgment parameter y according to the historical feature extraction result 1 and establish a calculation method for y1, and classify the historical UAV objects into two categories: stationary UAV objects and moving UAV objects;

[0047] S304: Update the database categories to a stationary UAV database and a moving UAV database according to the stationary UAV objects and moving UAV objects in the historical UAV objects.

[0048] Furthermore, the dynamic UAV target judgment parameter y in step S303 1 The calculation method is as follows:

[0049] Extract the corresponding UAV object, and calculate the gray mean value μ of the background pixels where the UAV is located g , convert the area where the pixel gray value in the image is higher than μ g to black, and convert the area where the pixel gray value in the image is lower than or equal to μ g to white to form a historical UAV binary image;

[0050] Calculate the area S of the black part in the historical UAV binary image o , perform edge detection on the black part of the historical UAV binary image, and calculate the area S enclosed by the formed closed boundary i ;

[0051] For the area S o of the black part in the historical UAV binary image and the area S i enclosed by the closed boundary, establish the dynamic UAV target judgment parameter y 1 :

[0052]

[0053] The dynamic UAV target judgment parameter y 1 The threshold is 2, that is: if y 1 ≥2, then judge that the object corresponding to this UAV target in the historical UAV image is a moving UAV object, if y 1 ≤2, then judge that the object corresponding to this UAV target in the historical UAV image is a stationary UAV object.

[0054] Furthermore, step S400 includes:

[0055] S401: Use the YoloV2 network for recognition and tracking, and draw a bounding box to determine the position distribution of the UAV object;

[0056] Step S500 includes:

[0057] S501: Establish an array Im of captured images, capture the first frame of the image for recognition, and determine whether the recognition of this frame of the image is successful. If the determination is successful, continue to capture the next frame of the image, and the previous frame of the image continues to be stored in the array without being deleted. If the determination of recognition is unsuccessful, clear the queue of the existing captured image array Im, and then the next frame of the image captured thereafter is stored as the first frame of the image in the captured image array Im;

[0058] S502: Establish a daemon process queue Pro, store the recognition program in the daemon process queue Pro, and determine whether an abnormal situation occurs. If an abnormal situation occurs, clear the daemon process queue Pro. If no abnormal situation occurs, do not clear the daemon process queue Pro.

[0059] Further, the specific determination method for "whether the recognition of this frame of the image is successful" in step S501 is as follows:

[0060] Assume that the recognition process of n - 1 frames of images has been completed at this time and all are determined to be recognized successfully. Currently, the nth frame of the image is being determined. The recognition time for each frame of the image is t i , and the correct recognition rate for each frame of the image recognition is P i , and the average recognition time for the previous n - 1 frames of images is T n-1 :

[0061]

[0062] If the recognition is completed when t n < T n-1 , it is determined that the recognition is successful; if the recognition is not completed when t n ≥ T n-1 , it is determined that the recognition is unsuccessful;

[0063] For the first frame of the image, it is stipulated that if the recognition is completed when t 1 < 2s, it is determined that the recognition is successful; if the recognition is not completed when t n ≥ 2s, it is determined that the recognition is unsuccessful.

[0064] Further, the specific implementation method of the daemon process queue Pro in step S502 is as follows:

[0065] The running recognition program is stored in the daemon process queue Pro, and it includes a set of monitors. The abnormal situations that the monitors can monitor are recorded as the abnormal feature vector Su(t i , I j-5 , I j ), where t i represents the recognition time of the i - th frame of the image, and I j-5 represents the average gray value of the pixels of the captured image 5s before the current time, and I j represents the average gray value of the pixels of the current time image;

[0066] Analyze for t i If t i ≥ 10s, it is determined that the current recognition program may be stuck, which is an abnormal situation, and the daemon process queue Pro immediately releases the recognition program; if t i < 10s, it is determined that the current recognition program is not stuck temporarily, which is a non-abnormal situation, and the daemon process queue Pro does not release the recognition program;

[0067] Analyze for I j-5 and I j Define the aiming scope freeze judgment parameter z 1 as:

[0068] z 1 = I j - I j-5

[0069] If z 1 = 0, it is determined that the current aiming scope may be frozen, which is an abnormal situation, and the daemon process queue Pro immediately releases the recognition program; if z 1 ≠ 0, it is determined that the current aiming scope is not frozen temporarily, which is a non-abnormal situation, and the daemon process queue Pro does not release the recognition program;

[0070] Analyze for t i and I j Define the aiming scope screen abnormality judgment parameter z 2 as:

[0071]

[0072] If z 2 = 0, it is determined that the current aiming scope screen may be abnormal, which is an abnormal situation, and the daemon process queue Pro immediately releases the recognition program; if z 2 ≠ 0, it is determined that the current aiming scope is not abnormal temporarily, which is a non-abnormal situation, and the daemon process queue Pro does not release the recognition program;

[0073] The abnormal feature vector Su(t i , I j-5 , I j ) is independently detected for each element. If any element triggers the above judgment mechanism, the daemon process queue Pro immediately releases the recognition program;

[0074] In addition to the abnormal feature vector Su obtained by the monitor, if the main program exits, the daemon process queue Pro immediately releases the recognition program.

[0075] Compared with the prior art, the remarkable advantages of the present invention are:

[0076] The present invention realizes the effective management and data integration of aiming devices by evaluating the quality of UAV image data and establishing a database; utilizes the UAV image data, combines with the position information of the aiming devices, and provides a basis for subsequent UAV identification and tracking; realizes the preliminary screening and feature extraction of UAV objects through the extraction and classification of historical UAV tracking records; combines the historical UAV object features with the UAV image data, realizes the accurate identification and position distribution analysis of target objects; determines whether to conduct video monitoring on the UAV according to the analysis results, and exits the video monitoring in a timely manner when an abnormality occurs in the video monitoring program. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is a flowchart of the UAV identification and tracking method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0078] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0079] Please refer to Figure 1 , the present invention provides a technical solution:

[0080] A UAV identification and tracking method in a complex urban background based on image processing technology, the method includes the following steps:

[0081] Step S100. Obtain UAV image data from the database, classify the UAV images according to the target background, evaluate the quality of the classified UAV image data, and establish a UAV database according to the quality evaluation results;

[0082] S101. Obtain UAV image data of a selected target background from the database, classify the UAV image data according to the target background, and one set of UAV image data of a selected target background corresponds to a UAV database;

[0083] S102. Evaluate the quality of the UAV image data for each target background, thereby establishing a UAV image quality score A, and the specific calculation formula of the UAV image quality score A is:

[0084]

[0085] Among them, M and N represent the image size of M×N, and F(i,j) represents the pixel gray value of the image to be evaluated at the coordinate (i,j). This formula is the root mean square of the average pixel gray value and the standard deviation of the pixel gray value of the UAV image, comprehensively considering the average brightness of the image and the degree of dispersion of the pixel gray value of the image relative to the mean value. The larger the calculation result of the formula, the higher the quality score A and the higher the quality of the UAV image.

[0086] S103. Randomly select some UAV images, and obtain the quality score A of this part of UAV images. Calculate the average value μ of the UAV image quality scores, compare the size of all UAV image quality scores A with the average value μ of the UAV image quality scores, use the UAV image data with A≥μ as the target image data, and delete the UAV image data with A<μ, so as to establish a UAV database, and different UAVs under different target backgrounds correspond to different UAV databases.

[0087] The calculation process of the average value μ of the UAV image quality score described in S103 is as follows

[0088] First, randomly select 3 scores from the quality scores A of the randomly selected part of UAV images, and record them as A 1 (the corresponding image is recorded as image 1), A 2 (the corresponding image is recorded as image 2), A 3 (the corresponding image is recorded as image 3), which are called the scoring benchmarks, and the corresponding 3 images are called the image benchmarks;

[0089] Except for the image benchmarks, the quality scores of the remaining randomly selected part of UAV images are respectively recorded as A i , where i represents the i-th randomly selected part of UAV images except for the randomly selected 3 scores;

[0090] Based on the shortest distance method for cluster analysis, the distance D i of the i-th randomly selected part of UAV images except for the image benchmarks is calculated as follows:

[0091] D i =min{|A i -A 1 |,|A i -A 2 |,|A i -A 3 |}

[0092] If:

[0093] D i =|A i -A 1 |

[0094] Then, except for the image benchmark, the $i$-th randomly selected part of the UAV images is classified into the same category as Image 1;

[0095] If:

[0096] D i = |A i - A 2 |

[0097] Then, except for the image benchmark, the $i$-th randomly selected part of the UAV images is classified into the same category as Image 2;

[0098] If:

[0099] D i = |A i - A 3 |

[0100] Then, except for the image benchmark, the $i$-th randomly selected part of the UAV images is classified into the same category as Image 3;

[0101] After clustering, three categories are obtained, denoted as category $j$ ($j = 1, 2, 3$), the number of images in category $j$ is denoted as $N$ j , the mean value of the image quality score $A$ in category $j$ is denoted as $\mu$ j , the corrected mean value of the image quality score $A$ in category $j$ is denoted as $v$ j , the number of randomly selected UAV images is denoted as $N$;

[0102] The corrected mean value $v$ of the image quality score $A$ in category $j$ j The specific calculation formula is:

[0103]

[0104] Then the specific calculation formula for the average value $\mu$ of the image quality scores of the randomly selected UAV images is:

[0105]

[0106] Step S200. Based on the UAV database, determine the position coordinates of the aiming device and the working area of the aiming device;

[0107] S201. According to the UAV database, establish a two-dimensional plane image of the aiming scope working area based on the target background, and establish a plane rectangular coordinate system on the two-dimensional plane image; mark the position when aiming at the UAV on the two-dimensional plane image of the aiming scope working area, so as to obtain the position coordinates of the aiming scope; different position coordinates are numbered, and different UAV databases are established for different position coordinates.

[0108] S202. Determine the position coordinates of the aiming device and the working area of the aiming device according to the average value A of the UAV image quality scores in the UAV database established based on different position coordinates.

[0109] The specific confirmation method for the position coordinates of the aiming device and the working area of the aiming device described in S202 is as follows:

[0110] The average value A of the UAV image quality scores in the UAV database established based on different position coordinates is respectively denoted as μs, and the arithmetic mean of all μs is denoted as x. The position coordinates with μs > x are called easy-to-aim positions, and the position coordinates with μ < x are called difficult-to-aim positions;

[0111] Select the coordinates of the easy-to-aim position on the left as the position coordinates for the aiming device to work. If there are multiple adjacent easy-to-aim positions, then the multiple adjacent easy-to-aim positions are used as the working area of the aiming device;

[0112] Aiming should be carried out at the position coordinates where the aiming device works or in the working area of the aiming device.

[0113] Step S300. Obtain the historical UAV tracking records, extract the historical UAV object features from the historical UAV tracking records, and classify the historical UAV objects into two categories: stationary UAV objects and moving UAV objects according to the historical UAV object features;

[0114] S301. Obtain the historical UAV tracking records from the UAV database, and read the relevant historical data according to the historical tracking records; the historical data includes the description and timestamp of the historical tracking records;

[0115] S302. According to the description of the historical tracking records in the historical data, extract the corresponding UAV objects, and analyze the features corresponding to the UAV objects based on the images captured by the aiming scope, so as to obtain the historical features of the UAV objects; the historical features of the UAV objects include position features and image blurring caused by movement; for each UAV object, the historical features of the UAV object are extracted;

[0116] S303. According to the historical feature extraction results, establish the dynamic UAV target decision parameter y 1 , and classify the historical UAV objects into two major categories: stationary UAV objects and moving UAV objects;

[0117] The dynamic UAV target decision parameter y described in S303 1 The calculation method is:

[0118] Extract the corresponding UAV objects, and calculate the average gray value μ of the pixels in the background where the UAV is located g , and for the pixels in the image with gray levels higher than μ gThe area where the pixel grayscale in the image is lower than or equal to μ is converted to black. g The area where the pixel grayscale is higher than μ is converted to white, forming a binary image of the historical drone.

[0119] Calculate the area S of the black part in the binary image of the historical drone. o Perform edge detection on the black part of the binary image of the historical drone, and calculate the area S enclosed by the formed closed boundary. i ;

[0120] For the area S of the black part in the binary image of the historical drone o and the area S enclosed by the closed boundary i , establish the dynamic drone target judgment parameter y 1 :

[0121]

[0122] The dynamic drone target judgment parameter y 1 The threshold is 2, that is: if y 1 ≥2, then it is judged that the object corresponding to this drone target in the historical drone image is a moving drone object; if y 1 ≤2, then it is judged that the object corresponding to this drone target in the historical drone image is a stationary drone object.

[0123] S304. Update the database categories to the stationary drone database and the moving drone database according to the stationary drone objects and moving drone objects in the historical drone objects.

[0124] Step S400. Based on the characteristics of the historical drone objects, combine the drone image data to identify the drone objects, and analyze the position distribution of the drone objects according to the identification results of the drone objects.

[0125] S401. Use the YoloV2 network for identification and tracking, and draw a bounding box to determine the position distribution of the drone objects.

[0126] Step S500. Evaluate whether to perform video monitoring on the drone according to the analysis results of the position distribution of the drone objects, and exit the video monitoring in time when an exception occurs in the video monitoring program.

[0127] S501. Establish an array Im of captured images, capture the first frame of the image for identification, and determine whether the identification of this frame of the image is successful. If the judgment is successful, continue to capture the next frame of the image, and the previous frame of the image continues to be stored in the array without deletion. If the identification is judged to be unsuccessful, clear the existing queue of the captured image array Im, and the next frame of the image captured thereafter is stored as the first frame of the image in the captured image array Im;

[0128] The specific method for determining whether the recognition of this frame of image is successful is as follows:

[0129] Assume that the recognition process of n - 1 frames of images has been completed at this time and all are judged to be recognized successfully. Currently, the nth frame of image is being judged. The recognition time for each frame of image is t i , and the correct recognition rate for each frame of image recognition is P i , and the average recognition time for the first n - 1 frames of images is T n-1 :

[0130]

[0131] If the recognition is completed when t n <T n-1 , it is determined that the recognition is successful; if the recognition is not completed when t n ≥T n-1 , it is determined that the recognition is not successful;

[0132] In particular, for the first frame of image, it is stipulated that if the recognition is completed when t 1 <2s, it is determined that the recognition is successful; if the recognition is not completed when t n ≥2s, it is determined that the recognition is not successful.

[0133] S502. Establish a daemon process queue Pro, store the recognition program in the daemon process queue Pro, and determine whether an abnormal situation occurs. If an abnormal situation occurs, clear the daemon process queue Pro; if no abnormal situation occurs, do not clear the daemon process queue Pro.

[0134] The specific implementation method of the daemon process queue Pro described in S502 is as follows:

[0135] The running recognition program is stored in the daemon process queue Pro, and it includes a set of monitors. The abnormal situations that the monitors can detect are recorded as the abnormal feature vector Su(t i ,I j-5 ,I j ), where t i represents the recognition time of the i - th frame of image, I j-5 represents the average gray value of the pixels of the image taken 5s before the current time, and I j represents the average gray value of the pixels of the current time image;

[0136] Analyze t i . If t i ≥10s, it is determined that the current recognition program may be stuck, which is an abnormal situation, and the daemon process queue Pro immediately releases the recognition program; if t i <10s, it is determined that the current recognition program does not have a stuck phenomenon temporarily, which is a non - abnormal situation, and the daemon process queue Pro does not release the recognition program

[0137] For I j-5 And I j Analyze and define the aiming scope freeze judgment parameter z 1 As:

[0138] z 1 = I j - I j-5

[0139] If z 1 = 0, it is determined that the current aiming scope may be frozen, which is an abnormal situation, and the daemon process queue Pro immediately releases the recognition program; if z 1 ≠0, it is determined that the current aiming scope is not frozen temporarily, which is a non-abnormal situation, and the daemon process queue Pro does not release the recognition program;

[0140] For t i And I j Analyze and define the aiming scope screen abnormality judgment parameter z 2 As:

[0141]

[0142] If z 2 = 0, it is determined that the current aiming scope screen may be abnormal, which is an abnormal situation, and the daemon process queue Pro immediately releases the recognition program; if z 2 ≠0, it is determined that the current aiming scope is not abnormal temporarily, which is a non-abnormal situation, and the daemon process queue Pro does not release the recognition program;

[0143] Each element of the abnormal feature vector Su(t i , I j-5 , I j ) is independently detected. If any element triggers the above judgment mechanism, the daemon process queue Pro immediately releases the recognition program;

[0144] In particular, in addition to the abnormal feature vector Su obtained by the monitor, if the main program exits, the daemon process queue Pro immediately releases the recognition program.

[0145] In this embodiment, assume that the current monitor obtains an abnormal feature vector representation Su(t i , I j-5 , I j ); Determine the size of t i , calculate the aiming scope freeze judgment parameter z 1 and the aiming scope screen abnormality judgment parameter z 2 ; Assume that if t i ≥10s or z 1 = 0 or z 2= 0 indicates that an abnormal event exists, and the daemon process queue Pro should release the recognition program; only when the assumption t i <is less than 10 s and z 1 ≠ 0 and z 2 ≠ 0 indicates that no abnormal event exists, and the daemon process queue Pro should not release the recognition program.

[0146] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device.

[0147] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for identifying and tracking drones in a complex urban background based on image processing, characterized in that: The steps include: Step S100: acquiring drone image data from a database, classifying the drone images according to target backgrounds, performing quality assessment on the classified drone image data, and establishing a drone database according to the quality assessment results; Step S200: determining the position coordinates of the aiming device and the working area of ​​the aiming device based on the drone database; Step S300: obtaining historical drone tracking records, extracting historical drone object features from the historical drone tracking records, and classifying historical drone objects into two categories: stationary drone objects and moving drone objects according to the historical drone object features; Step S400: Based on historical drone object features and combined with drone image data, drone object identification is performed, and location distribution analysis of the drone object is performed according to the drone object identification result; Step S5 00: Evaluate whether to conduct video surveillance on the drone based on the location distribution analysis results of the drone object, and exit video surveillance when an abnormality occurs in the video surveillance program.

2. The method according to claim 1, characterized in that Step S100 includes: S101: Obtain drone image data of a selected target background from a database, and classify the drone image data according to the target background. One drone image data of a selected target background corresponds to one drone database; S102: Performing a quality assessment on the drone image data of each target background, thereby establishing a drone image quality score A, and the specific calculation formula of the drone image quality score A is: Among them, M and N represent the image size of M×N, and F(i,j) represents the pixel grayscale value of the image to be evaluated at the coordinate (i,j). The formula is the square root of the pixel grayscale mean and the standard deviation of the pixel grayscale value of the drone image, which comprehensively considers the average brightness of the image and the discrete degree of the image pixel grayscale value relative to the mean. The larger the result of the formula calculation, the higher the quality score A, and the higher the quality of the drone image. S103: Randomly extract some drone images and obtain the quality scores A of these drone images, calculate the average quality score μ of the drone images, compare the quality scores A of all drone images with the average quality score μ of the drone images, use the drone image data with A≥μ as the target image data, and delete the drone image data with A<μ, so as to establish a drone database, and drones under different target backgrounds correspond to different drone databases.

3. The method according to claim 2, characterized in that The calculation process of the average value μ of the drone image quality score in step S103 is as follows: First, n scores are randomly selected from the quality scores A of a randomly selected portion of drone images, and are recorded as A1, the corresponding image is recorded as image 1; A2, the corresponding image is recorded as image 2; A i , the corresponding image is recorded as image i; ...An, the corresponding image is recorded as image n; it is called the scoring benchmark, and the corresponding n images are called image benchmarks; In addition to the image benchmark, the quality scores of the remaining randomly selected drone images are recorded as A i , i means that in addition to randomly selecting n ratings, the i-th randomly selected part of the drone images; Cluster analysis is performed based on the shortest distance method. In addition to the image benchmark, the distance D of the i-th randomly selected drone image is i The calculation formula is: <h2 style=";text-align:left;direction:ltr">D<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =min{|A<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> -A1|,|A<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> -A2|,|A<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> -A3|} like: D i =|A i -A1| Then, in addition to the image baseline, the i-th randomly selected part of the drone image is classified into the same category as image 1; like: D i =|A i -A2| Then, in addition to the image baseline, the i-th randomly selected part of the drone image is classified into the same category as image 2; like: <h2 style=";text-align:left;direction:ltr">D<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =|A<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> -A3| Then, in addition to the image baseline, the i-th randomly selected part of the drone image is classified into the same category as image 3; After clustering, we get three categories, denoted as category j (j=1, 2, 3), and the number of images in category j is denoted as N j , the mean value of the image quality score A in category j is denoted as μ j , the corrected mean of the image quality score A in category j is denoted as v j ,The number of randomly selected drone images is recorded as N; The corrected mean v of the image quality score A in category j j The specific calculation formula is: The specific calculation formula for the average quality score μ of randomly selected UAV images is:

4. The method according to claim 3, characterized in that Step S200 includes: S201: According to the UAV database and the target background, a two-dimensional plane image of the working area of ​​the sight is established, and a plane rectangular coordinate system is established on the two-dimensional plane image; the position of the sighting UAV is marked on the two-dimensional plane image of the working area of ​​the sight, so as to obtain the position coordinates of the sight; different position coordinates are numbered, and different UAV databases are established for different position coordinates; S202: Determine the position coordinates of the aiming device and the working area of ​​the aiming device according to the average value of the drone image quality score A in the drone database established at different position coordinates.

5. The method according to claim 4, characterized in that The step S202 of "determining the position coordinates of the aiming device and the working area of ​​the aiming device" is specifically: The average value A of the UAV image quality scores of the UAV databases established with different position coordinates is denoted as μ s , and the arithmetic mean of all μ s is denoted as x; for the position coordinates where μ s >x, they are called easy-to-aim positions, and the position coordinates where μ < x are called difficult-to-aim positions; The left coordinates of the easy-to-aiming position are selected as the position coordinates of the aiming device; if there are multiple easy-to-aiming positions adjacent to each other, the multiple adjacent easy-to-aiming positions are used as the working area of ​​the aiming device; Aiming should be carried out at the position coordinates where the aiming device works or in the working area of ​​the aiming device.

6. The method according to claim 5, characterized in that Step S300 includes: S301: Obtain historical drone tracking records from a drone database, and read relevant historical data according to the historical tracking records; the historical data includes a description and a timestamp of the historical tracking records; S302: extracting corresponding drone objects according to the description of the historical tracking records in the historical data, and analyzing the features corresponding to the drone objects based on the images taken by the sight, so as to obtain historical features of the drone objects; the historical features of the drone objects include position features and image blur caused by movement; extracting historical features of the drone objects for each drone object; S303: according to the historical feature extraction results, a dynamic UAV target decision parameter y1 is set, and a calculation method of y1 is established to classify historical UAV objects into two categories: stationary UAV objects and moving UAV objects; S304: According to the stationary drone objects and the moving drone objects in the historical drone objects, the database categories are updated into a stationary drone database and a moving drone database.

7. The method according to claim 6, characterized in that The calculation method of the dynamic UAV target decision parameter y1 in step S303 is: Extract the corresponding drone object and calculate the grayscale mean μ of the background pixels where the drone is located g , the pixels in the image with grayscale higher than μ g The area is converted to black, and the pixel grayscale in the image is less than or equal to μ g The area is converted to white to form a historical drone binary image; Calculate the area S of the black part in the historical drone binary image o , perform edge detection on the black part of the historical drone binary image and calculate the area S enclosed by the closed boundary i ; The area S of the black part in the historical drone binary image o The area S enclosed by the closed boundary i , establish the dynamic UAV target decision parameter y1: The threshold value of the dynamic UAV target judgment parameter y1 is 2, that is: if y1≥2, the object corresponding to this UAV target in the historical UAV image is judged to be a moving UAV object; if y1≤2, the object corresponding to this UAV target in the historical UAV image is judged to be a stationary UAV object.

8. The method according to claim 1, characterized in that Step S400 includes: S401: Use the YoloV2 network for identification and tracking, and draw a bounding box to determine the location distribution of the drone object; Step S500 includes: S501: Establishing a captured image array Im, capturing a first frame of image for recognition, and determining whether the recognition of the frame of image is successful. If the recognition is successful, continue capturing the next frame of image, and the previous frame of image continues to be stored in the array without being deleted. If the recognition is unsuccessful, clear the queue of the existing captured image array Im, and the next frame of image captured thereafter is stored as the first frame of image in the captured image array Im; S502: Establish a daemon process queue Pro, store the identification program in the daemon process queue Pro, and determine whether an abnormal situation occurs. If an abnormal situation occurs, clear the daemon process queue Pro; if no abnormal situation occurs, do not clear the daemon process queue Pro.

9. The method according to claim 8, characterized in that The specific judgment method of "whether the recognition of the frame image is successful" in step S501 is: Assume that the recognition process of n-1 frames of images has been completed and all of them are judged to be successful. The current image is the nth frame, and the recognition time of each frame is t i , the accuracy of each frame image recognition is P i , the average recognition time of the first n-1 frames is T n-1 : If n <T n-1 If the recognition is completed when t n ≥T n-1 If the recognition is still not completed when the time comes, it is judged as unsuccessful; For the first frame image, if the recognition is completed when t1<2s, it is considered as successful; if t n If the recognition is not completed within ≥2s, the recognition is considered unsuccessful.

10. The method according to claim 9, characterized in that The specific implementation of the daemon process queue Pro in step S502 is: The daemon queue Pro stores the running recognition program and contains a set of monitors. The abnormal conditions that the monitors can monitor are recorded as abnormal feature vectors Su(t i ,I j-5 ,I j ), where t i Indicates the recognition time of the i-th frame image, I j-5 Indicates the average grayscale value of the image pixels captured 5 seconds before the current time, I j Indicates the average gray value of the image pixels at the current time; For t i For analysis, if t i ≥10s, it is determined that the current recognition program may be stuck, which is an abnormal situation, and the daemon queue Pro immediately releases the recognition program; if t i <10s, it is determined that the current recognition program is not stuck temporarily, which is a normal situation, and the Daemon Queue Pro will not release the recognition program; For I j-5 with I j After analysis, the judgment parameter z1 of the scope crash is defined as: z1=I j -I j-5 If z1=0, it is determined that the current scope may be frozen, which is an abnormal situation, and the daemon queue Pro immediately releases the recognition program; if z1≠0, it is determined that the current scope is not frozen temporarily, which is a non-abnormal situation, and the daemon queue Pro does not release the recognition program; For t i with I j After analysis, the judgment parameter z2 of the sight image abnormality is defined as: If z2=0, it is determined that there may be an abnormality in the current sighting image. If it is an abnormality, the daemon queue Pro immediately releases the recognition program; if z2≠0, it is determined that there is no abnormality in the current sighting image temporarily. It is a non-abnormal situation, and the daemon queue Pro does not release the recognition program; abnormal The characteristic vector Su(t i ,I j-5 ,I j ) Each element is detected independently. If any element triggers the above judgment mechanism, the daemon queue Pro will immediately release the recognition program; In addition to the abnormal feature vector Su obtained by the monitor, if the main program exits, the daemon queue Pro immediately releases the identification program.

Images