A method for distinguishing drones from flying birds by integrating shape change features

Abstract

The present invention relates to a method for distinguishing drones from flying birds by integrating shape change features, comprising: S1. obtaining the target trajectory of a moving target in a surveillance video, and extracting a moving target sequence based on the target trajectory; S2. extracting a candidate target sequence from the moving target sequence based on a pre-trained drone target detection model; S3. selecting a candidate target at the i-th moment in the sequence based on the candidate target sequence, and calculating the shape change features between the candidate target and the candidate target at the moment before the i-th moment in the sequence; S4. obtaining the overall category confidence of the shape change features. If the overall category confidence is greater than a preset threshold, the candidate target at the i-th moment in the sequence is determined to be a drone. This solution can effectively distinguish birds in dynamic shapes from drones in fixed shapes by distinguishing shape change features, greatly improving the resolution capability and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method for distinguishing between drones and flying birds by integrating shape change features. Background Art

[0002] With the rapid growth of small consumer drones, effective regulation of these drones is becoming increasingly important. Among existing solutions, drone detection based on video image analysis is a major research hotspot. Known research results have enabled detection of drone targets with a scale range of 20 to 30 pixels.

[0003] For drone targets with a scale range of 20 to 30 pixels, a major problem faced by the actual application system is that flying bird targets in the same scale range are very likely to trigger false alarms, seriously affecting the user experience. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for distinguishing between drones and flying birds by integrating shape change features.

[0005] To achieve the above-mentioned object, the present invention provides a method for distinguishing between drones and flying birds by integrating shape change features, comprising:

[0006] S1. Obtain a moving target sequence of a moving target based on a surveillance video;

[0007] S2. extracting a candidate target sequence from the moving target sequence based on a pre-trained UAV target detection model;

[0008] S3. Selecting a candidate target at the i-th moment in the sequence based on the candidate target sequence, and calculating the shape change characteristics between the candidate target at the i-th moment before the moment in the sequence;

[0009] S4. Obtaining the overall category confidence of the shape change feature. If the overall category confidence is greater than a preset threshold, determining that the candidate target at the i-th moment in the sequence is a drone.

[0010] According to one aspect of the present invention, in step S1, the step of obtaining a moving target sequence of a moving target in a surveillance video based on the surveillance video includes:

[0011] S11 intercepts multiple frames of the surveillance video surveillance image, based on the acquisition of the surveillance image using Gaussian background modeling method to establish the current background model of the surveillance video;

[0012] S12. Using the background model to perform background removal on each frame of the surveillance video, extracting the motion pixel area in the surveillance image;

[0013] S13. Denoising the moving pixel area and extracting the retained connected area;

[0014] S14. Tracking the connected regions of adjacent frames using a correlation filtering method, determining the membership relationship of the connected regions extracted from different frames, and determining the order of the connected regions based on the membership relationship;

[0015] S15. Obtain the centroid coordinates and spatial masks of the connected regions extracted from different frames, and construct the moving target sequence based on the order of the connected regions.

[0016] According to one aspect of the present invention, in step S11, the first N frames of the surveillance image in the surveillance video are intercepted to establish the background model;

[0017] In step S12, the background model is used to perform background removal processing on the surveillance image after the Nth frame in the surveillance video, and the motion pixel area is extracted.

[0018] According to one aspect of the present invention, in step S3, the shape change feature includes: a perimeter change feature and an area change feature;

[0019] Step S3 includes:

[0020] S31. Obtaining the normalized perimeter of the spatial mask corresponding to the candidate target at the i-th moment, and obtaining the normalized area of ​​the spatial mask corresponding to the candidate target at the i-th moment;

[0021] S32. Constructing a perimeter change trend between the candidate targets at different times, and constructing an area change trend between the candidate targets at different times;

[0022] S33. Selecting multiple groups of candidate targets at different times and calculating corresponding perimeter change values ​​based on the perimeter change trend, and selecting multiple groups of candidate targets at different times and calculating corresponding area change values ​​based on the area change trend;

[0023] S34. Construct the shape change feature based on the obtained multiple perimeter change values ​​and multiple area change values.

[0024] According to one aspect of the present invention, in step S31, the normalized perimeter is expressed as:

[0025]

[0026] Among them, L i represents the normalized circumference at the i-th moment; l i represents the number of all edge pixels in the spatial mask corresponding to the candidate target at the i-th moment; wi and h i Respectively represent the width and height of the spatial mask corresponding to the candidate target at the i-th moment;

[0027] The normalized area is expressed as:

[0028]

[0029] Among them, S i represents the normalized area; s i represents the number of all pixels in the spatial mask corresponding to the candidate target at the i-th moment; w i and h i They represent the width and height of the spatial mask corresponding to the candidate target at the i-th moment respectively.

[0030] According to one aspect of the present invention, in step S32, the circumference change trend is expressed as:

[0031] ΔL [i][j] =abs(L i -L j )

[0032] Where, ΔL [i][j] Indicates the circumference change trend between the i-th moment and the j-th moment; L i represents the normalized circumference at the i-th moment; L j represents the normalized circumference at the jth moment;

[0033] The area change trend is expressed as:

[0034] ΔS [i][j] =abs(S i -S j )

[0035] Where, ΔS [i][j] Indicates the area change trend between the i-th moment and the j-th moment; S i represents the normalized area at the i-th moment; S j represents the normalized area at the jth moment.

[0036] According to one aspect of the present invention, in step S33, in the step of selecting multiple groups of candidate targets at different moments and calculating corresponding circumference change values ​​based on the circumference change trends, three groups of candidate targets at different moments are selected and corresponding circumference change values ​​are calculated based on the circumference change trends, wherein the intervals between the first group of different moments are smaller than the intervals between the second group of different moments, and the intervals between the second group of different moments are smaller than the intervals between the third group of different moments;

[0037] In the steps of selecting multiple groups of candidate targets at different moments and calculating corresponding area change values ​​based on the area change trends, three groups of candidate targets at different moments are selected and corresponding area change values ​​are calculated based on the area change trends, wherein the intervals between the first group of different moments are smaller than the intervals between the second group of different moments, and the intervals between the second group of different moments are smaller than the intervals between the third group of different moments.

[0038] According to one aspect of the present invention, in step S4, the overall category confidence of the shape change feature is obtained. If the overall category confidence is greater than a preset threshold, the step of determining that the candidate target at the i-th moment in the sequence is a drone includes:

[0039] Calculating the category confidence corresponding to the perimeter change feature and the area change feature at the i-th moment respectively;

[0040] Obtaining the overall category confidence at the i-th moment based on the calculated category confidence;

[0041] The overall category confidence is compared with the threshold. If the overall category confidence is greater than the preset threshold, it is determined that the candidate target at the i-th moment in the sequence is a drone.

[0042] According to one aspect of the present invention, the overall category confidence is expressed as:

[0043] Q1=q1

[0044] Q i =(q i-1 +q i ) / 2,i>1

[0045] Among them, Q i Represents the overall category confidence, q i Represents the category confidence.

[0046] According to one solution of the present invention, the solution of the present invention can effectively distinguish birds in dynamic shapes and drones in fixed shapes by distinguishing shape change characteristics, which greatly improves the resolution ability and resolution efficiency of the solution.

[0047] According to one solution of the present invention, the solution of the present invention can effectively distinguish between flying birds and drones within a small scale range. It is not only highly efficient, but also effectively reduces the probability of false triggering, greatly improving the user experience.

[0048] According to a solution of the present invention, the amount of calculation during the execution of the solution of the present invention is small, which effectively reduces the performance requirements for the processing equipment and effectively enhances the practicality and scope of application of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The figure schematically shows a flowchart of a method for distinguishing between drones and flying birds by fusing shape change features according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments cannot be described one by one here, but the embodiments of the present invention are not limited to the following embodiments.

[0051] like Figure 1 As shown, according to one embodiment of the present invention, a method of distinguishing between drones and flying birds by integrating shape change features includes:

[0052] S1. Obtain a moving target sequence of a moving target based on a surveillance video;

[0053] S2. Extract candidate target sequences from moving target sequences based on the pre-trained UAV target detection model;

[0054] S3. Selecting a candidate target at the i-th moment in the sequence based on the candidate target sequence, and calculating the shape change feature between the candidate target at the i-th moment and the candidate target at the moment before the i-th moment in the sequence;

[0055] S4. Obtain the overall category confidence of the shape change feature. If the overall category confidence is greater than a preset threshold, the candidate target at the i-th moment in the sequence is determined to be a drone.

[0056] According to one embodiment of the present invention, in step S1, the step of obtaining a moving target sequence of a moving target in a surveillance video based on the surveillance video includes:

[0057] S11. Capture surveillance images from multiple frames of surveillance video, and establish a background model of the current surveillance video based on the acquired surveillance images using a Gaussian background modeling method; in this embodiment, the first N frames of surveillance images in the surveillance video are captured to establish a background model; wherein the value range of N can be set between 5 and 30.

[0058] S12. Use the background model to perform background removal processing on each frame of the surveillance image in the surveillance video, and extract the moving pixel area in the surveillance image; in this embodiment, the background model is used in combination with the background subtraction method to perform background removal processing on the surveillance image after the Nth frame in the surveillance video, and the moving pixel area is extracted.

[0059] S13. De-noising the moving pixel area and extracting the retained connected areas; in this embodiment, mathematical morphological methods such as median filtering, dilation, and erosion are used to filter out discrete noise interference in the moving pixel area and extract the retained connected areas.

[0060] S14. Using correlation filtering to track the connected regions of adjacent frames, determine the membership relationship of the connected regions extracted from different frames, and determine the order of the connected regions based on the membership relationship;

[0061] S15. Obtain the centroid coordinates and spatial masks of the connected regions extracted from different frames, and construct a moving target sequence based on the order of the connected regions, that is, the centroid coordinates and spatial masks of the same moving pixel region together constitute the sequence of the moving target.

[0062] According to one embodiment of the present invention, in step S2, a pre-trained deep learning-based drone target detection model is used to detect the motion pixel area (which can also be the center of mass coordinates and spatial mask) contained in each sequence, find the area classified as the drone target, and use the sequence corresponding to the area as the candidate target sequence.

[0063] like Figure 1 As shown, according to an embodiment of the present invention, in step S3, the shape change feature includes: perimeter change feature and area change feature. In this embodiment, step S3 includes:

[0064] S31. Obtain a normalized perimeter of the spatial mask corresponding to the candidate target at the i-th moment, and obtain a normalized area of ​​the spatial mask corresponding to the candidate target at the i-th moment; wherein the normalized perimeter obtained based on the spatial mask corresponding to the candidate target is expressed as:

[0065]

[0066] Among them, L i represents the normalized circumference at the i-th moment; l i represents the number of all edge pixels in the spatial mask corresponding to the candidate target at the i-th moment; w i and h i Respectively represent the width and height of the spatial mask corresponding to the candidate target at the i-th moment;

[0067] The normalized area obtained based on the spatial mask corresponding to the candidate target is expressed as:

[0068]

[0069] Among them, S i represents the normalized area; s i represents the number of all pixels in the spatial mask corresponding to the candidate target at the i-th moment; w i and h i They represent the width and height of the spatial mask corresponding to the candidate target at the i-th moment respectively.

[0070] S32. Constructing the perimeter change trend between candidate targets at different times, and constructing the area change trend between candidate targets at different times; in this embodiment, for the i-th and j-th target areas in the candidate target sequence, the perimeter change trend is expressed as:

[0071] ΔL [i][j] =abs(L i -L j )

[0072] Where, ΔL [i][j] Indicates the circumference change trend between the i-th moment and the j-th moment; L i represents the normalized circumference at the i-th moment; L j represents the normalized circumference at the jth moment;

[0073] For the i-th and j-th target regions in the candidate target sequence, their area change trends are expressed as:

[0074] ΔS [i][j] =abs(S i -S j )

[0075] Where, ΔS [i][j] Indicates the area change trend between the i-th moment and the j-th moment; S i represents the normalized area at the i-th moment; S j represents the normalized area at the jth moment.

[0076] S33. Select multiple groups of candidate targets at different moments and calculate corresponding circumference change values ​​based on the circumference change trend, and select multiple groups of candidate targets at different moments and calculate corresponding area change values ​​based on the area change trend; in this embodiment, in the step of selecting multiple groups of candidate targets at different moments and calculating corresponding circumference change values ​​based on the circumference change trend, three groups of candidate targets at different moments are selected and corresponding circumference change values ​​are calculated based on the circumference change trend, wherein the interval between the first group of different moments is less than the interval between the second group of different moments, and the interval between the second group of different moments is less than the interval between the third group of different moments; specifically, the spatial mask corresponding to the candidate target at the i-th moment in the candidate target sequence and the candidate targets of the previous 15 frames is used as the calculation interval, and the following three circumference change trend values ​​corresponding to the candidate target at the current i-th moment are calculated. In this embodiment, trajectory points of three groups of different frame sequence intervals, namely instantaneous, short-term and long-term, are selected for calculation, wherein adjacent frames are selected for instantaneous calculation, points with a frame interval of 6 are selected for short-term calculation, and points with a frame interval of 13 are selected for long-term calculation, and furthermore, they are respectively:

[0077] Instantaneous circumference change

[0078]

[0079] Short-term circumference changes

[0080]

[0081] Long-term circumference changes

[0082]

[0083] In the step of selecting multiple groups of candidate targets at different moments and calculating the corresponding area change values ​​based on the area change trend, three groups of candidate targets at different moments are selected and the corresponding area change values ​​are calculated based on the area change trend, wherein the intervals between the first group of different moments are smaller than the intervals between the second group of different moments, and the intervals between the second group of different moments are smaller than the intervals between the third group of different moments. Specifically, the spatial masks corresponding to the candidate targets at the i-th moment in the candidate target sequence and the candidate targets of the previous 15 frames are used as calculation intervals to calculate the following three area change trend values ​​corresponding to the candidate targets at the current i-th moment. In this embodiment, trajectory points of three groups of different frame sequence intervals, instantaneous, short-term and long-term, are selected for calculation, wherein adjacent frames are selected for instantaneous calculation, points with a frame interval of 6 are selected for short-term calculation, and points with a frame interval of 13 are selected for long-term calculation. Furthermore, they are respectively:

[0084] Instantaneous area change

[0085]

[0086] Short-term area change

[0087]

[0088] Long-term area changes

[0089]

[0090] S34. Construct a shape change feature based on the obtained multiple perimeter change values ​​and multiple area change values.

[0091] like Figure 1 As shown, according to one embodiment of the present invention, in step S4, the overall category confidence of the shape change feature is obtained. If the overall category confidence is greater than a preset threshold, the step of determining that the candidate target at the i-th moment in the sequence is a drone includes:

[0092] Calculate the category confidence q corresponding to the perimeter change feature and area change feature at the i-th moment respectively i In this embodiment, based on the three perimeter change trend values ​​and three area change trend values ​​corresponding to time i in the candidate target sequence obtained in the previous step, a pre-trained SVM classifier is used to calculate the corresponding category confidence q i ;

[0093] Based on the calculated category confidence q i Get the overall category confidence Q of the candidate target moving to the i-th moment i In this embodiment, the overall category confidence is expressed as:

[0094] Q1=q1

[0095] Q i =(q i-1 +q i ) / 2,i>1

[0096] Among them, Q i Represents the overall category confidence, q i Represents the category confidence.

[0097] The overall category confidence Q i Compared with the threshold, if the overall category confidence is greater than the preset threshold, the candidate target at the i-th moment in the sequence is determined to be a drone.

[0098] The above contents are merely examples of specific solutions of the present invention. For devices and structures not described in detail, it should be understood that they can be implemented by adopting general devices and methods available in the art.

[0099] The above description is merely one embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for distinguishing between drones and birds by integrating shape change features, comprising: S1. Obtain a moving target sequence of a moving target based on a surveillance video; S2. extracting a candidate target sequence from the moving target sequence based on a pre-trained UAV target detection model; S3. Selecting a candidate target at the i-th moment in the sequence based on the candidate target sequence, and calculating a shape change feature between the candidate target at the i-th moment and the candidate target at the moment before the i-th moment in the sequence; wherein the shape change feature includes: a perimeter change feature and an area change feature; Step S3 includes: S31. Obtaining the normalized perimeter of the spatial mask corresponding to the candidate target at the i-th moment, and obtaining the normalized area of ​​the spatial mask corresponding to the candidate target at the i-th moment; S32. Constructing a perimeter change trend between the candidate targets at different times, and constructing an area change trend between the candidate targets at different times; S33. Selecting multiple groups of candidate targets at different moments and calculating corresponding perimeter change values ​​based on the perimeter change trends, and selecting multiple groups of candidate targets at different moments and calculating corresponding area change values ​​based on the area change trends; wherein, in the step of selecting multiple groups of candidate targets at different moments and calculating corresponding perimeter change values ​​based on the perimeter change trends, three groups of candidate targets at different moments are selected and corresponding perimeter change values ​​are calculated based on the perimeter change trends, wherein the intervals between the first group of different moments are smaller than the intervals between the second group of different moments, and the intervals between the second group of different moments are smaller than the intervals between the third group of different moments; In the step of selecting multiple groups of candidate targets at different moments and respectively calculating corresponding area change values ​​based on the area change trends, three groups of candidate targets at different moments are selected and respectively calculating corresponding area change values ​​based on the area change trends, wherein the intervals between the first group of different moments are smaller than the intervals between the second group of different moments, and the intervals between the second group of different moments are smaller than the intervals between the third group of different moments; S34. Constructing the shape change feature based on the obtained plurality of perimeter change values ​​and the plurality of area change values; S4. Obtaining the overall category confidence of the shape change feature. If the overall category confidence is greater than a preset threshold, determining that the candidate target at the i-th moment in the sequence is a drone.

2. The method according to claim 1, characterized in that In step S1, the step of obtaining a moving target sequence of a moving target in a surveillance video based on the surveillance video includes: S11 intercepts multiple frames of the surveillance video surveillance image, based on the acquisition of the surveillance image using Gaussian background modeling method to establish the current background model of the surveillance video; S12. Using the background model to perform background removal on each frame of the surveillance video, extracting the motion pixel area in the surveillance image; S13. Denoising the moving pixel area and extracting the retained connected area; S14. Tracking the connected regions of adjacent frames using a correlation filtering method, determining the membership relationship of the connected regions extracted from different frames, and determining the order of the connected regions based on the membership relationship; S15. Obtain the centroid coordinates and spatial masks of the connected regions extracted from different frames, and construct the moving target sequence based on the order of the connected regions.

3. The method according to claim 2, characterized in that In step S11, the first N frames of the surveillance image in the surveillance video are intercepted to establish the background model; In step S12, the background model is used to perform background removal processing on the surveillance image after the Nth frame in the surveillance video, and the motion pixel area is extracted.

4. The method according to claim 3, characterized in that In step S31, the normalized perimeter is expressed as: Among them, L i represents the normalized circumference at the i-th moment; l i represents the number of all edge pixels in the spatial mask corresponding to the candidate target at the i-th moment; w i and h i Respectively represent the width and height of the spatial mask corresponding to the candidate target at the i-th moment; The normalized area is expressed as: Among them, S i represents the normalized area; s i represents the number of all pixels in the spatial mask corresponding to the candidate target at the i-th moment; w i and h i They represent the width and height of the spatial mask corresponding to the candidate target at the i-th moment respectively.

5. The method according to claim 4, characterized in that In step S32, the circumference change trend is expressed as: ΔL [i][j] =abs(L i -L j ) Where, ΔL [i][j] Indicates the circumference change trend between the i-th moment and the j-th moment; L i represents the normalized circumference at the i-th moment; L j represents the normalized circumference at the jth moment; The area change trend is expressed as: ΔS [i][j] =abs(S i -S j ) Where, ΔS [i][j] Indicates the area change trend between the i-th moment and the j-th moment; S i represents the normalized area at the i-th moment; S j represents the normalized area at the jth moment.

6. The method according to claim 5, characterized in that In step S4, the overall category confidence of the shape change feature is obtained. If the overall category confidence is greater than a preset threshold, the step of determining that the candidate target at the i-th moment in the sequence is a drone includes: Calculating the category confidence corresponding to the perimeter change feature and the area change feature at the i-th moment respectively; Obtaining the overall category confidence of the candidate target moving to the i-th moment based on the calculated category confidence; The overall category confidence is compared with the threshold. If the overall category confidence is greater than the preset threshold, it is determined that the candidate target at the i-th moment in the sequence is a drone.

7. The method according to claim 6, characterized in that The overall category confidence is expressed as: Q1=q1 Q i =(q i-1 +q i ) / 2,i>1 Among them, Q i Represents the overall category confidence, q i Represents the category confidence.

Images