Bird detection, tracking and classification method and system based on double-spectrum photoelectricity

Abstract

The invention provides a bird detection, tracking and classification method and system based on double-spectrum photoelectric. The method comprises the following steps: S1, acquiring an image in a target area by using double-spectrum photoelectric equipment; the image comprises a visible light image and an infrared image; s2, detecting a flying bird target in the image based on a deep learning target detection algorithm, and locking the flying bird target; s3, obtaining the feature vector of the bird target, matching the feature vector with a preset database, obtaining a matching result of bird types, and calculating the position information of the bird; and S4, outputting the matching result of the bird types and the position information of the birds. According to the invention, a rapid target detection and tracking algorithm is used, the position of a bird can be detected and locked in real time when the bird enters a monitoring area, and the real-time performance is good; by adopting the bird classification technology based on deep learning, the type of the bird can be automatically identified according to the captured bird image, so that the behavior of the bird can be known conveniently and targeted measures can be taken conveniently.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a bird detection, tracking and classification method and system based on dual-spectrum photoelectricity. BACKGROUND

[0002] In the airport environment, birds pose a potential threat to the safe flight and landing of aircraft. Bird-aircraft collision accidents occur frequently, not only posing a serious threat to the safety of aircraft and passengers, but also causing huge economic losses to airlines. Therefore, timely and accurate detection, tracking and driving of birds to prevent them from approaching the airport runway area is a crucial task.

[0003] However, birds fly fast in the air and have high maneuverability, making it more difficult to detect birds than traditional detection targets such as ships or personnel. In addition, different types of birds differ significantly in size, color and behavior, and in practical applications, understanding the types of birds helps to take more effective bird driving measures. For example, large birds may require different driving strategies.

[0004] The present application aims to effectively improve the detection accuracy and classification efficiency of birds in the airport environment through a bird detection, tracking and classification method based on a dual-spectrum photoelectric system and deep learning technology, and to ensure the safety of flight in the airport and surrounding areas.

[0005] In Chinese patent document CN112907571A, a target determination method based on multi-spectral image fusion recognition is disclosed, the steps are as follows: S1, collect existing image samples; S2, import the image samples into the computer in a specific format, read and analyze the characteristics of the sample images, and label them according to the injection duration and characteristics after injection, and establish a sample database; S3, construct a multi-spectral image and a target recognition algorithm, respectively use an infrared camera and a visible light camera to collect image data, and perform image registration; use a deep learning model to automatically extract the time series characteristics of the multi-spectral image, input it into a classifier, and perform target detection, classification and positioning; S4, upload the target detection and classification results to the APP, and the medical staff judges the classification results to obtain the final result. This patent document helps nurses to make accurate judgments on the results of penicillin skin tests, which is essentially different from the technical problems to be solved by the present application. SUMMARY

[0006] In view of the defects in the prior art, the present application aims to provide a bird detection, tracking and classification method and system based on dual-spectrum photoelectricity.

[0007] According to the bird detection, tracking and classification method based on dual-spectrum photoelectricity provided by the present application, the method comprises the following steps:

[0008] Step S1: using a dual-spectrum photoelectric device to collect images in a target area;

[0009] The images include visible light images and infrared images;

[0010] Step S2: detecting a flying bird target in the images based on a deep learning target detection algorithm, and locking;

[0011] Step S3: obtaining a feature vector of the flying bird target and matching it with a preset database to obtain a matching result of the flying bird species, and calculating position information of the flying bird;

[0012] Step S4: outputting the matching result of the flying bird species and the position information of the flying bird.

[0013] Preferably, the step S3 includes the following sub-steps:

[0014] Step S3.1: regionally segmenting the flying bird target in the images to extract contour features of the flying bird;

[0015] Step S3.2: extracting a feature vector from the contour features and matching it with a preset database to obtain a matching result of the flying bird species;

[0016] Step S3.3: calculating a position deviation of the flying bird target from a center point of the image, and adjusting the dual-spectrum photoelectric device to realize flying bird tracking according to the position deviation;

[0017] Step S3.4: calculating position information of the flying bird according to the adjusted photoelectric device parameters.

[0018] Preferably, the preset database includes flying bird feature vectors extracted from recorded images of multiple flying bird species.

[0019] Preferably, the dual-spectrum photoelectric device includes a visible light camera and an infrared camera; the visible light camera is used to shoot visible light images during the day; and the infrared camera is used to shoot infrared images at night.

[0020] Preferably, the step S3.3 includes:

[0021] According to the size of the flying bird target, adjustment parameters of the lens are converted:

[0022] The pixel width of the flying bird target is calculated, and the proportion of the flying bird target relative to the width of the image is further calculated; the change amount of the lens parameters relative to the current parameters when the target is adjusted to 60% of the width of the image is calculated, and the lens is controlled according to the change amount;

[0023] The lens is adjusted to improve the proportion of the flying bird target in the image and reduce the proportion of the background area;

[0024] If the size of the flying bird target exceeds the field of view of the lens, the field of view of the lens is expanded until the target is completely included.

[0025] Preferably, the step S3.2 comprises matching the feature vector with the preset database multiple times, and taking the category with the most number of recognitions as the final matching result.

[0026] According to the flying bird detection and tracking classification system based on dual-spectrum photoelectricity provided by the application, the system comprises:

[0027] Module M1: using a dual-spectrum photoelectric device to collect images in a target area;

[0028] The images comprise visible light images and infrared images;

[0029] Module M2: detecting flying bird targets in the images based on a deep learning target detection algorithm, and locking;

[0030] Module M3: obtaining a feature vector of the flying bird target and matching the feature vector with a preset database to obtain a matching result of the flying bird category, and calculating position information of the flying bird;

[0031] Module M4: outputting the matching result of the flying bird category and the position information of the flying bird.

[0032] Preferably, the module M3 comprises the following sub-modules:

[0033] Module M3.1: performing region segmentation on the flying bird target in the images to extract contour features of the flying bird;

[0034] Module M3.2: extracting a feature vector from the contour features, and matching the feature vector with a preset database to obtain a matching result of the flying bird category;

[0035] Module M3.3: calculating a position deviation of the flying bird target from a center point of the image, and adjusting the dual-spectrum photoelectric device to realize flying bird tracking according to the position deviation;

[0036] Module M3.4: calculating position information of the flying bird according to the adjusted parameters of the photoelectric device.

[0037] Preferably, the preset database comprises flying bird feature vectors extracted from multiple types of flying bird images.

[0038] Preferably, the dual-spectrum photoelectric device comprises a visible light camera and an infrared camera; the visible light camera is used to shoot visible light images during the day; and the infrared camera is used to shoot infrared images at night.

[0039] Preferably, the module M3.3 comprises:

[0040] According to the size of the flying bird target, adjustment parameters of the lens are converted.

[0041] The pixel width of the flying bird target is calculated, and the proportion of the flying bird target relative to the picture width is further calculated; the change amount of the lens parameter relative to the current parameter when the target is adjusted to 60% of the picture width is calculated, and the lens is controlled according to the change amount;

[0042] The lens is adjusted to increase the proportion of the flying bird target in the picture and reduce the proportion of the background area.

[0043] If the size of the flying bird target exceeds the field of view of the lens, the field of view of the lens is expanded until the target is completely included.

[0044] Preferably, the module M3.2 includes multiple matching of the feature vector with a preset database, and the category with the most number of matching results is taken as the final matching result.

[0045] Compared with the prior art, the present application has the following beneficial effects:

[0046] 1. The present application uses a fast target detection and tracking algorithm, which can detect and lock the position of the flying bird as soon as it enters the monitoring area, and has good real-time performance; the flying bird classification technology based on deep learning can automatically identify the type of the captured flying bird image, which is convenient for understanding the behavior of the flying bird and taking targeted measures.

[0047] 2. The flying bird type database used in the present application can be freely increased or decreased in number according to demand, ensuring that the system adapts to the characteristics of birds in different airport environments, and has high expansibility; the use of dual-spectrum cameras and photoelectric fast tracking technology can work stably under complex background and lighting conditions, ensuring the accuracy of detection and tracking.

[0048] 3. The present application can be applied to different environmental scenarios, and the detection and driving of flying birds are particularly important when the plane takes off or lands, and the present application can be applied to the monitoring area of the airport to ensure flight safety; in the scenes of farmland or orchard, the system can be used to monitor the threat of birds to crops and provide real-time alarm or automatic driving measures; it can also be used to monitor the species and activity track of flying birds to help protect endangered birds and collect data.

[0049] Other beneficial effects of the present application will be described in the specific embodiments through the introduction of specific technical features and technical solutions, and those skilled in the art should be able to understand the beneficial technical effects brought by the technical features and technical solutions through the introduction of the technical features and technical solutions. BRIEF DESCRIPTION OF DRAWINGS

[0050] Other features, objects and advantages of the present application will become more apparent through reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0051] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION

[0052] The present application will be described in detail below with specific examples. The following examples will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present application. These are within the scope of protection of the present application.

[0053] Referring to Figure 1 The method shown in the figure is a dual-spectrum photoelectric-based flying bird detection, tracking and classification method, which comprises:

[0054] Firstly, based on the dual-spectrum photoelectric system, the advantages of visible light and infrared spectrum are combined to ensure that the image of the flying bird can be effectively captured under different light conditions. Through the deep learning target detection algorithm (YOLOv5 target detection algorithm), the flying bird in the air is quickly detected and locked.

[0055] After locking the flying bird, the flying bird target is adjusted to 60% of the picture width, and the deep learning target segmentation algorithm (YOLOv5 target segmentation algorithm) is used to segment the flying bird region image. The image of the flying bird target region is repaired and enhanced to make the segmented flying bird target more complete and realistic, and the feature vector data of the flying bird region is extracted. All types of flying bird feature data in the pre-constructed flying bird type feature vector database are matched to identify the specific type of the flying bird.

[0056] Since the flying bird has been locked by the target detection algorithm, the position deviation of the flying bird from the center point of the image is calculated by the position of the flying bird in the picture, and the photoelectric rotation angle is calculated to mobilize the photoelectric rapid rotation, so that the flying bird is always in the center of the picture, thereby realizing tracking. In the tracking process, the type of the flying bird is identified several times, and the type with the most number of identifications is the final classification result.

[0057] The present application uses fast target detection and tracking algorithms, which can detect and lock the position of the flying bird as soon as it enters the monitoring area, and has good real-time performance. The flying bird classification technology based on deep learning can automatically identify the type of the captured flying bird image, which is convenient for understanding the behavior of the flying bird and taking targeted measures.

[0058] The above is the basic embodiment of the present application, and the technical solution of the present application will be further described below through a preferred embodiment.

[0059] Example 1

[0060] Referring to Figure 1As shown, a dual-spectrum photoelectric-based flying bird detection, tracking and classification method includes:

[0061] Step 1, enter the flying bird species to be identified: enter multiple species of flying birds into the flying bird feature vector database; import images of flying birds of different species, and store the feature vector data of the flying birds in the images in the database.

[0062] Step 2, dual-spectrum photoelectric detection: the system captures images in the monitoring area through a dual-spectrum camera, including visible light and infrared images. The camera uses a camera composed of a visible light camera and an infrared camera.

[0063] During the day, the visible light camera can capture clear color information of the flying bird target; at night, the visible light camera cannot work, and the infrared camera is needed at this time. Since the color information of the flying bird target cannot be seen at night, the classification function cannot be supported, and only the flying bird target can be detected and tracked.

[0064] In this embodiment, since the visible light cannot work at night, a switching method needs to be set for the two working modes (infrared light mode, visible light mode):

[0065] 1) During the day, from 4 am to 8 pm, the visible light working mode is adopted, and the image in the visible light camera is subjected to target detection, segmentation, tracking and classification.

[0066] 2) At night, from 5 pm to 7 am, the infrared working mode is adopted, and the image in the infrared camera is subjected to target detection and tracking.

[0067] Step 3, flying bird detection: a deep learning target detection algorithm (YOLOv5) is used to detect the flying bird in the image, lock its position and start tracking.

[0068] The automatic control process of the lens is as follows:

[0069] 1) According to the target detection, the target size is obtained, and the adjustment parameters of the lens are correspondingly converted.

[0070] According to the target detection result (rectangular frame), the pixel width of the target is calculated, and the proportion of the target relative to the width of the picture is further calculated. Then, the change amount of the lens parameters relative to the current parameters when the target is adjusted to 60% of the picture width is calculated. The lens is controlled by the change amount.

[0071] The specific calculation process of the pixel width, the proportion relative to the width of the picture, and the change amount of the lens parameters is as follows:

[0072] (1) Target pixel width calculation

[0073] The target detection algorithm (e.g. YOLOv5) outputs a bounding box of the target, defined as (xmin, ymin, xmax, ymax).

[0074] The pixel width of the target Wtarget can be calculated by the following formula:

[0075] Wtarget = xmax - xmin

[0076] (2) The proportion relative to the width of the frame

[0077] Assuming the total width of the frame is Wframe, the proportion of the target in the frame width Ratio can be calculated as:

[0078]

[0079] (3) The amount of lens parameter change to adjust the target to 60% of the frame width

[0080] The target needs to be adjusted to 60% of the frame width, i.e. the new width of the target occupies TargetRatio = 0.6.

[0081] The calculation of the change amount relative to the current lens parameter is as follows:

[0082] Assuming the current focal length of the lens is Fcurrent (unit: mm).

[0083] The change ratio of the focal length is inversely proportional to the proportion of the frame width (inverse proportional relationship of magnification).

[0084] The new focal length Fnew needs to satisfy:

[0085]

[0086] Thus the new focal length is calculated as:

[0087]

[0088] The change amount of the lens parameter ΔF:

[0089] ΔF = Fnew - Fcurrent

[0090] Through these formulas, the lens can be precisely controlled to adjust the target to 60% of the frame width.

[0091] 2) Adjust the lens so that the target can show the full view in the frame as much as possible, increase the proportion of the target in the frame, and reduce the proportion of the background area as much as possible;

[0092] If the target size is too large and exceeds the field of view, the field of view should be enlarged to include the target.

[0093] Step 4, Bird Area Segmentation: After detecting the bird, adjust the bird target to 60% of the picture width; the system performs image segmentation on the area where the bird is located (YOLOv5); the segmentation algorithm sometimes causes rough edges, and using edge optimization and image smoothing techniques (Guided Filter) can improve the delicacy of the outline;

[0094] Image restoration and enhancement are performed on the bird target area to make the segmented bird target more complete and realistic; the image restoration model (DeepFill v2) of deep learning is used to fill in the missing areas in the bird image, especially the occluded parts.

[0095] Finally, the outline and features of the bird are extracted.

[0096] Step 5, Classification Matching: The system matches the extracted bird feature vector with the bird type feature vector in the database to find the most similar bird species.

[0097] Since the bird's shape changes greatly when flying in the air, the error rate of determining the classification result through one-time classification recognition is high. By adopting the scheme of "recognizing the species of the bird multiple times (more than 10 times) in the tracking process, and taking the species with the most recognition times as the final classification result", the accuracy of the classification result is greatly improved.

[0098] The specific implementation process of the bird classification technology is as follows:

[0099] 1) Feature Extraction: First, detect the target through the target detection algorithm (yolov5), then extract the features of the image through the deep learning network (ResNet) to generate the feature vector.

[0100] 2) Feature Storage: Use the open-source feature vector database (Milvus) to construct <id, feature vector> data and generate index storage for all image features uploaded to the image library.

[0101] 3) Retrieval and Classification: Extract the image feature vector of the bird image to be searched, and match it with the most similar bird species in the feature vector database through similarity (cosine distance / Hamming distance).

[0102] 4) Feature Matching Optimization: When matching features, first use the Hamming distance between hash features to quickly screen and form a preliminary screening set, and then use the cosine distance of floating-point features for further subdivision and filtering.

[0103] Step 6, Bird Tracking: After the bird has been locked by the target detection algorithm, the deviation of the bird's position from the center of the image is calculated based on the bird's position in the frame. The photoelectric rotation angle is then calculated, and the photoelectric fast rotation is adjusted to keep the bird in the center of the frame, thus achieving tracking.

[0104] Step 7, Bird Azimuth Information Calculation: Based on the camera's pitch angle, field of view angle, and other parameters, combined with the geographical information of the photoelectric station, the approximate azimuth information of the bird target is calculated. In addition, the resolution, frame rate, and other parameters of the camera need to be selected according to actual needs to meet the accuracy and real-time requirements of target detection.

[0105] To calculate the actual approximate azimuth information of the bird target, the following key parameters need to be considered:

[0106] 1) Camera pitch angle and azimuth angle: describes the current direction of the camera.

[0107] 2) Pitch angle (θ): the vertical angle of the camera relative to the horizontal plane, positive value indicates upward, negative value indicates downward.

[0108] 3) Azimuth angle (φ): the horizontal rotation angle of the camera relative to the north direction, clockwise is positive.

[0109] 4) Field of view angle: horizontal field of view angle (αh) and vertical field of view angle (αv) of the camera.

[0110] 5) Pixel coordinates of target position: pixel position (xtarget, ytarget) given by target detection, and frame resolution (Wframe, Hframe).

[0111] 1. Only calculate the angular direction of the target

[0112] In the case where the bird height is unknown, the line-of-sight angular direction of the target is calculated based on the pixel position of the target in the frame and the field of view angle of the camera:

[0113] 1) Azimuth angle of target (φtarget):

[0114] φtarget = φ + Δφ

[0115] The calculation formula of Δφ is:

[0116]

[0117] 2) Pitch angle of target (θtarget):

[0118] θtarget = θ + Δθ

[0119] The calculation formula of Δθ is:

[0120]

[0121] Through these formulas, the relative angular direction of the target can be determined, i.e., which direction within the camera's field of view the target is located.

[0122] 2. Calculate estimated distance using assumed height

[0123] Although the actual height of the flying bird cannot be known, an assumed height Hbird,assumed can be chosen to calculate the estimated distance of the target. The assumed height can be based on the height range at which flying birds usually fly, such as 50 meters or 100 meters commonly used for low-altitude flight.

[0124] Estimate the distance of the target:

[0125]

[0126] 3. Output approximate direction information of the target

[0127] 1) Relative azimuth angle of the target: φtarget, indicating the direction of the target relative to the camera.

[0128] 2) Relative elevation angle of the target: θtarget, indicating the angle of the target in the vertical direction.

[0129] 3) Estimated distance at assumed height: Dtarget,estimated, the distance calculated at the assumed height.

[0130] Selection of camera parameters

[0131] 1) Resolution:

[0132] The resolution determines the accuracy of target detection. High resolution can accurately detect the boundaries of the target, but will increase the amount of calculation. When selecting the resolution, the following needs to be considered:

[0133]

[0134] Where Min Pixel Size is the minimum pixel size that the detection algorithm can reliably distinguish.

[0135] 2) Frame rate:

[0136] The frame rate determines the real-time performance of detection. The moving speed of flying birds is fast, and the frame rate needs to meet:

[0137]

[0138] 3) Other parameters:

[0139] Optical zoom capability: used to adapt to targets at different distances.

[0140] Pixel sensitivity (ISO): Improve detection ability in low light conditions.

[0141] Dynamic range (HDR): Reduce the effects of strong light and shadows.

[0142] In actual project application, after the implementation of bird tracking, this step can calculate the approximate position information of the bird, and provide information data for automatic bird driving.

[0143] Step 8, result output: the category of the bird is identified multiple times in the tracking process, and the category with the most number of times of identification is the final classification result. The system outputs the type and position information of the bird in real time, and can selectively perform alarm or automatic bird driving.

[0144] In this embodiment, the Yolov5 deep network is used as the basis to train the bird samples collected in the foregoing steps to obtain a detection network; and the Yolov5 deep network is used as the basis to train the bird samples collected in the foregoing steps to obtain a segmentation network.

[0145] When deploying target detection, target segmentation, matching and other algorithms on a NVIDIA GPU, tensorRT technology is used as the basis to achieve the maximum inference speed on the hardware.

[0146] To improve the accuracy of target detection and recognition, the model training data can be expanded and the algorithm can be optimized to reduce false detection and missed detection in complex environments.

[0147] The bird type database used in the application can be freely increased or reduced in type according to requirements, ensuring that the system adapts to the characteristics of birds in different airport environments and has high expansibility; the dual-spectrum camera and the photoelectric fast tracking technology can work stably under complex background and lighting conditions, ensuring the accuracy of detection and tracking.

[0148] The application also provides a bird detection and tracking classification system based on dual-spectrum photoelectricity, which can be realized by executing the process steps of the bird detection and tracking classification method based on dual-spectrum photoelectricity, that is, the bird detection and tracking classification method based on dual-spectrum photoelectricity can be understood by those skilled in the art as the preferred embodiment of the bird detection and tracking classification system based on dual-spectrum photoelectricity.

[0149] Specifically, a bird detection and tracking classification system based on dual-spectrum photoelectricity comprises:

[0150] Module M1: using a dual-spectrum photoelectric device to collect images in a target area;

[0151] The images include visible light images and infrared images;

[0152] Module M2: detecting the flying bird target in the image based on a deep learning target detection algorithm, and locking;

[0153] Module M3: obtaining a feature vector of the flying bird target and matching the feature vector with a preset database to obtain a matching result of the flying bird species, and calculating position information of the flying bird;

[0154] Module M4: outputting the matching result of the flying bird species and the position information of the flying bird.

[0155] The module M3 comprises the following sub-modules:

[0156] Module M3.1: performing region segmentation on the flying bird target in the image, and extracting contour features of the flying bird;

[0157] Module M3.2: extracting a feature vector from the contour features, and matching the feature vector with a preset database to obtain a matching result of the flying bird species;

[0158] Module M3.3: calculating a position deviation of the flying bird target from a center point of the image, and adjusting a dual-spectrum photoelectric device to realize flying bird tracking according to the position deviation;

[0159] Module M3.4: calculating position information of the flying bird according to the adjusted parameters of the photoelectric device.

[0160] The preset database comprises flying bird feature vectors extracted from a plurality of species of flying bird images inputted.

[0161] The dual-spectrum photoelectric device comprises a visible light camera and an infrared camera; the visible light camera is used to capture a visible light image in the daytime; and the infrared camera is used to capture an infrared image at night.

[0162] The module M3.3 comprises:

[0163] According to the size of the flying bird target obtained, adjustment parameters of the lens are converted:

[0164] The pixel width of the flying bird target is calculated, and a proportion of the flying bird target relative to the width of the image is further calculated; the change amount of the lens parameters relative to the current parameters when the target is adjusted to 60% of the width of the image is calculated, and the lens is controlled according to the change amount;

[0165] The lens is adjusted to improve the proportion of the flying bird target in the image and reduce the proportion of the background area;

[0166] If the size of the flying bird target exceeds the field of view of the lens, the field of view of the lens is expanded until the target is completely included.

[0167] The module M3.2 includes multiple matches of the feature vector with a preset database, and the category with the most number of matches is taken as the final matching result.

[0168] Those skilled in the art know that, in addition to implementing the system provided by the present application and each device, module and unit thereof in the form of pure computer readable program code, the system provided by the present application and each device, module and unit thereof can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. by logically programming the method steps to achieve the same functions. Therefore, the system provided by the present application and each device, module and unit thereof can be considered as a hardware component, and the devices, modules and units included therein for achieving various functions can also be considered as structures within the hardware component; the devices, modules and units for achieving various functions can also be considered as both software modules implementing methods and structures within hardware components.

[0169] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be arbitrarily combined with each other without conflict.

Claims

1. A dual-spectrum photoelectric-based flying bird detection, tracking and classification method, characterized in that, The method comprises the following steps: Step S1: acquiring images in a target area by using a dual-spectrum photoelectric device; The images comprise visible light images and infrared images; Step S2: detecting a flying bird target in the images based on a deep learning target detection algorithm to perform locking; Step S3: obtaining a feature vector of the flying bird target and matching the feature vector with a preset database to obtain a matching result of a flying bird species and to calculate position information of the flying bird; Step S4: outputting the matching result of the flying bird species and the position information of the flying bird.

2. The dual-spectrum photoelectricity based flying bird detection, tracking and classification method according to claim 1, characterized in that, The step S3 comprises the following sub-steps: Step S3.1: performing region segmentation on the flying bird target in the images to extract contour features of the flying bird; Step S3.2: extracting a feature vector from the contour features and matching the feature vector with the preset database to obtain the matching result of the flying bird species; Step S3.3: calculating a position deviation of the flying bird target from a center point of an image frame, and adjusting the dual-spectrum photoelectric device to realize flying bird tracking according to the position deviation; Step S3.4: calculating the position information of the flying bird according to adjusted photoelectric device parameters.

3. The dual-spectrum photoelectricity based flying bird detection, tracking and classification method according to claim 2, characterized in that, The preset database comprises flying bird feature vectors extracted from a plurality of recorded flying bird images of different species.

4. The dual-spectrum photoelectricity based flying bird detection, tracking and classification method according to claim 2, characterized in that, The dual-spectrum photoelectric device comprises a visible light camera and an infrared camera; the visible light camera is used to capture visible light images during the day; and the infrared camera is used to capture infrared images at night.

5. The dual-spectrum photoelectricity based bird detection, tracking and classification method according to claim 2, characterized in that, The step S3.3 comprises the following steps: According to the size of the flying bird target obtained, adjustment parameters of a lens are calculated: The pixel width of the flying bird target is calculated, and a proportion of the flying bird target relative to the width of the image frame is further calculated; the change amount of the lens parameters relative to the current parameters when the target is adjusted to 60% of the width of the image frame is calculated, and the lens is controlled according to the change amount; The lens is adjusted to increase the proportion of the flying bird target in the image frame and to reduce the proportion of the background area; If the size of the flying bird target exceeds the field of view of the lens, the field of view of the lens is expanded until the target is completely included.

6. The dual-spectrum photoelectricity based flying bird detection, tracking and classification method according to claim 2, characterized in that, The step S3.2 comprises matching the feature vector with the preset database multiple times, and the species with the largest number of times of recognition is taken as the final matching result.

7. A dual-spectrum photoelectric-based flying bird detection, tracking and classification system, characterized in that, The method comprises the following steps: Module M1: acquiring images in a target area by using a dual-spectrum photoelectric device; The images comprise visible light images and infrared images; Module M2: detecting a flying bird target in the images based on a deep learning target detection algorithm to perform locking; Module M3: obtaining a feature vector of the flying bird target and matching the feature vector with a preset database to obtain a matching result of a flying bird species and to calculate position information of the flying bird; Module M4: outputting the matching result of the flying bird species and the position information of the flying bird.

8. The dual-spectrum photoelectricity based flying bird detection, tracking and classification system according to claim 7, characterized in that, The module M3 comprises the following sub-modules: Module M3.1: performing region segmentation on the flying bird target in the images to extract contour features of the flying bird; Module M3.2: extracting a feature vector from the contour features and matching the feature vector with the preset database to obtain the matching result of the flying bird species; Module M3.3: calculating a position deviation of the flying bird target from a center point of an image frame, and adjusting the dual-spectrum photoelectric device to realize flying bird tracking according to the position deviation; Module M3.4: calculating the position information of the flying bird according to adjusted photoelectric device parameters.

9. The dual-spectrum photoelectricity based flying bird detection, tracking and classification system according to claim 8, characterized in that, The preset database comprises flying bird feature vectors extracted from a plurality of recorded flying bird images of different species.

10. The dual-spectra photoelectric based flying bird detection, tracking and classifying system according to claim 8, wherein, The dual-spectrum photoelectric device comprises a visible light camera and an infrared camera; the visible light camera is used for shooting visible light images in the daytime; and the infrared camera is used for shooting infrared images at night.

Images