Wild animal identification and counting method and device based on multi-source unmanned aerial vehicle video

Abstract

The invention provides a wild animal identification and counting method and device based on a multi-source unmanned aerial vehicle video, and the method comprises the steps: planning a flight route of an unmanned aerial vehicle based on a delimited detection region, and setting a flight height; a wide-angle visible light image and an infrared image are collected through the unmanned aerial vehicle; unifying the sizes of the wide-angle visible light image and the infrared image; inputting the wide-angle visible light image and the infrared image which are unified in size into a target detection network, and obtaining an identification result output by the target detection network; and tracking and counting the identified target wild animals based on a ByteTrack target tracking algorithm. According to the wild animal identification and counting method and device based on the multi-source unmanned aerial vehicle video, the wide-angle visible light image and the infrared image are acquired through the unmanned aerial vehicle, detection and tracking are performed by using the visible light image data and the infrared image data, and the detection and tracking precision is improved through data fusion; and the efficiency and the accuracy of identifying and counting the wild animals are improved.

Description

Technical Field

[0001] The present invention relates to the field of wildlife protection technology, and in particular to a method and device for identifying and counting wildlife based on multi-source unmanned aerial vehicle videos. Background Art

[0002] For all kinds of ecological reserves, counting the number of wild animals is an important task. The current counting mainly relies on the sample survey method to estimate the population density. Usually, the number of animals in the sample is counted manually. In order to avoid disturbing wild animals, it is usually not deep into the wild environment.

[0003] In recent years, with the gradual maturity of drone control technology, drone remote sensing images have become an important data source in the field of low-altitude remote sensing research due to their cost-effectiveness and easy access. Wildlife identification and counting based on drone images has become a new research direction.

[0004] However, in the existing technology, the use of drones to assist in counting wild animals mainly takes advantage of the characteristics of drones to collect videos in the wild environment, while the counting work is still completed through manual analysis of videos, which is not only manpower-consuming but also has low accuracy. Summary of the invention

[0005] The present invention provides a method and device for identifying and counting wild animals based on multi-source unmanned aerial vehicle videos, which are used to solve the technical problems of low efficiency and inaccuracy in counting wild animals in the prior art.

[0006] The present invention provides a method for identifying and counting wild animals based on multi-source drone videos, comprising: Plan the flight route of the drone based on the designated detection area and set the flight altitude; the flight altitude of the drone is 70 to 100 meters, and the downward viewing angle of the drone when collecting images is 30 to 60 degrees; the side overlap rate of the planned drone flight route is less than or equal to 10%, and the route overlap rate is greater than 90%; Collect wide-angle visible light and infrared images through drones; Unifying the sizes of the wide-angle visible light image and the infrared image; and inputting the wide-angle visible light image and the infrared image after the unified sizes into a target detection network to obtain a recognition result output by the target detection network; The identified target wild animals are tracked and counted based on the ByteTrack target tracking algorithm.

[0007] In some embodiments, inputting the wide-angle visible light image and the infrared image of uniform size into the target detection network, and obtaining the recognition result output by the target detection network includes: The wide-angle visible light image and the infrared image after being unified in size are combined into a four-channel image; The four-channel image is input into the target detection network to obtain the recognition result output by the target detection network.

[0008] In some embodiments, inputting the unified-size wide-angle visible light image and infrared image into the target detection network to obtain the recognition result output by the target detection network includes: Inputting the uniformly sized wide-angle visible light image into the target detection network to obtain a first recognition result output by the target detection network; and inputting the uniformly sized infrared image into the target detection network to obtain a second recognition result output by the target detection network; Based on the first recognition result and the second recognition result, a final recognition result output by the object detection network is determined.

[0009] In some embodiments, inputting the uniformly sized wide-angle visible light image into the target detection network to obtain a first recognition result output by the target detection network includes: The wide-angle visible light image of uniform size is segmented by sliding cropping to obtain multiple sub-images; Inputting the multiple sub-graphs into the target detection network in sequence, respectively, and obtaining multiple recognition results output by the target detection network respectively; The multiple recognition results are combined to obtain the first recognition result.

[0010] In some embodiments, the flying altitude of the drone is 85 meters, and the downward viewing angle of the drone when collecting images is 45 degrees.

[0011] In some embodiments, the lateral overlap rate of the planned drone flight routes is 5%, and the route overlap rate is 95%.

[0012] The present invention also provides a wild animal identification and counting device based on multi-source drone video, comprising: A planning module is used to plan the flight route of the UAV based on the designated detection area and set the flight altitude; the flight altitude of the UAV is 70 to 100 meters, and the downward viewing angle of the UAV when collecting images is 30 to 60 degrees; the side overlap rate of the planned UAV flight route is less than or equal to 10%, and the route overlap rate is greater than 90%; A collection module, used to collect wide-angle visible light images and infrared images through drones; A recognition module is used to unify the sizes of the wide-angle visible light image and the infrared image; and input the unified wide-angle visible light image and the infrared image into a target detection network to obtain a recognition result output by the target detection network; The counting module is used to track and count the identified target wild animals based on the ByteTrack target tracking algorithm.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method for identifying and counting wild animals based on multi-source drone videos as described in any one of the above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for identifying and counting wild animals based on multi-source drone videos as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for identifying and counting wild animals based on multi-source drone videos.

[0016] The method and device for identifying and counting wild animals based on multi-source UAV videos provided by the present invention collect wide-angle visible light images and infrared images through UAVs, use visible light and infrared image data for detection and tracking, and improve detection and tracking accuracy through data fusion, thereby improving the efficiency and accuracy of wild animal identification and counting. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of a method for identifying and counting wild animals based on multi-source drone videos provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the shooting angle of the drone provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the UAV route planning provided by the present invention.

[0021] Figure 4 It is a schematic diagram of sliding cutting provided by the present invention.

[0022] Figure 5 This is one of the target detection flow charts provided by the present invention.

[0023] Figure 6 This is the second target detection flow diagram provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of a wild animal identification and counting device based on multi-source UAV video provided by the present invention.

[0025] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Figure 1 is a flow chart of a method for identifying and counting wild animals based on multi-source drone videos provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: Plan the flight route of the drone based on the designated detection area and set the flight altitude.

[0028] Specifically, in theory, the lower the flight altitude, the higher the pixel ratio of the detection object at the same resolution, and the easier it is to detect. However, if the flight altitude is too low, it will not only disturb wild animals, but also limit the scanning area, reducing efficiency. Therefore, the flight altitude needs to comprehensively consider the model detection capability and business scenario requirements. Considering the existence of obstructions such as grass and shade in the wild environment, the drone shooting should consider adjusting the viewing angle to avoid obstructions and cover the detection object as much as possible.

[0029] The closer the shooting distance, the larger the size of the captured target and the easier it is to detect; the higher the shooting angle, the flatter the image of the object in the picture, the fewer features, and the higher the difficulty of identification; if the shooting angle is too high, the object may be more severely obstructed by obstructions and the features of the identified object may be insufficient; if the shooting angle is too low, the shooting distance will become farther and the size of the target in the picture will become smaller.

[0030] Figure 2 Schematic diagram of the shooting angle of the drone provided by the present invention. Figure 2 As shown, the present invention takes into account the viewing angle, field of view, vertical distance and horizontal distance, and the flight altitude of the drone and the downward viewing angle when the drone collects images can be adjusted according to the application scenario.

[0031] In some embodiments, the flying altitude of the drone is 70 to 100 meters, and the downward viewing angle of the drone when collecting images is 30 to 60 degrees.

[0032] The embodiment of the present invention sets the flight altitude of the drone to 70 to 100 meters, and the downward viewing angle of the drone when collecting images is set to 30 to 60 degrees, which can ensure that the image collection has a high efficiency and the quality of the collected images can be guaranteed.

[0033] In some embodiments, the flying altitude of the drone is 85 meters, and the downward viewing angle of the drone when collecting images is 45 degrees.

[0034] In the embodiment of the present invention, the flight altitude of the drone is set to 85 meters, and the downward viewing angle of the drone when collecting images is set to 45 degrees, which can ensure that the image collection has a high efficiency and the quality of the collected images can be guaranteed.

[0035] In addition, in order to improve the detection capability, the drone is equipped with an infrared camera and a visible light camera. Infrared cameras are more likely to detect targets with heat and have lower lighting requirements. Visible light cameras have richer image information and higher resolution, but have higher lighting requirements. The present invention fuses the data of the two to achieve complementary advantages to obtain higher quality images and achieve better results.

[0036] For example, a feasible camera configuration parameter is shown in Table 1.

[0037] Table 1 Camera parameter configuration According to the set shooting height and angle as well as the camera parameters, the scanning area of ​​the drone on the ground can be calculated. Figure 3 This is a schematic diagram of the UAV route planning provided by the present invention, such as Figure 3 As shown in the figure, the route overlap rate is the overlap between two adjacent aerial photos of the same route, and the lateral overlap rate is the overlap between two adjacent aerial photos of the route. The calculation of lateral overlap affects the interval between routes. For a fixed surface, the wider the interval, the fewer routes there are, and the narrower the interval, the more routes there are. By setting the route overlap rate and lateral overlap rate, the flight route of the drone can be planned.

[0038] The design of the overlap rate is related to the application scenario. One solution is based on target detection + target tracking statistics. There are certain requirements for the lateral overlap rate to avoid repeated scanning of the side areas of the image. The route overlap rate should be as large as possible to ensure the continuity of the frame. In terms of flight time setting, it is necessary to consider the activity habits of wild animals and try to select the time period when the animals are active in open areas. At the same time, it is necessary to ensure high image quality for both visible light and infrared.

[0039] In some embodiments, the lateral overlap rate of the planned UAV flight routes is less than or equal to 10%, and the route overlap rate is greater than 90%.

[0040] The present invention sets the lateral overlap rate of the UAV flight route to be less than or equal to 10%, and the route overlap rate to be greater than 90%, which can not only avoid excessive lateral overlap of the collected images, but also ensure the continuity of the frames, thereby further improving the quality of the collected images.

[0041] In some embodiments, the lateral overlap rate of the planned drone flight routes is 5%, and the route overlap rate is 95%.

[0042] The present invention sets the lateral overlap rate of the UAV flight route to 5% and the route overlap rate to 95%, which can not only avoid excessive lateral overlap of the collected images, but also ensure the continuity of the frames, thereby further improving the quality of the collected images.

[0043] Step 102: collect wide-angle visible light images and infrared images by using a drone.

[0044] Specifically, according to the set shooting height and angle as well as the camera parameters, wide-angle visible light images and infrared images are collected by drones according to the planned route.

[0045] Step 103: unify the sizes of the wide-angle visible light image and the infrared image; and input the unified-size wide-angle visible light image and the infrared image into a target detection network to obtain a recognition result output by the target detection network.

[0046] Specifically, before detecting and identifying the target wild animal, the target detection network needs to be trained first.

[0047] For example, since the resolution of wide-angle visible light images is 3840×2160, for high-resolution images, if they are scaled to the common object detection network training size of 1280×1280 or 640×640, the pixels will be compressed, resulting in information loss, making small object detection more difficult. Figure 4 Schematic diagram of sliding cutting provided by the present invention, such as Figure 4 As shown in the figure, in order to avoid losing the pixel information of the original image and improve the model training efficiency, the visible light image is processed by sliding cropping. A sliding window with a size of 1280×1280 is designed to divide the original image into 8 sub-images. Target detection is performed in sequence. After the detection is completed, the detection results are merged and mapped to the original image. The sliding window is used for sliding cropping so that adjacent images retain a certain margin to prevent the loss of target detection information at the boundary.

[0048] For infrared images, the original resolution is 1280×1024. Using a 1280×1280 training size will not lose the original pixel information, and no preprocessing is required.

[0049] The object detection network in this application can be YOLO and DETR. The core idea of ​​YOLO is to regard the object detection task as a regression problem, divide the input image into grids, and finally predict the position of the bounding box and the category of the object in the bounding box; DETR does not rely on convolutional neural networks (CNN) to extract features, but uses the Transformer architecture to directly predict the category and position of the target.

[0050] The specific target detection network to be selected needs to be combined with the specific business scenario. For business scenarios where the UAV has strong onboard computing power and high real-time requirements, you can choose to train the YOLO target detection network for real-time target detection; for computing center deployment, for business scenarios without real-time requirements, you can choose the DETR target detection network.

[0051] After the training of the target detection network is completed, the trained target detection network is used to detect and identify target wild animals based on the wide-angle visible light image and the infrared image.

[0052] Since the resolution and focal length of infrared images are inconsistent with those of visible light, this application requires that the infrared image be magnified by a certain focal length ratio and distortion removed to be scaled to be consistent with the size of the visible light field of view. The overlapping area of ​​the infrared image and the visible light image is taken as the effective area and input into the target detection network for training and prediction to obtain the target detection result.

[0053] The present invention unifies the sizes of wide-angle visible light images and infrared images, further improving the efficiency and accuracy of wild animal identification and counting.

[0054] Step 104: Track and count the identified target wild animals based on the ByteTrack target tracking algorithm.

[0055] Specifically, multi-target tracking aims to estimate the bounding box and unique number of objects in the video. Most methods obtain unique numbers by associating detection boxes with scores higher than a threshold. However, for objects with low detection scores, especially those that are occluded in small target detection, general tracking methods simply discard this information, resulting in the loss of tracking targets and fragmentation of tracking trajectories. The current mainstream tracking algorithms are DeepSort and ByteTrack. DeepSort uses a re-identification algorithm to extract the appearance features of the detected object, and then calculates the similarity between the appearance features and the previously stored appearance features. It is difficult to track occluded objects, and the amount of calculation is large in some complex scenes, affecting real-time performance. ByteTrack mainly relies on the effect of target detection. The higher the detection accuracy, the better the tracking effect. This method uses a simple, effective and universal association method. For low-score detection boxes, it uses the similarity with the trajectory to restore the real object. Even if the detected object is occluded to a certain extent, it can still continue to track, and the real-time performance is good.

[0056] Since the tracking object photographed by the drone is small, lacks appearance features, and is obstructed, the target tracking effect based on appearance matching is not good. In the target detection link, this application can improve the detection capability by increasing the number of image channels for cross-correction. For deterministic detection objects, the detection accuracy can be improved by increasing the detection score. The target tracking effect based on ByteTrack is better based on its detection results. In order to ensure the stability of target tracking, in addition to improving the accuracy of target detection, the flight control of the drone needs to be as smooth as possible. For example, when turning, early turning is used to prevent the loss of the tracking target.

[0057] In order to further improve the stability of tracking calculations, it is necessary to adopt certain statistical strategies based on target tracking. Target wildlife identified in each frame of image (wide-angle visible light image and infrared image) is tracked and assigned a tracking number. After the planned route of the drone is completed, the number of times all tracking numbers appear is counted, considering the complexity of the collection area environment and the activity habits of the target animals, and the tracking numbers with fewer times are eliminated according to a certain threshold to obtain the number of target wildlife tracked in this flight plan. For the same designated detection area, the drone is used to fly multiple times, and the average number of target wildlife tracked each time is taken as the final statistical number of target wildlife.

[0058] The method for identifying and counting wild animals based on multi-source UAV videos provided by the present invention collects wide-angle visible light images and infrared images through UAVs, uses visible light and infrared image data for detection and tracking, and improves detection and tracking accuracy through data fusion, thereby improving the efficiency and accuracy of wild animal identification and counting.

[0059] In some embodiments, inputting the wide-angle visible light image and the infrared image of uniform size into the target detection network, and obtaining the recognition result output by the target detection network includes: The wide-angle visible light image and the infrared image after being unified in size are combined into a four-channel image; The four-channel image is input into the target detection network to obtain the recognition result output by the target detection network.

[0060] Specifically, in the process of training or predicting models based on visible light images and infrared images, the "pre-fusion" training and prediction method can be adopted.

[0061] Figure 5 This is one of the target detection flow diagrams provided by the present invention, such as Figure 5 As shown in the figure, "front fusion" means that the infrared image (single black and white channel) and the visible light image (three RGB channels) are overlapped and combined into four-channel image data, and the four-channel image is used as the input of the target detection network.

[0062] Infrared and visible light need to meet the following conditions to be better superimposed: the photographed object is at a certain distance from the drone, and the optical axes of the infrared camera and the visible light camera are completely parallel and close in position.

[0063] The method for identifying and counting wild animals based on multi-source UAV videos provided by the present invention adopts a "front fusion" training and prediction method to improve detection and tracking accuracy, thereby improving the efficiency and accuracy of wild animal identification and counting.

[0064] In some embodiments, inputting the unified-size wide-angle visible light image and infrared image into the target detection network to obtain the recognition result output by the target detection network includes: Inputting the uniformly sized wide-angle visible light image into the target detection network to obtain a first recognition result output by the target detection network; and inputting the uniformly sized infrared image into the target detection network to obtain a second recognition result output by the target detection network; Based on the first recognition result and the second recognition result, a final recognition result output by the object detection network is determined.

[0065] Specifically, in the process of training or predicting models based on visible light images and infrared images, a "post-fusion" training and prediction method can be adopted.

[0066] Figure 6 This is the second target detection flow diagram provided by the present invention, such as Figure 6As shown in the figure, "post-fusion" means taking the visible light image and infrared image as the input of the target detection algorithm respectively, and combining their respective output results. Train the infrared target detection and visible light target detection networks separately, and fuse the detection results of the two to obtain the final target detection result. For example, a fusion strategy of "visible light as the main and infrared as the auxiliary" is adopted, as follows: if the visible light detects the target object and the confidence is lower than 0.7, and the infrared does not detect the target object within a certain range, the detection result is invalid. If the visible light detects the target object and the confidence is higher than 0.3, and the infrared detects the target object within a certain range, the detection result is valid and the detection confidence is enhanced (increasing the confidence helps to improve the stability of subsequent target tracking).

[0067] The method for identifying and counting wild animals based on multi-source UAV videos provided by the present invention adopts a "post-fusion" training and prediction method to improve detection and tracking accuracy, thereby improving the efficiency and accuracy of wild animal identification and counting.

[0068] In some embodiments, inputting the uniformly sized wide-angle visible light image into the target detection network to obtain a first recognition result output by the target detection network includes: The wide-angle visible light image of uniform size is segmented by sliding cropping to obtain multiple sub-images; Inputting the multiple sub-graphs into the target detection network in sequence, respectively, and obtaining multiple recognition results output by the target detection network respectively; The multiple recognition results are combined to obtain the first recognition result.

[0069] Specifically, since the resolution of wide-angle visible light images is 3840×2160, for high-resolution images, if they are scaled to the common input image size of 1280×1280 or 640×640 for target detection networks, the pixels will be compressed, resulting in information loss, making small target detection more difficult. The requirements in the prediction process are similar to those in the training process. In order not to lose the pixel information of the original image and to improve the detection efficiency of the model, the visible light image is processed by sliding cropping. For example, a sliding window of size 1280×1280 is designed to divide the original image into 8 sub-images, and target detection is performed in sequence. After the detection is completed, the detection results are merged and mapped to the original image. The sliding window is used for sliding cropping so that adjacent images retain a certain margin to prevent the loss of target detection information at the boundary.

[0070] The wild animal identification and counting method based on multi-source UAV video provided by the present invention uses a sliding cropping method to process the collected image data, thereby improving the detection and tracking accuracy, thereby improving the efficiency and accuracy of wild animal identification and counting.

[0071] The wild animal identification and counting device based on multi-source drone video provided by the present invention is described below. The wild animal identification and counting device based on multi-source drone video described below and the wild animal identification and counting method based on multi-source drone video described above can be referenced to each other.

[0072] Figure 7 is a schematic diagram of the structure of a wild animal identification and counting device based on multi-source drone video provided by the present invention, such as Figure 7 As shown, the present invention provides a wild animal identification and counting device based on multi-source drone video, comprising: The planning module 701 is used to plan the flight route of the UAV based on the designated detection area and set the flight altitude; the flight altitude of the UAV is 70 to 100 meters, and the downward viewing angle of the UAV when collecting images is 30 to 60 degrees; the side overlap rate of the planned UAV flight route is less than or equal to 10%, and the route overlap rate is greater than 90%; The acquisition module 702 is used to acquire wide-angle visible light images and infrared images through the drone; The recognition module 703 is used to unify the sizes of the wide-angle visible light image and the infrared image; and input the unified wide-angle visible light image and the infrared image into the target detection network to obtain the recognition result output by the target detection network; The counting module 704 is used to track and count the identified target wild animals based on the ByteTrack target tracking algorithm.

[0073] The wild animal identification and counting device based on multi-source UAV video provided by the present invention collects wide-angle visible light images and infrared images through the UAV, uses the visible light and infrared image data for detection and tracking, and improves the detection and tracking accuracy through data fusion, thereby improving the efficiency and accuracy of wild animal identification and counting.

[0074] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communications interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute a method for identifying and counting wild animals based on multi-source drone videos, the method comprising: Plan the flight route of the drone based on the designated detection area and set the flight altitude; the flight altitude of the drone is 70 to 100 meters, and the downward viewing angle of the drone when collecting images is 30 to 60 degrees; the side overlap rate of the planned drone flight route is less than or equal to 10%, and the route overlap rate is greater than 90%; Collect wide-angle visible light and infrared images through drones; Unifying the sizes of the wide-angle visible light image and the infrared image; and inputting the wide-angle visible light image and the infrared image after the unified sizes into a target detection network to obtain a recognition result output by the target detection network; The identified target wild animals are tracked and counted based on the ByteTrack target tracking algorithm.

[0075] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0076] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the wildlife identification and counting method based on multi-source drone video provided by the above methods, the method includes: Plan the flight route of the drone based on the designated detection area and set the flight altitude; the flight altitude of the drone is 70 to 100 meters, and the downward viewing angle of the drone when collecting images is 30 to 60 degrees; the side overlap rate of the planned drone flight route is less than or equal to 10%, and the route overlap rate is greater than 90%; Collect wide-angle visible light and infrared images through drones; Unifying the sizes of the wide-angle visible light image and the infrared image; and inputting the wide-angle visible light image and the infrared image after the unified sizes into a target detection network to obtain a recognition result output by the target detection network; The identified target wild animals are tracked and counted based on the ByteTrack target tracking algorithm.

[0077] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the wildlife identification and counting method based on multi-source drone video provided by the above methods, the method comprising: Plan the flight route of the drone based on the designated detection area and set the flight altitude; the flight altitude of the drone is 70 to 100 meters, and the downward viewing angle of the drone when collecting images is 30 to 60 degrees; the side overlap rate of the planned drone flight route is less than or equal to 10%, and the route overlap rate is greater than 90%; Collect wide-angle visible light and infrared images through drones; Unifying the sizes of the wide-angle visible light image and the infrared image; and inputting the wide-angle visible light image and the infrared image after the unified sizes into a target detection network to obtain a recognition result output by the target detection network; The identified target wild animals are tracked and counted based on the ByteTrack target tracking algorithm.

[0078] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0079] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying and counting wild animals based on multi-source drone videos, characterized in that: include: Plan the flight route of the drone based on the designated detection area and set the flight altitude; the flight altitude of the drone is 70 to 100 meters, and the downward viewing angle of the drone when collecting images is 30 to 60 degrees; the side overlap rate of the planned drone flight route is less than or equal to 10%, and the route overlap rate is greater than 90%; Collect wide-angle visible light and infrared images through drones; Unifying the sizes of the wide-angle visible light image and the infrared image; and inputting the wide-angle visible light image and the infrared image after the unified sizes into a target detection network to obtain a recognition result output by the target detection network; The identified target wild animals are tracked and counted based on the ByteTrack target tracking algorithm.

2. The method for identifying and counting wild animals based on multi-source drone videos according to claim 1 is characterized in that: Inputting the wide-angle visible light image and the infrared image of uniform size into the target detection network, and obtaining the recognition result output by the target detection network, including: Combine the wide-angle visible light image and infrared image after unifying the size into a four-channel image; The four-channel image is input into the target detection network to obtain the recognition result output by the target detection network.

3. The method for identifying and counting wild animals based on multi-source drone videos according to claim 1 is characterized in that: The step of inputting the uniformly sized wide-angle visible light image and infrared image into the target detection network and obtaining the recognition result output by the target detection network comprises: Inputting the uniformly sized wide-angle visible light image into the target detection network to obtain a first recognition result output by the target detection network; and inputting the uniformly sized infrared image into the target detection network to obtain a second recognition result output by the target detection network; Based on the first recognition result and the second recognition result, a final recognition result output by the object detection network is determined.

4. The method for identifying and counting wild animals based on multi-source drone videos according to claim 3 is characterized in that: Inputting the uniformly sized wide-angle visible light image into the target detection network to obtain a first recognition result output by the target detection network includes: The wide-angle visible light image of uniform size is segmented by sliding cropping to obtain multiple sub-images; Inputting the multiple sub-graphs into the target detection network in sequence, respectively, and obtaining multiple recognition results output by the target detection network respectively; The multiple recognition results are combined to obtain the first recognition result.

5. The method for identifying and counting wild animals based on multi-source drone videos according to claim 1 is characterized in that: The flying altitude of the UAV is 85 meters, and the downward viewing angle of the UAV when collecting images is 45 degrees.

6. The method for identifying and counting wild animals based on multi-source drone videos according to claim 1 is characterized in that: The lateral overlap rate of the planned UAV flight routes is 5%, and the route overlap rate is 95%.

7. A wild animal identification and counting device based on multi-source drone video, characterized in that: include: A planning module is used to plan the flight route of the UAV based on the designated detection area and set the flight altitude; the flight altitude of the UAV is 70 to 100 meters, and the downward viewing angle of the UAV when collecting images is 30 to 60 degrees; the side overlap rate of the planned UAV flight route is less than or equal to 10%, and the route overlap rate is greater than 90%; A collection module, used to collect wide-angle visible light images and infrared images through drones; A recognition module, used to unify the sizes of the wide-angle visible light image and the infrared image; and inputting the wide-angle visible light image and the infrared image after the unified size into the target detection network to obtain the recognition result output by the target detection network; The counting module is used to track and count the identified target wild animals based on the ByteTrack target tracking algorithm.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for identifying and counting wild animals based on multi-source drone videos as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying and counting wild animals based on multi-source drone videos as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for identifying and counting wild animals based on multi-source drone videos as described in any one of claims 1 to 6 is implemented.

Images