Bird identification method and device and electronic equipment
By selecting stable and valid frames from the data frames to be identified and performing multi-frame statistical analysis, the problems of false detection and false identification in bird identification technology in complex scenarios are solved, and bird identification with high accuracy and low power consumption is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN MICROBT ELECTRONICS TECH CO LTD
- Filing Date
- 2024-10-15
- Publication Date
- 2026-04-17
AI Technical Summary
Existing bird identification technologies have high false detection and false recognition rates in complex scenarios, especially in complex outdoor weather conditions, and the identification accuracy is difficult to improve due to insufficient data on rare birds.
By selecting stable and valid frames from the data frames to be identified, and using multi-frame statistical and analysis window technology, a voting decision is made on the bird identification results, thereby reducing the false detection and false identification rates.
It improves the accuracy and robustness of bird identification, reduces power consumption, and enhances the stability of identification and user experience.
Smart Images
Figure CN121884205A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target recognition, and in particular to a bird recognition method, apparatus, and electronic device. Background Technology
[0002] Birdwatching devices are currently widely used, but many challenges remain. One major challenge is that bird detection and recognition algorithms often produce a large number of false detections and misidentifications in complex real-world scenarios.
[0003] Although the ability to detect objects has made great progress with the continuous development of deep convolutional networks, and bird detection algorithms can achieve very high recall rates with a suitable amount of training data, birdwatching devices are generally deployed in outdoor forests or on rooftops and are subject to various complex weather conditions. Moreover, even a few false alarms caused by occasional bird events can lead to a poor user experience, which poses a great challenge to the ability to prevent false detections in bird identification.
[0004] Furthermore, while recognition technologies such as facial recognition are relatively mature, bird recognition technology, due to its unique characteristics, cannot achieve the same level of accuracy as general target recognition. Currently, there are approximately 12,000 bird species worldwide, some of which are extremely rare, resulting in very limited training data. This poses a significant challenge to the accuracy of bird recognition. Summary of the Invention
[0005] This invention provides a bird identification method to improve the accuracy of bird identification.
[0006] The first aspect of this invention provides a bird identification method, the method comprising:
[0007] Based on the bird detection results, bird identification results, and continuous frame information of at least one consecutive frame of the data frame to be identified, at least one valid frame is obtained from the data frame to be identified. The valid frame is used to characterize a data frame in which a bird target is detected and the target is stable.
[0008] If the number of valid frames acquired is greater than the set number of frames, then search for valid frames according to the set number of frames.
[0009] The bird identification results corresponding to the currently searched valid frames are statistically analyzed to obtain statistical results. These results include: the frequency of each bird identification result; and the bird identification results used to characterize the bird category.
[0010] The bird identification result with the maximum number of times is selected from the statistical results. If the maximum number of times of the selected bird identification result is greater than the set first number threshold, the selected bird identification result is taken as the current identification result.
[0011] In one embodiment of the present invention, the step of searching for the obtained valid frames according to a set frame number further includes:
[0012] The process continues until the cumulative number of valid frames searched reaches the set maximum number of frames to be analyzed.
[0013] If the current identification result is not found in the statistical results of the valid frames searched according to the set frame number, the bird identification results corresponding to the cumulative searched valid frames are statistically analyzed, and the bird identification result with the maximum number of searches is selected from the statistical results, wherein:
[0014] If the maximum number of selected bird identification results exceeds the set second threshold, the selected bird identification result is taken as the current identification result. If any of the current identification results is obtained from the statistical results of the valid frames searched according to the set frame number, the current round of bird identification ends. Otherwise, at least one valid frame that has not been searched is searched from the acquired valid frames.
[0015] or
[0016] If the maximum number of selected bird identification results is not greater than the set second threshold, count whether the number of bird targets in the bird detection results corresponding to the cumulative search valid frames is greater than the set third threshold. If so, report the detection results used to characterize the presence of bird targets.
[0017] In one embodiment of the present invention, the process of continuing until the cumulative number of valid frames searched reaches a set maximum number of analysis frames further includes:
[0018] The acquired valid frames are arranged according to the time information of the data frames, where the time of the previous frame is earlier than the time of the subsequent frame.
[0019] The set frame number is used as the size of the analysis window for analyzing bird identification results, and the analysis window is used to search for the acquired valid frames;
[0020] The sliding analysis window searches for at least one backward frame of the currently searched valid frame to obtain the valid frames searched by the current analysis window.
[0021] Repeat the process until the cumulative number of valid frames searched reaches the maximum number of analysis frames.
[0022] In one embodiment of the present invention, obtaining at least one valid frame from the data frame to be identified based on the bird detection result and the bird identification result of the data frame to be identified includes:
[0023] Obtain bird detection results and bird identification results from the data frame to be identified. The bird detection results include a detection confidence score characterizing the probability of a bird target, and the bird identification results include an identification confidence score characterizing the probability of a bird category.
[0024] Based on the detection confidence level in the bird detection results and the bird identification confidence level in the bird identification results, it is determined whether the data frame to be identified is a valid frame.
[0025] In one embodiment of the present invention, the detection confidence score of the bird detection result corresponding to the data frame to be identified is greater than a set detection confidence score threshold, and
[0026] If the recognition confidence score of the bird recognition result corresponding to the data frame to be recognized is greater than the set recognition confidence score threshold, the data frame to be recognized is determined to be a valid frame.
[0027] In one embodiment of the present invention, the step of obtaining at least one valid frame from the data frame to be identified based on the bird detection result and the bird identification result of the data frame to be identified further includes:
[0028] Based on the bird detection results and bird identification results of the data frame to be identified, as well as the continuous frame information of at least one consecutive frame of the data frame to be identified, at least one valid frame is obtained from the data frame to be identified.
[0029] Among them, at least one consecutive frame of the data frame to be identified includes an adjacent next frame. The consecutive frame information refers to the detection of a bird target in the adjacent next frame and the matching degree between the data frame to be identified and the next frame is greater than a set matching degree threshold.
[0030] In one embodiment of the present invention, the data frame is an image frame, and the step of obtaining at least one valid frame from the data frame to be identified based on the bird detection result and bird identification result of the data frame to be identified includes:
[0031] The trained first neural network model is used to detect birds in the image frames to be identified. The bird detection results include: the image location information of the bird target box and the detection confidence of the bird target box.
[0032] Based on the bird target bounding box image location information, image information within the bird target bounding box is extracted from the image frame to be identified. The extracted image information is then used to perform category identification using a trained second neural network model to obtain bird identification results. These bird identification results include bird category information and the identification confidence level of the bird category information.
[0033] Calculate the intersection-union ratio (IUGR) between the bird target bounding box in the image frame to be identified and the bird target bounding box in the next frame, and consider the data frame as a valid frame if the IUGR is greater than the matching degree threshold, i.e., the IUGR threshold.
[0034] In one embodiment of the present invention, the data frame to be identified is obtained in the following manner:
[0035] Data frames from the data acquisition device are extracted according to a set control frame rate, and the extracted data frames are used as data frames to be identified.
[0036] A second aspect of the present invention provides a bird identification device, comprising:
[0037] The valid frame acquisition module is used to acquire at least one valid frame from the data frame to be identified based on the bird detection result, the bird identification result, and the continuous frame information of at least one consecutive frame of the data frame to be identified. The valid frame is used to characterize a data frame in which a bird target is detected and the target is stable.
[0038] The statistics module is used to search for valid frames according to the set number of frames when the number of valid frames acquired is greater than the set number of frames. It then performs statistics on the bird recognition results corresponding to the currently searched valid frames to obtain statistical results. The statistical results include the number of times each bird recognition result is obtained. The bird recognition result with the maximum number of times is selected from the statistical results. If the maximum number of times of the selected bird recognition result is greater than the set first number threshold, the selected bird recognition result is taken as the current recognition result.
[0039] A third aspect of the present invention provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to execute the computer program to implement the steps of any of the bird identification methods described above.
[0040] The bird identification method provided in this application reduces the power consumption of bird identification by obtaining valid frames from the data frames to be identified. The final identification result is determined by statistically analyzing the bird identification results of at least one valid frame, avoiding false detection and false identification caused by single-frame analysis, thus improving the accuracy and robustness of bird identification. Attached Figure Description
[0041] Figure 1 This is a schematic flowchart of a bird identification method according to an embodiment of this application.
[0042] Figure 2 This is a schematic diagram of the reporting of identification results in an embodiment of this application.
[0043] Figure 3 This is a flowchart illustrating a bird identification method based on images, as described in this embodiment.
[0044] Figure 4 This is a schematic diagram of a bird identification device according to an embodiment of this application.
[0045] Figure 5 This is another schematic diagram of a bird identification device or electronic device according to an embodiment of this application. Detailed Implementation
[0046] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.
[0047] The applicant has found that existing bird identification methods, which improve accuracy through improvements in the identification algorithm itself, have reached a bottleneck. The increased accuracy achieved at a high cost is not cost-effective and unlikely to gain market acceptance. This application provides a bird identification method that improves accuracy through post-processing of the identification results, effectively preventing false detections and misidentifications.
[0048] See Figure 1 As shown, Figure 1 This is a schematic flowchart of a bird identification method according to an embodiment of this application. The method includes:
[0049] Step 101: Based on the bird detection result, bird identification result, and continuous frame information of at least one consecutive frame of the data frame to be identified, obtain at least one valid frame.
[0050] Among them, the valid frame is used to represent the data frame in which a bird target is detected and the target is stable. Target stability means that the displacement change of the target is less than the set displacement change threshold. For example, the bird target is not in a state of flying around and moving quickly, but in a state of moving with small displacement.
[0051] The data frames to be identified include image frames and / or audio frames. The continuous frame information includes the bird detection results corresponding to the continuous frames, as well as the matching degree between the continuous frames and the data frames to be identified.
[0052] As an example, video data frames from the image acquisition device are sequentially extracted according to a set control frame rate. The extracted video data frames are used as the data frames to be identified. In this way, the data frames to be identified are skipped frames of the original video data frames acquired by the image acquisition device, which helps to reduce redundant video data frame information, reduce power consumption, and save computing resources. In the case where the data frames to be identified are skipped frames of the original video data frames, the consecutive frames are the skipped frames adjacent to the data frames to be identified in the extracted data frames. The number of adjacent skipped frames can be one frame, such as the next skipped frame adjacent to the data frames to be identified, or multiple frames, such as the first skipped frame adjacent to the data frames to be identified, the second skipped frame adjacent to the first skipped frame, the third skipped frame adjacent to the second skipped frame, and so on.
[0053] As an example, the bird detection results and bird identification results of the data frame to be identified are obtained. The bird detection results include a detection confidence score characterizing the probability of a bird target, and the bird identification results include an identification confidence score characterizing the probability of a bird category.
[0054] For example, if the data frame to be identified is an image frame, the trained first neural network model is used to detect the image frame and obtain bird detection results. These bird detection results include: the image location information of the bird target box and the detection confidence of the bird target box.
[0055] Based on the bird target bounding box image location information, image information within the bird target bounding box is extracted from the image frame to be identified. Then, using a trained second neural network model, the extracted image information is used to perform category identification, resulting in bird identification results. These results include bird category information and the confidence level of the bird category information.
[0056] As an example, based on the detection confidence level in the bird detection results, the bird recognition confidence level in the bird recognition results, and the information of the next frame adjacent to the data frame to be identified, it is determined whether the data frame to be identified is a valid frame, so as to filter out the data frames to be identified corresponding to the better bird detection results and the better bird recognition results.
[0057] in,
[0058] Valid frames can be determined as follows:
[0059] In the bird detection results corresponding to the data frame to be identified, the detection confidence is greater than the set detection confidence threshold, and
[0060] In the bird identification results corresponding to the data frame to be identified, the identification confidence score is greater than the set identification confidence score threshold, and
[0061] A bird target is detected in the next frame adjacent to the data frame to be identified, and the matching degree between the data frame to be identified and the next frame is greater than a set matching degree threshold. The bird target detection method in the next frame can be the same as the bird target detection method in the data frame to be identified, which helps improve the accuracy of valid frames. For example, the data frame to be identified is an image frame, the matching degree is the intersection-union ratio (IUGR) between the bird target bounding box in the image frame to be identified and the bird target bounding box in the next frame, and the matching degree threshold is the IUGR threshold.
[0062] As an example, the acquired valid frames are arranged according to the time information of the data frames. For example, as time goes by, the time information corresponding to the valid frames also increases, so that the time in the previous frame is earlier than the time in the later frame.
[0063] Step 102: If the number of valid frames acquired is greater than the set number of frames, search for valid frames according to the set number of frames to ensure that the number of valid frames currently being searched meets the set number of frames, so as to analyze the searched valid frames.
[0064] As an example, the analysis window size for analyzing bird recognition results is set to a set number of frames. The analysis window searches for valid frames to perform statistical analysis of bird recognition results window by window. If the cumulative number of valid frames searched has not yet reached the set maximum number of analysis frames, the analysis window is slid to search for at least one backward frame of the currently searched valid frame. This process is repeated after each analysis window's bird results are statistically analyzed, updating the valid frames in the analysis window until the cumulative number of searched valid frames reaches the maximum number of analysis frames. The maximum number of analysis frames can be determined based on the analysis window size (set number of frames) and the set number of slides. For example, if the analysis window size is i frames, the number of slides is j, and the distance between two adjacent analysis windows is p frames, then the maximum number of analysis frames is i + p × j frames. The maximum number of analysis frames can also be set according to the needs of the application scenario.
[0065] Furthermore, if the number of valid frames obtained is not greater than the set number of frames, then return to step 101 to obtain more valid frames.
[0066] The size of the analysis window can be fixed or dynamically changing.
[0067] Step 103: Statistical analysis is performed on the bird identification results corresponding to the currently searched valid frames. The statistical results include the frequency of each bird identification result.
[0068] As an example, the category of the currently searched valid frames is identified to obtain the bird identification results of the valid frames. The category identification method of the valid frames is the same as that of the data frames to be identified.
[0069] As an example,
[0070] If the cumulative number of valid frames searched has not yet reached the maximum number of analysis frames, the bird identification results of each valid frame in the current analysis window are counted in order to determine the bird identification results window by window.
[0071] If the cumulative number of valid frames searched reaches the maximum number of analysis frames, and if bird identification results are not obtained for all analysis windows, then the bird identification results of all accumulated valid frames are counted, that is, the bird identification results of all valid frames constituting the maximum number of analysis frames are counted.
[0072] Step 104: Select the bird identification result with the maximum number of times from the statistical results. If the maximum number of times of the selected bird identification result is greater than the set first number threshold, the selected bird identification result is taken as the current identification result.
[0073] As an example,
[0074] For the bird identification results of each valid frame within the current analysis window, the bird identification result with the maximum number of times is selected. If the maximum number of times of the selected bird identification result is greater than the set first number threshold, the selected bird identification result is used as the current identification result and reported. Otherwise, no identification result is reported.
[0075] After analyzing the bird identification results of all valid frames constituting the maximum number of analysis frames, the analysis further includes:
[0076] For all the accumulated valid frames of bird identification results, the bird identification result with the highest number of occurrences is selected.
[0077] If the maximum number of times the selected bird identification results exceed the set second threshold, the selected bird identification results will be used as the current identification results and reported.
[0078] If the maximum number of selected bird identification results does not exceed the set second threshold, the number of times a bird target exists in the bird detection results corresponding to the cumulative search valid frames is counted. If the count of bird target existence exceeds the set third threshold, the detection result representing the presence of a bird target is reported; otherwise, the report is abandoned. See also Figure 2 As shown, Figure 2This is a schematic diagram illustrating the reporting of recognition results in an embodiment of this application. n valid frames are obtained from m data frames. The analysis window is slid, and valid frames are searched window by window, with bird recognition results counted. Bird recognition results with a maximum count greater than a first threshold are selected and reported; that is, bird recognition results determined based on the current analysis window are reported. The frames corresponding to these bird recognition results are considered valid frames. When the analysis window slides and the cumulative number of valid frames searched reaches the maximum number of analysis frames without obtaining any bird recognition result, the bird recognition results of the cumulatively searched valid frames are counted, and bird recognition results with a maximum count greater than a second threshold are selected and reported; that is, bird recognition results determined based on the maximum number of analysis frames are reported. During the reporting of bird identification results, if the maximum number of bird identification results is not unique, for example, there are multiple maximum numbers corresponding to different bird identification results, all bird identification results with the maximum number of results can be reported. To avoid excessive information reporting and causing interference to users, the earlier valid frame involved in the maximum number of bird identification results can be selected for reporting. For example, the valid frames involved in the maximum number of bird identification results 1 include valid frames 1, 3, and 4, and the valid frames involved in the maximum number of bird identification results 2 include valid frames 2, 3, and 5. Since valid frame 1 is earlier than valid frame 2, the maximum number of bird identification results 1 will be reported.
[0079] As an example, when the data frame is an image frame and an audio frame, the bird identification method of this application can be used to identify birds based on the image frame and the audio frame respectively, and the results of the identification can be fused together to obtain the identification result. For example, a weighting coefficient can be set according to the confidence of the bird identification results of the image frame and the bird identification results of the audio frame, and the identification results can be weighted and summed to achieve fusion.
[0080] This application embodiment avoids false detection and false identification of single frames by statistically analyzing the bird identification results corresponding to several valid frames, effectively preventing occasional false detection and false identification, which helps to enhance the robustness of bird identification and greatly reduces the probability of false identification. Filtering valid frames can remove redundant bird information in the real-time video stream and reduce the power consumption caused by bird detection and identification. The reporting strategy that combines analysis window by window and cumulative maximum frame count reduces false alarms and improves the user experience.
[0081] To facilitate understanding of the embodiments of this application, video images are used as an example for illustration below. It should be understood that the embodiments of this application are not limited to this, and the data frame to be identified can also be an audio frame.
[0082] See Figure 3 As shown, Figure 3 This is a flowchart illustrating a bird identification method based on images, as described in this embodiment.
[0083] Step 301: Acquire video image frames, and extract the image frames to be identified from the acquired video image frames according to the set control frame rate to obtain skip frames.
[0084] For example, if the camera's frame rate is 15 frames per second, and the frame rate is controlled to 5 frames per second, that is, 5 frames can be extracted per second. Compared with the original video image frames, this reduces the video by two-thirds, allowing bird detection and recognition to be performed based on frame skipping, which helps reduce system overhead.
[0085] Step 302: Based on the extracted image frames to be identified, perform bird detection and bird identification to obtain bird detection results and bird identification results.
[0086] As an example, a frame of image to be identified is input into a bird detector used to detect bird images in the image frame, and the bird detection result of the image frame to be identified is obtained. The bird detection result includes: bird target box position information used to characterize the image position of the bird in the image frame, and detection confidence score det_score used to characterize the target box as a bird. The bird detector is implemented by a first neural network model after training.
[0087] Based on the target box location information of the image frame to be identified, the image information in the target box is extracted from the image frame to be identified, and the image information is input into the bird recognizer used to identify the bird category to obtain the bird recognition result. The bird recognition result includes: bird category result and recognition confidence score rec_score used to characterize the probability of the category result. The bird recognizer is implemented through a trained second neural network model.
[0088] Step 303: Based on the bird detection results, bird recognition results, and the matching results between consecutive frames of the image frame to be identified and the image frame to be identified, determine whether the image frame to be identified is a valid frame. The matching results are used to characterize the degree of matching between the image frame to be identified and consecutive frames.
[0089] If so, mark the image frame to be identified as a valid frame and record the frame information of the image frame to be identified, such as the ID of the image frame to be identified; otherwise, mark it as an invalid frame.
[0090] The criteria for determining a valid frame include:
[0091] In the bird detection results of the image frame to be identified, the detection confidence score is greater than the set detection confidence threshold `det_score_threshold`, and in the bird recognition results, the recognition confidence score is greater than the set recognition confidence threshold `rec_score_threshold`. Given that birds flap their wings and move rapidly during flight, recognition is very prone to errors. Therefore, the image frame to be identified under the above conditions is considered a stable frame; that is, the displacement change of the target in the image frame to be identified is less than the set displacement change threshold.
[0092] Whether a frame is stable is determined as follows:
[0093] If birds are detected in both the current frame and the next adjacent frame, and the Intersection over Union (IOU) of the bounding boxes is greater than a set IOU threshold, then the current frame is considered a stable frame. Therefore, when a bird is detected for the first time, the frame is not immediately declared a valid frame. Instead, it is checked whether birds are also detected in the next adjacent frame, and whether the IOU between the bounding boxes in the current frame and the next frame is greater than the IOU threshold. Only if the IOU is greater is the frame considered stable. To improve recognition accuracy, valid frames are sorted according to their temporal information, with earlier valid frames occurring earlier than later valid frames.
[0094] Step 304: Determine whether the number of currently valid frames is less than the set number of frames allowed in the analysis window. This set number of frames represents the size of the analysis window.
[0095] If so, it means there are not enough valid frames. Return to step 301 to obtain the next valid frame.
[0096] Otherwise, proceed to step 305.
[0097] Step 305: Search for valid frames according to the analysis window, and determine whether the current cumulative number of valid frames searched has reached the set maximum number of analysis frames.
[0098] If so, it means that the range of valid frames searched by the analysis window has reached the expected range. If a bird identification result is determined in any analysis window, the current round of voting is stopped to end the current round of identification. If no bird identification result is determined in any analysis window, the bird identification results of all valid frames searched cumulatively are counted, and the bird identification result with the most occurrences is selected.
[0099] If the number of times the selected bird identification result is greater than the set second threshold, then the selected bird identification result is taken as the voting result, and the bird corresponding to the voting result is directly reported, returning to step 304 to start the next round of voting.
[0100] If the number of selected bird identification results is not greater than the second threshold, then proceed to step 306; whereby the second threshold is used to characterize the number of valid frames for the same bird category under the maximum number of analysis frames.
[0101] Otherwise, count the bird identification results of each valid frame in the current analysis window, and select the bird identification result with the most occurrences.
[0102] If the number of times the selected bird identification result is greater than the set first threshold, the selected bird identification result is used as the voting result, and the bird corresponding to the voting result is directly reported. Then, the analysis window is slid to update the valid frames in the analysis window to start the next round of voting. Returning to step 305, the system searches for subsequent valid frames starting from the current valid frame according to the number of frames in the analysis window, so that the number of frames in the current analysis window reaches the set number of frames. Here, the first threshold is used to characterize the threshold of the bird with the most occurrences in the analysis window.
[0103] If the number of selected bird identification results is not greater than the threshold of the first count, then abandon the reporting, slide the analysis window to update the valid frames in the analysis window, and execute step 305.
[0104] Step 306: Determine whether the number of bird targets in the bird detection results of the currently accumulated valid frames is greater than the set third count threshold, where the third count threshold is used to characterize the number of times birds are classified as a category.
[0105] If so, a prompt indicating the presence of a bird will be reported, so that birds are output as a major category, which helps improve the user experience.
[0106] Otherwise, the vote will have no result and will not be reported.
[0107] For example, assuming the maximum number of analysis frames is 10, the analysis window size is 5 frames, and the effective frames are from frame 1 to frame 20, totaling 20 frames, then the search for effective frames is performed using the analysis window size. If frames 1 to 5 are searched and the number of times the bird identification result is "sparrow" is found is the highest and exceeds the first threshold, then this identification result is reported. Moving the analysis window, if frames 2 to 6 are searched and the number of times the bird identification result is "crow" is found is the highest but does not exceed the first threshold, then no report is made. Moving the analysis window, if frames 3 to 7 are searched, and so on, when frames 6 to 10 are searched, since the cumulative number of searched frames has reached the maximum number of analysis frames, if no bird identification result can be determined in any of the analysis windows, then the bird identification results of each effective frame from frame 1 to frame 10 are counted. For example, if the number of times the bird identification result is "sparrow" is found is greater than the second threshold, then this identification result is reported. Step 305 is repeated until all effective frames have been searched.
[0108] It should be understood that the spacing between adjacent analysis windows can be one frame or multiple frames, and can be fixed or dynamically changing. In this embodiment, the hyperparameters such as detection confidence threshold, identification confidence threshold, frame rate control, intersection-over-union (IoU) threshold, maximum number of analysis frames, analysis window size, first count threshold, second count threshold, and third count threshold can be adjusted and optimized according to the application scenario and user feedback.
[0109] In this embodiment, voting decision-making is achieved by statistically analyzing the bird recognition results of multiple valid frames, avoiding false detections and misidentifications caused by single-frame analysis. A sliding window is used to search for valid frames, ensuring continuous updates and thus enhancing the accuracy of bird recognition and significantly reducing the probability of false identification. Statistical analysis of bird recognition results for the maximum number of valid frames improves the robustness of bird recognition. Setting different thresholds for the sliding window and the maximum integer case improves the accuracy of reporting while reducing false alarms. When voting cannot output a recognition result, a detection result indicating the presence of a bird target is output, improving the user experience.
[0110] See Figure 4 As shown, Figure 4 This is a schematic diagram of a bird identification device according to an embodiment of this application. The bird identification device includes:
[0111] The data acquisition module is used to acquire the data frames to be identified.
[0112] The valid frame acquisition module is used to acquire at least one valid frame based on the bird detection result, bird recognition result, and continuous frame information of at least one data frame to be identified. The valid frame represents a data frame in which a bird target is detected and the target is stable.
[0113] The statistics module is used to search for valid frames according to the set number of frames when the number of valid frames acquired is greater than the set number of frames, so that the number of valid frames currently searched meets the set number of frames. The module then performs statistics on the bird recognition results corresponding to the currently searched valid frames to obtain statistical results. The statistical results include the number of times each bird recognition result is identified. The bird recognition result with the maximum number of times is selected from the statistical results. If the maximum number of times of the selected bird recognition result is greater than the set first number threshold, the selected bird recognition result is taken as the current recognition result.
[0114] As an example, the data acquisition module is also used to extract the acquired data frames according to a set control frame rate, and use the extracted data frames as data frames to be identified.
[0115] As an example, the bird identification device also includes:
[0116] The bird target detection module is used to detect bird targets in the data frames to be identified.
[0117] The bird identification module is used to identify the bird category in the data frame to be identified.
[0118] As another example, at least one of the above-mentioned effective frame acquisition module, statistics module, bird target detection module, and bird recognition module may be located on a server or in the cloud independently of the bird recognition device.
[0119] See Figure 5 As shown, Figure 5 This is another schematic diagram of an electronic device or bird identification apparatus according to an embodiment of this application. The electronic device or bird identification apparatus includes a memory and a processor. The memory stores a computer program, and the processor is configured to execute the computer program to implement the steps of the bird identification method.
[0120] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0121] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0122] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the bird identification method.
[0123] For the device / network-side equipment / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments.
[0124] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for bird identification, characterized in that, The method includes: Based on the bird detection results, bird identification results, and continuous frame information of at least one consecutive frame of the data frame to be identified, at least one valid frame is obtained from the data frame to be identified. The valid frame is used to characterize a data frame in which a bird target is detected and the target is stable. If the number of valid frames acquired is greater than the set number of frames, then search for valid frames according to the set number of frames. The bird identification results corresponding to the currently searched valid frames are statistically analyzed to obtain statistical results. These results include: the frequency of each bird identification result; and the bird identification results used to characterize the bird category. The bird identification result with the maximum number of times is selected from the statistical results. If the maximum number of times of the selected bird identification result is greater than the set first number threshold, the selected bird identification result is taken as the current identification result.
2. The bird identification method as described in claim 1, characterized in that, The step of searching for valid frames according to a set frame number further includes: The process continues until the cumulative number of valid frames searched reaches the set maximum number of frames to be analyzed. If the current identification result is not found in the statistical results of the valid frames searched according to the set frame number, the bird identification results corresponding to the cumulative searched valid frames are statistically analyzed, and the bird identification result with the maximum number of searches is selected from the statistical results, wherein: If the maximum number of selected bird identification results exceeds the set second threshold, the selected bird identification result is taken as the current identification result. If any of the current identification results is obtained from the statistical results of the valid frames searched according to the set frame number, the current round of bird identification ends. Otherwise, at least one valid frame that has not been searched is searched from the acquired valid frames. or If the maximum number of selected bird identification results is not greater than the set second threshold, count whether the number of bird targets in the bird detection results corresponding to the cumulative search valid frames is greater than the set third threshold. If so, report the detection results used to characterize the presence of bird targets.
3. The bird identification method as described in claim 2, characterized in that, The process continues until the cumulative number of valid frames searched reaches the set maximum number of analysis frames, further including: The acquired valid frames are arranged according to the time information of the data frames, where the time of the previous frame is earlier than the time of the subsequent frame. The set frame number is used as the size of the analysis window for analyzing bird identification results, and the analysis window is used to search for the acquired valid frames; The sliding analysis window searches for at least one backward frame of the currently searched valid frame to obtain the valid frames searched by the current analysis window. Repeat the process until the cumulative number of valid frames searched reaches the maximum number of analysis frames.
4. The bird identification method according to any one of claims 1 to 3, characterized in that, The step of obtaining at least one valid frame from the data frame to be identified based on the bird detection results and bird identification results includes: Obtain bird detection results and bird identification results from the data frame to be identified. The bird detection results include a detection confidence score characterizing the probability of a bird target, and the bird identification results include an identification confidence score characterizing the probability of a bird category. Based on the detection confidence level in the bird detection results and the bird identification confidence level in the bird identification results, it is determined whether the data frame to be identified is a valid frame.
5. The bird identification method as described in claim 4, characterized in that, In the bird detection results corresponding to the data frame to be identified, the detection confidence is greater than the set detection confidence threshold, and If the recognition confidence score of the bird recognition result corresponding to the data frame to be recognized is greater than the set recognition confidence score threshold, the data frame to be recognized is determined to be a valid frame.
6. The bird identification method as described in claim 5, characterized in that, The step of obtaining at least one valid frame from the data frame to be identified based on the bird detection result and bird identification result of the data frame to be identified further includes: Based on the bird detection results and bird identification results of the data frame to be identified, as well as the continuous frame information of at least one consecutive frame of the data frame to be identified, at least one valid frame is obtained from the data frame to be identified. Among them, at least one consecutive frame of the data frame to be identified includes an adjacent next frame. The consecutive frame information refers to the detection of a bird target in the adjacent next frame and the matching degree between the data frame to be identified and the next frame is greater than a set matching degree threshold.
7. The bird identification method as described in claim 6, characterized in that, The data frame is an image frame. The step of obtaining at least one valid frame from the data frame to be identified based on the bird detection results and bird identification results includes: The trained first neural network model is used to detect birds in the image frames to be identified. The bird detection results include: the image location information of the bird target box and the detection confidence of the bird target box. Based on the bird target bounding box image location information, image information within the bird target bounding box is extracted from the image frame to be identified. The extracted image information is then used to perform category identification using a trained second neural network model to obtain bird identification results. These bird identification results include bird category information and the identification confidence level of the bird category information. Calculate the intersection-union ratio (IUGR) between the bird target bounding box in the image frame to be identified and the bird target bounding box in the next frame, and consider the data frame as a valid frame if the IUGR is greater than the matching degree threshold, i.e., the IUGR threshold.
8. The bird identification method as described in claim 7, characterized in that, The data frame to be identified is obtained in the following manner: Data frames from the data acquisition device are extracted according to a set control frame rate, and the extracted data frames are used as data frames to be identified.
9. A bird identification device, characterized in that, include: The valid frame acquisition module is used to acquire at least one valid frame from the data frame to be identified based on the bird detection result, the bird identification result, and the continuous frame information of at least one consecutive frame of the data frame to be identified. The valid frame is used to characterize a data frame in which a bird target is detected and the target is stable. The statistics module is used to search for valid frames according to the set number of frames when the number of valid frames acquired is greater than the set number of frames. It then performs statistics on the bird identification results corresponding to the currently searched valid frames to obtain statistical results. The statistical results include the frequency of each bird identification result, which is used to characterize the bird category. The module selects the bird identification result with the highest frequency from the statistical results. If the maximum frequency of the selected bird identification result is greater than the set first frequency threshold, the selected bird identification result is taken as the current identification result.
10. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing a computer program and the processor being configured to execute the computer program to implement the steps of the bird identification method according to any one of claims 1 to 8.