Bird identification method and system based on negative sample feedback and large model knowledge enhancement

Abstract

The present application relates to the field of artificial intelligence and computer vision technology, and particularly relates to a bird identification method and system based on negative sample feedback and large model knowledge enhancement, the method comprising: when detecting that a monitoring area has a moving object with a heat source feature, waking up a camera to collect visual data and uploading the visual data to the cloud; performing target detection and preliminary classification on the uploaded visual data through a first-level identification model deployed on the cloud to obtain a preliminary identification result and a corresponding confidence; performing shunt decision and second-level identification based on the confidence to obtain a final identification result; pushing the final identification result to a user terminal and receiving negative sample feedback information submitted by the user based on the final identification result; and performing incremental training and updating on the first-level identification model based on the negative sample feedback information; through the two-level identification architecture of "first-level rapid screening + second-level knowledge enhancement", the present application significantly reduces the false positive rate and significantly improves the anti-interference ability in complex environments.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, and in particular to a bird recognition method and system based on negative sample feedback and large model knowledge enhancement. Background Technology

[0002] With increasing awareness of ecological protection, computer vision-based automated wildlife monitoring technologies have been widely applied. Currently, mainstream monitoring solutions typically deploy pre-trained deep learning models (such as the YOLO series and Faster R-CNN) in the cloud or at the edge to detect and identify birds in the collected images or videos. While these solutions achieve automation to some extent, they still have significant limitations.

[0003] First, the problem of model performance fixation is prominent. Most existing systems adopt an open-loop model of "offline training and online inference." Once the model is deployed, its parameters remain fixed and cannot learn from the actual operating environment. When encountering new disturbances not fully covered in the training data (such as leaves of specific shapes or small mammals), the model will produce continuous false alarms, which the system itself cannot correct. It is necessary to rely on manual data collection, labeling, and training of new models, which is time-consuming and costly.

[0004] Secondly, traditional models have a weak ability to identify unknown or rare species. Limited by the coverage of the training dataset, they often fail to correctly identify or miss rare species not present in the training set (zero-sample scenarios) or those found in long-tailed distributions. These models lack the ability to use external knowledge for reasoning, making them ill-equipped to handle fine-grained identification and knowledge-scarce scenarios.

[0005] Furthermore, there is a lack of effective interaction between the system and the user. Existing systems typically treat users as passive recipients, and users cannot easily provide feedback on identification errors they discover during use. These "difficult samples" contain valuable information, but they are wasted due to the lack of a recycling mechanism and fail to be transformed into resources for optimizing model performance.

[0006] To address the aforementioned issues, although some patents have proposed improvement solutions such as multimodal fusion, for example, by fusing image and voiceprint features to improve accuracy (such as patent CN120544239A, Bird Identification and Ecological Monitoring Method and System Based on Multimodal Data Fusion), they have not fundamentally broken through the framework of static models and unidirectional reasoning, and cannot achieve adaptive evolution and knowledge expansion of the system. Summary of the Invention

[0007] To address the technical problems of existing bird identification technologies, such as model performance invariability, poor adaptability to unknown scenes and species, and lack of self-optimization capabilities based on user feedback, this invention aims to solve at least one of the following technical problems:

[0008] Problem of poor adaptability to complex environments: Existing fixed models have a high false alarm rate under the interference of changes in outdoor lighting, target occlusion, and non-bird objects (such as squirrels, fallen leaves, and changes in light and shadow), and cannot effectively distinguish between targets and interference objects.

[0009] The problem of lack of fine-grained knowledge and difficulty in identifying unknown species: Traditional visual models can only output simple category labels and cannot provide fine-grained information such as bird habits and distribution areas. They also lack the ability to identify new species outside the training set (zero-shot scenes) and can only rely on data-driven feature matching, without logical reasoning ability.

[0010] The system lacks self-evolution capabilities: The existing monitoring system adopts an open-loop operation mode, which cannot use the erroneous identification samples (i.e. negative samples) found by users in actual use to optimize itself, resulting in long-term stagnation of model performance and high operation and maintenance costs.

[0011] The problem of balancing real-time performance and accuracy: Existing technologies either use lightweight models to ensure real-time performance but sacrifice accuracy, or use complex models to improve accuracy but cannot meet the real-time recognition requirements.

[0012] To achieve the objectives of this invention, the following technical solution is adopted:

[0013] The first aspect of this invention proposes a bird identification method based on negative sample feedback and large model knowledge enhancement, comprising the following steps:

[0014] Step S1: When a moving object with heat source characteristics is detected in the monitored area, the camera is activated to collect visual data and upload it to the cloud;

[0015] Step S2: Perform target detection and preliminary classification on the uploaded visual data using the first-level recognition model deployed in the cloud, and obtain preliminary recognition results and corresponding confidence scores;

[0016] Step S3: Based on the confidence level, perform a triage decision and secondary identification to obtain the final identification result;

[0017] Step S4: Push the final identification result to the user terminal and receive negative sample feedback information submitted by the user based on the final identification result;

[0018] Step S5: Based on the negative sample feedback information, perform incremental training and update on the first-level recognition model.

[0019] A further improvement is that, in step S1, the PIR sensor deployed on the front-end device works in conjunction with the image motion detection unit to detect whether there is a moving object with heat source characteristics in the monitored area. The PIR sensor is responsible for detecting heat source characteristics in the monitored area, and the image motion detection unit is responsible for detecting the movement state of the object.

[0020] A further improvement is that the specific method of step S2 includes: after the cloud-based data receiving gateway receives the uploaded visual data, it inputs it into the primary recognition model, quickly locates the target area through the primary recognition model, distinguishes between "bird" and "non-bird" categories, and outputs preliminary species classification results and corresponding confidence levels for the "bird" category.

[0021] A further improvement is that the specific method of step S3 includes:

[0022] Determine whether the confidence level is greater than or equal to a first preset threshold;

[0023] If so, the preliminary recognition result of the first-level recognition model shall be adopted as the final recognition result;

[0024] If not, then it is further determined whether the confidence level is greater than or equal to a second preset threshold, wherein the second preset threshold is less than the first preset threshold;

[0025] If the confidence level is greater than or equal to the second preset threshold, or if the preliminary identification result is "unknown bird", then the visual data and related information are input into the multimodal large language model for secondary identification, and the result of the secondary identification is used as the final identification result;

[0026] If the confidence level is less than the second preset threshold, the visual data is determined to be invalid data, and the subsequent process is terminated.

[0027] A further improvement lies in the specific method for inputting the visual data and associated information into a multimodal large language model for secondary recognition, which includes:

[0028] Based on the visual data, the preliminary recognition results of the first-level recognition model, the geographical location information and time information of the data collection, structured prompt words are constructed;

[0029] The structured prompts are input into the multimodal large language model, which then combines the knowledge base with the contextual information in the structured prompts to perform reasoning and output the corrected recognition result.

[0030] A further improvement is that the first-level recognition model is a pre-trained lightweight object detection model; and the multimodal large language model is a pre-trained model with visual understanding and common sense reasoning capabilities.

[0031] A further improvement lies in the fact that, in step S5, the specific method for incrementally training and updating the first-level recognition model based on the negative sample feedback information includes:

[0032] Collect negative sample data from user feedback and combine it with the original positive sample data to form a mixed training set;

[0033] The first-level recognition model is incrementally trained using a weighted loss function, wherein the negative sample data is given a higher weight than the original positive sample data in the loss calculation;

[0034] The performance of the newly trained model is evaluated, and the evaluation metrics include recognition accuracy, false positive rate, and recall rate. If the evaluation results meet the preset threshold, the weights of the new model are retained; otherwise, the weighting coefficients are adjusted and the model is retrained.

[0035] The hot update method is used to automatically deploy the new model weights after incremental training to the first-level recognition model in the cloud, replacing the original model weights.

[0036] A second aspect of this invention proposes a bird recognition system based on negative sample feedback and large model knowledge enhancement, comprising:

[0037] The front-end acquisition module is configured to wake up the camera to collect visual data and upload it to the cloud when a moving object with heat source characteristics is detected in the monitored area.

[0038] The first-level recognition module is deployed in the cloud and configured to run the first-level recognition model. It performs target detection and preliminary classification on the uploaded visual data to obtain preliminary recognition results and corresponding confidence scores.

[0039] The traffic splitting and secondary recognition module is deployed in the cloud and configured to perform traffic splitting decisions and secondary recognition based on the confidence level to obtain the final recognition result.

[0040] The user interaction and feedback module is configured to push the final identification result to the user terminal and receive negative sample feedback information submitted by the user.

[0041] The model optimization module is configured to perform incremental training and updates on the first-level recognition model based on the negative sample feedback information.

[0042] A third aspect of the present invention provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a bird recognition method based on negative sample feedback and large model knowledge enhancement as described in any of the first aspects.

[0043] A fourth aspect of the present invention provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform a bird identification method based on negative sample feedback and large model knowledge enhancement as described in any one of the first aspects.

[0044] The beneficial effects of this invention are as follows:

[0045] This invention employs a two-level recognition architecture: "first-level rapid screening + second-level knowledge enhancement." The first-level recognition ensures real-time performance for regular samples, while the second-level recognition utilizes the common-sense reasoning capabilities of a multimodal large language model to correct misjudgments caused by visual feature confusion and effectively eliminates non-avian interference, significantly reducing the false alarm rate and significantly improving the anti-interference capability in complex environments. The first-level recognition uses a lightweight target detection model with short single-frame image processing time, meeting real-time recognition requirements. The second-level recognition is triggered only for samples with medium confidence and samples of unknown species, avoiding the impact of high latency from large models on overall efficiency and achieving a balance between fast recognition of regular samples and accurate recognition of difficult samples.

[0046] This invention constructs a closed-loop optimization mechanism based on user negative sample feedback, provides a convenient false alarm feedback interface, transforms users from mere users into data annotators, and can automatically optimize models using user feedback without the need for manual data collection and annotation, significantly reducing operation and maintenance costs and solving the problem of performance solidification in existing models.

[0047] This invention integrates spatiotemporal contextual information with the massive knowledge base of a multimodal large language model through structured prompt words. For unknown bird species outside the training set, the multimodal large language model can perform logical reasoning based on morphological features and distribution patterns to output accurate recognition results and evidence, thus making up for the shortcomings of existing models in recognizing long-tailed samples and new species. Attached Figure Description

[0048] Figure 1 This is a flowchart of a bird identification method based on negative sample feedback and large model knowledge enhancement according to the present invention;

[0049] Figure 2 This is a system architecture diagram of a bird recognition system based on negative sample feedback and large model knowledge enhancement according to the present invention.

[0050] Figure 3 This is a logic diagram of a bird identification method based on negative sample feedback and large model knowledge enhancement according to the present invention.

[0051] Figure 4 This is a diagram showing the user terminal APP interface in this invention;

[0052] Figure 5This is a diagram showing the recognition results of the user terminal APP in this invention;

[0053] Figure 6 This is a diagram of the false alarm feedback interface of the user terminal APP in this invention;

[0054] Figure 7 This is a schematic diagram of an electronic device. Detailed Implementation

[0055] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0056] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0057] Please refer to the attached document. Figure 1 -Appendix Figure 7 The first aspect of this invention proposes a bird identification method based on negative sample feedback and large model knowledge enhancement, such as... Figure 1 As shown, it includes the following steps:

[0058] Step S1: When a moving object with heat source characteristics is detected in the monitored area, the camera is activated to collect visual data and upload it to the cloud.

[0059] Step S2: Perform target detection and preliminary classification on the uploaded visual data using a first-level recognition model deployed in the cloud to obtain preliminary recognition results and corresponding confidence levels.

[0060] Step S3: Perform triage decision and secondary identification based on the confidence level to obtain the final identification result.

[0061] Step S4: Push the final identification result to the user terminal and receive negative sample feedback information submitted by the user based on the final identification result.

[0062] Step S5: Based on the negative sample feedback information, perform incremental training and update on the first-level recognition model.

[0063] Specifically, in step S1, the PIR sensor deployed on the front-end device (intelligent bird feeder) works in conjunction with the image motion detection unit to detect whether there are moving objects with heat source characteristics in the monitored area. The PIR sensor is responsible for detecting heat source characteristics (such as the body temperature of birds and mammals) in the monitored area, and the image motion detection unit is responsible for detecting the movement status of the objects.

[0064] Understandably, the image motion detection unit can use the background subtraction method for motion detection. When both conditions (the PIR sensor detects a heat source signal and the image motion detection unit detects a moving object) are met simultaneously, it indicates that an active target has entered the monitoring area, which is considered a valid trigger, avoiding invalid triggers caused by wind, grass, or changes in light and shadow. After triggering, the front-end device wakes up the high-definition camera to capture high-definition images or short videos (the duration is set according to actual needs), forming visual data. The collected visual data is then compressed and uploaded to the cloud data receiving gateway through the IoT communication module.

[0065] Specifically, the method of step S2 includes: after receiving the uploaded visual data through the cloud data receiving gateway, inputting it into the primary recognition model, quickly locating the target area through the primary recognition model, distinguishing between "bird" and "non-bird" categories, and outputting preliminary species classification results and corresponding confidence levels for the "bird" category.

[0066] Specifically, the primary identification model is a pre-trained lightweight object detection model, such as YOLOv5 or YOLOv8, which has the advantages of low computational load and fast inference speed, and is suitable for real-time processing requirements in the cloud. The primary identification model mainly implements two functions: (1) fast screening: distinguishing between "birds" and "non-birds" (such as leaves, squirrels, static objects); (2) coarse classification: performing preliminary species classification on targets identified as "birds" and outputting candidate species names and confidence scores.

[0067] Understandably, the confidence level ranges from 0 to 100%, representing the level of confidence that the primary recognition model has in the recognition results, and is used to quantify the model's grasp of the recognition results.

[0068] Specifically, in this embodiment, the specific method of step S3 includes:

[0069] Determine whether the confidence level is greater than or equal to a first preset threshold.

[0070] If so, the preliminary recognition result of the first-level recognition model is adopted as the final recognition result.

[0071] If not, then it is further determined whether the confidence level is greater than or equal to a second preset threshold, wherein the second preset threshold is less than the first preset threshold.

[0072] If the confidence level is greater than or equal to the second preset threshold, or if the preliminary identification result is "unknown bird", then the visual data and related information are input into the multimodal large language model for secondary identification, and the result of the secondary identification is used as the final identification result.

[0073] If the confidence level is less than the second preset threshold, the visual data is determined to be invalid data, and the subsequent process is terminated to save cloud computing resources.

[0074] Specifically, in this embodiment, preferably, the first preset threshold is 85% and the second preset threshold is 45%; based on the first preset threshold and the second preset threshold, the scenarios can be divided into high confidence scenarios, medium confidence scenarios, and low confidence scenarios.

[0075] High confidence scenario: Confidence ≥ 85% indicates that the first-level recognition model is highly confident in the recognition result and is judged as a high-confidence recognition result. The second-level recognition is skipped directly, and the preliminary recognition result is used as the final recognition result to ensure the timeliness of recognition.

[0076] In medium confidence scenarios (45% ≤ confidence < 85%), the sample is judged as a difficult sample (such as a bird with an unclear shape or an easily confused species), or the preliminary identification result is "unknown bird". In this case, the visual data and related information need to be input into a multimodal large language model for secondary recognition to ensure the accuracy of the recognition.

[0077] Low confidence scenario: If the confidence level is less than 45%, the data is deemed invalid, the subsequent process is terminated, and no results are pushed to the user.

[0078] In this embodiment, the specific method for inputting the visual data and associated information into a multimodal large language model for secondary recognition includes:

[0079] Based on the visual data, the preliminary recognition results of the first-level recognition model, the geographical location information and time information of the data collection, structured prompt words are constructed.

[0080] It is understandable that the structured prompt (Prompt) structure = [Image Embedding] + [Geographical Location Information] + [Time and Season Information] + [Top-N Prediction Results of the Primary Model]. That is, the structured prompt (Prompt) specifically includes: (1) Image Embedding: Feature vectors extracted from keyframes of high-definition images or short videos through convolutional neural networks (CNN), which are used to provide visual feature information to the multimodal large language model; (2) Spatiotemporal context information: Geographical location information of the shooting (such as "Beijing", "subtropical region" or latitude and longitude coordinates) and seasonal information of the shooting time (such as "December 2025", "winter"); (3) Preliminary identification results: including preliminary species classification results (such as "sparrow") and corresponding confidence (such as 60%).

[0081] For example, the format of structured prompts could be: "Image Embedding: [Image Feature Vector]; Geographic Location: [Latitude and Longitude / Climate Zone, e.g., Beijing - Temperate Monsoon Climate]; Time and Season: [Year-Month-Day / Season, e.g., 2025-12-23-Winter]; Primary Recognition Top-3 Results: [Species A (Confidence X), Species B (Confidence Y), Species C (Confidence Z)]. Please confirm the species' accuracy by considering bird morphological characteristics, geographical distribution patterns, and migratory habits. If incorrect, please specify the species and describe its core characteristics, habits, and distribution range."

[0082] The structured prompts are input into the multimodal large language model, which combines the knowledge base with the contextual information in the structured prompts to perform reasoning, eliminate unreasonable results in the preliminary recognition results, output corrected recognition results, and may include fine-grained descriptions such as bird habits.

[0083] Understandably, multimodal large language models utilize a massive bird knowledge base (including species morphological characteristics, distribution areas, migration patterns, etc.) for logical reasoning. Through morphological feature analysis, verification of geographical distribution patterns, and judgment of migratory habits, they output accurate species names, correcting errors in preliminary identification results (such as mistaking "squirrel" for "bird" or "light-vented bulbul" for "sparrow"). They output corrected bird identification results and detailed species characteristic descriptions (including bird plumage characteristics, beak shape characteristics, body size parameters, habitat, food habits, and geographical distribution range, etc.), thus enhancing the user's birdwatching experience and educational value.

[0084] Specifically, the multimodal large language model is a pre-trained model (LLM) with visual understanding and common-sense reasoning capabilities, such as Doubao, Deepseek, and Tongyi Qianwen. For example, existing technologies might incorrectly identify "swallow" in northern winters (simply because of similar shape). This invention introduces semantic-level rule constraints and explicitly injects spatiotemporal contextual information into the reasoning process through structured prompts. By utilizing the encyclopedic knowledge (knowledge base) stored within the LLM as prior conditions, it can logically exclude recognition results that violate natural laws. This is a reasoning capability that purely statistical CNN / RNN models in existing technologies lack.

[0085] Specifically, in step S4, the user terminal is a mobile device with a dedicated APP installed, such as... Figure 6 As shown, the APP interface has an interactive button for false alarm feedback. Users can select the feedback type of "Identify Missed Reports" or "Identify Wrong Reports" and can optionally fill in the correct species name. The system associates the user-marked wrong samples with the shooting parameters and the identification results, marks them as negative samples, and stores them in the negative sample database.

[0086] Specifically, in step S5, the method for incrementally training and updating the first-level recognition model based on the negative sample feedback information includes:

[0087] Collect negative sample data from user feedback and combine it with the original positive sample data to form a mixed training set.

[0088] Specifically, negative samples from user feedback are extracted from the negative sample database, duplicate or invalid data are removed, and the collected negative samples are fused with the original training dataset (positive samples) to construct a hybrid training set.

[0089] The first-level recognition model is incrementally trained using a weighted loss function, wherein the negative sample data is given a higher weight than the original positive sample data in the loss calculation.

[0090] Specifically, the weighted loss function is either Weighted Cross-Entropy Loss or Focal Loss. The weighted loss function increases the weight of negative samples in the loss calculation, forcing the model to focus on learning features that lead to misclassification (such as the squirrel's tail features or the texture of leaves) during backpropagation, thus avoiding repeated misclassification. The weight of negative samples in the training set can be set to 1.5-3 times that of positive samples.

[0091] The performance of the newly trained model is evaluated, and the evaluation metrics include recognition accuracy, false positive rate, and recall rate. If the evaluation results meet the preset thresholds (e.g., accuracy ≥ 95%, false positive rate ≤ 3%), the weights of the new model are retained; otherwise, the weighting coefficients are adjusted and the model is retrained.

[0092] A hot update approach is adopted to automatically deploy the weights of the newly trained model to the first-level recognition model in the cloud, replacing the original model weights. Specifically, a rolling update strategy is used during the deployment process to keep the cloud recognition service running continuously without interrupting real-time recognition tasks, thus achieving imperceptible self-evolution of the model.

[0093] Specifically, based on the negative sample feedback information, incremental training and updating of the first-level recognition model can be performed once a week.

[0094] The second aspect of this invention proposes a bird recognition system based on negative sample feedback and large model knowledge enhancement, which corresponds to the bird recognition method based on negative sample feedback and large model knowledge enhancement provided in the above-mentioned embodiments of this invention. Since the bird recognition system based on negative sample feedback and large model knowledge enhancement provided in this embodiment corresponds to the bird recognition method based on negative sample feedback and large model knowledge enhancement provided in the above-mentioned embodiments of this invention, the implementation method of the aforementioned bird recognition method based on negative sample feedback and large model knowledge enhancement is also applicable to the bird recognition system based on negative sample feedback and large model knowledge enhancement provided in this embodiment.

[0095] Specifically, a bird recognition system based on negative sample feedback and large model knowledge enhancement includes:

[0096] The front-end acquisition module is configured to wake up the camera to collect visual data and upload it to the cloud when a moving object with heat source characteristics is detected in the monitored area.

[0097] The first-level recognition module, deployed in the cloud, is configured to run the first-level recognition model, perform target detection and preliminary classification on the uploaded visual data, and obtain preliminary recognition results and corresponding confidence scores.

[0098] The traffic splitting and secondary recognition module is deployed in the cloud and configured to perform traffic splitting decisions and secondary recognition based on the confidence level to obtain the final recognition result.

[0099] The user interaction and feedback module is configured to push the final identification result to the user terminal and receive negative sample feedback information submitted by the user.

[0100] The model optimization module is configured to perform incremental training and updates on the first-level recognition model based on the negative sample feedback information.

[0101] like Figure 2 The diagram shown illustrates the system architecture of this invention, comprising three core components: the front-end device (Edge), the cloud server (Cloud), and the user terminal (Client), forming a closed-loop system of "front-end data collection - cloud inference - user feedback - model evolution."

[0102] Front-end equipment: integrates a PIR sensor, an image motion detection unit, and a high-definition camera, responsible for target triggering and visual data acquisition.

[0103] Cloud server: Deploys the primary recognition module, the traffic splitting and secondary recognition module, the user interaction and feedback module, and the model optimization module, and is responsible for data processing, inference calculation and model updates;

[0104] User terminal: Presented in the form of an APP, responsible for displaying recognition results and user feedback interaction.

[0105] See Figure 7 The present invention also provides an electronic device and a computer-readable storage medium.

[0106] like Figure 7 The diagram illustrates an electronic device according to an embodiment of the present invention. This electronic device includes a processor 11, a memory 12, and a computer program stored in the memory and executable on the processor 11. When the processor 11 executes the computer program, it implements the steps described in the above embodiment of a bird recognition method based on negative sample feedback and large model knowledge enhancement. Alternatively, when the processor 11 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments.

[0107] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor 11 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0108] The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the schematic diagram is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0109] The processor 11 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting various parts of the electronic device through various interfaces and lines.

[0110] The memory 12 can be used to store the computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system 121, at least one application program 122 required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0111] Wherein, if the modules / units integrated in the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0112] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0113] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A bird identification method based on negative sample feedback and large model knowledge enhancement, characterized in that, Includes the following steps: Step S1: When a moving object with heat source characteristics is detected in the monitored area, the camera is activated to collect visual data and upload it to the cloud; Step S2: Use the first-level recognition model deployed in the cloud to perform target detection and preliminary classification on the uploaded visual data to obtain preliminary recognition results and corresponding confidence levels; Step S3: Based on the confidence level, perform a triage decision and secondary identification to obtain the final identification result; Step S4: Push the final identification result to the user terminal and receive negative sample feedback information submitted by the user based on the final identification result; Step S5: Based on the negative sample feedback information, perform incremental training and update on the first-level recognition model.

2. The bird identification method based on negative sample feedback and large model knowledge enhancement according to claim 1, characterized in that, In step S1, the PIR sensor deployed on the front-end device works in conjunction with the image motion detection unit to detect whether there is a moving object with heat source characteristics in the monitored area. The PIR sensor is responsible for detecting heat source characteristics in the monitored area, and the image motion detection unit is responsible for detecting the movement state of the object.

3. The bird identification method based on negative sample feedback and large model knowledge enhancement according to claim 1, characterized in that, The specific method of step S2 includes: after receiving the uploaded visual data through the cloud data receiving gateway, inputting it into the primary recognition model, quickly locating the target area through the primary recognition model, distinguishing between "bird" and "non-bird" categories, and outputting preliminary species classification results and corresponding confidence levels for the "bird" category.

4. The bird identification method based on negative sample feedback and large model knowledge enhancement according to claim 1, characterized in that, The specific methods for step S3 include: Determine whether the confidence level is greater than or equal to a first preset threshold; If so, the preliminary recognition result of the first-level recognition model shall be adopted as the final recognition result; If not, then it is further determined whether the confidence level is greater than or equal to a second preset threshold, wherein the second preset threshold is less than the first preset threshold; If the confidence level is greater than or equal to the second preset threshold, or if the preliminary identification result is "unknown bird", then the visual data and related information are input into the multimodal large language model for secondary identification, and the result of the secondary identification is used as the final identification result; If the confidence level is less than the second preset threshold, the visual data is determined to be invalid data, and the subsequent process is terminated.

5. The bird identification method based on negative sample feedback and large model knowledge enhancement according to claim 4, characterized in that, The specific methods for inputting the visual data and associated information into a multimodal large language model for secondary recognition include: Based on the visual data, the preliminary recognition results of the first-level recognition model, the geographical location information and time information of the data collection, structured prompt words are constructed; The structured prompts are input into the multimodal large language model, which then combines the knowledge base with the contextual information in the structured prompts to perform reasoning and output the corrected recognition result.

6. The bird identification method based on negative sample feedback and large model knowledge enhancement according to claim 4, characterized in that, The primary recognition model is a pre-trained lightweight object detection model; the multimodal large language model is a pre-trained model with visual understanding and common sense reasoning capabilities.

7. The bird identification method based on negative sample feedback and large model knowledge enhancement according to claim 1, characterized in that, In step S5, the specific method for incrementally training and updating the first-level recognition model based on the negative sample feedback information includes: Collect negative sample data from user feedback and combine it with the original positive sample data to form a mixed training set; The first-level recognition model is incrementally trained using a weighted loss function, wherein the negative sample data is given a higher weight than the original positive sample data in the loss calculation; The performance of the newly trained model is evaluated, and the evaluation metrics include recognition accuracy, false positive rate, and recall rate. If the evaluation results meet the preset threshold, the weights of the new model are retained; otherwise, the weighting coefficients are adjusted and the model is retrained. The hot update method is used to automatically deploy the new model weights after incremental training to the first-level recognition model in the cloud, replacing the original model weights.

8. A bird recognition system based on negative sample feedback and large model knowledge enhancement, characterized in that, include: The front-end acquisition module is configured to wake up the camera to collect visual data and upload it to the cloud when a moving object with heat source characteristics is detected in the monitored area. The first-level recognition module is deployed in the cloud and configured to run the first-level recognition model. It performs target detection and preliminary classification on the uploaded visual data to obtain preliminary recognition results and corresponding confidence scores. The traffic splitting and secondary recognition module is deployed in the cloud and configured to perform traffic splitting decisions and secondary recognition based on the confidence level to obtain the final recognition result. The user interaction and feedback module is configured to push the final identification result to the user terminal and receive negative sample feedback information submitted by the user. The model optimization module is configured to perform incremental training and updates on the first-level recognition model based on the negative sample feedback information.

9. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a bird identification method based on negative sample feedback and large model knowledge enhancement as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform a bird identification method based on negative sample feedback and large model knowledge enhancement as described in any one of claims 1 to 7.

Images