Closed set target detection and open vocabulary recognition fused intelligent recognition method and system for animal species in natural reserve

Abstract

The invention relates to a closed set target detection and open vocabulary recognition fused intelligent recognition method and system for animal species in natural reserves. The method comprises the following steps: firstly, carrying out initial detection and judgment on a monitoring image by utilizing a closed set target detection model; for a low-confidence target or a non-detected target, a high-quality target area is obtained through a prompt drive segmentation and self-adaptive cutting mechanism; matching the region with a text label set by using an open vocabulary cross-modal recognition model; and finally, carrying out confidence-based collaborative modeling and adaptive fusion on a closed set detection result and an open vocabulary recognition result, and judging and marking an uncertainty result according to a threshold value. According to the method, high-precision identification of known species and coverage of unseen species are both considered, the reliability and interpretability of an identification result in a complex environment are improved through multi-source fusion, and the method has good universality and expandability and is suitable for long-term continuous monitoring of natural reserves and other target identification scenes.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and ecological environment monitoring technology, specifically to an intelligent identification method and system for animal species in nature reserves that integrates closed-set target detection and open-vocabulary recognition. This invention is particularly applicable to scenarios such as nature reserves, wildlife monitoring, ecological surveys, and biodiversity assessment. Background Technology

[0002] Nature reserves are important carriers of biodiversity conservation, and wildlife species monitoring is a fundamental task in the management and ecological research of nature reserves. Currently, wildlife monitoring and ecological surveys in nature reserves largely rely on manual patrols, manual image interpretation, or analysis of monitoring data based on fixed-category automatic identification systems. With the large-scale deployment of wildlife cameras and unmanned monitoring equipment, nature reserves have generated massive amounts of image and video data during long-term operation. Manual analysis methods are inefficient and costly, making it difficult to meet the practical needs of continuous monitoring and rapid response.

[0003] Existing deep learning-based object detection models typically rely on predefined species categories (closed sets) for training. These models achieve high accuracy in identifying categories covered by the training set, but when species not found in the training set or rare species appear in the monitoring scenario, they often fail to provide effective identification results, easily leading to missed detections or misjudgments. This makes it difficult to meet the needs of nature reserves for "new species discovery" in long-term dynamic monitoring of biodiversity.

[0004] In recent years, with the development of artificial intelligence technologies such as visual-language models (e.g., CLIP) and multimodal learning, methods such as open-vocabulary recognition and cross-modal semantic matching have begun to be introduced to overcome the limitations of fixed categories in traditional closed-set object detection methods. For example, some existing techniques improve the model's adaptability to complex scenes by incorporating semantic information from visual-language models during the object detection model training phase. Other techniques attempt to expand the target category space and enhance the recognition of unknown categories by fusing multimodal features and combining them with dynamic knowledge graphs.

[0005] However, existing technological solutions still have shortcomings when facing long-term, open, and unattended monitoring scenarios in nature reserves. Some solutions focus on semantic enhancement during the model training phase, but their recognition capabilities are still limited by the existing model structure and training data, making it difficult to effectively identify species not covered during the training phase. Other solutions focus on expanding the category space through multimodal fusion, but lack a mechanism to ensure the stability of high-precision identification of known common species. Without a reliability gating system based on high-precision closed-set detection, the accuracy and statistical stability of common species identification can easily decline, thus affecting the reliable accumulation and trend analysis of long-term continuous monitoring data.

[0006] Therefore, existing technologies generally lack a comprehensive technical solution that can use closed-set target detection as the main path for high-precision recognition, integrate open vocabulary recognition capabilities on this basis, and simultaneously perform collaborative modeling, confidence fusion, and uncertainty discrimination on multi-source recognition results. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent identification method and system for animal species in nature reserves that integrates closed set target detection and open vocabulary recognition. This invention significantly improves the accuracy and stability of the identification of known species by collaboratively modeling and fusing the closed set detection results and the open vocabulary recognition results, while effectively enhancing the system's ability to identify unseen and rare species. It also enables effective discrimination and labeling of uncertainty in the identification results, thereby improving the intelligence level, coverage and reliability of long-term monitoring tasks in nature reserves.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: an intelligent identification method for animal species in nature reserves that integrates closed set target detection and open vocabulary recognition, comprising the following steps:

[0009] Step S1: Acquire image data collected by the monitoring equipment of the nature reserve, and preprocess the image data;

[0010] Step S2: Use a closed-set target detection model to detect targets in the preprocessed image data, obtain at least one target candidate region, and output the corresponding target category and target confidence P. det and the target bounding box;

[0011] Step S3: Determine the target candidate region based on the target confidence level. When the target confidence level P... det Reaching or exceeding a preset threshold T det When the target confidence level P is high, the corresponding candidate region is determined as a credible target; when the target confidence level P is high, the corresponding candidate region is determined as a credible target. det Below T det If the closed-set target detection model fails to detect the target, the open-vocabulary recognition process is triggered.

[0012] Step S4: Perform cue-driven segmentation and target cropping on the target candidate region to obtain the target image region for cross-modal recognition, wherein the cue information includes detection box cue and / or semantic text cue;

[0013] Step S5: Based on the open vocabulary cross-modal recognition model CLIP, the target image region is matched with a preset set of text labels to obtain the target recognition result and the corresponding recognition confidence under the open vocabulary meaning;

[0014] Step S6: Perform collaborative modeling and confidence fusion on the closed-set target detection results and the open-vocabulary recognition results. Output the final target recognition result based on the fusion result, and perform uncertainty labeling or backoff processing on targets whose fusion confidence fails to meet the conditions.

[0015] Furthermore, in step S1, the preprocessing includes image size normalization, noise suppression, and format conversion.

[0016] Furthermore, step S4 specifically includes:

[0017] Step S41: Construct a normalized geometric cue box b based on the target bounding box information output by the closed-set target detection model;

[0018] Step S42: Input the geometric cue box b and the original image into the cue-driven segmentation model to perform fine segmentation of the target candidate region, and select the mask with the highest score from the output candidate masks. As a final target segmentation mask;

[0019] Step S43: Based on the target segmentation mask The original image is cropped to generate a target image patch containing only the target region.

[0020] Step S44: When closed-set target detection does not output a valid target bounding box or the target confidence is lower than T det At the same time, semantic textual prompts are constructed, and the original image and the textual prompts are input into the prompt-driven segmentation model to obtain the animal mask. Open segmentation is performed on the entire image;

[0021] Step S45: Evaluate the segmentation results based on the mask quality evaluation function q(M) and set a quality threshold T. mask When the segmentation result satisfies q(M) * )≥T mask When the segmentation result is satisfied, the cropping result based on the segmentation mask is used as the target image region; when the segmentation result satisfies q(M * ) <T mask When necessary, the original image or the cropping result based on the detection box is used as the target image region.

[0022] Further, step S5 specifically includes:

[0023] Step S51: Construct a set of open-vocabulary candidate text labels ,in Let S(s) represent the k-th fine-grained candidate species. For each species, construct a set of synonyms S(s) containing scientific name, common name, local name, and different spellings. Fill the synonym entries into the preset prompt template to generate a text description.

[0024] Step S52: Using the image encoder and text encoder of the open-vocabulary cross-modal recognition model, feature encoding and normalization are performed on the target image region and each text description, respectively, to obtain the image feature vector v and the text feature vector v. ;

[0025] Step S53: Calculate the image feature vector v and the feature vector corresponding to each text description. The similarity score is calculated as follows: For each species, the maximum similarity score of all entries in its synonym set S(s) is aggregated to obtain the fine-grained similarity score score(s) for that species.

[0026] Step S54: Perform temperature scaling softmax normalization on the similarity scores of each species to obtain the category probability estimate and corresponding recognition confidence on the open vocabulary side;

[0027] Step S55: Construct a coarse-grained category set C based on the text tag set, and obtain the coarse-grained category probability estimate on the open vocabulary side.

[0028] Further, step S6 specifically includes:

[0029] S61: Obtain the Top-1 category labels and their confidence scores output by the closed-set object detection model, and map them onto the fine-grained candidate label set S through the alias matching projection function to obtain the closed-set evidence score;

[0030] S62: Obtain the fine-grained probability P of the open vocabulary output in step S5. fine Coarse-grained probability P coarse and the optimal category for coarse-grained particles;

[0031] S63: Construct a coarse-grained consistency gating factor based on the consistency between the coarse-grained category of the closed-set detection label and the optimal coarse-grained category of the open vocabulary. ;

[0032] S64: Using a smoothing function, adaptive weights are calculated based on the fine-grained confidence of open vocabulary and the confidence of closed set detection, and then normalized.

[0033] S65: Based on the adaptive weights, gating factors, open-vocabulary fine-grained probabilities, and closed-set evidence scores, a fusion score is constructed on the fine-grained candidate set S, and the species corresponding to the maximum fusion score is taken as the final identification result; according to the preset coarse-grained uncertainty threshold T... coarse Fine-grained uncertainty threshold T fine With closed set trusted backoff threshold T yolo When P is satisfied fine Less than T fine or Pcoarse Less than T coarse The result is marked as "uncertain" if P is satisfied. det Greater than or equal to T yolo Furthermore, the species label corresponding to the maximum fusion score can be mapped to a fine-grained species candidate in set S through aliases, and the mapped species is used as the final identification result backoff output.

[0034] This invention also provides an intelligent identification system for animal species in nature reserves that integrates closed-set target detection and open-vocabulary recognition, for implementing the above-mentioned method, comprising:

[0035] The data acquisition and preprocessing module is used to acquire monitoring data and perform preprocessing.

[0036] The closed-set target detection module is used to perform target detection on the preprocessed image and output the target candidate region, its category, confidence score, and bounding box.

[0037] The target candidate region determination module is used to determine the confidence level of the target based on the confidence level, and to trigger the open vocabulary recognition process when the confidence level is lower than a preset threshold or no effective region is detected.

[0038] The prompt-driven segmentation and target cropping module is used to segment and crop the target candidate region based on geometric prompt boxes or semantic text prompts to obtain the target image region;

[0039] An open vocabulary recognition module is used to perform cross-modal matching between the target image region and a preset set of text labels to obtain open vocabulary recognition results and confidence levels.

[0040] The confidence fusion and uncertainty assessment module is used to perform collaborative modeling and confidence fusion of closed-set detection results and open-vocabulary recognition results, output the final recognition result and perform uncertainty discrimination.

[0041] The recognition result output module is used to output the final recognition result, uncertainty markers, and related confidence information.

[0042] Furthermore, the prompt-driven segmentation and target cropping module is specifically used for:

[0043] When the closed-set target detection module outputs a valid bounding box, geometric cue information is constructed based on the bounding box to drive the segmentation model to obtain a fine target mask and perform cropping.

[0044] When the closed-set object detection module does not output a valid bounding box or the confidence level is too low, the segmentation model is driven by semantic text prompts to perform open segmentation of the image to obtain the target mask.

[0045] The prompt-driven segmentation and target cropping module also includes a quality evaluation unit, which evaluates the quality of the segmentation mask and adaptively reverts to using the original image or cropping results based on the detection box when the quality is lower than a preset threshold.

[0046] Furthermore, the open vocabulary recognition module includes:

[0047] The text tag configuration unit is used to build and manage a set of open vocabulary candidate text tags containing coarse-grained and fine-grained tags, and to build a set of synonyms for each tag;

[0048] The cross-modal matching unit uses a vision-language model to map target image regions and text labels to a unified semantic space and calculates image-text similarity.

[0049] The synonym aggregation unit is used to aggregate the similarity scores of different synonym entries for the same species;

[0050] The probability normalization unit is used to perform temperature-scaled softmax processing on the similarity score to obtain the open vocabulary recognition confidence score.

[0051] Furthermore, the confidence fusion and uncertainty assessment module is specifically used for:

[0052] Project closed-set detection evidence into a fine-grained label space for open vocabulary;

[0053] Calculate the coarse-grained and fine-grained confidence scores on the open vocabulary side;

[0054] A gating factor is constructed based on the consistency between the closed-set detection labels and the coarse-grained predictions of the open vocabulary;

[0055] Based on the smoothing function and confidence value, the fusion weight of the closed set detection path and the open vocabulary recognition path is adaptively calculated;

[0056] The final identification result is determined by calculating the fusion score based on the weights, gating factors, and evidence from both parties.

[0057] Uncertainty is marked on low-confidence fusion results according to preset multi-level thresholds, and a backoff mechanism based on high-confidence closed set detection results is triggered when the conditions are met.

[0058] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the above-described method.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] 1. Balancing the accuracy of known species identification with the ability to identify unseen species. This invention introduces a collaborative mechanism of closed-set target detection and open-vocabulary cross-modal recognition. While ensuring the accuracy of species identification within the training category, it effectively compensates for the shortcomings of traditional closed-set detection models in identifying unseen and rare species, achieving unified identification of known and unseen species and significantly improving the coverage capability of nature reserve monitoring tasks.

[0061] 2. By introducing a multi-source confidence fusion mechanism, the reliability of the recognition results is improved. This invention performs collaborative modeling and confidence fusion of closed-set target detection results and open-vocabulary recognition results. The weights of different recognition paths are adaptively adjusted according to the detection reliability, so that the system can effectively reduce the risk of false detection and false negative detection in complex natural environments and improve the stability and credibility of the overall recognition results.

[0062] 3. Explicitly discriminate the uncertainty of identification results, enhancing the interpretability of the system. By setting up an uncertainty discrimination mechanism, this invention can label and output identification results with low confidence or ambiguity, avoiding forced classification of unreliable results. This provides a basis for subsequent manual verification, new species confirmation, and sample accumulation, enhancing the interpretability and practicality of the system in long-term monitoring of nature reserves.

[0063] 4. Robust recognition capability applicable to complex natural environments: Through prompt-driven segmentation and adaptive pruning mechanism, this invention can still obtain stable target area input in typical monitoring scenarios of nature reserves, such as small target scale, complex background, and severe occlusion, thereby improving the robustness of the open vocabulary recognition stage and enhancing the system's adaptability in real application environments.

[0064] 5. Possesses good versatility and scalability. The technical solution proposed in this invention does not depend on any specific animal species or application scenario. Without changing the overall technical process and system architecture, it can be extended to other target recognition fields by changing the training weights of the closed-set target detection model and adjusting the text label configuration in the open-vocabulary recognition module, thus possessing good versatility and cross-domain application value.

[0065] 6. Applicable to long-term continuous monitoring tasks in nature reserves. This invention can identify and flag potential new species or abnormal targets during long-term, continuous monitoring, providing continuous and reliable technical support for ecological monitoring, biodiversity assessment, and scientific research in nature reserves, and helping to improve the level of intelligent management of nature reserves. Attached Figure Description

[0066] Figure 1 This is a schematic diagram illustrating the implementation principle of the intelligent identification system for animal species in nature reserves provided in an embodiment of the present invention.

[0067] Figure 2 This is a schematic diagram illustrating mask extraction based on the detection box prompt generated by the closed set target detection model in an embodiment of the present invention.

[0068] Figure 3 This is a schematic diagram of mask extraction based on semantic text prompts in an embodiment of the present invention.

[0069] Figure 4 This is a schematic diagram of the mask quality assessment and adaptive backoff strategy in an embodiment of the present invention.

[0070] Figure 5 This is a schematic diagram of cross-modal matching performed by the open vocabulary recognition module in an embodiment of the present invention.

[0071] Figure 6 This is a schematic diagram illustrating the collaborative modeling and gating fusion of the confidence fusion and uncertainty assessment modules in an embodiment of the present invention. Detailed Implementation

[0072] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0073] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0074] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0075] This invention provides an intelligent species identification method and system for nature reserves that integrates closed-set target detection and open-vocabulary recognition. This invention leverages the advantages of both closed-set target detection and open-vocabulary cross-modal recognition models to effectively identify targets in nature reserve monitoring images. While ensuring the accuracy of known species identification, it enhances the ability to identify unseen and rare species, thus providing more intelligent and reliable species identification support for long-term nature reserve monitoring tasks. Furthermore, this invention integrates the intelligent species identification method into an intelligent monitoring system for nature reserves. By uniformly processing image data collected by monitoring equipment, it achieves compatibility and adaptation with different monitoring terminals and application environments. Through collaborative modeling and confidence fusion of closed-set detection results and open-vocabulary recognition results, the system can effectively discern uncertainties in identification results under complex natural environments, improving the stability and robustness of the overall identification process. Furthermore, the intelligent animal species identification method proposed in this invention is not limited to specific species categories or application scenarios. The core technical framework of this invention has good versatility. Without changing the overall identification process, it can be extended to other target identification scenarios by changing the training weights of the closed-set target detection model and adjusting the text label configuration in the open vocabulary recognition module.

[0076] The intelligent identification method for animal species in nature reserves that integrates closed set target detection and open vocabulary recognition provided in this embodiment has the following implementation steps.

[0077] Step S1: Acquire image data collected by the monitoring equipment of nature reserves, and preprocess the image data, including image size normalization, noise suppression and format conversion.

[0078] Step S2: Use a closed-set object detection model (such as YOLO or DETR) to perform object detection on the preprocessed image data, obtain at least one object candidate region, and output the corresponding object category and object confidence P. det And the target bounding box.

[0079] Step S3: Determine the target candidate region based on the target confidence level. When the target confidence level P... det Reaching or exceeding a preset threshold T det When the target confidence level P is high, the corresponding candidate region is determined as a credible target; when the target confidence level P is high, the corresponding candidate region is determined as a credible target. det Below T det If the closed-set target detection model fails to detect the target, the open vocabulary recognition process is triggered.

[0080] Step S4: Perform cue-driven segmentation and target cropping on the target candidate region to obtain the target image region for cross-modal recognition, wherein the cue information includes detection box cue and / or semantic text cue.

[0081] Step S5: Based on the open vocabulary cross-modal recognition model CLIP, the target image region is matched with a preset set of text labels to obtain the target recognition result and the corresponding recognition confidence under the open vocabulary meaning.

[0082] Step S6: Perform collaborative modeling and confidence fusion on the closed-set target detection results and the open-vocabulary recognition results. Output the final target recognition result based on the fusion result, and perform uncertainty labeling or backoff processing on targets whose fusion confidence fails to meet the conditions.

[0083] In step S4, the prompt-driven segmentation and target cropping specifically include the following sub-steps:

[0084] Step S41: Based on the target bounding box information output by the closed-set target detection model, construct geometric cue information to indicate the approximate location of the target in the image; the bounding box information is based on the detection box output by the closed-set target model, and the pixel coordinates of its upper left and lower right corners. Get the center coordinates and width and height of the bounding box:

[0085]

[0086] Let the width and height of the original image be W and H, then the normalized geometric tooltip can be obtained:

[0087]

[0088] Step S42: Input the geometric cue information b and the original image into SAM3 (cue-driven segmentation model) to perform fine segmentation of the target candidate region and output a set of candidate masks and score vectors. Select the mask with the highest score. That is, a mask. It can be represented as:

[0089]

[0090] As the final target segmentation mask.

[0091] Step S43: Based on the target segmentation mask, crop the original image to generate a target image patch containing only the target region, so as to reduce the interference of background information on the subsequent recognition process.

[0092] Step S44: When closed-set target detection does not output a valid target bounding box or the target confidence is lower than T det At that time, semantic text-based prompts are constructed, and the original image (IMAGE) and the text prompts are input together into SAM3 to obtain the animal mask. That is, the mask M... *It can be represented as:

[0093]

[0094] Open segmentation is performed on the entire image to avoid missing potential targets.

[0095] Step S45: Define an adaptive pruning mechanism , indicating the use of a mask In the original image The target image patch with a transparent background is cropped out from the top. That is, the output target image patch is:

[0096]

[0097] The adaptive pruning mechanism constructs a mask quality evaluation function q(M) based on the mask score output by SAM3 and indicators such as mask area and connectivity. * The segmentation results are evaluated for quality, and a quality threshold T is set. mask When the segmentation result satisfies q(M) * )≥T mask When the segmentation result is satisfied, the cropping result based on the segmentation mask is used as the target image region; when the segmentation result satisfies q(M * ) <T mask When necessary, the original image or the cropping result based on the detection box is used as the target image region.

[0098] Through the aforementioned prompt-driven segmentation and adaptive cropping mechanism, the system can still obtain stable and reliable target image regions in natural protected area monitoring scenarios with complex backgrounds, severe occlusion, or small target scales, providing high-quality input for subsequent open vocabulary recognition.

[0099] Step S5 specifically includes the following sub-steps:

[0100] Step S51: Constructing the candidate text label set and synonym set by building an open vocabulary candidate label set. ,in Let represent the k-th fine-grained candidate species. For each species s, construct its set of synonyms / aliases. The set elements include scientific names, common names, local names, and different spellings. For each synonym entry... Fill it into the preset prompt template to generate a text description. (e.g., “a photo of a {t}”) to enhance the model’s robustness to multi-source naming and diverse expressions.

[0101] Step S52: Image and text feature encoding and normalization. Let the target image region output in step S4 be x (preferred). Let the CLIP image encoder be... The text encoder is The normalized image feature vector and text feature vector are obtained respectively:

[0102]

[0103] Among them l k This is a prompt for the k-th species or category.

[0104] Step S53: Image-text similarity calculation and synonym aggregation. Define the image-text cosine similarity for any text cue l:

[0105]

[0106] For a given species s, its set of synonyms S(s) will generate a set of similarity scores. The fine-grained similarity score for this species is obtained by using maximum aggregation:

[0107]

[0108] This synonym aggregation strategy is used to improve robustness to "professional naming / multi-source tags / different notations".

[0109] Step S54: Temperature scaling, softmax normalization, and open vocabulary confidence are applied to construct logits (denoted as z) from the matching scores of each candidate species. k During project implementation, it can be directly ordered... (Or obtained from CLIP forward output), and the category probability estimate (confidence) of the open vocabulary side is obtained by temperature-scaled softmax:

[0110]

[0111] in, Temperature is a parameter used to adjust the "sharpness" of the distribution. This leads to the Top-1 prediction for open vocabulary:

[0112]

[0113] Step S55: Coarse-grained prediction construction. To support "coarse-grained consistency gating" for closed and open vocabularies, a coarse-grained category set C is defined, and coarse-grained probabilities for open vocabularies are constructed. Coarse-grained processing can be achieved by directly configuring the coarse-grained hint set. Repeat steps S52-S54, or adjust the coarse-grained... According to preset mapping The results were obtained through aggregation. The final result was:

[0114]

[0115] This coarse-grained prediction will be used in step S6 to suppress the propagation of semantic conflicts.

[0116] The collaborative modeling and confidence fusion in step S6 specifically includes the following sub-steps:

[0117] Step S61: Project the closed-set evidence representation with the alias, and output the Top-1 class label through the closed-set target detection model (Step S2). and its confidence level To enable unified comparisons on the fine-grained candidate set S, an alias set is introduced. (The definition of the synonym set can be consistent with or mutually mapped from S51), define the closed-set evidence projection function:

[0118]

[0119] This projection allows closed-set evidence to participate in fusion as an "additivity score".

[0120] Step S62: Obtain fine-grained / coarse-grained confidence scores for open vocabulary, directly using the output of step S5: fine-grained probability. and coarse-grained probability And obtain the coarse-grained optimal category. and .

[0121] Step S63: Construct coarse-grained consistency gating factor, and label the closed set. Its coarse-grained category is obtained through a preset mapping. Constructing the basic gating factor:

[0122]

[0123] Furthermore, to avoid excessive suppression of highly reliable closed-set evidence due to "high confidence but bias in open-vocabulary coarse-classification," a protection term is introduced to achieve actual gating:

[0124]

[0125] when When the value is extremely high, increase the weight range of closed-set evidence and force g. coarse =1 to protect the closed set output; when When a coarse class in the middle-high range and an open-vocabulary class gives an extremely high confidence objection to another coarse class, restore the standard weight range and make the gating fall back to g. baseIt allows open lexicalities to correct misjudgments of closed sets; in other cases, standard balanced fusion is used.

[0126] This "suppression + protection" gating mechanism can suppress low-confidence closed-set evidence in the event of coarse-class conflicts, while preserving the dominance of high-confidence closed-set results.

[0127] Step S64: Smoothstep adaptive weight calculation and normalization. To avoid the failure of fixed weights in different confidence intervals, a smoothstep smoothing gate is used to construct adaptive weights. Let the smoothing function be... Satisfy within the interval The system exhibits monotonically increasing internally and saturates externally. A cubic smoothstep approach is used to calculate the original weights for both fine-grained open-vocabulary scores and closed-set confidence scores.

[0128]

[0129]

[0130] The final weights are then obtained by normalization:

[0131]

[0132] in, , , , These are configurable parameters.

[0133] Step S65: Constructing the fusion score, Top-1 decision and uncertainty discrimination / backoff, by constructing the fusion score on the fine-grained candidate set S:

[0134]

[0135] The species with the highest score is selected as the final candidate output.

[0136]

[0137] Set the coarse-grained uncertainty threshold T coarse Fine-grained uncertainty threshold T fine With closed set trusted backoff threshold T yolo When the following conditions are met:

[0138]

[0139] The result is then labeled as "Uncertain target / suspected unseen species". Under uncertain conditions, if the closed set evidence satisfies... Furthermore, if the closed set label can be mapped to a certain fine-grained candidate s′ through aliases, then the mapped species will be output as the fallback result.

[0140] This embodiment also provides a corresponding intelligent species identification system for nature reserves that integrates closed-set target detection and open-vocabulary recognition. This system is used for automated species identification of image data collected by monitoring equipment in nature reserves. Its implementation principle is as follows: Figure 1 As shown, the system includes, but is not limited to, the following functional modules: data acquisition and preprocessing module; closed-set target detection module; target candidate region determination module; cue-driven segmentation and target cropping module; open vocabulary recognition module; confidence fusion and uncertainty assessment module; and recognition result output module. These functional modules work collaboratively through data flow and control flow to achieve unified recognition of known and unseen species.

[0141] The data acquisition and preprocessing module is used to acquire image or video data collected by nature reserve monitoring equipment and to preprocess the data. The preprocessing includes, but is not limited to, at least one of image size normalization, format conversion, noise suppression, and brightness adjustment, to improve the stability of the subsequent recognition process.

[0142] The closed-set target detection module is used to perform target detection on the preprocessed image data and output at least one target candidate region. For each target candidate region, the closed-set target detection module outputs the corresponding target category, target confidence score, target bounding box, and target center point information to characterize the detection result of the target in the closed-set category space.

[0143] The target candidate region determination module is used to determine the reliability of target candidate regions based on the target confidence level output by the closed-set target detection module. When the target confidence level is higher than a preset threshold, the target candidate region determination module determines the target as a reliable candidate target; when the target confidence level is lower than the preset threshold or no valid target candidate region is detected, the module triggers the open vocabulary recognition process.

[0144] The prompt-driven segmentation and target cropping module, as follows: Figure 2 and Figure 3 As shown, this module is used for fine segmentation and cropping of target candidate regions. Specifically, it constructs geometric cue information based on the target bounding boxes output by the closed-set target detection module, and inputs the geometric cue information and the original image into the cue-driven segmentation model to obtain a target segmentation mask. If the closed-set target detection module does not output valid target bounding boxes, this module performs open segmentation of the image based on semantic text cue to avoid missing potential targets. Furthermore, this module also performs a quality assessment of the segmentation results and, if the segmentation results do not meet preset conditions, such as... Figure 4 As shown, adaptive regression uses the original image or the cropping result based on the detection box as the target image region.

[0145] The open vocabulary recognition module is used to perform cross-modal recognition on the cropped target image region. Specifically, such as... Figure 5 As shown, the open vocabulary recognition module maps the target image region to a preset set of text labels into a unified semantic embedding space, calculates the similarity between image features and text features, and thus obtains the target recognition result and corresponding recognition confidence under the meaning of open vocabulary. The set of text labels includes coarse-grained category labels and / or fine-grained species labels, and can be configured and adjusted according to the application scenario.

[0146] The confidence fusion and uncertainty assessment module, as follows: Figure 6 As shown, this module performs collaborative modeling and fusion processing on the target confidence scores output by the closed-set target detection module and the recognition confidence scores output by the open-vocabulary recognition module. This module adaptively adjusts the weights of different confidence scores in the fusion process based on the magnitude of the closed-set detection confidence score, and generates the final recognition confidence score of the target based on the fusion result. When the fusion confidence score is higher than a preset threshold, the module outputs the corresponding category as the final recognition result; when the fusion confidence score is lower than the preset threshold, the target is marked as an uncertain target or a suspected unseen species.

[0147] The identification result output module is used to output species identification results, uncertainty marker information and related confidence information, and provide the results to the upper-level nature reserve intelligent monitoring system or management platform for subsequent analysis, display or manual review.

[0148] System scalability description: Without changing the overall system structure and functional module division, by changing the training weights of the closed set target detection module and adjusting the text label configuration in the open vocabulary recognition module, the system described in this invention can be extended to other target recognition scenarios, including but not limited to urban street scene recognition, vehicle recognition and pedestrian analysis.

[0149] This invention addresses the practical problems of diverse animal species, imbalanced samples, and frequent occurrences of unseen species in complex monitoring scenarios of nature reserves. It proposes an intelligent recognition method and system that integrates closed-set target detection and open-vocabulary recognition. Key technologies include, but are not limited to, the following:

[0150] (1) Synergistic fusion mechanism of closed set object detection and open vocabulary recognition

[0151] This invention introduces an open-vocabulary cross-modal recognition mechanism based on the closed-set target detection results, and performs unified modeling and fusion processing on the two types of recognition results. While ensuring the species recognition accuracy within the training category, it significantly improves the system's ability to recognize unseen and rare species, and realizes effective synergy between closed-set recognition and open recognition in the monitoring scenario of nature reserves.

[0152] (2) Confidence-based multi-source identification result fusion and uncertainty discrimination mechanism

[0153] This invention improves the robustness and interpretability of the system in complex natural environments by jointly modeling the confidence scores of closed-set target detection and open-vocabulary recognition, and by introducing an adaptive weight allocation and threshold determination mechanism to explicitly evaluate and label the uncertainty of the recognition results. This avoids misjudgments or missed judgments caused by the failure of a single recognition path.

[0154] (3) A general target identification technology framework for monitoring nature reserves

[0155] The technical solution proposed in this invention is not limited to specific species categories or application scenarios. Its core lies in constructing a general target recognition technology framework of "closed-set target detection + open-vocabulary recognition + confidence fusion". Without changing the overall technical architecture, the method and system of this invention can be extended to other target recognition fields by changing the training weights of the closed-set target detection model and adjusting the text label configuration in the open-vocabulary recognition module.

[0156] The other target recognition fields include, but are not limited to, urban street scene target recognition, vehicle recognition, pedestrian analysis, and other target recognition scenarios based on visual perception, thereby giving the present invention good versatility, scalability, and cross-domain application value.

[0157] (4) Scalable identification capability suitable for long-term continuous monitoring scenarios

[0158] This invention, through open vocabulary recognition and uncertainty labeling mechanisms, enables the system to identify and label potential new categories of targets during long-term continuous monitoring, providing a basis for subsequent manual verification, sample accumulation, and model iteration. It is applicable to long-term monitoring and dynamic assessment of biodiversity in nature reserves.

[0159] This embodiment also provides a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method.

[0160] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0161] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0162] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0163] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0164] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for intelligent identification of animal species in nature reserves that integrates closed-set target detection and open-vocabulary recognition, characterized in that, Includes the following steps: Step S1: Acquire image data collected by the monitoring equipment of the nature reserve, and preprocess the image data; Step S2: Use a closed-set target detection model to detect targets in the preprocessed image data, obtain at least one target candidate region, and output the corresponding target category and target confidence P. det and the target bounding box; Step S3: Determine the target candidate region based on the target confidence level. When the target confidence level P... det Reaching or exceeding a preset threshold T det When the target candidate region is identified, it is determined as a credible target. When the target confidence level P det Below T det If the closed-set target detection model fails to detect the target, the open-vocabulary recognition process is triggered. Step S4: Perform cue-driven segmentation and target cropping on the target candidate region to obtain the target image region for cross-modal recognition, wherein the cue information includes detection box cue and / or semantic text cue; Step S5: Based on the open vocabulary cross-modal recognition model CLIP, the target image region is matched with a preset set of text labels to obtain the target recognition result and the corresponding recognition confidence under the meaning of open vocabulary; Step S6: Perform collaborative modeling and confidence fusion on the closed-set target detection results and the open-vocabulary recognition results. Output the final target recognition result based on the fusion result, and perform uncertainty labeling or backoff processing on targets whose fusion confidence fails to meet the conditions.

2. The intelligent identification method for animal species in nature reserves that integrates closed-set target detection and open-vocabulary recognition according to claim 1, characterized in that, In step S1, the preprocessing includes image size normalization, noise suppression, and format conversion.

3. The intelligent identification method for animal species in nature reserves that integrates closed-set target detection and open vocabulary recognition according to claim 1, characterized in that, Step S4 specifically includes: Step S41: Construct a normalized geometric cue box b based on the target bounding box information output by the closed-set target detection model; Step S42: Input the geometric cue box b and the original image into the cue-driven segmentation model to perform fine segmentation of the target candidate region, and select the mask with the highest score from the output candidate masks. As a final target segmentation mask; Step S43: Based on the target segmentation mask The original image is cropped to generate a target image patch containing only the target region. Step S44: When closed-set target detection does not output a valid target bounding box or the target confidence is lower than T det At the same time, semantic textual prompts are constructed, and the original image and the textual prompts are input into the prompt-driven segmentation model to obtain the animal mask. Open segmentation is performed on the entire image; Step S45: Evaluate the segmentation results based on the mask quality evaluation function q(M) and set a quality threshold T. mask When the segmentation result satisfies q(M) * )≥T mask When the segmentation result is satisfied, the cropping result based on the segmentation mask is used as the target image region; when the segmentation result satisfies q(M * ) <T mask When necessary, the original image or the cropping result based on the detection box is used as the target image region.

4. The intelligent identification method for animal species in nature reserves that integrates closed-set target detection and open vocabulary recognition according to claim 1, characterized in that, Step S5 specifically includes: Step S51: Construct a set of open-vocabulary candidate text labels ,in Let S(s) represent the k-th fine-grained candidate species. For each species, construct a set of synonyms S(s) containing scientific name, common name, local name, and different spellings. Fill the synonym entries into the preset prompt template to generate a text description. Step S52: Using the image encoder and text encoder of the open-vocabulary cross-modal recognition model, feature encoding and normalization are performed on the target image region and each text description, respectively, to obtain the image feature vector v and the text feature vector v. ; Step S53: Calculate the image feature vector v and the feature vector corresponding to each text description. The similarity score is calculated as follows: For each species, the maximum similarity score of all entries in its synonym set S(s) is aggregated to obtain the fine-grained similarity score score(s) for that species. Step S54: Perform temperature scaling softmax normalization on the similarity scores of each species to obtain the category probability estimate and corresponding recognition confidence on the open vocabulary side; Step S55: Construct a coarse-grained category set C based on the text tag set, and obtain the coarse-grained category probability estimate on the open vocabulary side.

5. The intelligent identification method for animal species in nature reserves that integrates closed-set target detection and open vocabulary recognition according to claim 1, characterized in that, Step S6 specifically includes: S61: Obtain the Top-1 category labels and their confidence scores output by the closed-set object detection model, and map them onto the fine-grained candidate label set S through the alias matching projection function to obtain the closed-set evidence score; S62: Obtain the fine-grained probability P of the open vocabulary output in step S5. fine Coarse-grained probability P coarse and the optimal category for coarse-grained particles; S63: Construct a coarse-grained consistency gating factor based on the consistency between the coarse-grained category of the closed-set detection label and the optimal coarse-grained category of the open vocabulary. ; S64: Using a smoothing function, adaptive weights are calculated based on the fine-grained confidence of open vocabulary and the confidence of closed set detection, and then normalized. S65: Based on the adaptive weights, gating factors, open-vocabulary fine-grained probabilities, and closed-set evidence scores, construct a fusion score on the fine-grained candidate set S, and take the species corresponding to the maximum fusion score as the final identification result; according to the preset coarse-grained uncertainty threshold T... coarse Fine-grained uncertainty threshold T fine With closed set trusted backoff threshold T yolo When P is satisfied fine Less than T fine or P coarse Less than T coarse The result is marked as "uncertain" if P is satisfied. det Greater than or equal to T yolo Furthermore, the species label corresponding to the maximum fusion score can be mapped to a fine-grained species candidate in set S through aliases, and the mapped species is used as the final identification result backoff output.

6. A smart identification system for animal species in nature reserves that integrates closed-set target detection and open-vocabulary recognition, used to implement the method described in any one of claims 1-5, characterized in that, include: The data acquisition and preprocessing module is used to acquire monitoring data and perform preprocessing. The closed-set target detection module is used to perform target detection on the preprocessed image and output the target candidate region, its category, confidence score, and bounding box. The target candidate region determination module is used to determine the confidence level of the target based on the confidence level, and to trigger the open vocabulary recognition process when the confidence level is lower than a preset threshold or no effective region is detected. The prompt-driven segmentation and target cropping module is used to segment and crop the target candidate region based on geometric prompt boxes or semantic text prompts to obtain the target image region; An open vocabulary recognition module is used to perform cross-modal matching between the target image region and a preset set of text labels to obtain open vocabulary recognition results and confidence levels. The confidence fusion and uncertainty assessment module is used to perform collaborative modeling and confidence fusion of closed-set detection results and open-vocabulary recognition results, output the final recognition result and perform uncertainty discrimination. The recognition result output module is used to output the final recognition result, uncertainty markers, and related confidence information.

7. The intelligent identification system for animal species in nature reserves that integrates closed-set target detection and open vocabulary recognition according to claim 6, characterized in that, The prompt-driven segmentation and target cropping module is specifically used for: When the closed-set target detection module outputs a valid bounding box, geometric cue information is constructed based on the bounding box to drive the segmentation model to obtain a fine target mask and perform cropping. When the closed-set object detection module does not output a valid bounding box or the confidence level is too low, the segmentation model is driven by semantic text prompts to perform open segmentation of the image to obtain the target mask. The prompt-driven segmentation and target cropping module also includes a quality evaluation unit, which evaluates the quality of the segmentation mask and adaptively reverts to using the original image or cropping results based on the detection box when the quality is lower than a preset threshold.

8. The intelligent identification system for animal species in nature reserves that integrates closed-set target detection and open vocabulary recognition according to claim 6, characterized in that, The open vocabulary recognition module includes: The text tag configuration unit is used to build and manage a set of open vocabulary candidate text tags containing coarse-grained and fine-grained tags, and to build a set of synonyms for each tag; The cross-modal matching unit uses a vision-language model to map target image regions and text labels to a unified semantic space and calculates image-text similarity. The synonym aggregation unit is used to aggregate the similarity scores of different synonym entries for the same species; The probability normalization unit is used to perform temperature-scaled softmax processing on the similarity score to obtain the open vocabulary recognition confidence score.

9. The intelligent identification system for animal species in nature reserves that integrates closed-set target detection and open vocabulary recognition according to claim 6, characterized in that, The confidence fusion and uncertainty assessment module is specifically used for: Project closed-set detection evidence into a fine-grained label space for open vocabulary; Calculate the coarse-grained and fine-grained confidence scores on the open vocabulary side; A gating factor is constructed based on the consistency between the closed-set detection labels and the coarse-grained predictions of the open vocabulary; Based on the smoothing function and confidence value, the fusion weight of the closed set detection path and the open vocabulary recognition path is adaptively calculated; The final identification result is determined by calculating the fusion score based on the weights, gating factors, and evidence from both parties. Uncertainty is marked on low-confidence fusion results according to preset multi-level thresholds, and a backoff mechanism based on high-confidence closed set detection results is triggered when the conditions are met.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by a processor, the method described in any one of claims 1-5 is implemented.

Images