Hierarchical Classification Method for Wildlife Monitoring Images Based on Species Classification Tree
Through a hierarchical classification method based on species classification tree, the hierarchy structure and soft decision strategy of the image classification model are solved, and more efficient species classification and accuracy are achieved.
Patent Information
- Application Number
- CN202411768368.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-07-25
- Estimated Expiration
- 2044-12-04
AI Technical Summary
The lack of interpretability of existing wildlife monitoring image classification models, which leads to high cost for ecologists when reviewing model results, and the output information of the model is insufficient to confirm species, which increases the workload of manual review.
Using a hierarchical classification method based on species classification trees, we use image classification models to construct an image classification model and introduce hierarchical structures and soft decision strategies among categories to provide multi-level classification results. Using the hierarchical relationships and path correction strategies of classification trees, we gradually refine species classification and weaken the impact of early errors.
It improves the usability and interpretability of image classification models, reduces the workload of ecologists to review model results, improves classification accuracy and consistency, and reduces labor costs.
Smart Images
Figure CN119723171B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of hierarchical species classification, and specifically to a hierarchical classification method for wildlife monitoring images based on a species classification tree. Background Art
[0002] The scale of wildlife resource surveys and biodiversity monitoring is constantly expanding, and the number of monitoring images is increasing day by day. In recent years, wildlife surveys in China, especially those targeting terrestrial vertebrates, have achieved rapid development in terms of survey scale (geographical area, animal group coverage) and survey methods. Technologies such as infrared-triggered cameras provide effective means for data collection, and wildlife resource surveys and monitoring carried out by various national parks, nature reserves at all levels, and forestry departments provide rich and diverse data sources.
[0003] In the classification of wildlife monitoring images, the cost of model result review is high. Ecologists need to review the model results to ensure accurate and rigorous downstream ecological research. Since the model does not provide more information beneficial to species confirmation except for the specific categories output, for misclassified samples, the reviewers need to reassign labels, which is tantamount to relabeling.
[0004] In addition, existing species classification models usually lack interpretability, which also raises concerns among ecologists about the reliability of the models and brings additional manual review costs. The research on this issue in existing wildlife monitoring image classification methods is relatively limited. Therefore, how to make the model provide more information beneficial to species classification and reduce the workload of ecological experts in reviewing model results is an issue to be solved. In view of this, the present invention proposes a hierarchical classification method for wildlife monitoring images based on a species classification tree. Summary of the Invention
[0005] (I) Technical Problems to be Solved
[0006] The purpose of the present invention is to propose a hierarchical classification method for wildlife monitoring images based on a species classification tree to solve the problems raised in the background art. The present invention can provide more information helpful for species judgment and improve the usability and interpretability of the image classification model.
[0007] (II) Technical Solutions
[0008] To achieve the above object, the present invention proposes a hierarchical classification method for wildlife monitoring images based on a species classification tree, including the following steps:
[0009] S1. Continuously monitor to obtain wildlife monitoring image data, form a monitoring image data set, and construct an image classification model for classifying the obtained wildlife monitoring image data;
[0010] S2. Thoughts on constructing an image classification model based on a species classification tree. Input image samples into the model and output classification results at multiple levels of the corresponding species classification tree, specifically including the following:
[0011] S2.1. Design text prompts according to the subordinate relationship between categories, so that the image classification model can use the hierarchical structure between categories for classification;
[0012] S2.2. Based on the top-down hierarchical classification of the species classification tree, along the path from the root node to the leaf node of the classification tree, refine the classification of species;
[0013] S2.3. Design a correction strategy for the classification path, use the classification probability of soft decision, and use the probability score of the previous level to correct the logits score of the current level, weakening the influence of the decision result of the previous level;
[0014] S3. Based on the classification strategy designed in S2, use the monitoring image dataset constructed in S1 to train the image classification model, and complete the hierarchical classification of wildlife monitoring images through the stable model obtained by training.
[0015] Preferably, the specific content of S2.1 includes the following:
[0016] Carry out prompt engineering on the basis of the text prompts for single-level classification, add descriptions of adjacent classification levels to the text of each level to improve the matching degree between the image and the text of each level; the addition of category names at adjacent levels establishes the connection between different-level categories, so that the image classification model can use the hierarchical structure between categories for classification.
[0017] Preferably, the specific content of S2.2 includes the following:
[0018] According to the structure of the classification tree, adopt a layer-by-layer classification strategy from coarse to fine to refine the species classification. The category granularity is from coarse to fine, and the semantics is from abstract to specific. Use the classification results of the coarse-grained level to exclude incorrect options in the fine-grained categories;
[0019] Allow the image classification model to gradually understand the features in the image, start from more general categories, and then gradually refine to more specific categories, so as to improve the classification accuracy and generate classification results containing multi-level information.
[0020] Preferably, the calculation formula for the classification probability of soft decision in S2.3 is:
[0021] score l = softmax(map(score l-1 ,l)·logits l / τ)
[0022] Among them, l represents the current classification level; score0 = 1 N ; logits l represents a vector of the l-th layer category similarity scores after the masking operation, with a dimension of 1×N; map(·) is used to map the scores score of the upper level l-1 to the positions of the sub-categories of the l-th layer and multiply them correspondingly with the logits scores of this layer; τ represents an independent coefficient, which is used to control the influence degree of the classification probability of the upper level on the classification of the current level; when τ is large, score is uniformly distributed, and each level makes independent decisions, and the upper-level classification result has no influence on subsequent classifications; when τ is small, the score of each layer is close to the one-hot vector, changing the classification range of the next level, and the soft decision degenerates into a top-down hierarchical classification.
[0023] (III) Beneficial effects
[0024] Compared with the prior art, the present invention provides a hierarchical classification method for wildlife monitoring images based on a species classification tree, having the following beneficial effects:
[0025] Aiming at the problem of the high cost in the result review process of the image classification model in the prior art, the present invention proposes a species zero-shot hierarchical classification method for wildlife monitoring images based on a classification tree, enabling the model to provide more information helpful for species judgment, thereby improving the usability and interpretability of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The present invention is described with the aid of the following drawings:
[0027] Figure 1 is a flowchart of the hierarchical classification method for wildlife monitoring images based on a species classification tree proposed by the present invention;
[0028] Figure 2 is the hierarchical structure of the dataset labels of the hierarchical classification method for wildlife monitoring images based on a species classification tree proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0029] In order to better understand the above technical solutions, the exemplary embodiments of the present invention will be described in more detail below with reference to the drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and the scope of the present invention can be fully conveyed to those skilled in the art.
[0030] The hierarchical classification method of wildlife monitoring images based on the species classification tree proposed according to the present invention will be described below with reference to the accompanying drawings.
[0031] Embodiment 1:
[0032] The present invention provides a hierarchical classification method of wildlife monitoring images based on the species classification tree, which classifies at five classification levels of class, order, family, genus and species in the species classification tree, and reduces the labor cost of reviewing the model results by providing richer species determination information. This method utilizes the hierarchical relationship between categories, introduces soft decision-making and path correction strategies, and improves the accuracy and consistency of hierarchical classification. When the classification is incorrect at the fine-grained species level, the coarse-grained results provided by the model are also of reference value.
[0033] Please refer to Figure 1 , Figure 1 , which is a schematic flowchart of the hierarchical classification method of wildlife monitoring images based on the species classification tree according to an embodiment of the present invention.
[0034] As Figure 1 shown, the hierarchical classification method of wildlife monitoring images includes the following steps:
[0035] Taking the white-tailed deer image in Figure 1 as an example, when the model gives an incorrect prediction result of black-tailed deer at the species level, the labeling personnel can refer to the results at the family level and genus level to quickly confirm the correct category as white-tailed deer from similar species.
[0036] First, for each input sample, the model gives classification results at multiple levels of the corresponding species classification tree, uses the species classification tree to establish the thinking path of the model, obtains multi-level labels by matching the image with text prompts containing hierarchical information from top to bottom according to the species classification tree levels, explicitly expands the open vocabulary model to hierarchical classification, and introduces soft decision-making and path correction strategies to improve the accuracy and consistency of the multi-level labels. The multi-level classification results provide more information conducive to species confirmation, thereby reducing the labor cost of expert review and label query. When the classification is incorrect at the fine-grained species level, the coarse-grained results provided by the model can also be used as a reference, and by enhancing the interpretability of the species classification model, the trust level and willingness to use of ecologists for deep learning models are improved.
[0037] Secondly, design text prompts according to the subordinate relationship between categories. Since hierarchical classification can be regarded as a combination of multiple single-level classification tasks, methods to improve single-level classification performance can also be incorporated into hierarchical classification methods. Based on the text prompts for single-level classification, conduct prompt engineering by adding descriptions of adjacent classification levels to the text of each level to improve the matching degree between the image and the text of each level. The addition of adjacent-level category names establishes the connection between different-level categories, enabling the model to use the hierarchical structure between categories for classification.
[0038] Then, for the top-down hierarchical classification based on the species classification tree, along the path from the root node to the leaf nodes of the classification tree, the classification of species gradually becomes more refined, the category granularity changes from coarse to fine, and the semantics change from abstract to specific. According to the structure of the classification tree, adopt a layer-by-layer classification strategy from coarse to fine, using the classification results of the coarse-grained levels to exclude incorrect options in the fine-grained categories, thereby improving the accuracy of species classification, allowing the model to gradually understand the features in the image, starting from more general categories and then gradually refining to more specific categories, so as to improve the classification accuracy and generate a more interpretable classification result containing multi-level information.
[0039] Finally, for the correction strategy of the classification path, the top-down hierarchical classification pre-excludes incorrect options at deeper levels in the classification tree, gradually narrowing the label space. However, whether the excluded categories are reasonable depends on the correctness of the current classification result. Therefore, early classification errors will accumulate to deeper levels, resulting in the accuracy of the next level being impossible to be higher than the previous level. As the levels deepen, the exponential accumulation of classification errors severely limits the accuracy of deep-level classification. Use the classification probability of soft decision-making to correct the logits score of the current level using the probability score of the previous level, retain but weaken the influence of the decision result of the previous level, avoid directly narrowing the classification scope, and give deeper-level decisions the opportunity to correct early errors. The formula for the classification probability of soft decision-making is:
[0040] score l = softmax(map(score l-1 , l)·logits l / τ)
[0041] where l represents the current classification level, score0 = 1 N , logits l represents the vector of the l-th layer category similarity scores after masking operation, with a dimension of 1×N. map(·) is used to map the score of the previous level score l-1It is mapped to the positions of the sub-categories at the l-th layer, and thus multiplied correspondingly with the logits scores of this layer. The temperature τ is called the independence coefficient here, which is used to control the influence degree of the classification probability of the previous level on the classification of the current level. When the independence coefficient is extremely large, the score is uniformly distributed, and each level makes independent decisions, and the classification result of the upper level has no influence on the subsequent classification; when the independence coefficient is very small, the score of each layer is close to a one-hot vector, which will directly change the classification range of the next level, and the soft decision degenerates into a top-down hierarchical classification.
[0042] Example 2:
[0043] Based on Example 1 but with differences, this method classifies the monitoring images at five levels: class, order, family, genus, and species. Please refer to Figure 2 , introduce text prompts containing hierarchical relationships between categories and perform top-down layer-by-layer matching according to the species classification tree. At the same time, a soft decision and path correction strategy are proposed to enable the model to recover classification errors of previous levels in deeper-level classification and maintain the consistency of multi-level results. Thus, the zero-shot classification of the open-vocabulary model is extended to hierarchical classification. Design a comparative experiment to compare the performance of the method proposed in the present invention with that of the multi-level classification baseline. The results are as follows.
[0044] Table 1 Performance comparison between this method and the multi-level classification baseline
[0045]
[0046] Table 2 Ablation of the classification levels and independence coefficients of this method
[0047]
[0048] The results show that compared with the baseline method enhanced by the path correction strategy, this method improves the accuracy by 0.27%, 1.44%, 1.42%, 1.39%, and 1.03% respectively at the five classification levels from class to genus. In addition, it also reduces the severity of errors by 1.1%. The ablation experiment explores the influence of the number of levels in hierarchical classification and the independence degree between classifications at each level on the method performance. Reasonably reducing the classification levels and constraining the influence degree between levels helps to further improve the hierarchical classification performance.
[0049] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.
[0050] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions.
[0051] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of other elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the claims listing several means, several of these means can be embodied by the same piece of hardware. The use of the words first, second, third, etc. is only for convenience of expression and does not denote any order. These words can be understood as part of the name of the element.
[0052] In addition, it should be noted that in the description of this specification, the descriptions of terms such as "an embodiment", "some embodiments", "embodiments", "examples", "specific examples" or "some examples" mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.
[0053] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications after learning the basic creative concept. Therefore, the claims should be construed to include the preferred embodiments and all changes and modifications falling within the scope of the present invention.
[0054] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention should also cover these modifications and variations.
Claims
1. A hierarchical classification method for wildlife monitoring images based on a species classification tree, characterized in that It includes the following steps: S1. Continuously monitor and obtain wildlife monitoring image data, form a monitoring image dataset, and build an image classification model for classifying the obtained wildlife monitoring image data; S2. Based on the thinking path of the image classification model constructed from the species classification tree, input image samples into the model and output classification results at multiple levels of the corresponding species classification tree. Specifically, it includes the following content: S2.
1. Design text prompts according to the subordinate relationship between categories, so that the image classification model can use the hierarchical structure between categories for classification; S2.
2. Based on the top-down hierarchical classification of the species classification tree, along the path from the root node to the leaf node of the classification tree, refine the classification of species; S2.
3. Design a correction strategy for the classification path, use the classification probability of soft decision-making, and use the probability score of the previous level to correct the logits score of the current level, weakening the influence of the decision result of the previous level; The calculation formula for the classification probability of the soft decision-making is: score l = softmax(map(score l-1 , l) · logits l / τ) where l represents the current classification level; score0 = 1 N ; logits l represents a vector of the l-th layer's class similarity scores after masking operation, with a dimension of 1×N; map(·) is used to map the scores score l-1 from the previous level to the positions of the sub-categories at the l-th layer and multiply them with the logits scores of this layer correspondingly; τ represents an independent coefficient used to control the influence degree of the classification probability of the previous level on the classification of the current level; S3. Based on the classification strategy designed in S2, use the monitoring image dataset constructed in S1 to train the image classification model, and complete the hierarchical classification of wildlife monitoring images through the stable model obtained from the training.
2. The hierarchical classification method for wildlife monitoring images based on a species classification tree according to claim 1, characterized in that The specific content of S2.1 includes the following: Carry out prompt engineering based on the text prompts of single-level classification, add descriptions of adjacent classification levels to the text of each level to improve the matching degree between the image and the text of each level; The addition of the category names of adjacent levels establishes the connection between different levels of categories, so that the image classification model can use the hierarchical structure between categories for classification.
3. The hierarchical classification method for wildlife monitoring images based on a species classification tree according to claim 2, wherein The specific content of S2.2 includes the following: According to the structure of the classification tree, adopt a step-by-step classification strategy from coarse to fine to refine the species classification. The category granularity is from coarse to fine, and the semantics is from abstract to specific. Use the classification results of the coarse-grained level to exclude incorrect options in the fine-grained categories; Allow the image classification model to gradually understand the features in the image, start from the coarse-grained category, and then gradually refine to the fine-grained category, so as to improve the classification accuracy and generate classification results containing multi-level information.