Hierarchical classification method for wild animal monitoring images based on species classification tree

Abstract

The invention relates to a wild animal monitoring image hierarchical classification method based on a species classification tree, and the method comprises the steps: carrying out the classification of five classification levels, namely the outline, the objective, the family, the genus and the species, of the species classification tree, and reducing the labor cost of a rechecking model result through providing richer species judgment information. According to the method, a soft decision and path correction strategy is introduced by utilizing a hierarchical relationship between categories, so that the accuracy and consistency of hierarchical classification are improved. When classification at fine-grained species levels is wrong, a coarse-grained result provided by the model also has a reference value.

Description

Technical Field

[0001] The present invention relates to the technical field of hierarchical species classification, and in particular to a hierarchical classification method for wild animal monitoring images based on a species classification tree. Background Art

[0002] The scale of wildlife resource surveys and biodiversity monitoring is constantly expanding, and the number of monitoring images is growing. In recent years, my country's wildlife surveys, especially those on terrestrial vertebrates, have made rapid progress in both survey scale (geographical area, animal group coverage) and survey methods. Technologies such as infrared triggered cameras provide an effective means of data collection, and wildlife resource surveys and monitoring conducted by national parks, nature reserves at all levels, and forestry departments provide a rich and diverse source of data.

[0003] In the classification of wildlife monitoring images, the cost of reviewing model results is high, and ecologists need to review model results to ensure accurate and rigorous downstream ecological research. Since the model does not provide more information that is helpful for confirming species in addition to the specific category of the output, for misclassified samples, reviewers need to reassign labels, a process that is tantamount to re-labeling.

[0004] In addition, existing species classification models usually lack interpretability, which also causes ecologists to worry about the reliability of the model and brings additional manual review costs. The existing wildlife monitoring image classification methods have limited research on this issue. Therefore, how to make the model provide more information that is beneficial to species classification and reduce the workload of ecological experts to review the model results is a problem to be solved. In view of this, the present invention proposes a hierarchical classification method for wildlife monitoring images based on a species classification tree. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] The purpose of the present invention is to propose a hierarchical classification method for wildlife monitoring images based on a species classification tree to solve the problems raised in the background technology. The present invention can provide more information that is helpful for judging species and improve the availability and interpretability of the image classification model.

[0007] (II) Technical solution

[0008] To achieve the above object, the present invention proposes a hierarchical classification method for wildlife monitoring images based on a species classification tree, comprising the following steps:

[0009] S1. Continuously monitor and acquire wildlife monitoring image data to form a monitoring image data set, and construct an image classification model for classifying the acquired wildlife monitoring image data;

[0010] S2. The thinking path of building an image classification model based on the species classification tree is to input image samples into the model and output the classification results of multiple levels of the corresponding species classification tree, which specifically includes the following contents:

[0011] S2.1. Design textual prompts based on the affiliation between categories so that the image classification model can use the hierarchy between categories for classification.

[0012] S2.2. Based on the top-down hierarchical classification of the species classification tree, refine the classification of species along the path from the root node to the leaf node of the classification tree;

[0013] S2.3, design a correction strategy for the classification path, use the classification probability of soft decisions, and use the probability score of the previous level to correct the logits score of the current level to weaken the influence of the decision results of the previous level;

[0014] S3. Based on the classification strategy designed in S2, the monitoring image dataset constructed in S1 is used to train the image classification model, and the hierarchical classification of wildlife monitoring images is completed through the stable model obtained through training.

[0015] Preferably, the S2.1 specifically includes the following contents:

[0016] Based on the text prompts of single-level classification, a prompt project is carried out to add descriptions of adjacent classification levels to the text of each level to improve the matching degree between the image and the text of each level; the addition of category names of adjacent levels establishes the connection between categories of different levels, so that the image classification model can use the hierarchical structure between categories for classification.

[0017] Preferably, the S2.2 specifically includes the following contents:

[0018] According to the structure of the classification tree, a layer-by-layer classification strategy from coarse to fine is adopted to refine the species classification, with the category granularity from coarse to fine and the semantics from abstract to concrete. The classification results of the coarse-grained level are used to eliminate the wrong options in the fine-grained category.

[0019] Allows image classification models to gradually understand features in images, starting from more general categories and then gradually refining to more specific categories, thereby improving classification accuracy and generating classification results that contain multiple layers of information.

[0020] Preferably, the calculation formula for the classification probability of the soft decision in S2.3 is:

[0021] score l =softmax(map(score l-1 ,l)·logits l / τ)

[0022] Where l represents the current classification level; score0 = 1 N ;logits l The vector representing the similarity scores of the categories at the lth level after the mask operation, with a dimension of 1×N; map(·) is used to convert the scores of the previous level l -1 is mapped to the position of the subcategory of the lth layer and multiplied by the corresponding logits score of the layer; τ represents the independence coefficient, which is used to control the influence of the classification probability of the previous layer on the classification of the current layer; when τ is large, the score is evenly distributed, each layer makes independent decisions, and the classification results of the upper layer have no effect on the subsequent classification; when τ is small, the score of each layer is close to the one-hot vector, which changes the classification range of the next layer, and the soft decision degenerates into a top-down hierarchical classification.

[0023] (III) Beneficial effects

[0024] Compared with the prior art, the present invention provides a hierarchical classification method for wildlife monitoring images based on species classification trees, which has the following beneficial effects:

[0025] In response to the problem of high cost in the review process of image classification model results in the prior art, the present invention proposes a zero-sample hierarchical classification method for species in wildlife monitoring images based on a classification tree, so that the model can provide more information that is helpful for judging species, thereby improving the usability and interpretability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention is described with the aid of the following drawings:

[0027] Figure 1 A flow chart of the hierarchical classification method for wildlife monitoring images based on species classification tree proposed by the present invention;

[0028] Figure 2 This is the hierarchical structure of the dataset labels for the hierarchical classification method for wildlife monitoring images based on the species classification tree proposed in the present invention. DETAILED DESCRIPTION

[0029] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0030] The following describes the hierarchical classification method for wildlife monitoring images based on species classification trees proposed in accordance with the present invention with reference to the accompanying drawings.

[0031] Embodiment 1:

[0032] The present invention provides a hierarchical classification method for wildlife monitoring images based on a species classification tree, which performs classification at the five classification levels of class, order, family, genus and species of the species classification tree, and reduces the labor cost of reviewing model results by providing richer species determination information. The method utilizes the hierarchical relationship between categories, introduces soft decision and path correction strategies, and improves the accuracy and consistency of hierarchical classification. When classification errors are made at the fine-grained species level, the coarse-grained results provided by the model are also of reference value.

[0033] See also Figure 1 , Figure 1 The figure is a flow chart of a hierarchical classification method for wildlife monitoring images based on a species classification tree according to an embodiment of the present invention.

[0034] like Figure 1 As shown, the hierarchical classification method for wildlife monitoring images includes the following steps:

[0035] by Figure 1 For example, when the model gives an incorrect prediction result of black-tailed deer at the species level, the labeler can refer to the results at the family and genus levels to quickly confirm the correct category as white-tailed deer from similar species.

[0036] First, for each input sample, the model gives the classification results of multiple levels of the corresponding species classification tree, and uses the species classification tree to establish the model's thinking path. Multi-level labels are obtained by matching images with text prompts containing hierarchical information from top to bottom according to the species classification tree levels. The open vocabulary model is explicitly extended to hierarchical classification, and soft decision and path correction strategies are introduced to improve the accuracy and consistency of multi-level labels. The multi-level classification results provide more information that is conducive to species confirmation, thereby reducing the labor cost of expert review and label query. When the model makes a classification error at the fine-grained species level, the coarse-grained results provided by the model can also be used for reference. By enhancing the interpretability of the species classification model, ecologists' trust in and willingness to use deep learning models will be improved.

[0037] Secondly, text prompts are designed according to the subordinate relationship between categories. Since hierarchical classification can be regarded as a combination of multiple single-level classification tasks, the methods to improve the performance of single-level classification can also be incorporated into the hierarchical classification method. Prompt engineering is carried out on the basis of text prompts for single-level classification. Descriptions of adjacent classification levels are added to the text of each level to improve the matching degree between the image and the text of each level. The addition of category names of adjacent levels establishes the connection between categories at different levels, so that the model can use the hierarchical structure between categories for classification.

[0038] Then, based on the top-down hierarchical classification of the species classification tree, along the path from the root node to the leaf node of the classification tree, the classification of species is gradually refined, the category granularity is from coarse to fine, and the semantics is from abstract to concrete. According to the structure of the classification tree, a coarse-to-fine layer-by-layer classification strategy is adopted, and the classification results of the coarse-grained level are used to eliminate the wrong options in the fine-grained categories, thereby improving the accuracy of species classification, allowing the model to gradually understand the features in the image, starting from more general categories and then gradually refining to more specific categories, so as to improve the classification accuracy and generate more interpretable classification results containing multi-level information.

[0039] Finally, the correction strategy of the classification path, top-down hierarchical classification pre-excludes erroneous options at deeper levels in the classification tree, gradually reducing the label space. However, whether the excluded categories are reasonable depends on the correctness of the current classification results. Therefore, early classification errors will accumulate to deeper levels, resulting in the next level The accuracy cannot be higher than the previous level. As the level deepens, the exponential accumulation of classification errors seriously limits the accuracy of deep-level classification. Using the classification probability of soft decisions, the probability score of the previous level is used to correct the logits score of the current level, retaining but weakening the impact of the decision results of the previous level, avoiding directly narrowing the scope of classification, and giving deeper-level decisions the opportunity to correct early errors. The classification probability formula using soft decisions is:

[0040] score l =softmax(map(score l-1 ,l)·logits l / τ)

[0041] Among them, l represents the current classification level, score0 = 1 N , logits l A vector representing the similarity scores of the categories at the lth level after the mask operation, with a dimension of 1×N. map(·) is used to convert the scores of the previous level l-1 is mapped to the position of the subcategory of the lth layer, and thus multiplied by the corresponding logits score of the layer. The temperature τ is called the independence coefficient here, which is used to control the influence of the classification probability of the previous layer on the classification of the current layer. When the independence coefficient is extremely large, the score is evenly distributed, and each layer makes independent decisions. The classification result of the upper layer does not affect the subsequent classification; when the independence coefficient is very small, the score of each layer is close to the one-hot vector, which will directly change the classification range of the next layer, and the soft decision degenerates into a top-down hierarchical classification.

[0042] Embodiment 2:

[0043] Based on Example 1, but different in that this method classifies monitoring images at five levels: class, order, family, genus and species. Figure 2 , introduces text prompts containing hierarchical relationships between categories and performs top-down layer-by-layer matching according to the species classification tree. At the same time, soft decision and path correction strategies are proposed to enable the model to recover classification errors of previous levels in deeper classification and maintain the consistency of multi-level results, thereby extending the zero-shot classification of the open vocabulary model to hierarchical classification. A comparative experiment is designed to compare the performance of the method proposed in this invention with the multi-level classification baseline, and the results are as follows.

[0044] Table 1 Performance comparison of this method with multi-level classification baselines

[0045]

[0046] Table 2 Ablation of classification level and independence coefficient of this method

[0047]

[0048] The results show that compared with the baseline method enhanced by the path correction strategy, this method improves the accuracy by 0.27%, 1.44%, 1.42%, 1.39% and 1.03% at the five classification levels from class to genus, and also reduces the error severity by 1.1%. The ablation experiment explores the impact of the number of levels in the hierarchical classification and the degree of independence between the classifications at each level on the performance of the method. Reasonably reducing the classification levels and constraining the degree of influence between the levels can help further improve the performance of the hierarchical classification.

[0049] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0050] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0051] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0052] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0053] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0054] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A hierarchical classification method for wildlife monitoring images based on species classification trees, characterized in that: The following steps are involved: S1. Continuously monitor and acquire wildlife monitoring image data to form a monitoring image data set, and construct an image classification model for classifying the acquired wildlife monitoring image data; S2. The thinking path of building an image classification model based on the species classification tree is to input image samples into the model and output the classification results of multiple levels of the corresponding species classification tree, which specifically includes the following contents: S2.

1. Design textual prompts based on the affiliation between categories so that the image classification model can use the hierarchy between categories for classification. S2.

2. Based on the top-down hierarchical classification of the species classification tree, refine the classification of species along the path from the root node to the leaf node of the classification tree; S2.3, design a correction strategy for the classification path, use the classification probability of soft decisions, and use the probability score of the previous level to correct the logits score of the current level to weaken the influence of the decision results of the previous level; S3. Based on the classification strategy designed in S2, the monitoring image dataset constructed in S1 is used to train the image classification model, and the hierarchical classification of wildlife monitoring images is completed through the stable model obtained through training.

2. The hierarchical classification method for wildlife monitoring images based on species classification tree according to claim 1 is characterized in that: The S2.1 specifically includes the following contents: Based on the text prompts of the single-level classification, the prompt engineering is carried out to add the description of the adjacent classification level to the text of each level to improve the matching degree between the image and the text of each level; The addition of category names at adjacent levels establishes a connection between categories at different levels, so that the image classification model can use the hierarchical structure between categories for classification.

3. The hierarchical classification method for wildlife monitoring images based on species classification tree according to claim 2 is characterized in that: S2.2 specifically includes the following contents: According to the structure of the classification tree, a layer-by-layer classification strategy from coarse to fine is adopted to refine the species classification, with the category granularity from coarse to fine and the semantics from abstract to concrete. The classification results of the coarse-grained level are used to eliminate the wrong options in the fine-grained category. Allows image classification models to gradually understand features in images, starting from more general categories and then gradually refining to more specific categories, thereby improving classification accuracy and generating classification results that contain multiple layers of information.

4. The hierarchical classification method for wildlife monitoring images based on species classification tree according to claim 3 is characterized in that: The calculation formula for the classification probability of the soft decision described in S2.3 is: score l =softmax(map(score l-1 ,l)·logits l / τ) Where l represents the current classification level; score0 = 1 N ;logits l The vector representing the similarity scores of the categories at the lth level after the mask operation, with a dimension of 1×N; map(·) is used to convert the scores of the previous level l -1 is mapped to the position of the subcategory of the lth layer and multiplied by the corresponding logits score of the layer; τ represents the independence coefficient, which is used to control the influence of the classification probability of the previous layer on the classification of the current layer; when τ is large, the score is evenly distributed, each layer makes independent decisions, and the classification results of the upper layer have no effect on the subsequent classification; when τ is small, the score of each layer is close to the one-hot vector, which changes the classification range of the next layer, and the soft decision degenerates into a top-down hierarchical classification.

Images