Species classification method for wildlife monitoring images based on point context

Abstract

This invention relates to a method for species classification in wildlife monitoring images based on point context, belonging to the field of image classification technology. The method involves modeling the correlation between monitoring points and species, assigning weights to samples based on the strength of the correlation, and decoupling the point classification and species classification tasks to learn environmental and species characteristics. By using the environmental information represented by the points as context for species classification, the method enhances the model's understanding of the relationship between species and habitat environments, achieving effective generalization to out-of-distribution data.

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification methods, and in particular relates to a method for species classification of wildlife monitoring images based on point context. Background Art

[0002] The scale of wildlife resource surveys and biodiversity monitoring continues to expand, and the number of monitored images is increasing. In recent years, wildlife surveys in my country, particularly those targeting terrestrial vertebrates, have seen rapid progress in both scale (geographical coverage and animal group coverage) and methods. Technologies such as infrared-triggered cameras provide effective means of data collection, and wildlife resource surveys and monitoring conducted by national parks, nature reserves at all levels, and forestry departments provide a rich and diverse source of data.

[0003] The shift in the distribution of wildlife monitoring data leads to reduced model performance. Wildlife monitoring spans long periods of time and covers a wide geographical area. The data collection process is affected by objective, uncontrollable factors such as weather, lighting, and the natural environment of the monitoring site, as well as subjective differences such as the specific camera installation location and shooting parameters. This leads to differences in the data distribution of wildlife monitoring images collected by different monitoring points or different cameras. In addition, due to the diverse living habits and population sizes of different wild animals, monitoring images are often biased in terms of sample size. Categories with smaller sample sizes may have higher research value (such as rare wildlife). A shift in the category distribution can easily bias the model towards the leading categories, affecting the classification performance of the tail categories and resulting in lost opportunities to discover and observe species.

[0004] Deep learning models typically assume that training and test data come from the same distribution. However, in wildlife monitoring image species classification, the data distributions of the training and test sets are not consistent. This distribution shift makes it difficult for the model to generalize to new application environments. Training a specific model for each monitoring environment or camera requires a large amount of annotated data and computing resources, which is cost-prohibitive. The challenge of generalizing to unknown data distributions hinders the large-scale application of species classification models. Therefore, improving the out-of-distribution generalization ability of wildlife monitoring image species classification models to make them applicable to monitoring images from different environments is an urgent problem that needs to be solved. Summary of the Invention

[0005] (1) Technical issues to be solved

[0006] The purpose of the present invention is to propose a species classification method for wildlife monitoring images based on point context to solve the problems raised in the background technology. The present invention enhances the model's understanding of the relationship between species and habitat environment and achieves effective generalization of out-of-distribution data.

[0007] (2) Technical solution

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] The method for species classification of wildlife monitoring images based on point context includes the following steps:

[0010] S1. Input wildlife monitoring images into the encoder for feature extraction, and analyze and process the acquired features to facilitate model learning;

[0011] S2. Divide the image features obtained in S1 into species features and point features, and design a method to assign sample weights based on species features and point features to enhance the learning process of the model;

[0012] S3. Input the species characteristics and point characteristics into the species classifier and point classifier respectively, use the classifier to classify the input species characteristics and point characteristics to obtain the prediction results, calculate the species classification loss and point classification loss, and reweight the classification loss and point classification loss to form the final LoCo loss;

[0013] S4. Based on the operations described in S1 to S3, a wildlife monitoring image classification model is constructed, and the learning and training of the model are completed; the trained model is used to estimate the loss weights of samples of each species in different environments and to classify the species.

[0014] Preferably, the S1 specifically includes the following contents:

[0015] The correlation between habitat environment and species category is modeled based on the uncertainty of data distribution. The strength of the correlation is used to describe the difficulty of the sample, and difficult samples with repeated pseudo-correlation features are distinguished. The corresponding sample weights are assigned, and the learning of environmental information represented by the point and species characteristics are decoupled. The point is used as the context information of the species classification task, and then generalization outside the distribution is achieved, forming the point context loss, which guides the model to perceive the point differences and learn generalizable features.

[0016] Preferably, the method of allocating sample weights according to species characteristics and site characteristics in S2 specifically includes the following:

[0017] The species uncertainty of point distribution and the point uncertainty of species distribution are proposed to describe the prior knowledge contained in the point information. The calculation formula is as follows:

[0018]

[0019] Among them, U t is the species’ location distribution uncertainty, U c is the species category uncertainty of the point, P tFor a given point Y c The species label distribution when P c For a given species Y c The point distribution at the time, N is the number of species, M is the number of camera points;

[0020] The point information is used as the context to guide the model to learn the relationship between species and background or habitat environment during training. The correlation is used to enhance the learning process of the species classification model. The normalized mutual information correlation score is calculated as follows:

[0021]

[0022] Where p(x,y) is the joint probability distribution function of X and Y; p(x) and p(y) are marginal probability distribution functions; MI is the mutual information, H(·) is the entropy calculation function, and NMI is the normalized mutual information.

[0023] Preferably, the species classification loss and the point classification loss are calculated in S3, and the classification loss and the point classification loss are reweighted to form the final LoCo loss, which specifically includes the following:

[0024] Use weighted loss for optimization and select softmax as the weight scaling function. The specific calculation formula is:

[0025]

[0026] Among them, w t is the weight of species classification loss, w c is the weight of the point classification loss, Λ(·) represents the weight scaling function, w t ′ and w c ′ together constitute the input vector of softmax, adjust the weights of the above two tasks to make the model focus on species classification, and introduce a hyperparameter to control the degree of weight scaling at the sample level;

[0027] The supervision process of CoLoCo is decoupled to become LoCo, which clearly separates the environmental features used for point classification and the animal features used for species classification. The feature extraction process, model optimization process and the specific input of the classifier are:

[0028] x t ,x c =φ(x)

[0029] p t ,p c =f t (concat(x t ,detach(xc ))),f c (x c )

[0030] L 1c (p t ,p c ,y t ,y c )=L t (p t ,y t )+L c (p c ,y c )

[0031] Among them, x t is the species characteristic, x c is the point feature, represents the encoder that extracts features from the input image x, f t is the species classifier, f c is the point classifier, p t and p c represents the result after classification; y represents the supervised label, the concat(·) operation represents feature concatenation, and the detach(·) operation represents the cancellation of the back propagation of species classification loss in point features to decouple the optimization process of species features and point features.

[0032] (3) Beneficial effects

[0033] (1) This paper addresses the problem that the distribution bias of wildlife monitoring data reduces the generalization performance of the model. It uses point labels to learn environmental information to improve the robustness of the model to out-of-distribution data. Then, it qualitatively and quantitatively analyzes the prior correlation between point labels and species labels observed in wildlife monitoring data, and points out the nature of its pseudo-correlation and its two-sided nature.

[0034] (2) This paper proposes two uncertainty models to model the relationship between point context and species categories, and formally describes the proposed point context-guided species classification method.

[0035] (3) This paper demonstrates the effectiveness of this method through experiments from various perspectives. Compared to the best baseline method, the Macro F1 score on the out-of-distribution test sets of the iWildCam and TerraInc datasets improves by 3.1% and 1.9%, respectively, while the accuracy improves by 0.5% and 3.2%, respectively. Finally, in closed-set scenarios, this method improves the model's recognition performance for known categories and its generalization performance to out-of-distribution data. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction to the drawings involved in the embodiments is now provided. It is obvious that the drawings described below are only schematic illustrations of some embodiments of the present invention. Those skilled in the art can construct other forms of drawings based on these drawings without inventive effort.

[0037] Figure 1 This is a flow chart of the method for species classification of wildlife monitoring images based on point context proposed by the present invention. DETAILED DESCRIPTION

[0038] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0039] This paper provides a method for species classification in wildlife monitoring images based on point context. This method models the correlation between monitoring points and species, assigns weights to samples based on the strength of the correlation, and simultaneously decouples the point and species classification tasks to learn environmental and species characteristics. By using the environmental information represented by the points as context for species classification, this method enhances the model's understanding of the relationship between species and habitat environments, improving the model's recognition of known categories in closed-set scenarios and its generalization performance to out-of-distribution data.

[0040] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0041] The following describes a method for species classification of wildlife monitoring images based on point context according to an embodiment of the present invention with reference to the accompanying drawings.

[0042] Example 1:

[0043] like Figure 1 As shown, the wildlife monitoring image species classification method includes the following steps:

[0044] First, the input image is put into the encoder for feature extraction. The correlation between habitat environment and species category is modeled based on the uncertainty of data distribution. The strength of the correlation is used to describe the difficulty of the sample. Difficult samples that are challenging due to repeated pseudo-correlation features are distinguished, and corresponding sample weights are assigned. The environmental information represented by the point is decoupled from the learning of species characteristics. The point is used as context information for the species classification task, thereby achieving out-of-distribution generalization, forming point context loss, and guiding the model to perceive point differences and learn generalizable features.

[0045] Secondly, the obtained features are composed of species features and site features. The prior knowledge contained in the site information is described according to the two proposed uncertainty measures, including species uncertainty of site distribution and site uncertainty of species distribution. Based on the site information as the context that guides the model to learn the relationship between species and background or habitat environment during training, the proposed site context-guided species classification method is formally described, explaining how the species classification model uses observed correlations to enhance the learning process, and calculating the normalized mutual information correlation score:

[0046]

[0047] Where p(x,y) is the joint probability distribution function of X and Y, p(x) and p(y) are the marginal probability distribution functions, MI is the mutual information, H(·) is the entropy calculation function, and NMI is the normalized mutual information.

[0048] The species classifier and the location classifier then classify the features to obtain predictions. The two losses are reweighted to form the final LoCo loss. The detach(·) operation is used to cancel the backpropagation of the species classification loss in the location features, thereby decoupling the optimization process of species and location features. This prevents task competition during feature extraction from hindering the expression of environmental information. Using a method that uses location, i.e., environmental information, as the context for the species classification problem, we propose the uncertainty of species location distribution and the uncertainty of species category at the location. The formulas are:

[0049]

[0050] Among them, U t is the species’ location distribution uncertainty, U c is the species category uncertainty of the point, P t For a given point Y c The species label distribution when P c For a given species Y c The point distribution at the time of , N is the number of species, and M is the number of camera points.

[0051] Using weighted loss for optimization, softmax is selected as the function of weight scaling, and the formula is:

[0052]

[0053] Among them, w t is the weight of species classification loss, w c is the weight of the point classification loss, and Respectively abbreviated as w t 'and w' c , together constitute the input vector of softmax, adjust the weights of the two tasks to make the model focus on species classification, and introduce a hyperparameter to control the degree of weight scaling at the sample level.

[0054] We use CoLoCo, which decouples the supervision process, to become LoCo. We clearly separate the environmental features used for point classification and the animal features used for species classification. The feature extraction process, model optimization process, and classifier input are:

[0055] x t ,x c =φ(x)

[0056] p t ,p c =f t (concat(x t ,detach(x c ))),f c (x c )

[0057] L 1c (p t ,p c ,y t ,y c )=L t (p t ,y t )+L c (p c ,y c )

[0058] Among them, x t is the species characteristic, x c is the point feature, f t is the species classifier, f c is a point classifier, and the result after classification is p t and p c .

[0059] Finally, the two proposed uncertainties were used to pre-estimate the sample loss weights of each species in different environments. In the loss calculation stage, the corresponding weights can be directly queried by species labels and point labels to perform loss weighting.

[0060] Example 2:

[0061] Based on Example 1, but different in that, the present invention proposes a method that uses point location, i.e., environmental information, as the context for species classification problems, aiming to improve the model's species classification ability by utilizing the correlation between the image background (environment) and the foreground (wildlife). Next, the task definition and the formal description of the proposed method are given. Samples with a monotonous background and a lack of foreground variation (i.e., categories distributed in fewer monitoring points and points with fewer species) are considered difficult samples for species classification, and a greater weight is assigned to the species classification loss. The CoLoCo that decouples the supervision process is called LoCo (i.e., LocationContext Loss). The method in this chapter follows widely used evaluation protocols such as the WILDS out-of-distribution generalization evaluation benchmark, i.e., using an out-of-distribution validation set containing 75 categories for model selection, and reporting the standard deviation and mean of the results through repeated experiments with 3 different random seeds. This method uses accuracy and Macro F1 score as the main evaluation indicators and compares with existing methods. Similar to WILDS, this chapter focuses on the Macro F1 score of the out-of-distribution test set to evaluate the out-of-distribution generalization performance of the model. Given that it is difficult to evaluate the recognition effect of tail categories based on the overall accuracy, this chapter also reports the accuracy on non-overlapping category subsets divided by sample frequency (i.e., the overall accuracy of the head category, middle category, and tail category test samples) to evaluate the robustness of the model to the category distribution shift of the samples.

[0062] This method demonstrates its effectiveness through multiple experiments. Compared to the best baseline, it achieves improvements in Macro F1 scores of 3.1% and 1.9% on the out-of-distribution test sets of the iWildCam and TerraInc datasets, respectively, while also improving accuracy by 0.5% and 3.2%, respectively. Finally, in closed-set scenarios, this method improves the model's recognition of known categories and generalization to out-of-distribution data (see Tables 1–3).

[0063]

[0064] Table 1

[0065]

[0066] Table 2

[0067]

[0068] Table 3

[0069] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions.

[0071] It should be noted that, in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims enumerating several means, several of these means may be embodied by one and the same hardware. The use of the words first, second, third etc. is for convenience only and does not indicate any order. These words may be understood as part of the component name.

[0072] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0073] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments after learning the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0074] Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if such modifications and variations fall within the scope of the claims and their equivalents, the present invention shall also include such modifications and variations.

Claims

1. A species classification method for wildlife monitoring images based on point context, characterized by: The following steps are involved: S1. Input wildlife monitoring images into the encoder for feature extraction, and analyze and process the acquired features to facilitate model learning; S2. Divide the image features obtained in S1 into species features and point features, and design a method to assign sample weights based on species features and point features to enhance the learning process of the model; S3. Input the species characteristics and site characteristics into the species classifier and site classifier respectively, use the classifier to classify the input species characteristics and site characteristics to obtain prediction results, calculate the species classification loss and site classification loss, and reweight the classification loss and site classification loss to form the final LoCo loss; specifically, it includes the following: Use weighted loss for optimization and select softmax as the weight scaling function. The specific calculation formula is: in, w t is the weight of species classification loss, w c is the weight of the point classification loss, Λ(∙) represents the weight scaling function, ; , , and Together they form the softmax input vector. The weights of the two tasks are adjusted to make the model focus on species classification. A hyperparameter is introduced to control the degree of weight scaling at the sample level. The supervision process of CoLoCo is decoupled to become LoCo, which clearly separates the environmental features used for point classification and the animal features used for species classification. The feature extraction process, model optimization process and the specific input of the classifier are: p t , p c = f t ( concat ( x t , detach ( x c ))), f c ( x c ) L 1c ( p t , p c , y t , y c )= L t ( p t , y t )+ L c ( p c , y c ) in, x t For species characteristics, x c is the point feature, φ(∙) represents the encoder that extracts features from the input image x, f t is the species classifier, f c is the point classifier, p t and p c Represents the result after classification; y represents the supervised label, and the concat (∙) operation represents feature concatenation. detach The (∙) operation cancels the back propagation of species classification loss in point features to decouple the optimization process of species features and point features; S4. Based on the operations described in S1 to S3, a wildlife monitoring image classification model is constructed, and the learning and training of the model are completed; the trained model is used to estimate the sample loss weights of each species in different environments and to classify the species.

2. The method for species classification of wildlife monitoring images based on point context according to claim 1 is characterized in that: The S1 specifically includes the following contents: The correlation between habitat environment and species category is modeled based on the uncertainty of data distribution. The strength of the correlation is used to describe the difficulty of the sample, and difficult samples with repeated pseudo-correlation features are distinguished. The corresponding sample weights are assigned, and the learning of environmental information represented by the point and species characteristics are decoupled. The point is used as the context information of the species classification task, and then generalization outside the distribution is achieved, forming the point context loss, which guides the model to perceive the point differences and learn generalizable features.

3. The method for species classification of wildlife monitoring images based on point context according to claim 1, characterized in that: The method described in S2 for allocating sample weights based on species characteristics and site characteristics specifically includes the following: The species uncertainty of point distribution and the point uncertainty of species distribution are proposed to describe the prior knowledge contained in the point information. The calculation formula is as follows: in, U t is the species' point distribution uncertainty, U c is the species category uncertainty of the point, P t For a given point Y c The species label distribution at P c For a given species Y c The point distribution at N is the number of species, M is the number of camera points; The point information is used as the context to guide the model to learn the relationship between species and background or habitat environment during training. The correlation is used to enhance the learning process of the species classification model. The normalized mutual information correlation score is calculated as follows: in, p ( x , y )for X and Y The joint probability distribution function of p ( x ), p ( y ) is the marginal probability distribution function; MI is mutual information, H (∙) is the entropy calculation function, NMI is the normalized mutual information.

Images