Wild animal species identification method based on incremental learning

Abstract

A wild animal species identification method based on incremental learning belongs to the field of target detection, and comprises the following steps: constructing a known category species detection data set, training a known category species detection model, selecting representative samples of each category species, and storing expression characteristics of each category species; constructing a species discrimination data set, and training a species discrimination model; manually checking new species sample data; and constructing a species detection fine tuning data set, updating a known category species detection model based on a feature playback class increment strategy, and updating a species discrimination model based on a feature-level sample playback class increment strategy. The method has the capability of identifying whether species belong to known categories in a training set; according to the method, after the data of the newly-added species category is acquired, the ability of identifying the new species can be learned at the minimum calculation cost; according to the method, the intermediate expression features of partial representative samples of old data are stored locally and then merged with the new category of data, and then the network is finely adjusted, so that the computing resource consumption is low.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a method for identifying wild animal species based on incremental learning. Background Art

[0002] Wild animals play an important role in the ecosystem, such as the composition of the food chain, the circulation of nutrients, soil conservation and water conservation. Through species detection, we can understand the number, distribution, behavior and habits of wild animal populations, evaluate their impact on the function of the ecosystem, and take corresponding measures to maintain the ecological balance.

[0003] Existing wildlife species recognition methods generally build end-to-end deep neural networks based on classification tasks or target detection tasks, and judge wildlife species through sample annotation and model training. Due to the limitations of the end-to-end deep neural network architecture, this method will identify species that have not appeared in the training set as the category with the most similar features in the training set when predicting them, and does not have the ability to identify new wildlife species. Therefore, providing a wildlife species recognition method capable of identifying new wildlife species has become a technical problem that needs to be solved urgently by those skilled in the art.

[0004] Chinese patent CN116863250A discloses "an open scene target detection method involving multimodal unknown class recognition". The patent proposes a method for identifying unknown classes in open scenes. The method first trains the target detection model UC-OSOD for open scene unknown recognition; uses RPN to generate background frames, and marks the background frames with the highest scores as potential unknown categories; then uses the comparative clustering method to separate known categories from unknown categories; uses the CLIP model based on multimodality to optimize the target detection model UC-OSOD for open scene unknown recognition, identifies unknown categories and classifies them; finally, based on the unknown class labels provided, uses the incremental learning method to learn new classes, and then cyclically realizes the recognition of unknown classes in open scenes. This method adopts a class incremental learning strategy based on sample playback, randomly samples the data in the training set and merges it with the new class samples to fine-tune the detection network. When selecting samples, the importance of different samples to the task is not considered, and there is a problem of computational redundancy.

[0005] In summary, existing wildlife species identification methods have the following shortcomings:

[0006] 1) Limited detection categories: Existing methods for wildlife species identification based on end-to-end deep neural network architectures only have the ability to identify species of known categories and do not have the ability to expand categories.

[0007] 2) Computational redundancy: When a new species category is added to the task, the neural network needs to be retrained, which consumes a lot of resources. Summary of the invention

[0008] In order to solve the problems of limited detection categories and computational redundancy in existing wildlife species identification methods, the present invention provides a wildlife species identification method based on incremental learning.

[0009] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0010] The present invention provides a method for identifying wildlife species based on incremental learning, comprising the following steps:

[0011] Step S1: construct a known category species detection dataset, train a known category species detection model, select representative samples of each category of species, and save the expression characteristics of each category of species;

[0012] Step S2: construct a species discrimination dataset and train a species discrimination model;

[0013] Step S3: Manually verify the new species sample data;

[0014] Step S4: Construct a species detection fine-tuning dataset, update the known category species detection model based on the class increment strategy of feature playback, and update the species discrimination model based on the feature-level sample playback class increment strategy.

[0015] Furthermore, in step S1, the label of the known category species detection data set includes species category and coordinate position information; the known category species detection model includes: a feature extraction module, a regression module, a classification module and a post-processing module; the feature extraction module extracts the features of the wild animals in the image, and inputs the extracted features of the wild animals into the regression module, locates the position of the target wild animals through the regression module, and at the same time inputs the extracted features of the wild animals into the classification module, predicts the category of the target wild animals through the classification module, and finally inputs the position of the target wild animals and the category of the target wild animals into the post-processing module, and removes the predicted redundant prediction boxes through the post-processing module.

[0016] Furthermore, in step S1, the specific process of selecting representative samples of each category of species and saving the expression characteristics of each category of species is as follows: obtaining the inference detection results of samples of all species categories in the training set on the known category species detection model; for each category of species, the samples with correct detection are formed into a queue, and the sample queues are sorted according to the level of classification confidence, and the first 1% samples of the sample queue, the last 4% samples of the sample queue and their corresponding classification confidence are recorded; the expression feature set F corresponding to the selected representative sample set of species is 1 ,f 2 ,...,f c}Save to local.

[0017] Furthermore, the process of acquiring the expression feature set is expressed as:

[0018] f i =Det backbone (x i ),i∈C

[0019] Among them, Det backbone represents the feature extraction module of the known category species detection model; i represents the category information; C represents the total number of species categories; x i represents the representative sample set of species in the i-th category; f i Represents the expression feature set corresponding to the representative sample set of the i-th species.

[0020] Furthermore, in step S2, the specific process of constructing the species discrimination dataset is as follows:

[0021] The expression features corresponding to the representative sample sets of the obtained species are combined into pairs, and soft labels are constructed for the expression feature pairs of the same category according to the classification confidence, and the expression feature pairs of different categories are marked as 0.

[0022] Furthermore, the construction rules of the soft label are as follows:

[0023]

[0024] Among them, L represents the marked label; i and j both represent category information; C represents the total number of species categories; f i represents the expression characteristics corresponding to the samples of the i-th species; f j represents the expression characteristics corresponding to the sample of the jth species; x i represents the representative sample of the i-th species; x j represents a representative sample of the jth species; conf represents the classification confidence value of the sample in the inference detection result of the known category species detection model.

[0025] Furthermore, the species discrimination model includes a feature mapping branch 1, a feature mapping branch 2, a feature aggregation layer and a classification layer; the feature mapping branch 1 and the feature mapping branch 2 respectively map the expression feature pairs to the same feature space; the feature aggregation layer aggregates two features in the same feature space to generate intermediate features for classification; the classification layer processes the intermediate features to obtain the confidence of whether the two features belong to the same category.

[0026] Furthermore, the specific process of step S3 is as follows:

[0027] After deploying the known category species detection model and the species discrimination model, during inference, the known category species detection model is first used to obtain the location, category information and expression characteristics of the species; then the expression characteristics corresponding to the sample instances ranked in the top 1% of the classification confidence of different species are randomly selected and paired with the expression characteristics of the inference samples one by one; finally, the species discrimination model is used to identify all the expression feature pairs. If all the identification results are judged not to be of the same category, the inference sample is marked as a new species; the number of sample data identified as new category species by the species discrimination model is recorded, and the corresponding expression characteristics are saved. When the number of sample data of new category species reaches the preset number of samples, the species category information in the sample data is manually verified.

[0028] Furthermore, the specific process of updating the known category species detection model based on the feature playback class incremental strategy is as follows:

[0029] The expression features of the constructed representative samples of known species and the expression features of the samples of new species categories after manual verification are summarized to form a species detection fine-tuning dataset; during the fine-tuning process, the feature extraction module of the known category species detection model does not participate in the calculation, and the fine-tuning model part uses the expression features as input; the model weights of the regression module are frozen, and the species position information obtained after the expression features pass through the regression module remains unchanged; the number of output heads of the classification module is changed from the original number of species categories to the number of categories after including the new species, and the expression features pass through the classification module to obtain the new category information of the species; a new known category species detection model is obtained by fine-tuning on the species detection fine-tuning dataset to replace the original known category species detection model.

[0030] Furthermore, the specific process of updating the species discrimination model based on the feature-level sample replay incremental strategy is as follows:

[0031] The expression features corresponding to the constructed species representative sample set are paired with the expression features of the new category species samples that have been manually verified to reconstruct the species discrimination dataset; a new species discrimination model is obtained by fine-tuning on the reconstructed species discrimination dataset to replace the original species discrimination model.

[0032] The beneficial effects of the present invention are:

[0033] The present invention provides a method for identifying wildlife species based on incremental learning, which is divided into two stages. In the first stage, a known category species detection data set is constructed, a known category species detection model is obtained by training, representative samples of species in each category are selected, and the expression characteristics of species in each category are saved; according to the results of the classification module of the known category species detection model, a preset number of representative samples of species in each category are selected according to the confidence level of the species category, and the expression characteristics of the representative samples of the species are saved before classification by the known category species detection model; finally, a species discrimination data set pair is constructed based on the expression characteristics corresponding to the selected representative samples of the species, and a species discrimination model is obtained by training. In the second stage, the data identified as new species by the species discrimination model are manually labeled; then the expression characteristics corresponding to the new species samples are merged with the expression characteristics corresponding to the representative samples of the species to construct a species detection fine-tuning data set, and the known category species detection model and the species discrimination model are updated, thereby realizing the category expansion of wildlife species recognition with relatively small computing resources.

[0034] Compared with existing wildlife species detection methods, the present invention brings about improved effects in the following aspects:

[0035] 1) Detection categories are expandable: The present invention has the ability to identify whether a species belongs to a known category in the training set;

[0036] 2) Continuous learning capability: After acquiring data on newly added species categories, the present invention can learn the ability to identify new species at minimal computational cost.

[0037] 3) Small amount of computation: Compared with fine-tuning on the entire data, the present invention saves the intermediate expression features of some representative samples of the old data locally, and then merges them with the new category of data to fine-tune the network, which consumes less computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flow chart of a method for identifying wild animal species based on incremental learning provided by the present invention.

[0039] Figure 2 Diagram of the structure of the model for detecting species of known categories.

[0040] Figure 3 This is the structural diagram of the species discrimination model. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0042] The present invention provides a method for identifying wildlife species based on incremental learning, comprising the following steps:

[0043] 1) Construct a known species detection dataset, train a known species detection model, select representative samples of each species category, and save the expression characteristics of each species category;

[0044] 2) Construct a species discrimination dataset and train a species discrimination model;

[0045] 3) Manual verification of new species sample data;

[0046] 4) Construct a species detection fine-tuning dataset and update the known category species detection model and species discrimination model.

[0047] See also Figure 1 To illustrate, the present invention provides a method for identifying wild animal species based on incremental learning, and its specific implementation process is as follows:

[0048] Step S1: construct a known category species detection dataset, train a known category species detection model, select representative samples of each category of species, and save the expression characteristics of each category of species;

[0049] (1) Collect existing high-quality images of wild animals, with at least 1,000 images of each species, and manually annotate the species and coordinate information of the animals in the images;

[0050] (2) Construct a detection model for species of known categories such as Figure 2 As shown, the known category species detection model mainly includes feature extraction module, regression module, classification module and post-processing module.

[0051] Among them, the feature extraction module is mainly used to extract the features of wild animals in the image, and input the extracted features of wild animals into the regression module, and locate the position of the target wild animal through the regression module. At the same time, the extracted features of wild animals are input into the classification module, and the category of the target wild animal is predicted through the classification module. Finally, the position of the target wild animal and the category of the target wild animal are input into the post-processing module, and the redundant prediction boxes are removed through the post-processing module; finally, a known category species detection model is trained based on the constructed known category species detection dataset.

[0052] (3) Select representative samples of species in each category and preserve the expression characteristics of species in each category;

[0053] Obtain the inference test results of samples of all species categories in the training set on the known species detection model; for each species category, the correctly detected samples are grouped into queues, and the sample queues are sorted according to the level of classification confidence, and the first 1% samples of the sample queue, the last 4% samples of the sample queue and their corresponding classification confidence are recorded; the expression feature set corresponding to the selected representative sample set F = {f 1 ,f2 ,...,f c} is saved locally, where the process of obtaining the expression feature set can be expressed as:

[0054] f i =Det backbone (x i ),i∈C

[0055] Among them, Det backbone represents the feature extraction module of the known category species detection model; i represents the category information; C represents the total number of species categories; x i represents the representative sample set of species in the i-th category; f i Represents the expression feature set corresponding to the representative sample set of the i-th species.

[0056] Step S2: construct a species discrimination dataset and train a species discrimination model;

[0057] (1) Construct a species discrimination dataset;

[0058] The expression features corresponding to the representative sample sets of the obtained species are combined into pairs, and the expression feature pairs of the same category are constructed into soft labels according to the classification confidence, and the expression feature pairs of different categories are marked as 0. The soft label construction rules are as follows:

[0059]

[0060] Among them, L represents the marked label; i and j both represent category information; C represents the total number of species categories; f i represents the expression characteristics corresponding to the samples of the i-th species; f j represents the expression characteristics corresponding to the sample of the jth species; x i represents the representative sample of the i-th species; x j represents a representative sample of the jth species; conf represents the classification confidence value of the sample in the inference detection result of the known category species detection model.

[0061] (2) Training species discrimination models;

[0062] Constructing species discrimination model Figure 3 As shown, the species discrimination model mainly includes two feature mapping branches (feature mapping branch 1 and feature mapping branch 2), a feature aggregation layer and a classification layer.

[0063] Among them, feature mapping branch 1 and feature mapping branch 2 are used to respectively map the expression feature pairs to the same feature space; the feature aggregation layer is used to aggregate two features in the same feature space to generate intermediate features for classification; the classification layer can specifically use the sigmoid function, which is mainly used to process the intermediate features to obtain the confidence of whether the two features belong to the same category; finally, the species discrimination model is trained based on the constructed species discrimination dataset.

[0064] Step S3: Manually verify the new species sample data;

[0065] After deploying the known category species detection model and the species discrimination model, during inference, the known category species detection model is first used to obtain the location, category information, and expression characteristics of the species; then the expression characteristics corresponding to the sample instances ranked in the top 1% of the classification confidence of different species are randomly selected and paired with the expression characteristics of the inference samples one by one; finally, the species discrimination model is used to identify all the expression feature pairs. If all the identification results are judged to be not of the same category, the inference sample is marked as a new species.

[0066] The number of sample data identified as new species by the species discrimination model is recorded, and the corresponding expression characteristics are saved. When the number of sample data of new species reaches the preset number of samples, the species category information in the sample data is manually verified.

[0067] Step S4: construct a species detection fine-tuning dataset and update the known category species detection model and species discrimination model;

[0068] (1) Update the known category species detection model based on the feature playback incremental strategy. Specifically:

[0069] The expression features of representative samples of known species constructed in step S1(3) and the expression features of samples of new species that have been manually verified in step S3 are aggregated to form a species detection fine-tuning dataset.

[0070] During the fine-tuning process, the feature extraction module of the known category species detection model does not participate in the calculation, such as Figure 2 As shown in the dashed box in the middle, the fine-tuned model part uses the expression features as input; the model weights of the regression module are frozen, and the species position information obtained after the expression features pass through the regression module remains unchanged; the number of output heads of the classification module is changed from the original number of species categories to the number of categories after including the new species, and the expression features pass through the classification module to obtain the new category information of the species; fine-tuning on the species detection fine-tuning dataset obtains a new known category species detection model to replace the original known category species detection model.

[0071] (2) Update the species discrimination model based on the feature-level sample replay incremental strategy. Specifically:

[0072] The expression features corresponding to the representative sample set constructed in step S2(1) are paired with the expression features of the new category species samples after manual verification in step S3 to reconstruct the species discrimination data set; a new species discrimination model is obtained by fine-tuning on the reconstructed species discrimination data set to replace the original species discrimination model.

[0073] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for identifying wildlife species based on incremental learning, characterized in that: The following steps are involved: Step S1: construct a known category species detection dataset, train a known category species detection model, select representative samples of each category of species, and save the expression characteristics of each category of species; Step S2: construct a species discrimination dataset and train a species discrimination model; Step S3: Manually verify the new species sample data; Step S4: Construct a species detection fine-tuning dataset, update the known category species detection model based on the class increment strategy of feature playback, and update the species discrimination model based on the feature-level sample playback class increment strategy.

2. A method for identifying wildlife species based on incremental learning according to claim 1, characterized in that: In step S1, the label of the known category species detection data set includes species category and coordinate location information; The known category species detection model includes: a feature extraction module, a regression module, a classification module and a post-processing module; The feature extraction module extracts the features of wild animals in the image, and inputs the extracted features of wild animals into the regression module, which locates the position of the target wild animal. At the same time, the extracted features of wild animals are input into the classification module, which predicts the category to which the target wild animal belongs. Finally, the position of the target wild animal and the category to which the target wild animal belongs are input into the post-processing module, and the redundant prediction boxes are removed by the post-processing module.

3. A method for identifying wildlife species based on incremental learning according to claim 2, characterized in that: In step S1, the specific process of selecting representative samples of each category of species and saving the expression characteristics of each category of species is as follows: obtain the inference detection results of samples of all species categories in the training set on the known category species detection model; for each category of species, form a queue with the samples that have been detected correctly, and sort the sample queues according to the level of classification confidence, record the first 1% samples of the sample queue, the last 4% samples of the sample queue and their corresponding classification confidence; the expression feature set F = {f 1 , f 2 , ..., f c }Save to local.

4. A method for identifying wildlife species based on incremental learning according to claim 3, characterized in that: The process of acquiring the expression feature set is expressed as follows: f i =That backbone (x i ),i∈C Among them, Det backbone represents the feature extraction module of the known category species detection model; i represents the category information; C represents the total number of species categories; x i represents the representative sample set of species in the i-th category; f i Represents the expression feature set corresponding to the representative sample set of the i-th species.

5. The method for identifying wildlife species based on incremental learning according to claim 1, characterized in that: In step S2, the specific process of constructing the species discrimination dataset is as follows: The expression features corresponding to the representative sample sets of the obtained species are combined into pairs, and soft labels are constructed for the expression feature pairs of the same category according to the classification confidence, and the expression feature pairs of different categories are marked as 0.

6. A method for identifying wildlife species based on incremental learning according to claim 5, characterized in that: The construction rules of the soft label are as follows: Among them, L represents the marked label; i and j both represent category information; C represents the total number of species categories; f i represents the expression characteristics corresponding to the samples of the i-th species; f j represents the expression characteristics corresponding to the sample of the jth species; x i represents the representative sample of the i-th species; x j represents a representative sample of the jth species; conf represents the classification confidence value of the sample in the inference detection result of the known category species detection model.

7. The method for identifying wildlife species based on incremental learning according to claim 1, characterized in that: The species discrimination model includes a feature mapping branch 1, a feature mapping branch 2, a feature aggregation layer and a classification layer; the feature mapping branch 1 and the feature mapping branch 2 respectively map the expression feature pairs to the same feature space; the feature aggregation layer aggregates two features in the same feature space to generate intermediate features for classification; the classification layer processes the intermediate features to obtain the confidence of whether the two features belong to the same category.

8. A method for identifying wildlife species based on incremental learning according to claim 2 or 5, characterized in that: The specific process of step S3 is as follows: After deploying the known category species detection model and the species discrimination model, during inference, the known category species detection model is first used to obtain the location, category information and expression characteristics of the species; then the expression characteristics corresponding to the sample instances ranked in the top 1% of the classification confidence of different species are randomly selected and paired with the expression characteristics of the inference samples one by one; finally, the species discrimination model is used to identify all the expression feature pairs. If all the identification results are judged not to be of the same category, the inference sample is marked as a new species; the number of sample data identified as new category species by the species discrimination model is recorded, and the corresponding expression characteristics are saved. When the number of sample data of new category species reaches the preset number of samples, the species category information in the sample data is manually verified.

9. A method for identifying wildlife species based on incremental learning according to claim 8, characterized in that: The specific process of updating the known category species detection model based on the feature playback class incremental strategy is as follows: The expression features of the constructed representative samples of known species and the expression features of the samples of new species that have been manually verified are summarized to form a species detection fine-tuning dataset; during the fine-tuning process, the feature extraction module of the known category species detection model does not participate in the calculation, and the fine-tuning model part uses the expression features as input; Freeze the model weights of the regression module, and the species location information obtained after the expression features pass through the regression module remains unchanged; change the number of output heads of the classification module from the original number of species categories to the number of categories after including the new species, and the expression features pass through the classification module to obtain the new category information of the species; fine-tune on the species detection fine-tuning dataset to obtain a new known category species detection model to replace the original known category species detection model.

10. The method for identifying wildlife species based on incremental learning according to claim 8, characterized in that: The specific process of updating the species discrimination model based on the feature-level sample playback incremental strategy is as follows: The expression features corresponding to the constructed species representative sample set are paired with the expression features of the new category species samples that have been manually verified to reconstruct the species discrimination dataset; a new species discrimination model is obtained by fine-tuning on the reconstructed species discrimination dataset to replace the original species discrimination model.

Images