Pest image recognition method based on joint adversarial transfer learning

Abstract

The present application relates to the technical field of artificial intelligence, and more particularly to a pest image recognition method based on joint adversarial transfer learning, comprising the following steps: step S1, performing stylization processing on an open-source agricultural and forestry pest dataset to form a transfer learning dataset; step S2, training all combinations of source domains, style domains and blank domains contained in the field-test field; step S3, introducing a center perception constraint in the field adversarial neural network; and step S4, introducing a related alignment constraint in the last fully connected layer of the label recognizer. The pest image classification algorithm method based on joint adversarial transfer learning can better solve the problem of the influence of negative transfer on pest classification accuracy, thereby achieving effective recognition of agricultural and forestry pests.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for pest image recognition based on joint adversarial transfer learning. Background Technology

[0002] Pest identification is crucial for the healthy growth of crops and forests, as pests affect crop yield and quality. Insecticides can be used to control pests, but improper control or overuse can be harmful to plants and humans. Significant losses can be avoided if pests are detected early and properly prevented. Currently, pest identification and monitoring technologies mainly include manual statistical methods, attractant trapping, and satellite remote sensing. However, existing monitoring methods still have certain limitations: manual statistical methods are inefficient and costly; the effectiveness of attractants is affected by the hanging height of the traps and the type of ecological environment; and satellite remote sensing cannot accurately collect local microscopic information. To overcome these limitations, deep learning-based image recognition methods have gradually gained widespread attention. Among them, methods based on deep convolutional neural networks have advantages such as fast response and less susceptibility to environmental factors, making them the mainstream approach for pest image recognition.

[0003] Chinese Patent Publication No. CN115861729A discloses a method for identifying forestry pests using a few samples based on a deep learning algorithm. The method includes the following steps: constructing a prototype network suitable for few-sample learning, using a ResNet-18 neural network pre-trained on the ImageNet dataset as the backbone of the prototype network; expanding the dataset using rotation, flipping, and random cropping to achieve data augmentation; using a few-sample dataset of 70 other forestry pests as a meta-dataset to train the prototype grid so that users can acquire prior knowledge of few-sample feature measurements; dividing the images to be identified into training and testing sets according to a predetermined ratio, and after training, making the feature vectors of similar images closer together and the feature vectors of dissimilar images farther apart to effectively classify unknown images; and introducing information entropy regularization to assist training and enhance the classification effect.

[0004] Therefore, it can be seen that the pest image recognition method has the problem of negative transfer in pest recognition under complex scenarios. Summary of the Invention

[0005] To address this issue, the present invention provides a pest image recognition method based on joint adversarial transfer learning, which overcomes the problem of negative transfer in pest recognition under complex scenarios in the prior art.

[0006] To achieve the above objectives, this invention provides a pest image recognition method based on joint adversarial transfer learning, comprising:

[0007] Step S1: Stylize the open-source agricultural and forestry pest dataset to form a transfer learning dataset;

[0008] Step S2, the training domain-test domain includes all combinations of the three domains: source domain, style domain, and blank domain;

[0009] Step S3: Introduce center-aware constraints into the domain adversarial neural network;

[0010] Step S4: Introduce relevant alignment constraints in the final fully connected layer of the tag recognizer.

[0011] Further, in step S1, the transfer learning dataset is the Pest-TL transfer learning dataset for agricultural and forestry pests, and its main steps include:

[0012] Collect open-source pest species datasets and delete non-adult data;

[0013] The organized data is divided into blank background areas and normal background areas according to different backgrounds;

[0014] Based on the DNN-based image transfer method, the normal domain background is transferred into the style domain image.

[0015] Furthermore, in step S2, the specific steps of the training process include:

[0016] The training process organizes the data into source domain, style domain, and blank domain;

[0017] The six migration tasks are set up as follows: migration from source domain to style domain, migration from source domain to blank domain, migration from style domain to blank domain, migration from style domain to source domain, migration from blank domain to source domain, and migration from blank domain to style domain.

[0018] The training domain and the test domain encompass all combinations of these three domains;

[0019] During model inference, labeled source domain data and unlabeled source domain data are simultaneously input into two feature extractors, which share parameters.

[0020] Each feature extractor generates a center-perception loss, which is then jointly processed by a domain discriminator to generate an adversarial loss.

[0021] The label classifier generates a classification loss and a joint discriminant loss, with the label classifiers sharing parameters.

[0022] Furthermore, in step S3, the introduction of center-aware constraints is defined as follows:

[0023]

[0024] Where, xi For the sample, L c Perceive loss at the center The y-th feature represents i Each category center, R d Let y represent a real number. i y represents the class center ordinal number. i ∈{1,2,...,c}, where c represents the size of the class center, m represents the size of the mini-batch, and i is a variable.

[0025] Furthermore, in step S3, the center-aware loss is performed based on mini-batch updates when updating the centers of the entire training set.

[0026] Furthermore, in step S3, when updating the center perception loss, a scalar is used to control the learning rate of the center to avoid large perturbations caused by a few mislabeled samples.

[0027] Further, in step S4, the introduced correlation alignment constraint, the calculation formula for the correlation alignment of the output features of the label recognizer across the source and target domains is as follows:

[0028]

[0029] Among them, L d To determine the loss, F represents the feature extractor, and d represents the dimension of the feature. Let C denote the Frobenius norm of a square matrix. s The correlation matrix representing the samples in the source domain, C T The correlation matrix of samples in the target domain.

[0030] Furthermore, in step S4, relevant alignment is used to constrain the covariance of source and target domain features to reduce domain discrepancies.

[0031] Furthermore, a joint optimization approach is adopted, namely, using a min-max optimization method to optimize the parameters of the feature extractor and the label recognizer to train a source domain discriminative model.

[0032] Furthermore, objective functions are established for the feature extractor and label recognizer, and corresponding hyperparameters are set to optimize the objective functions.

[0033] Compared with the prior art, the beneficial effects of the present invention are that it introduces the domain adversarial neural network model into the center perception constraint module and the related alignment constraint module, uses the transfer learning dataset of agricultural and forestry pests as the transfer basis of the domain adversarial neural network model, and performs joint optimization of the domain in the introduction module, thereby improving the accuracy of pest image recognition tasks.

[0034] Furthermore, this invention modifies the database of pest species by designing the Pest-TL transfer learning dataset for agricultural and forestry pests.

[0035] Furthermore, this invention verifies and tests the training process of data organization, repeatedly updating the data in the model, thereby iterating the pest image recognition method based on joint adversarial transfer learning within the domain adversarial neural network model, ensuring the efficiency of data processing when dealing with complex scenarios.

[0036] Furthermore, this invention balances the data within the domain adversarial neural network model by introducing a center-perception constraint, ensuring accurate discrimination even for a small number of categories during neural network training.

[0037] Furthermore, in order to avoid large perturbations caused by a few mislabeled samples, the present invention introduces scalar control learning rate when updating the center perception constraint, so that the model can adaptively adjust the learning rate during training.

[0038] Furthermore, the constraint objects in the relevant constraint processing are restricted to the covariance of features in the source domain and the target domain, reducing domain differences and thus achieving cross-domain feature transferability.

[0039] Furthermore, this invention introduces relevant alignment constraints in the final fully connected layer of the label recognizer, which further helps to align the results during classification, reduces domain differences, and thus achieves cross-domain feature transferability.

[0040] Furthermore, a joint optimization method is adopted to optimize the parameters of the feature extractor and the tag recognizer to achieve the task of pest image recognition. Attached Figure Description

[0041] Figure 1 This is a flowchart of the pest image recognition method based on joint adversarial transfer learning provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the Pest-TL pest image dataset, which is a transfer learning dataset for agricultural and forestry pests provided in an embodiment of the present invention.

[0043] Figure 3 This is a schematic diagram of the training domain-testing process of an embodiment of the pest image recognition method based on joint adversarial transfer learning provided by the present invention. Detailed Implementation

[0044] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0045] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0046] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0047] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0048] Please see Figure 1 As shown, the flowchart of the pest image recognition method based on joint adversarial transfer learning provided in this embodiment of the invention includes,

[0049] Step S1: Stylize the open-source agricultural and forestry pest dataset to form a transfer learning dataset;

[0050] Step S2, the training domain-test domain includes all combinations of the three domains: source domain, style domain, and blank domain;

[0051] Step S3: Introduce center-aware constraints into the domain adversarial neural network;

[0052] Step S4: Introduce relevant alignment constraints in the final fully connected layer of the tag recognizer.

[0053] This invention introduces a domain adversarial neural network model into a center perception constraint module and a related alignment constraint module, and uses a transfer learning dataset of agricultural and forestry pests as the transfer basis for the domain adversarial neural network model. Furthermore, joint optimization of the domain is performed in the introduction module to improve the accuracy of pest image recognition tasks.

[0054] Please see Figure 2As shown, this is Pest-TL, a transfer learning dataset for agricultural and forestry pests.

[0055] Specifically, in step S1, the transfer learning dataset is the Pest-TL transfer learning dataset for agricultural and forestry pests, and its main steps include:

[0056] Step 101: Collect an open-source pest species dataset and delete non-adult data;

[0057] Step 102: Divide the organized data into blank background areas and normal background areas according to different backgrounds;

[0058] Step 103: Based on the DNN image transfer method, transfer the normal domain background into a style domain image.

[0059] This invention uses the Pest-TL dataset, a transfer learning dataset for agricultural and forestry pests, to regulate the database of pest species.

[0060] Please refer to Figure 3 This invention provides a flowchart illustrating an embodiment of a pest image recognition method based on joint adversarial transfer learning. The pest image recognition method based on joint adversarial transfer learning includes:

[0061] This invention introduces a center perception constraint module and a related alignment constraint module on the basis of the DANN model, which increases the complementarity of the model knowledge transfer process and improves the image recognition accuracy of unsupervised transfer learning.

[0062] Specifically, in real-time example step 201, the pest image recognition method based on joint adversarial transfer learning is characterized by introducing a center-perception constraint into the domain adversarial neural network, which is defined as follows:

[0063]

[0064] Where, x i For the sample, L c Perceive loss at the center The y-th feature represents i Each category center, R d Let y represent a real number. i y represents the class center ordinal number. i ∈{1,2,...,c}, where c represents the size of the class center, m represents the size of the mini-batch, and i is a variable.

[0065] This invention balances the data within a domain adversarial neural network model by introducing a center-perception constraint, ensuring accurate discrimination even for a small number of categories during neural network training.

[0066] Specifically, in real-time example step 201, the center-aware loss is performed based on mini-batch updates when updating the centers of the entire training set.

[0067] To address the challenge that center-aware loss cannot be directly applied to deep network models, two necessary improvements were made. In this embodiment, when updating the centers of the entire training set, a mini-batch update is performed.

[0068] In each iteration, the present invention calculates the class center by averaging the depth features of the current batch (in which case some centers may not be updated).

[0069] Specifically, in real-time example step 201, when updating the center perception loss, a scalar is used to control the learning rate of the center to avoid large perturbations caused by a few mislabeled samples.

[0070] To avoid large disturbances caused by a few mislabeled samples, this invention introduces a scalar control learning rate when updating the center-aware constraints, enabling the model to adaptively adjust the learning rate during training.

[0071] This embodiment uses a scalar δ to control the learning rate of the center.

[0072] therefore, Update using the following formula at each iteration:

[0073]

[0074]

[0075] Among them, c j Δc represents the center of each category. j Indicates each time c j The result of the iteration, L c For center-perceived loss, x i As a sample, The y-th feature represents i Each category center, the condition y i =j indicates whether the predicted category is the same as the true category, and i is a variable.

[0076] If the condition is met, δ(y) i =j)=1, otherwise, δ(y) i =j)=0. Typically, the scalar δ takes values ​​in the range [0,1].

[0077] Step 202,

[0078] Specifically, the relevant alignment constraint introduced in the final fully connected layer of the label recognizer is the relevant alignment of the output features of the label recognizer across the source and target domains. The specific calculation formula is as follows:

[0079]

[0080] Where d represents the dimension of the feature, Let C denote the Frobenius norm of a square matrix. s The correlation matrix representing the samples in the source domain, C T The correlation matrix of the target domain samples, L d Output relevant alignment values ​​for features to the cross-source and target domain label recognizers.

[0081] This invention introduces relevant alignment constraints in the final fully connected layer of the tag recognizer, which further helps to align the results during classification, reduces domain differences, and thus achieves cross-domain feature transferability.

[0082] Specifically,

[0083]

[0084]

[0085] By restricting the constraint objects in the relevant constraint processing to the covariance of features in the source and target domains, the domain differences are reduced, thereby achieving cross-domain feature transferability.

[0086] Among them, C s The correlation matrix representing samples from the source domain; C T The correlation matrix of the target domain samples; N s Size of the source domain samples; N T Size of the target domain samples; B S B represents the source domain feature representation output by label recognizer C; T This represents the target domain feature representation output by the tag recognizer C; 1 T This represents a row vector consisting entirely of 1s.

[0087] Then feature B S and B T The gradient can be calculated using the chain rule:

[0088]

[0089]

[0090] in, Represents the j-th dimension of the i-th source domain sample; N represents the j-th dimension of the i-th target domain sample.s Size of the source domain samples; N T Size of the target domain samples; C s The correlation matrix representing samples from the source domain; C T The correlation matrix of the target domain samples; The matrix transpose represents the feature table of the source domain output by the tag recognizer C. T represents the matrix transpose of the feature table of the target domain output by the tag recognizer C; T is the matrix transpose symbol.

[0091] Specifically, a joint optimization approach is adopted, which involves using a min-max optimization method to optimize the parameters of the feature extractor and the label recognizer to train a source-domain discriminative model.

[0092] Specifically, objective functions are established for the feature extractor and label recognizer, and corresponding hyperparameters are set to optimize the objective functions.

[0093] Step 203: A joint optimization method is adopted, that is, the source domain discriminative model is trained by optimizing the parameters of the feature extractor and the label recognizer to achieve the pest image recognition task.

[0094] The source domain classification model C(F(x)) is trained by optimizing the parameters of the feature extractor F and the label classifier C. Therefore, the recognition loss function for the source domain data is as follows:

[0095]

[0096] Among them, L y n represents the classification loss in the source domain; s This represents the number of samples in the source domain; s represents the source domain. Represents samples in the source domain; Indicates the category of the source domain; L CCS (.) represents the cross-entropy loss function.

[0097] By combining the unsupervised transfer learning benchmark method with the joint adversarial transfer learning algorithm proposed in this embodiment, the final discriminative features and adversarial learning transfer learning model can be obtained.

[0098] The objective function of the pest image recognition algorithm based on joint adversarial transfer learning is expressed as follows:

[0099] E(θ f θ y θ a θ c θ d ) = L y +α*L a +β*L c +μ*L d.

[0100] Where, E(θ) f θ y θ a θ c θ d ) represents about θ f θ y θ a θ c θ d The function, E, is the objective function symbol of the pest image recognition algorithm, representing the functional relationship; θ f θ represents the F-parameters of the feature extractor. y L represents the classification loss y Parameter; θ d The parameters of the discrimination loss of the neighborhood discriminator D; θ a θ represents the parameters of the adversarial loss of the neighborhood discriminator D. c Represents the label classifier C; L y L represents the classification loss in the source domain; a L represents the adversarial loss, which is the misclassification from the domain discriminator D; c Perceived loss at the center; L d This represents the discriminative loss; the hyperparameter α is used to control the adversarial loss L. a The hyperparameter β is used to control the center sensing loss L. c The hyperparameter μ is used to control the discrimination loss L. d The extent of the impact.

[0101] In this embodiment, these three parameters are set through experimental verification. The pest image recognition network with joint adversarial transfer learning optimizes the network parameters. and get:

[0102]

[0103]

[0104] in, This represents the relationship between the optimized feature extractor F parameters and θ. y The functional relationship between them;

[0105] This represents the D parameters of the optimized neighborhood discriminator.

[0106] By employing a joint optimization approach, the parameters of the feature extractor and the label recognizer are optimized to achieve the task of pest image recognition and improve the network's ability to recognize pest images.

[0107] Please see Figure 3This invention provides a schematic diagram of the training domain-testing process for an embodiment of a pest image recognition method based on joint adversarial transfer learning. The specific steps of the training process for this pest image recognition method based on joint adversarial transfer learning include:

[0108] Step 301: The training process organizes the data into the source domain (R), style domain (S), and blank domain (W);

[0109] Step 302: Set up 6 migration tasks: R→S, R→W, S→W, S→R, W→R, and W→S;

[0110] Step 303, the training domain-test domain includes all combinations of these three domains.

[0111] Step 304: During model inference, labeled source domain data and unlabeled source domain data are simultaneously input into two feature extractors, which share parameters.

[0112] Step 305: Each feature extractor generates a center-perception loss, and the feature extractors jointly generate an adversarial loss through a domain discriminator.

[0113] Step 306: Generate classification loss and jointly generate discriminative loss through the label classifier, wherein the label classifier shares parameters.

[0114] This invention verifies and tests the training process of data organization, and repeatedly updates the data in the model, thereby iterating the pest image recognition method based on joint adversarial transfer learning in the domain adversarial neural network model, ensuring the efficiency of data processing when dealing with complex scenarios.

[0115] The center perception constraint module and related alignment constraint module of the pest image recognition method based on joint adversarial transfer learning proposed in this embodiment are also applicable to models such as ResNet50 and DAN.

[0116] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0117] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A pest image recognition method based on joint adversarial transfer learning, characterized in that, Includes, includes, Step S1: Stylize the open-source agricultural and forestry pest dataset to form a transfer learning dataset; Step S2, the training domain-test domain includes all combinations of the three domains: source domain, style domain, and blank domain; Step S3: Introduce center-aware constraints into the domain adversarial neural network; Step S4: Introduce relevant alignment constraints in the final fully connected layer of the tag recognizer; In step S1, the transfer learning dataset is the Pest-TL transfer learning dataset for agricultural and forestry pests, and its main steps include: Collect open-source pest species datasets and delete non-adult data; The organized data is divided into blank background areas and normal background areas according to different backgrounds; Based on the DNN image transfer method, normal domain backgrounds are transferred into style domain images; In step S2, the specific steps of the training process include: The training process organizes the data into source domain, style domain, and blank domain; The six migration tasks are set up as follows: migration from source domain to style domain, migration from source domain to blank domain, migration from style domain to blank domain, migration from style domain to source domain, migration from blank domain to source domain, and migration from blank domain to style domain. The training domain and the test domain encompass all combinations of these three domains; During model inference, labeled source domain data and unlabeled source domain data are simultaneously input into two feature extractors, which share parameters. Each feature extractor generates a center-perception loss, which is then jointly processed by a domain discriminator to generate an adversarial loss. The label classifier generates a classification loss and a joint discriminant loss, with the label classifiers sharing parameters.

2. The pest image recognition method based on joint adversarial transfer learning according to claim 1, characterized in that, In step S3, the center-aware constraint is introduced, which is defined as follows: , in, As a sample, Perceive loss at the center The first characteristic represents the feature Each category center Represent real numbers, c represents the size of the class center, m represents the size of the mini-batch, and i is a variable.

3. The pest image recognition method based on joint adversarial transfer learning according to claim 2, characterized in that, In step S3, the center-aware loss is performed based on mini-batch updates when updating the centers of the entire training set.

4. The pest image recognition method based on joint adversarial transfer learning according to claim 3, characterized in that, In step S3, when updating the center perception loss, a scalar is used to control the learning rate of the center to avoid large perturbations caused by a few mislabeled samples.

5. The pest image recognition method based on joint adversarial transfer learning according to claim 1, characterized in that, In step S4, the relevant alignment constraint introduced, and the calculation formula for the relevant alignment of the output features of the label recognizer across the source and target domains, are as follows: , in, To determine the loss, F represents the feature extractor, and d represents the dimension of the feature. Denotes the Frobenius norm of a square matrix. The correlation matrix representing samples from the source domain. The correlation matrix of samples in the target domain.

6. The pest image recognition method based on joint adversarial transfer learning according to claim 1, characterized in that, In step S4, relevant alignment is used to constrain the covariance of source and target domain features to reduce domain discrepancies.

7. The pest image recognition method based on joint adversarial transfer learning according to claim 6, characterized in that, A joint optimization approach is adopted, that is, by using the min-max optimization method to optimize the parameters of the feature extractor and the label recognizer to train the source domain discriminative model.

8. The pest image recognition method based on joint adversarial transfer learning according to claim 7, characterized in that, Objective functions are established for the feature extractor and label recognizer, and corresponding hyperparameters are set to optimize the objective functions.

Images