A method, device, electronic device and storage medium for identifying harmful organisms

Abstract

The present invention relates to the field of image recognition technology, and discloses a method, device, equipment and storage medium for identifying harmful organisms, including obtaining an image to be identified; inputting the image to be identified into a pre-trained target identification model, determining whether the image to be identified is a harmful organism image, and identifying the image to be identified with a confidence level lower than a threshold as other categories, and classifying the image to be identified with a confidence level higher than a threshold into a corresponding category; the training process of the pre-trained target identification model includes taking the harmful insect classification data set as a closed set data set, performing closed set training on the initial classifier model, and obtaining an initial identification model; using synthetic images generated by a generative adversarial network and synthetic features generated by a feature replicator adversarial training network to simulate an open data set, and performing open set training on the initial identification model to obtain a target identification model. It is beneficial to improving the intelligent and accurate identification and rapid processing capabilities of common pests in imported wood.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device, equipment and storage medium for identifying harmful organisms. Background Art

[0002] Identification of imported pests is a vital link in maintaining domestic agricultural production and ecological security. Among them, the "List of Quarantine Pests of Plants Entering the People's Republic of China" has covered 446 pests, which may pose a threat to my country's agricultural production and ecological security.

[0003] At present, the identification of inbound pests mainly relies on manual identification by experts in the customs laboratory. Inbound pests include insects, mollusks, nematodes, weeds, etc., which are difficult to identify from multiple angles, in multiple forms, in different environments, and with subtle differences. Manual identification methods have problems such as a shortage of experts and samples, a long customs clearance cycle, subjective misjudgment, and large differences in identification levels at various ports.

[0004] Therefore, there is an urgent need to provide a pest identification and recognition method, device, equipment and storage medium, which will help improve the ability to intelligently and accurately identify and quickly handle common pests in imported wood. Summary of the invention

[0005] Based on this, it is necessary to provide a pest identification and recognition method, device, equipment and storage medium to address existing problems, which will help improve the ability to intelligently and accurately identify and quickly handle common pests in imported wood.

[0006] The first aspect of the present application provides a method for identifying harmful organisms, which comprises:

[0007] Obtain an image to be recognized;

[0008] Input the image to be identified into a pre-trained target identification model to determine whether the image to be identified is a harmful biological image, and identify the image to be identified with a confidence level lower than a threshold as other categories, and classify the image to be identified with a confidence level higher than the threshold into a corresponding category;

[0009] Among them, the training process of the pre-trained target identification and recognition model includes taking the harmful insect classification data set as a closed set data set, performing closed set training on the initial classifier model, and obtaining the initial identification and recognition model; using the synthetic image generated by the adversarial generative network and the synthetic features generated by the feature replicator adversarial training network to simulate the open data set, and performing open set training on the initial identification and recognition model to obtain the target identification and recognition model.

[0010] In some embodiments, before the harmful insect classification dataset is used as a closed set dataset and the initial classifier model is trained in a closed set to obtain an initial identification model, the method includes:

[0011] Preprocessing the collected pest specimen images to obtain a preprocessed image set adapted to network input, wherein the preprocessing includes at least one of size adjustment and normalization;

[0012] Based on the preprocessed image set, image rotation, scaling, and cropping are used to increase data diversity and obtain an enhanced image set;

[0013] The enhanced image set is classified and labeled to obtain the harmful insect classification dataset, and the harmful insect classification dataset is constructed as the closed set dataset.

[0014] In some embodiments, the feature replicator adversarial training network includes:

[0015] Feature Copier Network W ′ , the feature replicator network W ′ Used to generate virtual feature F i ′ ;

[0016] The first classifier network W is used to classify the virtual feature F i ′ Classification is performed to enhance the virtual feature F through adversarial generation training i ′ Open set classification performance;

[0017] The first classifier network W before adversarial generation training is obtained based on the initial identification model after closed set training, and the virtual feature is F i ′ Used to construct a first open subset, the synthetic image generated by the adversarial generative network and the first open subset constitute the open data set.

[0018] In some embodiments, the initial classifier model is constructed based on ResNet18, and the feature replicator network W ′ It is built with ResNet18 as the network backbone.

[0019] In some embodiments, the feature replicator network W ′ Used to generate virtual feature F i ′ include:

[0020] The feature replicator network W ′The outputs of the first residual block, the third residual block, and the fourth residual block are used as the virtual feature F i ′ , for training the first classifier network W;

[0021] Among them, the feature replicator network W ′ The outputs of the first residual block and the third residual block are corrected by the loss function to make them close to the real intermediate features of the corresponding network layer of the first classifier network W; and the feature replicator network W ′ The first virtual feature generated by the first residual block and the third residual block of the feature replicator network W' is used as the first easily distinguishable virtual feature, and the synthetic image generated by the adversarial generation network is used as the second easily distinguishable virtual feature. i ′ It includes the difficult-to-distinguish virtual feature and the first easily-distinguishable virtual feature.

[0022] In some embodiments, the feature replicator network W ′ The loss function is expressed as:

[0023] L copy =L reg +L imi

[0024] Among them, L reg is the cross entropy loss between the predicted label and the true label, L imi is the feature replicator network W ′ L1 loss of the first virtual features generated in the first residual block and the third residual block and the real intermediate features of the corresponding layer of the first classifier network W;

[0025] The loss function of the first classifier network W is expressed as:

[0026] L cla =L close +λ·L open

[0027] Among them, L close is the cross entropy loss between the predicted label and the true label in closed-set training, and λ is L open The weight, L open is the cross entropy loss between the predicted labels and the fitted labels on the open set training.

[0028] In some embodiments, for different types of open set data, the fitting label The fitting label is calculated in different ways. The calculation process includes:

[0029] For the first easily resolvable virtual feature and the second easily resolvable virtual feature, the fitting labels of the first easily resolvable virtual feature and the second easily resolvable virtual feature The calculation method is:

[0030]

[0031] Wherein, K is the number of categories of the closed set classification task, and u is a vector whose values ​​are all 1, that is, the loss function will correct the first classifier network W so that it predicts the generated first easily distinguishable virtual features and second easily distinguishable virtual features as average probabilities;

[0032] For the indistinguishable virtual feature, the fitting label of the indistinguishable virtual feature The calculation method is:

[0033]

[0034] Among them, α is the smoothing coefficient, y is the true label of the input image, K is the number of categories of the closed-set classification task, and u is a vector whose values ​​are all 1, that is, the loss function will correct the first classifier network W to predict the generated indistinguishable virtual features as a smoothed probability distribution.

[0035] The second aspect of the present application provides a device for identifying harmful organisms, comprising:

[0036] A pattern acquisition module, used to acquire the image to be identified;

[0037] An identification and analysis module, comprising a target identification and recognition model, wherein the target identification and recognition model is used to determine whether the image to be identified is a harmful biological image, and to identify the image to be identified with a confidence level lower than a threshold as other categories, and to classify the image to be identified with a confidence level higher than the threshold into a corresponding category;

[0038] Among them, the training process of the pre-trained target identification and recognition model includes taking the harmful insect classification data set as a closed set data set, performing closed set training on the initial classifier model, and obtaining the initial identification and recognition model; using the synthetic image generated by the adversarial generative network and the synthetic features generated by the feature replicator adversarial training network to simulate the open data set, and performing open set training on the initial identification and recognition model to obtain the target identification and recognition model.

[0039] According to a third aspect of the present application, an electronic device is provided, which includes at least one processor and a memory for storing processor-executable instructions, and the processor implements the method described in any of the above embodiments when executing the instructions.

[0040] A fourth aspect of the present application provides a storage medium, in which a computer program or computer instructions are stored. When the computer program or computer instructions are executed by a processor, the method described in any one of the above embodiments is implemented.

[0041] Beneficial effects of the present invention:

[0042] The pest identification and recognition method of the present invention inputs the image to be identified into a pre-trained target identification and recognition model to determine whether the image to be identified is a pest image, and identifies the image to be identified with a confidence level lower than a threshold as other categories, and classifies the image to be identified with a confidence level higher than the threshold into a corresponding category, which is beneficial to improving the intelligent and accurate identification and rapid processing capabilities of common pests of imported wood, ensuring the biological safety of ports, and preventing and controlling the invasion of foreign pests.

[0043] Due to the large number of imported pest species, it is difficult to collect enough pictures for each species to train a deep learning model. In an embodiment of the present invention, the training process of the pre-trained target identification and recognition model includes taking the harmful insect classification data set as a closed set data set, performing closed set training on the initial classifier model, and obtaining an initial identification and recognition model; using the synthetic image generated by the adversarial generative network and the synthetic features generated by the feature replicator adversarial training network to simulate the open data set, and performing open set training on the initial identification and recognition model to obtain the target identification and recognition model. Its target identification and recognition model has undergone closed set training and open set training. On the one hand, it can ensure the classification accuracy of known species, and on the other hand, it has the function of classifying pictures of unknown pest species into other categories.

[0044] In addition, it uses synthetic images generated by adversarial generative networks and synthetic features generated by feature replicator adversarial training networks to simulate open data sets for open set training, improving the difficulty of constructing open set data. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, drawings of other embodiments can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of a method for identifying harmful organisms provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of the structure of a harmful organism identification and recognition device provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0050] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0051] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0052] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0053] In the present invention, unless otherwise clearly specified and limited, a feature "above" or "below" a second feature may be in direct contact with the second feature, or in indirect contact with the second feature through an intermediate medium. Moreover, a feature "above", "above" and "above" a second feature may mean that the feature is directly above or obliquely above the second feature, or simply means that the feature is higher in level than the second feature. A feature "below", "below" and "below" a second feature may mean that the feature is directly below or obliquely below the second feature, or simply means that the feature is lower in level than the second feature.

[0054] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "upper", "lower", "left", "right" and similar expressions used herein are for illustrative purposes only and are not intended to be the only implementation method.

[0055] refer to Figure 1 The embodiment of the present application provides a method for identifying harmful organisms, which includes:

[0056] Step 100: Obtain an image to be identified;

[0057] Step 200: Input the image to be identified into a pre-trained target identification model to determine whether the image to be identified is a harmful organism image, and identify the image to be identified with a confidence level lower than a threshold as other categories, and classify the image to be identified with a confidence level higher than the threshold into a corresponding category;

[0058] Among them, the training process of the pre-trained target identification and recognition model includes step 010, taking the harmful insect classification data set as a closed set data set, performing closed set training on the initial classifier model, and obtaining an initial identification and recognition model; step 020, using the synthetic image generated by the adversarial generative network and the synthetic features generated by the feature replicator adversarial training network to simulate the open data set, and performing open set training on the initial identification and recognition model to obtain the target identification and recognition model.

[0059] The implementation manner of the present application inputs the image to be identified into a pre-trained target identification model to determine whether the image to be identified is a harmful biological image, and identifies the image to be identified with a confidence level lower than a threshold as other categories, and classifies the image to be identified with a confidence level higher than a threshold into a corresponding category. This is beneficial to improving the intelligent and accurate identification and rapid processing capabilities of common pests in imported wood, ensuring the biological safety of ports, and preventing and controlling the invasion of foreign harmful organisms.

[0060] In addition, due to the large number of imported pest species, it is difficult to collect enough images for each species to train a deep learning model. In an embodiment of the present invention, the training process of the pre-trained target identification model includes taking the harmful insect classification data set as a closed set data set, performing closed set training on the initial classifier model, and obtaining an initial identification model;

[0061] Furthermore, it uses synthetic images generated by the adversarial generative network and synthetic features generated by the feature replicator adversarial training network to simulate an open data set, and performs open set training on the initial identification model to obtain the target identification model. Its target identification model has been trained with closed set training and open set training, which can ensure the classification accuracy of known species on the one hand, and has the function of classifying unknown species of pests into other categories on the other hand.

[0062] In addition, it uses synthetic images generated by adversarial generative networks and synthetic features generated by feature replicator adversarial training networks to simulate open data sets for open set training, improving the difficulty of constructing open set data.

[0063] Explanation: Open set data and closed set data generally refer to the openness or closedness of the categories in the dataset. In closed set data, all the categories are pre-defined and the model is only expected to recognize these pre-defined categories during training and testing. Closed set learning is the traditional supervised learning setting where the model is provided with labeled data at training time and is expected to accurately classify these known categories at test time. A typical example of closed set data is an image classification task where the model is trained to recognize a specific set of animals such as cats, dogs, and birds and is only expected to recognize these animals at test time. Open set data refers to the fact that during the test time, the model may encounter some new categories that it has not seen during the training phase. In open set learning, the model needs to be able to recognize the categories in the training set and also be able to recognize and distinguish unknown categories. An example of open set learning is anomaly detection where the model is trained to recognize normal data and at test time needs to recognize any abnormal data that deviates from the normal pattern and may not have been seen during training.

[0064] Furthermore, in some embodiments, the threshold mentioned above may be 0.95 for comparison and classification.

[0065] In addition, adversarial generative networks can also be called GAN networks. Adversarial training usually involves two networks: a generator and a discriminator. The task of the generator is to generate data that is as realistic as possible, while the task of the discriminator is to distinguish between generated data and real data. The two networks compete with each other during the training process. The generator constantly learns how to generate more realistic data, while the discriminator constantly learns how to better distinguish between real and fake data.

[0066] In some embodiments, in step 010, the harmful insect classification dataset is used as a closed set dataset, and the initial classifier model is trained in a closed set to obtain an initial identification model, which includes:

[0067] Step 007: preprocessing the collected pest specimen images to obtain a preprocessed image set adapted to network input, wherein the preprocessing includes at least one of size adjustment and normalization;

[0068] Step 008: Based on the preprocessed image set, image rotation, scaling, and cropping are used to increase data diversity to obtain an enhanced image set;

[0069] Step 009: classify and label the enhanced image set to obtain the harmful insect classification dataset, and construct the harmful insect classification dataset as the closed set dataset.

[0070] Through the above steps, a closed set data set that meets the training requirements can be obtained. In addition, the specific division of the closed set data set will not be repeated, and the division can be performed according to the closed set training requirements.

[0071] The following is a detailed introduction to the feature replicator adversarial training network and the specific content of simulating open datasets using synthetic images generated by the adversarial generative network and synthetic features generated by the feature replicator adversarial training network in open set training.

[0072] In some embodiments, the feature replicator adversarial training network includes a feature replicator network W ′ and the first classifier network W. The feature replicator network W ′ Used to generate virtual feature F i ′ The first classifier network W is used to classify the virtual feature F of the input i ′ Classification is performed to enhance the virtual feature F through adversarial generation training i ′ The open set classification performance; wherein the first classifier network W before adversarial generation training is obtained based on the initial identification recognition model after closed set training, and the virtual feature is F i′ Used to construct a first open subset, the synthetic image generated by the adversarial generative network and the first open subset constitute the open data set.

[0073] The implementation method of the present application creatively constructs a feature replicator adversarial training network, and uses the synthetic image generated by the feature replicator adversarial training network and the first open subset to construct open set data, thereby solving the problem of difficulty in constructing open set data.

[0074] In addition, the general recognition and classification model performs poorly in the pest classification task. Therefore, the convolutional network (CNN) / attention mechanism (Transformer) network is first selected as the initial classifier model for training. The network structure includes advanced network structures such as ResNet18, Vit, and swin_v2. After experimental screening and verification, the superiority of ResNet18 and swin_v2 in the closed-set pest image recognition task; further, in some embodiments, the initial classifier model is constructed based on ResNet18, and the feature replicator network W ′ ResNet18 is used as the network backbone to solve the problem of poor performance of the model in pest classification tasks. The synthetic images generated by the adversarial generative network and the synthetic features generated by the feature replicator adversarial training network, namely the first open subset, are used to simulate the open set for training, thereby improving the accuracy and generalization ability of the pest recognition model on the closed set.

[0075] Specifically, in some embodiments, the feature replicator network W ′ Used to generate virtual feature F i ′ include:

[0076] The feature replicator network W ′ The outputs of the first residual block, the third residual block, and the fourth residual block are used as the virtual feature F i ′ , for training the first classifier network W;

[0077] Among them, the feature replicator network W ′ The outputs of the first residual block and the third residual block are corrected by the loss function to make them close to the real intermediate features of the corresponding network layer of the first classifier network W; and the feature replicator network W ′ The first virtual feature generated by the first residual block and the third residual block of the feature replicator network W' is used as the first easily distinguishable virtual feature, and the synthetic image generated by the adversarial generation network is used as the second easily distinguishable virtual feature. i′ It includes the difficult-to-distinguish virtual feature and the first easily-distinguishable virtual feature.

[0078] Illustratively, in some embodiments, the feature replicator network W ′ It generally includes a base network layer and a high-level layer. The first few layers of the base network are used as feature extractors, which can capture low-level features of the image, such as edges, textures, etc. Then, the high-level layers (such as the residual block of ResNet18) further extract and fuse features to form a more abstract feature representation. The feature replicator network W ′ The first virtual feature generated by the first residual block and the third residual block is used as a difficult-to-distinguish virtual feature; the feature replicator network W ′ The second virtual feature generated by the fourth residual block of is used as the first easily distinguishable virtual feature. Feature replicator network W ′ The hard-to-distinguish virtual features and the first easy-to-distinguish virtual features with different degrees of difficulty can be generated to form an open data set, which is beneficial to improving the open set recognition performance of the first classifier network W.

[0079] Explanatory, in some embodiments, a feature replicator network W′ is designed with the same structure as the first classifier network W, which is capable of generating virtual features that are similar to real features but have certain differences. Then, the parameters of the feature replicator network W′ are initialized, and training data is prepared, the training data including images of known categories and corresponding labels. The first classifier network W is then used to forward propagate the real image to extract intermediate features. The feature replicator network W′ receives the real image x and generates virtual features Fi′. These features are intended to simulate the intermediate features that are difficult to distinguish in the first classifier network W. Next, the feature replicator network W′ is subjected to adversarial training with the first classifier network W. The feature replicator network W′ attempts to generate virtual features that are increasingly difficult to distinguish by the first classifier network W, while the first classifier network W attempts to improve the recognition ability of these virtual features. In the adversarial generation training process, the feature replicator network W is then used to generate virtual features that are increasingly difficult to distinguish by the first classifier network W, while the first classifier network W attempts to improve the recognition ability of these virtual features. ′ The first virtual features generated by the first residual block and the third residual block are used as difficult-to-distinguish virtual features, and the feature replicator network W ′ The second virtual feature generated by the fourth residual block and the synthetic image generated by the adversarial generative network are used together as the easily distinguishable virtual feature. The easily distinguishable virtual feature may include the first easily distinguishable virtual feature and the second easily distinguishable virtual feature.

[0080] In the embodiment of the present application, the feature replicator adversarial training network is composed of the feature replicator network W ′ And the first classifier network W.

[0081] For a given image x and a first classifier network W, the first classifier network W can be viewed as a continuous multi-layer network (W1, W2, ..., W n , W c ), where W c is a fully connected layer and a softmax layer, which is used to output the final prediction probability, while W1,…,W n Represents several sets of continuous network structures. Therefore, the predicted probability P of the image can be expressed as:

[0082] P=W c *W n *…*W1(x)

[0083] Among them, we define the intermediate features generated by the image during the network inference process as F i :

[0084] F i =W i *W i-1 *…*W1(x)

[0085] The main task of the feature replicator network W′ is to generate virtual features of varying degrees of difficulty. W′ is denoted as the feature replicator network (its structure is the same as that of the classifier), W1 ′ , …, W n ′ , W c ′ is the network of each layer of the feature replicator network W′, and the virtual features generated by the network are F i ′ :

[0086] F′ i =W′ i *W′ i-1 *…*W′1(x)

[0087] The virtual features generated by the feature replicator network W′ are input into the corresponding network layer of the first classifier network W, so that the first classifier network W obtains training data of a simulated open set, thereby enhancing the open set classification performance of the classifier for intermediate features.

[0088] In some embodiments, the loss function of the feature replicator network W′ is expressed as:

[0089] L copy =L reg +L imi

[0090] Among them, L reg is the cross entropy loss between the predicted label and the true label, L imi is the feature replicator network W ′The L1 loss of the first virtual feature generated in the first residual block and the third residual block and the real intermediate feature of the corresponding layer of the first classifier network W; during the optimization process, optimization algorithms such as back propagation and gradient descent can be used to adjust the network weights to minimize the loss function.

[0091] The loss function of the first classifier network W is expressed as:

[0092] L cla =L close +λ·L open

[0093] Among them, L close is the cross entropy loss between the predicted label and the true label in closed-set training, and λ is L open The weight, L open is the cross entropy loss between the predicted labels and the fitted labels on the open set training. By designing the loss function to optimize the feature replicator network W ′ The generated virtual features and the recognition ability of the first classifier network W.

[0094] In addition, there are no objective labels in open set training, so fitting labels is required In some embodiments, for different types of open set data, the fitting label The fitting label is calculated in different ways. The calculation process includes:

[0095] For the first easily resolvable virtual feature and the second easily resolvable virtual feature, the fitting labels of the first easily resolvable virtual feature and the second easily resolvable virtual feature The calculation method is:

[0096]

[0097] Wherein, K is the number of categories of the closed set classification task, and u is a vector whose values ​​are all 1, that is, the loss function will correct the first classifier network W so that it predicts the generated first easily distinguishable virtual features and second easily distinguishable virtual features as average probabilities;

[0098] For the indistinguishable virtual feature, the fitting label of the indistinguishable virtual feature The calculation method is:

[0099]

[0100] Among them, α is the smoothing coefficient, y is the real label of the input image, K is the number of categories of the closed-set classification task, and u is a vector whose values ​​are all 1, that is, the loss function will correct the first classifier network W to predict the generated indistinguishable virtual features as a smoothed probability distribution. Specifically, in the experiment, α=0.5.

[0101] The implementation method of the present application realizes that in the process of constructing a model for identifying multiple pests in a list, while accurately identifying pests of known categories, unknown categories can be identified as others, thereby realizing the scalability and iterativeness of the method. In addition, the method of quantitative comparison and confusion matrix can be used to select the optimal identification model. The optimal model trained by the current optimal hyperparameters can achieve an accuracy rate of 98% on a closed set and an accuracy rate of 91% on an open set, and the optimal model has been verified through model training experiments.

[0102] The pest identification and recognition method provided in the implementation mode of the present application can, firstly, realize the accuracy and reliability of automatic identification of pests at the front-line ports, solve the problems of shortage of insect identification experts, insufficient manpower and samples, and difficulty of manual identification at customs, and improve the time efficiency of customs clearance and release of goods; secondly, the method has been verified by experiments to be accurate, scalable and iterative; furthermore, it is conducive to enabling the customs plant quarantine initial screening laboratory to scientifically and orderly accumulate image data of imported pests to construct an imported pest identification knowledge base, and has the ability to gradually cover 446 pests in the "List of Quarantine Pests of Plants Entering the People's Republic of China", build a new model of customs supervision, and gradually promote the construction of smart customs and a smart customs power for the realization of quarantine at the front-line ports.

[0103] The embodiment of the present application also provides a harmful organism identification and recognition device, see Figure 2 The embodiment of the present application is shown as a schematic diagram of a structure of a device for identifying and recognizing harmful organisms. The device for identifying and recognizing harmful organisms comprises:

[0104] A pattern acquisition module, used to acquire the image to be identified;

[0105] An identification and analysis module, comprising a target identification and recognition model, wherein the target identification and recognition model is used to determine whether the image to be identified is a harmful biological image, and to identify the image to be identified with a confidence level lower than a threshold as other categories, and to classify the image to be identified with a confidence level higher than the threshold into a corresponding category;

[0106] Among them, the training process of the pre-trained target identification and recognition model includes taking the harmful insect classification data set as a closed set data set, performing closed set training on the initial classifier model, and obtaining the initial identification and recognition model; using the synthetic image generated by the adversarial generative network and the synthetic features generated by the feature replicator adversarial training network to simulate the open data set, and performing open set training on the initial identification and recognition model to obtain the target identification and recognition model.

[0107] In addition, the pest identification and recognition device may also include a storage module, which is used to store data such as harmful insect classification data sets as a database for training or further optimizing the target identification and recognition model, which can be called an imported pest identification knowledge base.

[0108] It should be understood that the device corresponds to the above-mentioned harmful organism identification method embodiment and can perform the various steps involved in the above-mentioned method embodiment. The specific functions of the device can be found in the above description. To avoid repetition, the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or fixed in the operating system (OS) of the device.

[0109] An embodiment of the present application further provides an electronic device, which includes at least one processor and a memory for storing processor-executable instructions, and the processor implements the method described in any of the above embodiments when executing the instructions.

[0110] See also Figure 3 The electronic device provided by the embodiment of the present application is shown in the structural diagram. The electronic device provided by the embodiment of the present application includes a processor and a memory, the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions execute the above method when executed by the processor.

[0111] An embodiment of the present application further provides a storage medium, in which a computer program or computer instructions are stored. When the computer program or computer instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0112] Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0113] Finally, it should be noted that the various technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] The above-mentioned embodiment only expresses one implementation mode of the present invention, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the attached claims.

Claims

1. A method for identifying harmful organisms, characterized in that: include: Obtain an image to be recognized; Input the image to be identified into a pre-trained target identification model to determine whether the image to be identified is a harmful biological image, and identify the image to be identified with a confidence level lower than a threshold as other categories, and classify the image to be identified with a confidence level higher than the threshold into a corresponding category; The training process of the pre-trained target identification model includes taking the harmful insect classification data set as a closed set data set, performing closed set training on the initial classifier model, and obtaining the initial identification model; using the synthetic image generated by the adversarial generative network and the synthetic features generated by the feature replicator adversarial training network to simulate the open data set, and performing open set training on the initial identification model to obtain the target identification model; The feature replicator adversarial training network includes: Feature Copier Network W ′ , the feature replicator network W ′ Used to generate virtual feature F i ′ ; The first classifier network W is used to classify the virtual feature F i ′ Classification is performed to enhance the virtual feature F through adversarial generation training i ′ Open set classification performance; The first classifier network W before adversarial generation training is obtained based on the initial identification model after closed set training, and the virtual feature F i ′ Used to construct a first open subset, the synthetic image generated by the generative adversarial network and the first open subset constitute the open data set; The feature replicator network W ′ Used to generate virtual feature F i ′ include: The feature replicator network W ′ The outputs of the first residual block, the third residual block, and the fourth residual block are used as the virtual feature F i ′ , for training the first classifier network W; Among them, the feature replicator network W ′ The outputs of the first residual block and the third residual block are corrected by the loss function to make them close to the real intermediate features of the corresponding network layer of the first classifier network W; and the feature replicator network W ′ The first virtual feature generated by the first residual block and the third residual block of the feature replicator network W' is used as the first easily distinguishable virtual feature, and the synthetic image generated by the adversarial generation network is used as the second easily distinguishable virtual feature. i ′ It includes the difficult-to-distinguish virtual feature and the first easily-distinguishable virtual feature.

2. The method for identifying harmful organisms according to claim 1, characterized in that: In the method of using the harmful insect classification dataset as a closed set dataset, performing closed set training on the initial classifier model to obtain an initial identification model, the method includes: Preprocessing the collected pest specimen images to obtain a preprocessed image set adapted to network input, wherein the preprocessing includes at least one of size adjustment and normalization; Based on the preprocessed image set, image rotation, scaling, and cropping are used to increase data diversity and obtain an enhanced image set; The enhanced image set is classified and labeled to obtain the harmful insect classification dataset, and the harmful insect classification dataset is constructed as the closed set dataset.

3. The method for identifying harmful organisms according to claim 1, characterized in that: The initial classifier model is constructed based on ResNet18, and the feature replicator network W ′ It is built with ResNet18 as the network backbone.

4. The method for identifying harmful organisms according to claim 1, characterized in that: The feature replicator network W ′ The loss function is expressed as: L copy =L reg +L imi Among them, L reg is the cross entropy loss between the predicted label and the true label, L imi is the feature replicator network W ′ L1 loss of the first virtual features generated in the first residual block and the third residual block and the real intermediate features of the corresponding layer of the first classifier network W; The loss function of the first classifier network W is expressed as: L cla =L close +λ·L open Among them, L close is the cross entropy loss between the predicted label and the true label in closed-set training, and λ is L open The weight, L open is the cross entropy loss between the predicted labels and the fitted labels on the open set training.

5. The method for identifying harmful organisms according to claim 4, characterized in that: For different kinds of open set data, the fitting label The fitting label is calculated in different ways. The calculation process includes: For the first easily resolvable virtual feature and the second easily resolvable virtual feature, the fitting labels of the first easily resolvable virtual feature and the second easily resolvable virtual feature The calculation method is: Wherein, K is the number of categories of the closed set classification task, and u is a vector whose values ​​are all 1, that is, the loss function will correct the first classifier network W so that it predicts the generated first easily distinguishable virtual features and second easily distinguishable virtual features as average probabilities; For the indistinguishable virtual feature, the fitting label of the indistinguishable virtual feature The calculation method is: Among them, α is the smoothing coefficient, y is the true label of the input image, K is the number of categories of the closed-set classification task, and u is a vector whose values ​​are all 1, that is, the loss function will correct the first classifier network W to predict the generated indistinguishable virtual features as a smoothed probability distribution.

6. A harmful organism identification and recognition device, characterized in that: include: A pattern acquisition module, used to acquire the image to be identified; An identification and analysis module, comprising a target identification and recognition model, wherein the target identification and recognition model is used to determine whether the image to be identified is an image of a harmful organism, and to identify the image to be identified with a confidence level lower than a threshold as other categories, and to classify the image to be identified with a confidence level higher than the threshold into a corresponding category; The training process of the pre-trained target identification model includes taking the harmful insect classification data set as a closed set data set, performing closed set training on the initial classifier model, and obtaining the initial identification model; using the synthetic image generated by the adversarial generative network and the synthetic features generated by the feature replicator adversarial training network to simulate the open data set, and performing open set training on the initial identification model to obtain the target identification model; The feature replicator adversarial training network includes: Feature Copier Network W ′ , the feature replicator network W ′ Used to generate virtual feature F i ′ ; The first classifier network W is used to classify the virtual feature F i ′ Classification is performed to enhance the virtual feature F through adversarial generation training i ′ Open set classification performance; The first classifier network W before adversarial generation training is obtained based on the initial identification model after closed set training, and the virtual feature F i ′ Used to construct a first open subset, the synthetic image generated by the generative adversarial network and the first open subset constitute the open data set; The feature replicator network W ′ Used to generate virtual feature F i ′ include: The feature replicator network W ′ The outputs of the first residual block, the third residual block, and the fourth residual block are used as the virtual feature F i ′ , for training the first classifier network W; Among them, the feature replicator network W ′ The outputs of the first residual block and the third residual block are corrected by the loss function to make them close to the real intermediate features of the corresponding network layer of the first classifier network W; and the feature replicator network W ′ The first virtual feature generated by the first residual block and the third residual block of the feature replicator network W' is used as the first easily distinguishable virtual feature, and the synthetic image generated by the adversarial generation network is used as the second easily distinguishable virtual feature. i ′ It includes the difficult-to-distinguish virtual feature and the first easily-distinguishable virtual feature.

7. An electronic device, characterized in that: The invention comprises at least one processor and a memory for storing processor-executable instructions, wherein when the processor executes the instructions, the method according to any one of claims 1 to 5 is implemented.

8. A storage medium, characterized in that: The storage medium stores a computer program or a computer instruction. When the computer program or the computer instruction is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images