Pest image recognition method and device under sample scarcity condition
By generating adversarial networks and contrast learning modules to generate pseudo-visual features in pest recognition and aligning with semantic information, the problem of existing models insufficient recognition ability for unseen pests is solved, and stronger generalization ability and reduced labeling costs are achieved. effect.
Patent Information
- Application Number
- CN202410992246.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-26
AI Technical Summary
The existing supervised pest recognition model cannot identify pest categories that have not appeared in the training sample data, and its dependence on sample data leads to insufficient generalization ability and high labeling costs.
Generative adversarial networks are used to combine contrast learning modules and visual semantic alignment modules to build a training set by obtaining semantic information of visible and invisible pests. The generator generates pseudo-visual features and aligns with semantic information, and trains zero-sample classifiers to identify unsuccessful ones. See pests.
The model's ability to identify unseen pests has been improved, the labeling cost has been reduced, and the generalization ability has been improved, which is suitable for the identification and control of new pests in agriculture.
Smart Images

Figure CN119027718A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and in particular to a method and device for identifying pest images under a condition of scarce samples. Background Art
[0002] Pest identification has always been one of the key points of precision crop prevention and control, which is directly related to the yield and quality of crops. Due to the complex structure and small size of pests, many pests have extremely high similarities in appearance, making it a very difficult task to accurately identify the pest species.
[0003] At present, pest identification methods based on deep learning have made great progress in efficiency and accuracy, but they generally use supervised learning methods and rely heavily on a large amount of training sample data that needs to be carefully labeled. This also makes traditional supervised pest identification models unable to identify pest categories that have not appeared in the training sample data.
[0004] At present, there is an urgent need for a pest image recognition method and device under the condition of scarce samples to solve the problems existing in the related technology. Summary of the invention
[0005] The present invention provides a pest image recognition method and device under the condition of scarce samples, which are used to solve the problem that the traditional supervised pest recognition model cannot recognize the pest categories that have not appeared in the training sample data.
[0006] The present invention provides a pest image recognition method under the condition of sample scarcity, the method comprising: Obtain image data and semantic information of visible pests and semantic information of invisible pests to construct a training set; The training set is used to train a generator in a generative adversarial network to obtain a trained generator, wherein a contrast learning module and a visual semantic alignment module are added to the generative adversarial network; Using the trained generator to train a zero-shot classifier to obtain a trained zero-shot classifier; Apply the trained zero-shot classifier to identify the pest species of invisible pests or visible pests.
[0007] According to the pest image recognition method under the condition of sample scarcity provided by the present invention, the generator in the generative adversarial network is trained using the training set to obtain the trained generator, including: The semantic information and random noise of each type of pest are input into the generator to obtain pseudo visual features; splicing the pseudo visual features and the semantic information of each type of pests to obtain a first input of a discriminator in a generative adversarial network; splicing the visual features of visible pests and the semantic information of visible pests to obtain a second input of a discriminator in a generative adversarial network, wherein the visual features of the visible pests are obtained according to corresponding image data; and the discriminator outputs a first loss function; The semantic information, visual features and pseudo-visual features of each type of pest are input into the visual semantic alignment module to obtain the second loss function; Input the visual features and pseudo-visual features into the contrastive learning module to obtain the third loss function; According to the first loss function, the second loss function and the third loss function, the generative adversarial network is optimized to obtain a trained generator.
[0008] According to the pest image recognition method under the condition of sample scarcity provided by the present invention, the acquisition of semantic information of invisible pests and visible pests includes: For invisible pests and visible pests, the word vector corresponding to each type of pest is obtained according to the textual descriptive corpus to encode the semantic information of each type of pest.
[0009] According to the pest image recognition method under the condition of sample scarcity provided by the present invention, the acquisition of semantic information of invisible pests and visible pests includes: For visible pests, image data of visible pests are labeled according to preset pest feature definitions to obtain attribute vectors of each visible pest to construct semantic information of each visible pest.
[0010] According to the pest image recognition method under sample scarcity conditions provided by the present invention, the discriminator is used to identify real visual features and generated pseudo visual features in the input, and the first loss function is used to adjust the parameters of the discriminator.
[0011] According to the pest image recognition method under sample scarcity conditions provided by the present invention, the generator is used to generate pseudo visual features similar to real visual features, and the second loss function and the third loss function are used to adjust the parameters of the generator.
[0012] The present invention provides a pest image recognition device under the condition of sample scarcity, the device comprising: A data acquisition unit, used to acquire image data and semantic information of visible pests and semantic information of invisible pests to construct a training set; A first training unit is used to train a generator in a generative adversarial network using the training set to obtain a trained generator, wherein a contrastive learning module and a visual semantic alignment module are added to the generative adversarial network; A second training unit is used to train the zero-shot classifier using the trained generator to obtain a trained zero-shot classifier; The model application unit is used to apply the trained zero-shot classifier to identify the pest types of invisible pests or visible pests.
[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described methods when executing the program.
[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any of the methods described above is implemented.
[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, any of the above-described methods is implemented.
[0016] The present invention provides a pest image recognition method and device under the condition of scarce samples, which obtains semantic information of invisible pests and visible pests to construct a training set, and introduces zero-sample learning into the field of pest recognition to solve the problem that the existing model cannot recognize pest species that do not appear in the training set. Based on the generative adversarial network framework, a contrastive learning module is introduced to encourage the generator to generate more discriminative visual features, thereby improving the distinguishing ability of the generated features. A visual-semantic alignment module is added to align the generated visual features with predefined semantic attribute knowledge, constraining the generator to generate feature representations that are more consistent with the semantic description, thereby improving the quality of the generated features. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 This is one of the flow charts of a pest image recognition method under a sample scarcity condition provided by the present invention.
[0019] Figure 2 It is a schematic diagram of the principle of labeling image data of visible pests provided by the present invention.
[0020] Figure 3 It is a schematic diagram of the principle of a generative adversarial network provided by the present invention.
[0021] Figure 4 This is the second flow chart of a pest image recognition method under sample scarcity conditions provided by the present invention.
[0022] Figure 5 It is a structural schematic diagram of a visual semantic alignment module provided by the present invention.
[0023] Figure 6 It is a structural schematic diagram of a comparative learning module provided by the present invention.
[0024] Figure 7 The present invention provides a schematic diagram of the principle of training a zero-sample classifier.
[0025] Figure 8 It is a schematic diagram of the principle of the trained zero-sample classifier provided by the present invention.
[0026] Fig. 9 It is a structural schematic diagram of a pest image recognition device under a sample-scarce condition provided by the present invention.
[0027] Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0029] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchanged where appropriate, so that the embodiments can be implemented in a sequence other than that illustrated or described in the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps that appear in the present application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.
[0030] At present, pest identification methods based on deep learning have made great progress in efficiency and accuracy, but they generally adopt the supervised learning paradigm and rely heavily on a large amount of carefully labeled training sample data. The core idea of this method is to learn visual feature patterns from limited labeled samples and then generalize them to new test samples for identification and classification.
[0031] However, this approach has some serious flaws in practice. First, supervised learning has a huge demand for training data. Taking the field of pests as an example, there are millions of known pest species in the world, and it is almost impossible to build a training set containing all categories. Even if a lot of manpower and material resources are invested, it is difficult to cover new species that may appear in the future, which seriously limits the generalization ability of the model. Secondly, the workload of labeling training data is very large. Due to the wide variety of pests and the subtle differences in individual morphology, professionals are required to label them one by one, which consumes a lot of manpower and material costs and is inefficient. Third, the supervised learning model can only recognize categories that have appeared in the training set during the testing phase. Once a category outside the training set is encountered, the recognition accuracy of the model will drop sharply due to the mismatch of feature patterns, and it is difficult to meet the actual needs of agricultural production.
[0032] Therefore, the present invention provides a pest image recognition method under the condition of scarce samples, which can be based on limited known category training data and use external semantic knowledge to infer and identify categories that do not appear in the training set, thereby having stronger generalization ability. This can not only reduce the annotation cost, but more importantly, it can effectively respond to new pests that may appear at any time in agricultural production, providing a prerequisite for precise prevention and control.
[0033] Figure 1 One of the flowchart diagrams of a pest image recognition method under a sample-scarce condition provided by the present invention includes steps S101-S104.
[0034] Step S101, obtaining image data and semantic information of visible pests and semantic information of invisible pests to construct a training set.
[0035] Specifically, various types of pest images are collected in advance, and encyclopedic description information of various types of pests is collected at the same time to obtain a pest data set. The encyclopedic description information is used to generate subsequent semantic information of invisible pests and visible pests to construct a training set. In an embodiment of the present invention, during the collection stage of pest images and encyclopedic description information, pests are divided into invisible pests and visible pests. Among them, there is no image data for invisible pests, and there is image data for visible pests.
[0036] In the present invention, the method for acquiring the semantic information of invisible pests and visible pests is as described in the following embodiments.
[0037] In a possible implementation, in step S101, obtaining semantic information of invisible pests and visible pests specifically includes: For invisible pests and visible pests, the word vector corresponding to each type of pest is obtained according to the textual descriptive corpus to encode the semantic information of each type of pest.
[0038] In one embodiment of the present invention, text descriptive corpus related to target pest species is retrieved and collected in advance from literature resources such as agricultural encyclopedias and insect books. These corpus include descriptions of morphological characteristics of pests, introductions to living habits, prevention and control methods, etc. The collected original corpus is segmented, the continuous text in the original corpus is segmented into word sequences, and stop words in the corpus are removed. The processed corpus data set is sorted out.
[0039] Subsequently, the word vector model Word2Vec is trained. The processed corpus dataset is converted into an acceptable text format for input into the model. The relevant hyperparameters of the word vector model Word2Vec are set, such as the word vector dimension, window size, number of training iterations, etc. The word vector model is trained on the processed corpus dataset so that the word vector model Word2Vec learns the semantic relationship and contextual information between different words.
[0040] After training is completed, the word vector model Word2Vec will generate a fixed-dimensional word vector representation for each category term to encode the semantic information of the type of pest.
[0041] The above embodiment can be applied to the process of acquiring semantic information of invisible pests and visible pests.
[0042] In a possible implementation, in step S101, semantic information of invisible pests and visible pests is obtained, specifically including: for visible pests, image data of visible pests can also be labeled according to preset pest feature definitions to obtain attribute vectors of each visible pest to construct semantic information of each visible pest.
[0043] In another embodiment of the present invention, Figure 2 As shown in the figure, starting from the antennae, back, tail, feet, wings and overall body of the pest, attributes are defined in terms of color, pattern, size, shape, etc., and then each picture in the dataset is labeled to form a 65-dimensional attribute vector for each type of pest. Finally, the attribute vectors of all pest categories are put together to form an N×65-dimensional semantic attribute matrix (where N is the number of pest categories), as shown in the figure. Figure 2 As shown. Each row of the matrix corresponds to a category, and each column contains a value corresponding to an attribute. The value represents the proportion of the attribute in a given class, that is, the proportion of the number of images with the attribute in the class to the total number of images in the class. Through the above method, the attribute vector of each visible class of pests is obtained, and the attribute vector is further used as the semantic information of each visible class of pests.
[0044] Step S102, using the training set to train the generator in the generative adversarial network to obtain a trained generator, wherein a contrast learning module and a visual semantic alignment module are added to the generative adversarial network.
[0045] In an embodiment of the present invention, a schematic diagram of the principle of generating an adversarial network is exemplarily given, such as Figure 3 As shown in Figure 1, a contrastive learning module and a visual semantic alignment module are added to the generative adversarial network.
[0046] Figure 4 The second flowchart of the pest image recognition method under the condition of scarce samples provided by the present invention.
[0047] In a possible implementation, Figure 4 As shown, the generator in the generative adversarial network is trained using a training set to obtain a trained generator, including steps S401-S405.
[0048] Step S401, inputting semantic information and random noise of each type of pest into a generator to obtain pseudo visual features.
[0049] Step S402, concatenating the pseudo visual features and the semantic information of each type of pests to obtain a first input of the discriminator in the generative adversarial network; concatenating the visual features of visible pests and the semantic information of visible pests to obtain a second input of the discriminator in the generative adversarial network, wherein the visual features of visible pests are obtained based on the corresponding image data; and the discriminator outputs a first loss function.
[0050] In a possible implementation, the discriminator is used to identify real visual features and generated pseudo visual features in the input, and the first loss function is used to adjust parameters of the discriminator.
[0051] In steps S401-S402, if Figure 3 As shown, the generator G takes semantic information and random noise as input to generate pseudo visual features , then Will and real samples (i.e., visual features), concatenated with the corresponding semantics and input into the discriminator D.
[0052] Specifically, the last layer of the discriminator is a sigmoid function, which is used to determine whether the input features are real or synthetic. The generator G generates real visual features (i.e., pseudo visual features) as much as possible. ) to deceive the discriminator D, while D tries to accurately distinguish the real visual features from the generated pseudo visual features, and the two are constantly optimized in this way.
[0053] In an embodiment of the present invention, the first loss function output by the discriminator D is The definition is as follows:
[0054] in, For visual features, is a pseudo-visual feature, is the penalty coefficient, is the semantic matrix, for The corresponding class, for The corresponding semantic attributes, represents the output probability of the real visual feature, For the corresponding expectations.
[0055] Step S403: inputting the semantic information, visual features and pseudo-visual features of each type of pest into a visual semantic alignment module to obtain a second loss function.
[0056] Specifically, if Figure 5 As shown, by constructing semantic embedding space and visual embedding space , mapping visual features and pseudo-visual features into visual embedding space, and mapping semantic information into semantic embedding space Given a batch of N (image, semantic) pairs in a sample batch, to maximize the cosine similarity of the visual and semantic embeddings of the N positive sample pairs in the batch, while minimizing the cosine similarity of the embeddings of (N^2-N) negative sample pairs, Figure 5 The sample pairs on the diagonal are positive sample pairs, while those outside the diagonal are negative sample pairs. Both the visual embedding and the semantic embedding contain only one linear mapping layer, which maps the visual feature vector and the semantic vector into the same dimension. During training, for each sample pair ( , ), pass them through the linear mapping layer respectively, and the output is ( , ), and The training process contains two parts of loss, one is the visual to semantic alignment loss, and the other is the semantic to visual alignment loss. Therefore, for the i-th pair of samples, its loss function is As shown below:
[0057] is the loss of the i-th pair of samples; represents cosine similarity; is the temperature hyperparameter; k is the sample subscript. For N samples in a batch of samples, the total alignment loss function is , that is, the second loss function Then:
[0058] Step S404: input the visual features and the pseudo-visual features into a contrastive learning module to obtain a third loss function.
[0059] Specifically, if Figure 3 As shown in Figure 1, the contrastive learning module encourages the generator G to generate more discriminative pseudo visual features from two aspects: instance-based contrast and class-level contrast. The instance-based contrast starts from the pest instance and compares the image features of each pest, while the class-level contrast starts from the pest category and compares the semantic attributes of each type of pest.
[0060] In the present invention, Figure 6 As shown in the figure, due to the small size of pests, some pests have similar sizes, colors, shapes and semantic descriptions, but some are very different. The core idea of contrastive learning is to make two similar positive samples as close as possible and to separate two dissimilar negative samples as far as possible. Therefore, introducing contrastive learning into pest zero-shot recognition can enable the generative network G to generate more discriminative visual features, thus achieving better classification results. In a batch, for each sample ,make As close to the positive sample as possible , and stay away from all negative samples ,in is the visual feature or synthesized visual feature of the i-th category. and have the same category, It belongs to other categories except the i-th category. A large number of negative samples can enable the generator G to capture strong discriminative information in different categories, thus making the generation effect better. The third loss function as follows:
[0061] Among them, k is the total number of negative samples, is the temperature hyperparameter, greater than 0. By sample Through a non-linear projection head Mapping obtained.
[0062] In a possible implementation, the generator is used to generate pseudo visual features similar to real visual features, and the second loss function and the third loss function are used to adjust parameters of the generator.
[0063] Step S405, optimizing the generative adversarial network according to the first loss function, the second loss function and the third loss function to obtain a trained generator.
[0064] Specifically, the first loss function is used to call the discriminator D, the second loss function and the third loss function are used to optimize the generator G. After the generative adversarial network competes with each other and is optimized, the trained generator is obtained.
[0065] Exemplarily, the initial parameters of the generative adversarial network before training are: Both the generator G and the discriminator D contain a hidden layer of 4096 units with LeakyReLU activation. The optimizer uses the Adam optimizer, the learning rate is 0.00001, and the number of iterations is 1000. In the contrastive learning module, the dimension of the output h of the nonlinear projection F is set to 512. The output dimensions of the visual embedding and the semantic embedding in the visual-semantic alignment module are both 512. The zero-shot classifier consists only of the input layer, the output layer, and the Softmax layer. The learning rate is 0.001 and the number of iterations is 100. The random noise dimension and the semantic vector dimension are consistent, both of which are 65 dimensions.
[0066]
[0067] in, is the value of the i-th dimension in the vector, is the probability corresponding to the i-th category.
[0068] In an embodiment of the present invention, the experimental environment is based on the Ubuntu 20.04.5 LTS operating system, the hardware is NVIDIARTX 3090 GPU, the memory is 64G, the CUDA version is 11.7, and the Python 3.8 programming language and Pytorch 2.0.1 deep learning framework are used.
[0069] Step S103, using the trained generator to train the zero-shot classifier to obtain a trained zero-shot classifier.
[0070] Specifically, if Figure 7 As shown, the semantic encoding of invisible pests and random noise are used as input, the trained generator is used to generate labeled pseudo visual features, and the zero-shot classifier is trained based on these labeled pseudo visual features to obtain the trained zero-shot classifier; Step S104: using the trained zero-sample classifier to identify the types of invisible pests or visible pests.
[0071] Specifically, if Figure 8 As shown, the visual features of the pest images obtained during the application phase of invisible pests or visible pests are input into the trained zero-shot classifier to identify the pest types of the invisible pests or visible pests.
[0072] In the embodiment of the present invention, the evaluation index of the trained zero-sample classifier specifically includes the following contents: In the zero-shot setting, the evaluation index is the average Top-1 index of all categories. This index refers to the ratio of all correctly predicted samples to the total number of test set samples:
[0073] in, is the Top-1 accuracy, is the total number of test set samples, is the indicator function, is the true category of the sample, is the sample prediction category. When the trained zero-shot classifier is used to identify pest images, a correct identification is 1 and an incorrect identification is 0.
[0074] In the generalized zero-shot setting, the Top-1 accuracy of the model on the seen and unseen classes is calculated separately, and the overall performance is measured by their harmonic mean H:
[0075] Where S is the Top-1 accuracy on the visible class; U is the Top-1 accuracy on the invisible class.
[0076] In summary, the zero-shot classifier trained in the present invention is targeted at the task of zero-shot identification of pests, that is, invisible pests. By improving the generative adversarial network framework, introducing a contrastive learning module to improve the discriminability of generated features, and adding a visual-semantic alignment module to constrain the consistency of generated features with semantic information. The problem that the existing model cannot identify insect species that do not appear in the training set is solved. As shown in Table 1, the experimental results show that when evaluated on a dataset of 20 types of collected pests, a recognition accuracy of 77.4% was achieved under a zero-shot setting, and a reconciliation accuracy of 78.3% was achieved under a generalized zero-shot setting. The best performance was also achieved on a public dataset.
[0077] By adopting the above method, the present invention effectively improves the generalization ability of the existing pest detection model, provides technical support for the timely discovery and prevention of pests, is of great significance to maintaining the health of crops and ensuring food security, and can be widely used in agricultural practice.
[0078] Table 1
[0079] The present invention can achieve the following beneficial effects by adopting the above method: 1. By obtaining semantic information of invisible and visible pests to construct a training set, and by introducing zero-shot learning into the field of pest identification, the problem that the existing model cannot identify pest species that do not appear in the training set is solved.
[0080] 2. Based on the generative adversarial network framework, a contrastive learning module is introduced to encourage the generator to produce more discriminative visual features, thereby improving the distinguishing ability of the generated features.
[0081] 3. Add a visual-semantic alignment module to align the generated visual features with predefined semantic attribute knowledge, constrain the generator to generate feature representations that are more consistent with the semantic description, and improve the quality of the generated features.
[0082] The following is a description of a pest image recognition device under sample-scarce conditions provided by the present invention. The pest image recognition device under sample-scarce conditions described below and the pest image recognition method under sample-scarce conditions described above can be referenced to each other, and the corresponding processes in the aforementioned method embodiments can be referred to, and the specific contents will not be repeated here.
[0083] The present invention provides a pest image recognition device under the condition of scarce samples, the device comprising: A data acquisition unit 910 is used to acquire image data and semantic information of visible pests and semantic information of invisible pests to construct a training set; A first training unit 920 is used to train a generator in a generative adversarial network using a training set to obtain a trained generator, wherein a contrast learning module and a visual semantic alignment module are added to the generative adversarial network; A second training unit 930 is used to train the zero-shot classifier using the trained generator to obtain a trained zero-shot classifier; The model application unit 940 is used to apply the trained zero-shot classifier to identify the types of invisible pests or visible pests.
[0084] In a possible implementation, the first training unit 920 includes: A feature generation subunit is used to input the semantic information and random noise of each type of pest into the generator to obtain pseudo visual features; The first training subunit is used to splice the pseudo visual features and the semantic information of each type of pests to obtain a first input of the discriminator in the generative adversarial network; splice the visual features of visible pests and the semantic information of visible pests to obtain a second input of the discriminator in the generative adversarial network, wherein the visual features of visible pests are obtained according to corresponding image data; and the discriminator outputs a first loss function; A second training subunit is used to input the semantic information, visual features and pseudo-visual features of each type of pest into the visual semantic alignment module to obtain a second loss function; A third training subunit, used for inputting the visual features and the pseudo-visual features into the contrastive learning module to obtain a third loss function; The model optimization subunit is used to optimize the generative adversarial network according to the first loss function, the second loss function and the third loss function to obtain a trained generator.
[0085] In a possible implementation, the data acquisition unit 910 includes: The first data acquisition subunit is used to obtain the word vector corresponding to each type of pests according to the text descriptive corpus for the invisible pests and the visible pests, so as to encode the semantic information of each type of pests.
[0086] In a possible implementation, the data acquisition unit 910 further includes: The second data acquisition subunit annotates the image data of the visible pests according to a preset pest feature definition, obtains an attribute vector of each visible pest, and constructs semantic information of each visible pest.
[0087] In a possible implementation, the discriminator is used to identify real visual features and generated pseudo visual features in the input, and the first loss function is used to adjust parameters of the discriminator.
[0088] In a possible implementation, the generator is used to generate pseudo visual features similar to real visual features, and the second loss function and the third loss function are used to adjust parameters of the generator.
[0089] Fig.10 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.10As shown, the electronic device may include: a processor 1010, a communications interface 1020, a memory 1030 and a communication bus 1040, wherein the processor 1010, the communications interface 1020 and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute a method for identifying pest images under a sample scarcity condition, the method comprising: Obtain image data and semantic information of visible pests and semantic information of invisible pests to construct a training set; The training set is used to train the generator in the generative adversarial network to obtain a trained generator, in which a contrastive learning module and a visual semantic alignment module are added; The trained generator is used to train the zero-shot classifier to obtain a trained zero-shot classifier; Apply the trained zero-shot classifier to identify the pest species of invisible pests or visible pests.
[0090] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0091] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a pest image recognition method under a sample-scarce condition provided by the above methods, and the method includes: Obtain image data and semantic information of visible pests and semantic information of invisible pests to construct a training set; The training set is used to train the generator in the generative adversarial network to obtain a trained generator, in which a contrastive learning module and a visual semantic alignment module are added; The trained generator is used to train the zero-shot classifier to obtain a trained zero-shot classifier; Apply the trained zero-shot classifier to identify the pest species of invisible pests or visible pests.
[0092] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform a pest image recognition method under a sample scarcity condition provided by the above methods, the method comprising: Obtain image data and semantic information of visible pests and semantic information of invisible pests to construct a training set; The training set is used to train the generator in the generative adversarial network to obtain a trained generator, in which a contrastive learning module and a visual semantic alignment module are added; The trained generator is used to train the zero-shot classifier to obtain a trained zero-shot classifier; Apply the trained zero-shot classifier to identify the pest species of invisible pests or visible pests.
[0093] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.
[0094] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.
[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A pest image recognition method under the condition of sample scarcity, characterized in that: The method comprises: Obtain image data and semantic information of visible pests and semantic information of invisible pests to construct a training set; The training set is used to train a generator in a generative adversarial network to obtain a trained generator, wherein a contrast learning module and a visual semantic alignment module are added to the generative adversarial network; Using the trained generator to train a zero-shot classifier to obtain a trained zero-shot classifier; Apply the trained zero-shot classifier to identify the pest species of invisible pests or visible pests.
2. The pest image recognition method under the condition of sample scarcity according to claim 1, characterized in that: The training set is used to train the generator in the generative adversarial network to obtain a trained generator, including: The semantic information and random noise of each type of pest are input into the generator to obtain pseudo visual features; splicing the pseudo visual features and the semantic information of each type of pests to obtain a first input of a discriminator in a generative adversarial network; splicing the visual features of visible pests and the semantic information of visible pests to obtain a second input of a discriminator in a generative adversarial network, wherein the visual features of the visible pests are obtained according to corresponding image data; and the discriminator outputs a first loss function; The semantic information, visual features and pseudo-visual features of each type of pest are input into the visual semantic alignment module to obtain the second loss function; Input the visual features and pseudo-visual features into the contrastive learning module to obtain the third loss function; According to the first loss function, the second loss function and the third loss function, the generative adversarial network is optimized to obtain a trained generator.
3. The pest image recognition method under the condition of sample scarcity according to claim 1, characterized in that: Obtain semantic information of invisible pests and visible pests, including: For invisible pests and visible pests, the word vector corresponding to each type of pest is obtained according to the textual descriptive corpus to encode the semantic information of each type of pest.
4. The pest image recognition method under the condition of sample scarcity according to claim 1, characterized in that: Obtain semantic information of invisible pests and visible pests, including: For visible pests, image data of visible pests are labeled according to preset pest feature definitions to obtain attribute vectors of each visible pest to construct semantic information of each visible pest.
5. The pest image recognition method under the condition of sample scarcity according to claim 2, characterized in that: The discriminator is used to identify real visual features and generated pseudo visual features in the input, and the first loss function is used to adjust the parameters of the discriminator.
6. The pest image recognition method under the condition of sample scarcity according to claim 2, characterized in that: The generator is used to generate pseudo visual features similar to real visual features, and the second loss function and the third loss function are used to adjust the parameters of the generator.
7. A pest image recognition device under the condition of sample scarcity, characterized in that: The device comprises: A data acquisition unit, used to acquire image data and semantic information of visible pests and semantic information of invisible pests to construct a training set; A first training unit is used to train a generator in a generative adversarial network using the training set to obtain a trained generator, wherein a contrastive learning module and a visual semantic alignment module are added to the generative adversarial network; A second training unit is used to train the zero-shot classifier using the trained generator to obtain a trained zero-shot classifier; The model application unit is used to apply the trained zero-shot classifier to identify the pest types of invisible pests or visible pests.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
