Pest monitoring method and apparatus, device, and readable medium

Abstract

A pest monitoring method and apparatus, a device, and a readable medium. The method comprises: collecting an insect image and performing feature extraction on the insect image to obtain a first feature image (S101); and inputting the first feature image into a pre-trained classification model, and classifying and counting insects in the first feature image, to determine the types of insects in the insect image and corresponding quantities (S102).

Description

Insect monitoring method, device, equipment and readable medium

[0001] Cross-references

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on November 27, 2023, with application number 202311599315.8, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of image processing technology, and in particular to an insect monitoring method, device, equipment and readable medium. Background Art

[0004] Pest prevention and control are widely used in traditional Chinese medicine, grain storage, tobacco, food processing, forest pest control, crop cultivation, and other industries. Effective pest prevention methods can guide scientific chemical control in the field, reduce pesticide usage, thereby reducing production costs and significantly improving product quality.

[0005] Related technologies combine traditional pest traps with IoT information technology to monitor and biologically control insects encountered in agricultural and forestry production. However, traditional deep learning methods are often trained and deployed only on closed sets of insect images, and are unable to robustly respond to test data beyond the training data, nor can they identify new, unknown insect species.

[0006] Therefore, how to provide a technical solution that can accurately and effectively monitor insect infestations is a technical problem that needs to be urgently solved by those skilled in the art.

[0007] Summary of the Invention

[0008] One purpose of the present application is to provide an insect monitoring method, device, equipment and readable medium.

[0009] To achieve the above objectives, some embodiments of the present application provide the following aspects:

[0010] In a first aspect, some embodiments of the present application further provide a method for monitoring insect infestation, the method comprising:

[0011] Collecting an insect image and performing feature extraction on the insect image to obtain a first feature image;

[0012] The first feature image is input into a pre-trained classification model, the insects in the first feature image are classified and the number is counted, and the types of insects in the insect image and the corresponding number are determined.

[0013] In a second aspect, some embodiments of the present application further provide an insect monitoring device, the device comprising:

[0014] a feature extraction module, configured to collect an insect image and perform feature extraction on the insect image to obtain a first feature image;

[0015] The species recognition module is used to input the first feature image into a pre-trained classification model, classify the insects in the first feature image and count the number, and determine the insect species and corresponding number in the insect image.

[0016] In a third aspect, some embodiments of the present application further provide a computer device, comprising:

[0017] one or more processors; and

[0018] A memory storing computer program instructions, which, when executed, cause the processor to perform the insect monitoring method as described above.

[0019] In a fourth aspect, some embodiments of the present application further provide a computer-readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the insect monitoring method as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG1 is a flow chart of an insect monitoring method according to an embodiment of the present invention;

[0021] FIG2 is a flow chart of a training method based on a teacher-student network according to a second embodiment of the present application;

[0022] FIG3 is a flow chart of a training method based on a semi-supervised clustering network provided in Example 3 of the present application;

[0023] FIG4 is a schematic structural diagram of an insect monitoring device provided in Example 4 of the present application;

[0024] FIG5 is a schematic diagram of the structure of a computer device provided in Example 5 of the present application. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] The technical solutions provided by the embodiments of the present application are described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0027] Example 1

[0028] FIG1 is a flow chart of the insect monitoring method provided in Example 1 of the present application. As shown in FIG1 , the method specifically includes the following steps:

[0029] Step S101 : collecting an insect image and performing feature extraction on the insect image to obtain a first feature image.

[0030] The insect image may be an image captured by capturing insects trapped by an insect presence visual collection device. Specifically, the insects may be first attracted and collected using an intelligent insect presence detection light, and then captured using a high-resolution CCD camera to obtain the insect image. It should be noted that the number of insects in the insect image may be at least one, and the number of insect species may be at least one.

[0031] In this solution, features of insect images can be extracted using a deep model. The deep model can be a convolutional neural network. Specifically, the convolutional neural network may include convolutional layers, pooling layers, and fully connected layers. The insect image is input into the convolutional neural network. The convolutional layers extract local features from the insect image while retaining the spatial location information of the features, resulting in a convolutional feature map. The pooling layers reduce the spatial size of the convolutional feature map, reducing the number of model parameters and avoiding overfitting. The fully connected layers expand the features obtained by the convolution and pooling operations into a one-dimensional vector, resulting in a first feature image.

[0032] Step S102: input the first feature image into a pre-trained classification model, classify the insects in the first feature image and count the number, and determine the types of insects in the insect image and the corresponding number.

[0033] The classification model can be trained based on historically collected insect images. After inputting an image, the model can predict the insect species. This prediction can be a specific insect species or the probability of the insect being of each species. For example, using an image to identify the animal category, the trained model can output the animal category. For example, given an image of an animal, the model can output the probability of the animal being a dog with 0.7, a cat with 0.2, and a cow with 0.1.

[0034] In this solution, the insects in the insect image are classified to obtain the species corresponding to each insect. Then, according to the insect species, the number of each type of insect is counted to obtain the number corresponding to each insect species.

[0035] In some embodiments of the present application, the training process of the classification model includes:

[0036] Input the first insect image set with the insect species labels into the DINO model for training to obtain an initial model;

[0037] Based on the teacher-student network and / or the semi-supervised clustering network, the initial model is adjusted to obtain a classification model.

[0038] The first insect image set labeled with insect species labels may be images obtained from a public insect image library on the Internet.

[0039] To improve model performance, the model is trained using a pre-trained model based on DINO (DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection) rather than random parameters. Based on this pre-trained model, a first set of insect images labeled with their species is fed into the DINO model, where it is trained using supervised learning to establish a preliminary ability to identify insect species.

[0040] In the process of training the DINO model, in order to minimize the cross entropy loss function between the predicted label and the true label, the following formula can be used as the loss function of the DINO model:

[0041] Where, Represents the loss value of the DINO model, y represents the true label, Represents the predicted label, and n represents the number of categories.

[0042] Furthermore, the initial model can be fine-tuned to obtain a classification model with accurate classification. Specifically, the initial model can be fine-tuned using a teacher-student network, a semi-supervised clustering network, or a combination of a teacher-student network and a semi-supervised clustering network. This embodiment of the present invention does not limit this, and can be set according to actual needs.

[0043] After adjusting the initial model, you can also use AdamW as an optimizer to improve the training effect of the model and improve the classification accuracy of the classification model.

[0044] The advantage of this arrangement of the present invention is that by extracting and classifying the features of the collected insect images, it is possible to identify the newly emerged insect species, thereby more effectively monitoring and controlling the number of insects.

[0045] Example 2

[0046] FIG2 is a flow chart of a training method based on a teacher-student network provided in Example 2 of the present application. As shown in FIG2 , the method specifically includes the following steps:

[0047] Step S201: Use the initial model as a student model, and perform exponential moving average processing on the student model to obtain a teacher model.

[0048] The student model and the teacher model have the same network structure, and the parameters of the teacher model are obtained by exponential moving average of the parameters of the student model. The advantage of this setting is that it can stabilize the training process and enhance the generalization ability of the teacher model.

[0049] Step S202: Input the second insect image set without insect species labels into the teacher model for training to obtain the predicted probability of the insect species corresponding to each insect image in the second insect image set.

[0050] The second set of insect images without insect species labels may be actually captured and collected insect images. Specifically, by inputting the second set of insect images into the teacher model for unsupervised training, a temperature sharpener may be used to refine the prediction results.

[0051] In some embodiments of the present application, the insect image set is input into the teacher model for training to obtain the predicted probability of the insect species, including:

[0052] The predicted probability of the insect species is determined using the following formula:

[0053] Where, P sharper (y i ) represents the predicted probability of a sharper temperature, y i represents the predicted probability that the insect image is the i-th insect species, T represents the hyperparameter that controls the sharpness of the probability distribution, and y j represents the predicted probability that the insect image is the jth insect species.

[0054] In step S203 , each predicted probability is used as a pseudo label, and the second insect image set labeled with the pseudo labels is used to train the student model to obtain a classification model.

[0055] Ideally, the teacher model performs better than the student model, so the predicted probabilities obtained by the teacher model can be used as pseudo labels to guide the training of the student model.

[0056] For each training session, on the one hand, the student model uses the first set of insect images labeled with insect species for supervised training and calculates the supervised loss value; on the other hand, the student model uses the second set of insect images labeled with pseudo labels for unsupervised training and calculates the unsupervised loss value; the supervised loss value and the unsupervised loss value are weightedly summed to update the parameters of the student model, and then the updated student model is subjected to exponential moving average processing to update the parameters of the teacher model to obtain the classification model.

[0057] This solution uses the initial model as the student model and performs an exponential moving average on the student model to generate a teacher model. A second set of insect images, unlabeled with species, is fed into the teacher model for training, resulting in predicted probabilities for the species corresponding to each insect image in the second set. Each predicted probability is used as a pseudo-label, and the student model is trained using the pseudo-labeled second set of insect images to generate a classification model. This technical solution uses a teacher-student network to reduce the number of model parameters and transfer knowledge learned by the more capable teacher model to the less capable student model, enhancing the student model's generalization ability and ultimately improving insect classification accuracy.

[0058] Example 3

[0059] FIG3 is a flow chart of a training method based on a semi-supervised clustering network provided in Example 3 of the present application. As shown in FIG3 , the method specifically includes the following steps:

[0060] Step S301: input the first insect image set into a semi-supervised clustering network for training to obtain a first cluster center.

[0061] Specifically, cluster centers may be initialized for a first set of insect images that have been labeled with insect species labels to obtain at least one first cluster center.

[0062] Step S302: Input the second insect image set without insect species labels into a semi-supervised clustering network for training to obtain a second cluster center.

[0063] Specifically, if the number of first cluster centers is less than the preset number of clusters of the semi-supervised clustering network, the roulette wheel method is used to input the second insect image set without insect species labels into the semi-supervised clustering network for training to find the remaining number of second cluster centers.

[0064] Step S303: Iteratively train the first cluster center and the second cluster center to obtain a classification model.

[0065] Specifically, first, based on the divided cluster centers, the cluster centers closest to each insect image in the first insect image set and the second insect image set can be determined; secondly, based on the divided clusters, the cluster centers are recalculated; finally, the above steps are repeated until the training termination condition is met to obtain a classification model, where the training termination condition is achieved by reaching the number of training times or model convergence.

[0066] In some embodiments of the present application, the cluster center includes the first cluster center and the second cluster center, wherein the process of determining the cluster center includes:

[0067] The cluster center is iteratively updated using the following formula:

[0068] Where C k represents the kth cluster center, S k represents the sample set assigned to the kth cluster label, x n Represents the sample set S k The nth sample in .

[0069] In some embodiments of the present application, before inputting the first insect image set and the second insect image set without insect species labels into a semi-supervised clustering network for training, the method further includes: inputting the first insect image set and the second insect image set into an adversarial training network to perform domain adversarial training to obtain the trained first insect image set and second insect image set.

[0070] Without domain adaptation, if a model is trained solely on raw data, it can easily underperform when presented with target data whose feature distribution differs significantly from the raw data. For example, a classification model trained on a collection of photos of apples and pears will often fail to accurately predict whether an input photo is of a cat or a dog.

[0071] Therefore, in order to bridge the domain gap between the first set of insect images with labeled insect species and the second set of insect images without labeled insect species, a domain adversarial network can be used for domain alignment.

[0072] Specifically, the domain adversarial network includes a domain discriminator and a generative network. It maximizes the difference between the feature distributions of the source domain and the target domain so that the domain discriminator cannot distinguish between the source domain and the target domain, thereby aligning the source domain and the target domain in distribution.

[0073] Among them, the objective function of the adversarial training network can be expressed by the following formula:

[0074] Where L represents the objective function value of the adversarial training network, G represents the generative network, which is used to learn feature representations in the input data, and F represents the domain discriminator, which is used to distinguish the feature representations of the source domain and the target domain. represents the classification loss of the generative network G, which is used for supervised training of source domain data. It represents the domain loss between the generative network G and the domain discriminator D, which is used to distinguish the feature representations of the source domain and the target domain. λ is a hyperparameter used to control the weight between the classification loss and the domain loss.

[0075] Among them, the field loss It is usually defined as the binary classification loss of the domain discriminator D between the source domain and the target domain on the feature representation produced by the generator G, which can be expressed as follows:

[0076] Where, P STC represents the data distribution in the source domain, P tgt Represents the data distribution of the target domain, x represents the input sample of the corresponding domain, G(x) represents the output of the generative network G, that is, the feature representation, and D(G(x)) represents the output of the domain discriminator D on the feature representation.

[0077] The advantage of this setting is that the data features extracted from the source domain and the target domain will not be significantly different, and the insect images can be accurately classified and predicted, solving the problem of data mismatch between the source domain and the target domain.

[0078] This solution provides a training method based on a semi-supervised clustering network. This method involves inputting a first set of insect images into the semi-supervised clustering network for training to obtain first cluster centers. A second set of insect images, unlabeled with insect species, is then input into the semi-supervised clustering network for training to obtain second cluster centers. The first and second cluster centers are then iteratively trained to produce a classification model. This technical solution can effectively improve clustering accuracy.

[0079] Example 4

[0080] FIG4 is a schematic diagram of the structure of the insect monitoring device provided in Example 4 of the present application. As shown in FIG4 , it specifically includes the following:

[0081] A feature extraction module 410 is configured to collect an insect image and perform feature extraction on the insect image to obtain a first feature image;

[0082] The species recognition module 420 is used to input the first feature image into a pre-trained classification model, classify the insects in the first feature image and count the number, and determine the insect species and corresponding number in the insect image.

[0083] The insect monitoring device in the embodiments of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., which is not specifically limited in the embodiments of the present application.

[0084] The insect monitoring device in the embodiment of the present application can be a device having an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0085] The insect monitoring device provided in the embodiment of the present application can implement each process implemented in the above method embodiment. To avoid repetition, it will not be described here.

[0086] Example 5

[0087] In addition, an embodiment of the present application further provides a computer device. FIG5 is a schematic diagram of the structure of the computer device provided in Example 5 of the present application. The structure of the device is shown in FIG5 . The device includes a memory 51 for storing computer-readable instructions and a processor 52 for executing the computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor is triggered to execute the method described.

[0088] The methods and / or embodiments in the embodiments of the present application can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. When the computer program is executed by a processing unit, the above-mentioned functions defined in the method of the present application are performed.

[0089] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. Computer-readable media may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0090] In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0091] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0092] The flow chart or block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the equipment, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code include one or more executable instructions for realizing the logical function of the specification. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs the function or operation of the specification, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] As another aspect, embodiments of the present application further provide a computer-readable medium, which may be included in the device described in the above embodiments, or may exist independently and not incorporated into the device. The computer-readable medium carries one or more computer-readable instructions, which can be executed by a processor to implement the steps of the methods and / or technical solutions of the above embodiments of the present application.

[0094] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPUs), input / output interfaces, network interfaces and memories.

[0095] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0096] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc-read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0097] In addition, an embodiment of the present application further provides a computer program, which is stored in a computer device, so that the computer device executes the method for controlling code execution.

[0098] It should be noted that the present application can be implemented in a combination of software and / or software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In certain embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including related data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0099] It is obvious to those skilled in the art that the present application is not limited to the details of the above-mentioned exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. Words such as first and second are used to indicate names and do not indicate any particular order.

Claims

1. A pest situation monitoring method, wherein, the method includes: collecting pest images and extracting features from the pest images to obtain a first feature image; inputting the first feature image into a pre-trained classification model, classifying the pests in the first feature image and counting the quantity, and determining the types and corresponding quantities of the pests in the pest image.

2. The method according to claim 1, wherein, the training process of the classification model includes: inputting a first set of pest images with labeled pest type labels into a DINO model for training to obtain an initial model; adjusting the initial model based on a teacher-student network and / or a semi-supervised clustering network to obtain a classification model.

3. The method according to claim 2, wherein, adjusting the initial model based on a teacher-student network to obtain a classification model includes: taking the initial model as a student model and performing exponential moving average processing on the student model to obtain a teacher model; inputting a second set of pest images without labeled pest type labels into the teacher model for training to obtain the predicted probabilities of the pest types corresponding to each pest image in the second set of pest images; using each predicted probability as a pseudo-label and training the student model with the second set of pest images with labeled pseudo-labels to obtain a classification model.

4. The method according to claim 3, wherein, inputting the set of pest images into the teacher model for training to obtain the predicted probabilities of the pest types includes: The predicted probability of the bug type is determined using the following formula: Wherein, P sharper (y i ) represents the predicted probability with a sharper temperature, y i represents the predicted probability that the bug image is of the i-th bug species, T represents the hyperparameter that controls the sharpness of the probability distribution, y j represents the predicted probability that the bug image is of the j-th bug species.

5. The method according to claim 2, wherein, adjusting the initial model based on a semi-supervised clustering network to obtain a classification model includes: inputting the first set of pest images into a semi-supervised clustering network for training to obtain a first clustering center; inputting a second set of pest images without labeled pest type labels into the semi-supervised clustering network for training to obtain a second clustering center; performing iterative training on the first clustering center and the second clustering center to obtain a classification model.

6. The method according to claim 5, wherein, the clustering centers include the first clustering center and the second clustering center, and the determination process of the clustering centers includes: The clustering centers are iteratively updated using the following formula: where C k represents the k-th clustering center, S k represents the sample set assigned the k-th clustering label, and x n represents the n-th sample in the sample set S k .

7. The method according to claim 5, wherein, before inputting the first set of pest images and the second set of pest images without labeled pest type labels into the semi-supervised clustering network for training, the method further includes: inputting the first set of pest images and the second set of pest images into an adversarial training network for domain adversarial training to obtain a trained first set of pest images and second pest images.

8. A pest situation monitoring device, wherein, the device includes: a feature extraction module for collecting pest images and extracting features from the pest images to obtain a first feature image; a type recognition module for inputting the first feature image into a pre-trained classification model, classifying the pests in the first feature image and counting the quantity, and determining the types and corresponding quantities of the pests in the pest image.

9. A computer device, wherein, The device includes: one or more processors; and a memory storing computer program instructions which, when executed, cause the processors to execute the pest situation monitoring method according to any one of claims 1-7.

10. A computer-readable medium, wherein, computer program instructions are stored thereon, and the computer program instructions can be executed by a processor to implement the pest situation monitoring method according to any one of claims 1-7.

Images