A corn field pest discrimination method based on underground pest image feature recognition

Abstract

The application discloses a corn field pest identification method based on underground pest image feature recognition. The method effectively suppresses uneven internal soil illumination through homomorphic filtering, highlights pest edge details, improves detection stability, introduces causal feature decoupling and intervention learning, makes the model learn the morphological characteristics of pests rather than background pseudo-correlation, maintains high recognition rate under different soil types and humidity conditions, and fuses near-infrared texture features, uses the spectral difference between biological tissues and soil in the near-infrared wave band, and enhances the distinguishing ability of pests and sundries with similar colors.

Description

Technical Field

[0001] This invention relates to the field of agricultural pest and disease control technology, and more specifically, to an automatic identification method for underground pests in cornfields based on image processing and deep learning. Background Technology

[0002] Rice is one of my country's main food crops, and underground pests (such as grubs, wireworms, cutworms, and mole crickets) cause serious yield losses year after year. These pests inhabit the soil, damaging corn roots and seeds, and are characterized by their strong concealment, difficulty in detection, and uneven spatial distribution. Traditional survey methods rely on manual soil digging and sampling, which is not only labor-intensive and inefficient but also damages soil structure and cannot achieve continuous monitoring. Insect-attracting lamps can only monitor adult insects and cannot directly reflect larval damage. Ground-penetrating radar and other technologies have low resolution and are expensive.

[0003] In recent years, machine vision-based pest identification technology has developed, but it mainly targets images of surface pests captured by insect-attracting lamps or specimen images against a laboratory background. When identifying underground pests in in-situ soil environments, the body color of the pests is similar to that of soil particles and plant debris. Uneven lighting after the probe penetrates the soil produces light spots and shadows, making it difficult to accurately extract the target and affecting image quality.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a method for identifying cornfield pests based on image features of underground pests. This method addresses the current problem that when identifying underground pests in situ, the pests' body color is similar to that of soil particles and plant debris, and the uneven illumination after the probe penetrates the soil produces light spots and shadows, making it difficult to accurately extract the target and affecting image quality.

[0006] To achieve the above objectives, this invention provides a method for identifying cornfield pests based on underground insect image feature recognition. The method includes: Collect RGB images of soil profiles from cornfields; The sample RGB image is enhanced by homomorphic filtering to obtain an enhanced image; The enhanced image is input into a feature pyramid network including an attention module to extract candidate regions of samples that may contain pests. A feature extraction model is constructed, wherein the features of the sample candidate region image are decoupled into sample causal features and sample original environment features. The sample intervention environment features are obtained by randomly perturbing the sample original environment features. An intervention sample is generated based on the sample causal features and the sample intervention environment features. The feature extraction model is then trained adversarially based on the intervention sample and the original sample to obtain the trained feature extraction model. The original sample includes the sample candidate region image. Acquire multimodal image data of the soil profile inside a cornfield, wherein the multimodal image data includes a current RGB image and a near-infrared image of the same field of view; Based on the current RGB image, extract the current candidate region image, and extract the texture features corresponding to the spatial location of the current candidate region image from the near-infrared image to obtain the near-infrared texture features; The trained feature extraction model is used to extract the current causal features from the current candidate region image, and the near-infrared texture features are fused with the current causal features to obtain fused features; The fused features are input into the final classifier, which outputs the discrimination results of underground pests in cornfields.

[0007] Furthermore, the step of performing homomorphic filtering enhancement on the current RGB image includes: The current RGB image is logarithmically transformed and then subjected to Fourier transform. A homomorphic filter is applied in the frequency domain for processing, followed by inverse Fourier transform and exponential transform. The transfer function of the homomorphic filter is: in, This is the high-frequency boost factor, with a value ranging from 2.0 to 2.5. This is the low-frequency suppression coefficient, with a value ranging from 0.3 to 0.6; is the cutoff frequency, ranging from 30 to 60; c is the sharpening factor, ranging from 0.8 to 1.5. , is the frequency point Distance to the origin The spatial frequency in the horizontal direction. The spatial frequency is in the vertical direction. It is the transfer function of a homomorphic filter, which is a filter defined in the frequency domain and used to weight the spectrum of an image.

[0008] Furthermore, the feature pyramid network includes: The backbone network is used to perform convolution operations on the enhanced image and extract the first multi-scale feature map. The network includes C3, C4, and C5 in order of increasing network depth, where C3 is a low-level feature map, C4 is a mid-level feature map, and C5 is a high-level feature map. The downsampling factor of C5 is greater than that of C4, and the downsampling factor of C4 is greater than that of C3. The feature pyramid structure is used to upsample the high-level feature map C5 layer by layer through a top-down path, and then fuse the upsampled high-level feature map C5 with the mid-level backbone feature map C4 and the low-level feature map C3 after being weighted by the attention module through lateral connection to generate a second multi-scale feature map, wherein the second multi-scale feature map includes P3, P4 and P5.

[0009] An attention module, embedded in the lateral connections of the feature pyramid structure, is used to sequentially perform channel attention weighting and spatial attention weighting on the corresponding first multi-scale feature map before fusion. A region proposal network is used to generate candidate boxes on the second multi-scale feature map to filter out the sample candidate region images that may contain pests.

[0010] Furthermore, in the feature pyramid structure, C5 is designated as P5; The C4 weighted by the attention module is fused with the upsampled P5 to generate P4; The C3 weighted by the attention module is fused with the upsampled P4 to generate P3.

[0011] Furthermore, the feature extraction model includes: An encoder is used to encode the sample candidate region image into latent features; The feature decoupling network contains two independent branches for decomposing the latent features into the sample causal features and the sample original environment features; Construct the following modules to assist training: A generator is used to combine the causal features of the sample and the original environmental features of the sample to reconstruct the candidate region image of the sample; The intervention module is used to randomly perturb the original environmental features of all candidate region images in the current training batch, generate intervention environmental features, and generate intervention samples, wherein the pest morphology in the intervention samples is the same as that in the candidate region images, while the background environment is replaced. An auxiliary classifier is used to classify pests based on the causal features of the samples. A discriminator is used to distinguish between the candidate region image of the sample and the intervention sample.

[0012] Furthermore, the step of constructing the feature extraction model also includes: The causal features of the candidate region image are input into the auxiliary classifier to obtain the first classification probability distribution; The causal features of the intervention sample are input into the auxiliary classifier to obtain the second classification probability distribution; Based on the loss function, the feature extraction model, the generator, the auxiliary classifier, and the discriminator are jointly optimized using the backpropagation algorithm.

[0013] Furthermore, the loss function includes: Classification loss: ,in, This represents the number of samples in the training batch. The total number of pest categories. For the first The true label of the candidate region image of the sample, if the candidate region image of the sample is the first Class, then Otherwise, it is 0. The first output of the auxiliary classifier The sample belongs to the first The probability of a class.

[0014] Loss of invariance: ,in, For the first The first classification probability distribution of each sample For the first The second classification probability distribution of each sample To represent the KL divergence, it is used to indicate the difference between two probability distributions; Reconstruction loss: ,in, For the first Each sample candidate region image, The first reconstructed generator One image, represents the square of the L2 norm, and represents the sum of squares of the pixel-by-pixel differences.

[0015] Combat losses: Total loss: ,in, , , This is the preset balance coefficient.

[0016] Further, the step of extracting texture features corresponding to the spatial location of the current candidate region from the near-infrared image includes: In the near-infrared image, the region corresponding to the spatial location of the current candidate region is extracted to obtain local binary mode histogram features and gray-level co-occurrence matrix features. Principal component analysis is performed on the local binary pattern histogram features and the gray-level co-occurrence matrix features to reduce dimensionality and obtain the near-infrared texture features.

[0017] Further, the step of extracting the current causal feature from the current candidate region image using the feature extraction model and fusing the near-infrared texture feature with the current causal feature includes: The attention weights are calculated using the following formula: ,in, As a current causal characteristic, This is the weight matrix of the current causal features. This is the weight matrix for near-infrared features. It is a near-infrared texture feature. For bias vectors, For activation function, Attention weights The fusion features are calculated using the following formula: ,in, This is a feature of fusion.

[0018] Furthermore, the final classifier includes an input layer, at least two hidden layers, and an output layer. The hidden layers are followed by a ReLU activation function, and the output layer uses a softmax function to convert the output values ​​into class probabilities of underground pests. This is the activation function for the softmax function.

[0019] This application effectively suppresses uneven lighting within the soil through homomorphic filtering, highlights the edge details of pests, and improves detection stability. It introduces causal feature decoupling and intervention learning, enabling the model to learn the morphological features of the pests themselves rather than background pseudo-correlation, maintaining a high recognition rate under different soil types and humidity conditions. Furthermore, it integrates near-infrared texture features, utilizing the spectral differences between biological tissues and soil in the near-infrared band to enhance the ability to distinguish pests and debris with similar colors. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating an embodiment of the cornfield pest identification method based on underground insect image feature recognition of this application; The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0022] This invention further provides a method for identifying cornfield pests based on image feature recognition of underground insect pests. (Refer to...) Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the cornfield pest identification method based on underground insect image feature recognition according to the present invention.

[0023] In this embodiment, the main body executing the cornfield pest identification method based on underground pest image feature recognition is a cornfield pest identification method system based on underground pest image feature recognition. This system includes a cornfield pest identification device or equipment based on underground pest image feature recognition, which can be a PC, PDA, or other terminal device. This application effectively suppresses uneven lighting within the soil through homomorphic filtering, highlighting pest edge details and improving detection stability. It introduces causal feature decoupling and intervention learning, enabling the model to learn the morphological features of the pest itself rather than background pseudo-correlation, maintaining a high recognition rate under different soil types and humidity conditions. Furthermore, it integrates near-infrared texture features, utilizing the spectral differences between biological tissues and soil in the near-infrared band to enhance the ability to distinguish pests and debris with similar colors.

[0024] The steps of this method for identifying cornfield pests based on underground insect image features include: Step S10: Collect RGB images of soil profiles inside the cornfield. In this embodiment, a microporous imaging probe with a transparent protective cover is buried in the cornfield. The probe integrates an RGB camera and a near-infrared camera (850nm band), and is equipped with a ring-shaped LED supplementary light. The probe extends 10-30cm into the soil to acquire RGB and near-infrared images of the same field of view, with an image resolution of no less than 1920×1080 pixels. A spectrophotometric imaging system or image registration algorithm is used to ensure that each pixel in the two images corresponds to the same spatial location in the soil. When constructing the feature extraction model, the microporous imaging probe can acquire sample RGB images for model training. During the application phase, when identifying underground pests in the cornfield, the microporous imaging probe can acquire the current RGB and infrared images.

[0025] Step S20: Perform homomorphic filtering enhancement on the sample RGB image to obtain the enhanced image; In this embodiment, when performing homomorphic filtering enhancement on the sample RGB image, the logarithm of the sample RGB image is first taken, followed by a Fourier transform. A homomorphic filter is then applied in the frequency domain, followed by an inverse Fourier transform and an exponential transform. The sample RGB image can be represented as an illumination component. and reflection component The product of ,in Slow changes correspond to low-frequency information (uneven illumination). It includes detailed edges, corresponding to high-frequency information. The purpose of homomorphic filtering is to compress low frequencies and enhance high frequencies, thereby suppressing uneven lighting and enhancing pest details. Specifically: 1. Take the logarithm to transform the multiplicative relationship into an additive relationship.

[0026] in, is the x-coordinate of a pixel in the sample RGB image, representing the pixel's position along the image's width. Let be the ordinate of the pixels in the sample RGB image. Indicates the sample RGB image in The pixel grayscale value at that location.

[0027] 2. Fourier transform converts an image from the spatial domain to the frequency domain.

[0028] in, For the Fourier transform of the illuminance component, For the Fourier transform of the reflection component, , . The spatial frequency in the horizontal direction represents the image in... The rate of grayscale change in the horizontal direction, usually measured in "period / image width", with a value range of 0 ≤ ≤M−10≤ ≤M−1, where M is the number of sampling points in the frequency domain after Fourier transform (usually equal to the image width W). The spatial frequency in the vertical direction represents the image's frequency at that direction. The rate of grayscale change in the vertical direction, with a value range of 0 ≤ ≤N−10≤ ≤N−1, where N equals the image height H. The frequency spectrum of the image is a complex number that contains amplitude and phase information.

[0029] 3. Frequency domain filtering, using homomorphic filters. right Process it.

[0030] The transfer function of the homomorphic filter is: in, This is the high-frequency boost factor, with a value ranging from 2.0 to 2.5. This is the low-frequency suppression coefficient, with a value ranging from 0.3 to 0.6; is the cutoff frequency, ranging from 30 to 60; c is the sharpening factor, ranging from 0.8 to 1.5. , is the frequency point The distance to the origin is used to measure the frequency.

[0031] 4. Inverse Fourier transform, converting the processed spectrum back to the spatial domain. in, This is the filtered logarithmic domain image.

[0032] 5. Take the exponent and restore the logarithmic domain image to its normal grayscale value to obtain the enhanced image.

[0033] in, The enhanced image in coordinates for the final output The grayscale value of the pixel at that location, for all pixels. To create an enhanced image.

[0034] Step S30: Input the enhanced image into a feature pyramid network including an attention module to extract candidate region images of samples that may contain pests; In this embodiment, the feature pyramid network includes a backbone network, a feature pyramid structure, an attention module, and a region proposal network.

[0035] The backbone network performs convolutional operations on the enhanced image to extract the first multi-scale feature map. It consists of C3, C4, and C5 layers in ascending order of network depth, where C3 is the low-level feature map, C4 is the mid-level feature map, and C5 is the high-level feature map. The downsampling factor of C5 is greater than that of C4, and the downsampling factor of C4 is greater than that of C3. For example, if C3 has a downsampling factor of 8x, the corresponding feature map size is 64. 64, C4 has a downsampling factor of 16x, corresponding to a feature map size of 32. 32. The downsampling factor of C5 is 32x, and the corresponding feature map size is 16. 16.

[0036] A feature pyramid structure is used to upsample the high-level feature map C5 layer by layer through a top-down path. Then, through lateral connections, the upsampled high-level feature map C5 is fused layer by layer with the mid-level backbone feature map C4 (weighted by the attention module) and the low-level feature map C3 to generate a second multi-scale feature map, which includes P3, P4, and P5. Specifically, C5 is used as P5, which can be understood as applying 1 to C5. 1. Convolution, unifying the number of channels, yields P5; then, the C4 weighted by the attention module is fused with the upsampled P5 to generate P4. Understandably, C4 is first applied with 1... 1. Convolution is performed to obtain a temporary feature map. P5 is then upsampled by a factor of 2, making its feature map size the same as C4's. The upsampled P5 is then added element-wise to the temporary feature map of C4, which has been weighted by the attention module. The sum is then applied to the 3... 3. Convolution to eliminate aliasing, resulting in P4; then, the weighted C3 after the attention module is fused with the upsampled P4 to generate P3. Understandably, first, 1... 1. Convolution is performed to obtain a temporary feature map. P4 is upsampled by a factor of 2 so that the feature map size of P4 is the same as that of C3. The upsampled P4 is then added element-wise to the temporary feature map of C3 after being weighted by the attention module. The result is then applied to 3. 3 convolutions yield P3.

[0037] The attention module is embedded in the lateral connections of the feature pyramid structure. Before fusion, it sequentially applies channel attention weighting and spatial attention weighting to the corresponding first multi-scale feature maps. The first multi-scale feature maps are then subjected to global average pooling and global max pooling to obtain two 1×1×C channel descriptors. These are fed into a shared multilayer perceptron, and the outputs are summed and activated by a sigmoid function to obtain channel weights. These channel weights are then multiplied channel-wise by the feature vectors in the first multi-scale feature maps to obtain the attention-weighted first multi-scale feature maps. The attention-weighted first multi-scale feature maps are then subjected to average pooling and max pooling along the channel axes to obtain two H×W×1 feature maps. These are concatenated, convolved, and activated by a sigmoid function to obtain spatial weights. These spatial weights are then multiplied spatially by the feature vectors in the attention-weighted first multi-scale feature maps to obtain the final output, which is the candidate region image.

[0038] The Region Proposal Network (RPN) is used to generate candidate bounding boxes on the second multi-scale feature map, filtering out candidate region images that may contain pests. The RPN slides across the generated second multi-scale feature maps P3, P4, and P5 to generate candidate bounding boxes. Through foreground / background classification and bounding box regression, the RPN filters out the top 200 candidate bounding boxes with high confidence as candidate region images that may contain pests.

[0039] Step S40: Construct a feature extraction model; In this embodiment, the features of the candidate region image are decoupled into causal features and original environmental features. The original environmental features are randomly perturbed to obtain intervention environmental features. Intervention samples are then generated based on these causal and intervention environmental features. The feature extraction model is then trained adversarially against the intervention samples to obtain the trained model. The original samples include the candidate region image. This feature extraction model includes an encoder and a feature decoupling network. In this embodiment, by constructing and training the feature extraction model, it is able to extract causal features from the candidate region image that are independent of the background environment.

[0040] The encoder consists of four convolutional layers, encoding the candidate region images of the samples into latent features. The feature decoupling network contains two independent branches, which decompose the latent features into sample causal features and sample original environment features, respectively. The sample causal features include the essential morphological attributes of the pests (such as the number of body segments, the distribution of setae, the width of the head shell, etc.), and the sample original environment features include background soil information.

[0041] In addition, the following modules are constructed to assist training: a generator, an intervention module, an auxiliary classifier, and a discriminator. The generator, composed of a deconvolutional network, reconstructs candidate region images by taking a combination of causal features of the input samples and original environmental features of the samples, thus validating the decoupling effect. The intervention module randomly perturbs the original environmental features of all candidate region images in the current training batch, generating intervention environmental features. The original causal features of the candidate region images are combined with the intervention environmental features and input into the generator to obtain the intervention sample. In this intervention sample, the pest morphology is the same as the original sample, but the background environment is replaced with the background environment of other samples. The auxiliary classifier is used to classify pests based on the causal features of the samples. It applies invariance constraints during training. The causal features of the candidate region images are input into the auxiliary classifier to obtain a first classification probability distribution, and the causal features of the intervention sample are input into the auxiliary classifier to obtain a second classification probability distribution.

[0042] During training, the feature extraction model, generator, auxiliary classifier, and discriminator are jointly optimized using backpropagation based on a loss function, which includes classification loss, invariant loss, reconstruction loss, adversarial loss, and total loss. After training, the feature extraction model can extract causal features from any candidate region image that are independent of the background environment. The auxiliary classifier is only used to apply constraints during training and is no longer used during inference. The specific loss function is as follows: Classification loss: ,in, This represents the number of samples in the training batch. The total number of pest categories. For the first The true label of the candidate region image of the sample, if the candidate region image of the sample is the first Class, then Otherwise, it is 0. The first output of the auxiliary classifier The sample belongs to the first The probability of a class.

[0043] Loss of invariance: ,in, For the first The first classification probability distribution of each sample For the first The second classification probability distribution of each sample To represent the KL divergence, it is used to indicate the difference between two probability distributions; Reconstruction loss: ,in, For the first Each sample candidate region image, The first reconstructed generator One image, represents the square of the L2 norm, and represents the sum of squares of the pixel-by-pixel differences.

[0044] Combat losses: ,in, For the discriminator to the first Candidate region images of samples The output value, For the discriminator pair Output values ​​for each intervention sample.

[0045] Total loss: ,in, , , This is the preset balance coefficient.

[0046] Step S50: Obtain multimodal image data of the soil profile inside the cornfield, wherein the multimodal image data includes the current RGB image and near-infrared image of the same field of view.

[0047] In this embodiment, a microporous imaging probe with a transparent protective cover is buried in a cornfield. The probe integrates an RGB camera and a near-infrared camera (850nm band) and is equipped with a ring LED supplementary light. The probe extends into the soil to a depth of 10-30cm to acquire the current RGB image and near-infrared image in the same field of view, with an image resolution of no less than 1920×1080 pixels. Through a spectroscopic imaging system or image registration algorithm, it is ensured that each pixel in the current RGB image and near-infrared image corresponds to the same spatial location in the soil.

[0048] Step S60: Extract the current candidate region image based on the current RGB image, and extract the texture features corresponding to the spatial location of the current candidate region image from the near-infrared image to obtain the near-infrared texture features.

[0049] In this embodiment, the current candidate region image is obtained in the same way as during training, through homomorphic filtering enhancement and feature pyramid network, which will not be described in detail here.

[0050] In step S50, by utilizing a spectroscopic imaging system or image registration algorithm, a region corresponding to the spatial location of the current candidate region can be found in the near-infrared image, thereby extracting local binary mode histogram features and gray-level co-occurrence matrix features. Specifically, for each pixel, with it as the center, the gray values ​​of neighboring pixels are compared with the center value to generate a binary code, and a histogram is statistically analyzed to obtain 256-dimensional features; the co-occurrence matrix in four directions (0°, 45°, 90°, 135°) is calculated, and 16-dimensional features of contrast, correlation, energy, and homogeneity are extracted; after the above features are concatenated, the dimensionality is reduced to 64 dimensions through principal component analysis to obtain near-infrared texture features.

[0051] Step S70: Use the trained feature extraction model to extract the current causal features from the current candidate region image, and fuse the near-infrared texture features with the current causal features to obtain fused features.

[0052] In this embodiment, a trained feature extraction model is used to extract current causal features from the current candidate region image. These current causal features can be 128-dimensional. An attention mechanism is then used to fuse the current causal features with near-infrared texture features. Specifically: The attention weights are calculated using the following formula: ,in, As a current causal characteristic, This is the weight matrix of the current causal features. , This is the weight matrix for near-infrared features. , It is a near-infrared texture feature. For bias vectors, , For activation function, For attention weights, each component of w (the value in each dimension) takes a value in the range of (0, 1), that is, the open interval between 0 and 1.

[0053] The fusion features are calculated using the following formula: ,in, As a feature of fusion, This indicates element-wise multiplication. .

[0054] Step S80: Input the fused features into the final classifier and output the discrimination results of underground pests in cornfields.

[0055] In this embodiment, the fused features are input to the final classifier, which can be a three-layer perceptron with the following structure: 64-dimensional input layer → 128-dimensional hidden layer (ReLU) → 64-dimensional hidden layer (ReLU) → 5-dimensional output layer (Softmax). The output corresponds to the probabilities of five categories: grubs, wireworms, cutworms, mole crickets, and background. The one with the highest probability is taken as the discrimination result, and the bounding box position and confidence score are also output. If the highest probability corresponds to the background, it means that the candidate region is free of pests. Since the entire method is based on candidate region detection, and each candidate region corresponds to a rectangle in the image, the discrimination result usually needs to include the coordinates of the current candidate region image in the current RGB image. Because the fused features include causal features that are independent of the background environment learned in step S40, the final discrimination result naturally inherits the background invariance.

[0056] In this embodiment, the final classifier can be obtained by training the feature extraction model independently. Specifically, the trained feature extraction model can extract causal features that are independent of the background environment. The causal features of the training samples are extracted using the trained feature extraction model, and near-infrared texture features are extracted from the near-infrared images of the training samples. The causal features and near-infrared texture features are fused to train a new classifier. The input is the fused features, and the output is the probability of 5 classes. The training process is simple and does not require complex joint optimization.

[0057] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0058] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0059] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0060] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for identifying cornfield pests based on image feature recognition of underground insect pests, characterized in that, Includes the following steps: Collect RGB images of soil profiles from cornfields; The sample RGB image is enhanced by homomorphic filtering to obtain an enhanced image; The enhanced image is input into a feature pyramid network including an attention module to extract candidate regions of samples that may contain pests. A feature extraction model is constructed, wherein the features of the sample candidate region image are decoupled into sample causal features and sample original environment features. The sample intervention environment features are obtained by randomly perturbing the sample original environment features. An intervention sample is generated based on the sample causal features and the sample intervention environment features. The feature extraction model is then trained adversarially based on the intervention sample and the original sample to obtain the trained feature extraction model. The original sample includes the sample candidate region image. Acquire multimodal image data of the soil profile inside a cornfield, wherein the multimodal image data includes a current RGB image and a near-infrared image of the same field of view; Based on the current RGB image, extract the current candidate region image, and extract the texture features corresponding to the spatial location of the current candidate region image from the near-infrared image to obtain the near-infrared texture features; The trained feature extraction model is used to extract the current causal features from the current candidate region image, and the near-infrared texture features are fused with the current causal features to obtain fused features; The fused features are input into the final classifier, which outputs the discrimination results of underground pests in cornfields.

2. The method as described in claim 1, characterized in that, The step of performing homomorphic filtering enhancement on the current RGB image includes: The current RGB image is logarithmically transformed and then subjected to Fourier transform. A homomorphic filter is applied in the frequency domain for processing, followed by inverse Fourier transform and exponential transform. The transfer function of the homomorphic filter is: in, This is the high-frequency boost factor, with a value ranging from 2.0 to 2.

5. This is the low-frequency suppression coefficient, with a value ranging from 0.3 to 0.6; is the cutoff frequency, ranging from 30 to 60; c is the sharpening factor, ranging from 0.8 to 1.

5. , is the frequency point Distance to the origin The spatial frequency in the horizontal direction. The spatial frequency is in the vertical direction.

3. The method as described in claim 1, characterized in that, The feature pyramid network includes: The backbone network is used to perform convolution operations on the enhanced image and extract the first multi-scale feature map. The network includes C3, C4, and C5 in order of increasing network depth, where C3 is a low-level feature map, C4 is a mid-level feature map, and C5 is a high-level feature map. The downsampling factor of C5 is greater than that of C4, and the downsampling factor of C4 is greater than that of C3. The feature pyramid structure is used to upsample the high-level feature map C5 layer by layer through a top-down path, and then fuse the upsampled high-level feature map C5 with the mid-level backbone feature map C4 and the low-level feature map C3 after being weighted by the attention module through lateral connection to generate a second multi-scale feature map, wherein the second multi-scale feature map includes P3, P4 and P5. An attention module, embedded in the lateral connections of the feature pyramid structure, is used to sequentially perform channel attention weighting and spatial attention weighting on the corresponding first multi-scale feature map before fusion. A region proposal network is used to generate candidate boxes on the second multi-scale feature map to filter out the sample candidate region images that may contain pests.

4. The method as described in claim 3, characterized in that, In the aforementioned feature pyramid structure, C5 is designated as P5; The C4 weighted by the attention module is fused with the upsampled P5 to generate P4; The C3 weighted by the attention module is fused with the upsampled P4 to generate P3.

5. The method as described in claim 1, characterized in that, The feature extraction model includes: An encoder is used to encode the sample candidate region image into latent features; The feature decoupling network contains two independent branches for decomposing the latent features into the sample causal features and the sample original environment features; Construct the following modules to assist training: A generator is used to combine the causal features of the sample and the original environmental features of the sample to reconstruct the candidate region image of the sample; The intervention module is used to randomly perturb the original environmental features of all candidate region images in the current training batch, generate intervention environmental features, and generate intervention samples, wherein the pest morphology in the intervention samples is the same as that in the candidate region images, while the background environment is replaced. An auxiliary classifier is used to classify pests based on the causal features of the samples. A discriminator is used to distinguish between the candidate region image of the sample and the intervention sample.

6. The method as described in claim 5, characterized in that, The steps for constructing the feature extraction model also include: The causal features of the candidate region image are input into the auxiliary classifier to obtain the first classification probability distribution; The causal features of the intervention sample are input into the auxiliary classifier to obtain the second classification probability distribution; Based on the loss function, the feature extraction model, the generator, the auxiliary classifier, and the discriminator are jointly optimized using the backpropagation algorithm.

7. The method as described in claim 6, characterized in that, The loss function includes: Classification loss: ,in, This represents the number of samples in the training batch. The total number of pest categories. For the first The true label of the candidate region image of the sample, if the candidate region image of the sample is the first Class, then Otherwise, it is 0. The first output of the auxiliary classifier The sample belongs to the first The probability of a class. Loss of invariance: ,in, For the first The first classification probability distribution of each sample For the first The second classification probability distribution of each sample To represent the KL divergence, it is used to indicate the difference between two probability distributions; Reconstruction loss: ,in, For the first Each sample candidate region image, The first reconstructed generator One image, represents the square of the L2 norm, and represents the sum of squares of the pixel-by-pixel differences. Combat losses: Total loss: ,in, , , This is the preset balance coefficient.

8. The method as described in claim 1, characterized in that, The step of extracting texture features corresponding to the spatial location of the current candidate region from the near-infrared image includes: In the near-infrared image, the region corresponding to the spatial location of the current candidate region is extracted to obtain local binary mode histogram features and gray-level co-occurrence matrix features. Principal component analysis is performed on the local binary pattern histogram features and the gray-level co-occurrence matrix features to reduce dimensionality and obtain the near-infrared texture features.

9. The method as described in claim 8, characterized in that, The step of extracting the current causal feature from the current candidate region image using the feature extraction model and fusing the near-infrared texture feature with the current causal feature includes: The attention weights are calculated using the following formula: ,in, As a current causal characteristic, This is the weight matrix of the current causal features. This is the weight matrix for near-infrared features. It is a near-infrared texture feature. For bias vectors, For activation function, Attention weights The fusion features are calculated using the following formula: ,in, This is a feature of fusion.

10. The method as described in claim 9, characterized in that, The final classifier includes an input layer, at least two hidden layers, and an output layer. The hidden layers are followed by a ReLU activation function, and the output layer uses a softmax function to convert the output values ​​into class probabilities for underground pests. This is the activation function for the softmax function.

Images