Rice pest image acquisition and high-fidelity reconstruction detection method

Abstract

The application discloses a rice pest image acquisition and high-fidelity reconstruction detection method, and relates to the technical field of agricultural intelligent monitoring and machine vision. Field images are acquired through a fixed rod type device, and an end-to-end network including a main trunk network, a multi-scale feature fusion module, a channel separation type high-fidelity feature reconstruction module and a target detection decoding module is constructed. The reconstruction module generates channel response weights and their complementary weights according to channel description information, modulates the complementary response of complete input features, and obtains high-response weighted features and low-response weighted features with the same number of channels and input features. High-fidelity features are obtained through high-response local enhancement, low-response information compensation, channel fusion and residual reconstruction. The target detection decoding module outputs pest category labels, predicted bounding box coordinates and confidence. The application is beneficial to improving the expression of pest fine-grained features and the stability of detection in a complex field environment.

Description

Technical Field

[0001] This invention relates to the fields of agricultural intelligent monitoring, digital image processing and machine vision target detection technology, specifically to a method for rice pest image acquisition and high-fidelity reconstruction detection. Background Technology

[0002] Rice is a vital food crop in my country, and its stable production is crucial for ensuring food security. However, rice is frequently attacked by various pests throughout its growth cycle, including brown planthoppers, black-tailed leafhoppers, rice leaf rollers, rice stem borers, rice green bugs, and rice leaf miners. These pests disrupt normal rice growth by sucking sap, feeding on leaves, or boring into stems, leading to reduced yields and even large-scale crop failures in severe cases. Therefore, achieving rapid and accurate identification of rice pests is essential for implementing precision control, reducing agricultural losses, and promoting the development of smart agriculture.

[0003] Currently, rice pest monitoring methods mainly include manual field surveys, monitoring with insect-attracting devices, and automatic identification methods based on computer vision. Traditional manual surveys rely on the experience and judgment of agricultural personnel, resulting in low detection efficiency, high subjectivity, and difficulty in achieving continuous dynamic monitoring. With the development of deep learning technology, using image acquisition equipment to obtain pest images and automatically identifying pests through target detection algorithms has gradually become an important technical approach.

[0004] However, existing image-based rice pest detection methods still suffer from the following problems in practical field applications: First, the field environment is affected by factors such as changes in natural light, leaf movement, and background interference, resulting in unstable brightness, significant changes in target scale, and insufficient effective texture in the acquired images, leading to a decline in the quality of the input images for the detection model. Second, rice pests are typically small in size, with body color similar to leaves and indistinct edge textures. During the multi-layer feature extraction and scale fusion process in deep neural networks, their fine-grained features, such as contours and textures, are easily weakened by background interference, causing missed detections or localization errors. Third, existing networks are prone to loss of detailed information during multi-scale feature fusion. Some existing feature enhancement methods process all features uniformly or divide the input channels into fixed segments according to a preset number and process them separately, making it difficult to differentiate and compensate for different response features while maintaining complete channel information.

[0005] Therefore, in order to address the problems of insufficient stability in rice pest image acquisition under complex field conditions, easy loss of features of small targets, and weak ability to express fine-grained information in multi-scale detection, it is urgent to propose an intelligent detection method for rice pests that combines stable image acquisition with high-fidelity feature reconstruction, so as to improve the accuracy and reliability of pest target identification under complex conditions. Summary of the Invention

[0006] This invention aims to provide a method for rice pest image acquisition and high-fidelity reconstruction detection, in order to solve the problems of insufficient image acquisition stability, easy attenuation of target features, loss of detail information in multi-scale fusion, and difficulty in differentiating and compensating for different response features while preserving complete channel information in natural field environments, small target pests, and similar background interference conditions.

[0007] The present invention achieves the above objectives through the following technical solutions:

[0008] A method for image acquisition and high-fidelity reconstruction detection of rice pests, the method comprising the following steps:

[0009] S1: Acquire field images containing information about rice plants, pest targets, and natural environmental background through a rice pest image acquisition device;

[0010] S2: Filter and label the field images collected in step S1 to generate a target detection label file that includes pest target category information and bounding box information, construct a rice pest target detection dataset, and divide the rice pest target detection dataset into a training dataset and a validation dataset according to a preset ratio.

[0011] S3: Construct an end-to-end detection network for rice pests, which includes a feature extraction backbone network, a multi-scale feature fusion module, a channel-separated high-fidelity feature reconstruction module, and a target detection decoding module.

[0012] The feature extraction backbone network is used to extract rice pest features at different spatial resolutions.

[0013] The multi-scale feature fusion module is used to perform channel mapping and cross-scale fusion on the rice pest features with different spatial resolutions, and output fused features at least two scales.

[0014] The channel-separated high-fidelity feature reconstruction module is used to sequentially extract channel description information, generate channel response weights, modulate complementary responses, enhance high-response local spatial features, compensate low-response information, and reconstruct features at each scale of the fused features, thereby generating high-fidelity reconstructed features at the corresponding scale.

[0015] The target detection and decoding module is used to output the category prediction result and location prediction result of the pest target based on the high-fidelity reconstruction features;

[0016] S4: Using the rice pest target detection dataset constructed in step S2, train the rice pest end-to-end detection network constructed in step S3; during the training data loading stage, simultaneously adjust the size of the input image and its corresponding bounding box information, and convert the adjusted image into the input data format required by the rice pest end-to-end detection network; update the network parameters by jointly optimizing the pest category prediction error, bounding box regression error, predicted bounding box overlap error, and localization distribution related error to obtain the trained rice pest detection model;

[0017] S5: Deploy the trained rice pest detection model to the backend processing equipment, perform inference calculations on the collected rice field images, and output pest category labels, predicted bounding box coordinates, and corresponding confidence scores.

[0018] As one of the preferred technical solutions of the present invention, the rice pest image acquisition device in step S1 adopts a fixed rod acquisition structure, including a fixed support module, an image acquisition module, an auxiliary lighting module, an attitude adjustment module, a distance detection module, and a data transmission module.

[0019] The image acquisition module is mounted on the fixed support module and is used to acquire images of rice plants and insect targets on their surface.

[0020] The attitude adjustment module is used to adjust the azimuth and pitch angles of the image acquisition module so that the rice target area to be monitored is within the preset shooting field of view.

[0021] The distance detection module is used to detect the spatial distance between the image acquisition module and the rice target area, and to determine whether the spatial distance is within a preset imaging distance range. The distance detection result is used to help determine the installation position and shooting posture of the image acquisition module.

[0022] The auxiliary lighting module is used to provide supplemental lighting to the image acquisition area, so as to reduce the impact of changes in natural lighting on the brightness and clarity of the acquired image, and improve the imaging recognition of pest targets whose body color is similar to that of rice leaf background.

[0023] The data transmission module is used to transmit the collected field images to back-end processing equipment or data storage equipment.

[0024] As one of the preferred technical solutions of the present invention, in step S2, the six types of rice pests appearing in the image are manually labeled one by one using a labeling tool. The labels uniformly use the English category identifiers of the pest species, including brown planthopper, green leafhopper, rice leaf folder, stem borer, rice bug, and whorl maggot, generating a standard target detection format labeling file containing the target category and bounding box position.

[0025] As one of the preferred technical solutions of this invention, the English name of the Channel-Separated High-Fidelity Feature Reconstruction Module in step S3 is Channel-Separated High-Fidelity Feature Reconstruction Module, abbreviated as CSHF module. The channel separation described in this invention refers to using channel response weights and their complementary weights to modulate the complete input feature into two complementary response components, each with the same number of channels as the input feature, rather than physically segmenting the input feature according to a fixed number of channels based on the channel index.

[0026] The CSHF module first performs global average pooling on the input features to obtain channel description vectors, and then generates channel response weights sequentially through a first channel mapping, ReLU activation, a second channel mapping, and a Sigmoid gated activation. These channel response weights and their complementary weights are then used to weight the complete input features, resulting in high-response weighted features and low-response weighted features, which are element-wise summed to equal the input features.

[0027] The high-response weighted features are subjected to deep convolutional local spatial enhancement to obtain high-response enhanced features; the low-response weighted features and high-response enhanced features are added element-wise, and then subjected to channel mapping, batch normalization and SiLU activation to obtain low-response compensated features; the high-response enhanced features and low-response compensated features are then concatenated along the channel dimension, and after channel fusion, residual fusion is performed with the input features to obtain high-fidelity output features.

[0028] As one of the preferred technical solutions of this invention, step S4 employs a joint loss function to optimize the parameters of the rice pest end-to-end detection network. The joint loss function includes variable focus classification loss, bounding box regression loss, generalized cross-union loss, fine-grained localization loss, and decoupled distillation focus loss.

[0029] The variable focus classification loss is used to constrain the difference between the predicted pest target category and the true category label; the bounding box regression loss is used to constrain the positional and size deviations between the predicted bounding box parameters and the true bounding box parameters; the generalized intersection-union loss is used to constrain the overlap relationship between the predicted bounding box and the true bounding box; the fine-grained localization loss is used to constrain the fine-grained prediction results of the bounding box localization distribution; and the decoupled distillation focus loss is used to perform self-distillation constraints on the localization distribution between different decoding layers of the target detection decoding module.

[0030] As one of the preferred technical solutions of the present invention, the back-end processing device in step S5 includes a memory, a processor, and a detection program stored in the memory and executable by the processor.

[0031] The detection program is used to load the rice pest detection model trained in step S4, perform input preprocessing, network forward computation and detection result parsing on field images acquired in real time or stored in advance, and output rice pest identification results including predicted bounding boxes, pest category labels and confidence scores.

[0032] Compared with the prior art, the beneficial effects of the present invention are: (1) The present invention monitors the image acquisition distance status by setting a distance detection module, a posture adjustment module and an auxiliary lighting module, and adjusts the shooting direction and lighting conditions of the acquisition area, which helps to reduce the impact of changes in natural light, shooting posture and target imaging scale on the consistency of field images. (2) The CSHF module of the present invention generates channel response weights based on the global channel description information of the input features, and uses the channel response weights and their complementary weights to perform complementary response modulation on the complete input features, so that both the high response weighted features and the low response weighted features retain all channels of the input features, avoids physical segmentation according to the channel index, and maintains the complementary relationship between the two response components. (3) The present invention performs deep convolutional local spatial enhancement on the high response weighted features and compensates the high response enhancement features to the low response weighted features; through channel concatenation, channel fusion and residual reconstruction, while maintaining the consistency of input and output spatial size and number of channels, it helps to retain the edge, texture and local morphological information of the pest target, and improves the integrity and detection stability of rice pest feature expression in complex field environments. Attached Figure Description

[0033] Figure 1 This is a flowchart of a method for rice pest image acquisition and high-fidelity reconstruction detection provided by the present invention;

[0034] Figure 2 This is a schematic diagram of the rice pest image acquisition device in this invention;

[0035] Figure 3This is a schematic diagram of the overall structure of the end-to-end rice pest detection network in this invention.

[0036] Figure 4 This is a schematic diagram of the internal structure and data flow of the channel-separated high-fidelity feature reconstruction module in this invention;

[0037] Figure 5 This is an example of original images of rice pests collected in this invention;

[0038] Figure 6 This is a schematic diagram of rice pest detection results in an embodiment of the present invention. Detailed Implementation

[0039] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0040] Example: This example provides a method for rice pest image acquisition and high-fidelity reconstruction detection, such as... Figure 1 As shown, it includes the following steps:

[0041] S1: Acquire field images containing information about rice plants, pest targets, and natural environmental background through a rice pest image acquisition device;

[0042] S2: Screen and label the collected images of rice pests in the field, generate corresponding target detection label files, and divide the labeled image data into training dataset and validation dataset according to a preset ratio.

[0043] S3: Construct an end-to-end detection network for rice pests based on an improved D-FINE. This network includes a feature extraction backbone network, a multi-scale feature fusion encoder, a channel-separated high-fidelity feature reconstruction module, and a target detection decoder. The feature extraction backbone network extracts feature maps at different spatial resolutions. The multi-scale feature fusion encoder performs channel mapping and cross-scale fusion on P4-scale and P5-scale features, generating P4-scale and P5-scale basic fusion features with 128 channels respectively. Before the P4-scale and P5-scale features enter the target detection decoder, CSHF modules are set to perform high-fidelity reconstruction of the corresponding scale's basic fusion features.

[0044] S4: During the training data loading phase, the input images and their corresponding bounding box information in the training dataset are simultaneously resized, and the resized input images are converted into the input data format required by the rice pest end-to-end detection network; the rice pest end-to-end detection network constructed in step S3 is trained using the processed training data to obtain the trained rice pest detection model.

[0045] S5: Deploy the validated rice pest detection model from step S4 onto the backend processing equipment to perform inference calculations on real-time acquired or pre-stored field images, and output pest category labels, predicted bounding box coordinates, and corresponding confidence scores.

[0046] Step S1 includes the following steps: Figure 2 As shown, this embodiment uses a rice pest image acquisition device with a fixed pole-type acquisition structure to acquire field images. The rice pest image acquisition device includes a fixed support module, an image acquisition module, an auxiliary lighting module, an attitude adjustment module, a distance detection module, and a data transmission module.

[0047] The fixed support module includes a fixed base, a vertical support rod, and a mounting connector. The fixed base is fixed to a preset monitoring position in the paddy field via the anchor connector. The lower end of the vertical support rod is fixedly connected to the fixed base, and the mounting connector is positioned at a preset height on the vertical support rod. The attitude adjustment module is mounted on the mounting connector. The image acquisition module, auxiliary lighting module, and distance detection module are supported by the attitude adjustment module, thereby reducing changes in the shooting position caused by wind, ground vibration, and field operation disturbances.

[0048] The attitude adjustment module adopts a dual-axis motor gimbal structure, including a horizontal rotation base, a pitch support, a first drive motor, a second drive motor, a motor driver, and an attitude controller.

[0049] The horizontal rotating base is rotatably mounted on the fixed support module. The output shaft of the first drive motor is directly or via a transmission mechanism connected to the horizontal rotating base, driving the horizontal rotating base to rotate about a vertical rotation axis that is substantially parallel to the vertical support rod. When the horizontal rotating base rotates, it drives the pitch support and the image acquisition module to rotate in the horizontal direction, thereby adjusting the azimuth angle of the image acquisition module.

[0050] The pitch support is rotatably mounted on a horizontal rotating base. The output shaft of the second drive motor is directly or via a transmission mechanism connected to the pitch support, driving the pitch support to rotate about a horizontal rotation axis that is substantially perpendicular to the vertical rotation axis. When the pitch support rotates, it causes the image acquisition module to deflect upwards or downwards, thereby adjusting the pitch angle of the image acquisition module.

[0051] In this embodiment, the first drive motor and the second drive motor are respectively connected to a motor driver, which is in turn connected to an attitude controller. Both the first and second drive motors are capable of rotational position control. The current rotation angles of the horizontal swivel mount and the pitch support are obtained by a position feedback unit inside the motor or by angle detection units respectively located on the two rotation axes.

[0052] When the device is started, the attitude controller controls the first drive motor and the second drive motor to perform initial position calibration, so that the horizontal rotating seat and the pitch support reach the preset reference position respectively, and sets the azimuth angle and pitch angle corresponding to the preset reference position as the initial angle for attitude control.

[0053] The attitude controller receives attitude control commands sent by the back-end processing device or pre-stored. These attitude control commands include at least the target azimuth angle and the target pitch angle. The attitude controller reads the current azimuth angle of the horizontal swivel mount and the current pitch angle of the pitch support, and calculates them respectively:

[0054]

[0055]

[0056] in, Indicates azimuth deviation. Indicates the target azimuth. Indicates the current azimuth angle; Indicates pitch angle deviation. Indicates the target pitch angle. Indicates the current pitch angle.

[0057] When the absolute value of the azimuth angle deviation is greater than the preset azimuth angle allowable error, the attitude controller controls the first drive motor to rotate in the forward or reverse direction according to the positive or negative direction of the azimuth angle deviation; when the absolute value of the pitch angle deviation is greater than the preset pitch angle allowable error, the attitude controller controls the second drive motor to rotate in the forward or reverse direction according to the positive or negative direction of the pitch angle deviation.

[0058] The attitude controller continuously reads the current azimuth and pitch angles during motor rotation. When:

[0059]

[0060] and:

[0061]

[0062] When the first and second drive motors stop, the first drive motor and the second drive motor are stopped. and These represent the preset azimuth angle tolerance and the preset pitch angle tolerance, respectively. This ensures that the image acquisition module's shooting direction is towards the rice target area to be monitored, and that this area is within the preset shooting field of view.

[0063] The image acquisition module is fixedly mounted on a tilting bracket, and its lens optical axis changes direction synchronously with the horizontal rotation mount and the tilting bracket. After receiving an image acquisition command, the image acquisition module acquires field images containing information about rice plants, pest targets, and the natural environment background, according to preset image resolution, exposure parameters, and acquisition time.

[0064] The auxiliary lighting module is positioned around the image acquisition module, and in one specific embodiment, it comprises multiple lighting units arranged around the lens. The illumination direction of each lighting unit is directed towards the field of view of the image acquisition module. An attitude controller or back-end processing device sends lighting control commands according to preset lighting parameters to control the auxiliary lighting module to turn on, off, or adjust its brightness. The auxiliary lighting module is activated when natural light is insufficient to improve the imaging effect of the pest's outline, texture, and body color information.

[0065] The distance detection module is positioned near the image acquisition module and rotates synchronously with the attitude adjustment module. The detection axis of the distance detection module is substantially parallel to the optical axis of the lens of the image acquisition module, ensuring that the target area detected by the distance detection module corresponds to the image capture area of ​​the image acquisition module. In one specific embodiment, the distance detection module employs a time-of-flight ranging unit or a laser ranging unit.

[0066] The distance detection module continuously acquires multiple distance measurements before image acquisition, performs outlier removal and averaging on these measurements, and obtains the effective spatial distance between the image acquisition module and the rice target area. The attitude controller or back-end processing device will effectively control the spatial distance. With respect to the preset imaging distance range Compare them.

[0067] when:

[0068]

[0069] When the target area of ​​the rice paddy is determined to be within the preset imaging distance range, the image acquisition module is allowed to perform image acquisition; when or If the target area of ​​the rice field is determined to be outside the preset imaging distance range, the current image acquisition is paused, and a prompt for adjusting the installation position of the fixed support module or a prompt for checking the shooting posture is output to the backend processing equipment.

[0070] Since the attitude adjustment module mainly changes the shooting direction of the image acquisition module, rather than directly changing the physical distance between the fixed support module and the rice target area, when the distance detection result continuously exceeds the preset imaging distance range, the problem is addressed by adjusting the installation position of the fixed support module or the imaging parameters of the image acquisition module, rather than describing the adjustment of the azimuth or pitch angle as a distance adjustment.

[0071] The data transmission module includes a data interface, a data buffer unit, and a communication unit. The data interface is connected to the image acquisition module to receive field images output by the image acquisition module; the data transmission module is also connected to the distance detection module and the attitude controller to obtain the distance detection results, azimuth angle, and pitch angle corresponding to the corresponding field images.

[0072] After receiving a field image, the data transmission module associates the field image with an image identifier, acquisition time, distance detection result, azimuth angle, and elevation angle, and forms the field image and its associated information into data to be transmitted. The communication unit then sends the data to be transmitted to the backend processing device or data storage device via wired or wireless communication.

[0073] When the communication connection is normal, the data transmission module sends the data to be transmitted and determines whether the data transmission is complete based on the reception confirmation information returned by the backend processing device. When the communication connection is interrupted or no reception confirmation information is received, the data caching unit temporarily saves the corresponding field images and their associated information and retransmits them after the communication connection is restored, so as to reduce the loss of image data caused by unstable field communication conditions.

[0074] The complete working process of the rice pest image acquisition device includes:

[0075] First, the attitude controller completes the initial position calibration of the first drive motor and the second drive motor;

[0076] Subsequently, the distance detection module acquires the effective spatial distance between the image acquisition module and the rice target area, and determines whether the effective spatial distance is within the preset imaging distance range;

[0077] After the effective spatial distance meets the requirements, the attitude controller controls the first drive motor and the second drive motor to rotate according to the target azimuth angle and the target pitch angle, so that the rice target area enters the preset shooting field of view.

[0078] Control the auxiliary lighting module to turn on or adjust the lighting brightness according to the lighting conditions of the shooting area;

[0079] The image acquisition module performs image acquisition, and the data transmission module associates the acquired field images with the acquisition time, distance detection results, azimuth angle and elevation angle, and transmits them to the back-end processing device or data storage device.

[0080] Step S2 includes the following steps: screening the acquired images and removing invalid images that do not meet the quality requirements; using the VisioFirm tool to manually classify and annotate the six types of rice pests (each using English category identifiers), and other targets not included in the preset pest categories are not labeled as positive sample categories; generating a COCO format annotation file, which includes image identifiers, pest categories, bounding box positions, and bounding box size information; and dividing the annotated images and corresponding labels into training sets and validation sets according to a preset ratio, which are used for network parameter updates and model performance evaluation, respectively.

[0081] In step S3, the rice pest end-to-end detection network is implemented based on an end-to-end architecture. The backbone network is used to extract multi-scale features from the input 640×640×3 rice pest image. The specific process is as follows: The StemBlock module performs initial feature extraction and downsampling on the input image to generate an initial feature map with a spatial size of 160×160 and 16 channels, which is used to extract low-level features such as edges and textures.

[0082] The first HG_Stage module performs further convolution processing to generate a 160×160×64 P2 scale feature map;

[0083] The second HG_Stage module performs downsampling extraction with a stride of 2 to generate a P3 scale feature map of 80×80×256.

[0084] The third HG_Stage module performs downsampling extraction with a stride of 2 to generate a 40×40×512 P4 scale feature map;

[0085] The fourth HG_Stage module performs downsampling extraction with a stride of 2, generating a 20×20×1024 P5 scale deep feature map.

[0086] Among these, the P4 and P5 scale feature maps contain intermediate and high-level semantic information, respectively, and are selected by the network as feature outputs, which are then passed to the subsequent encoder for fusion processing.

[0087] The encoder is used to perform cross-scale fusion processing on the P4-scale and P5-scale features output by the backbone network.

[0088] First, the first ConvNormLayer_fuse module is used to perform channel mapping on the P4 scale feature map with a spatial size of 40×40 and 512 channels to obtain a P4 scale mapped feature map with a spatial size of 40×40 and 128 channels. Then, the second ConvNormLayer_fuse module is used to perform channel mapping on the P5 scale feature map with a spatial size of 20×20 and 1024 channels to obtain a P5 scale mapped feature map with a spatial size of 20×20 and 128 channels.

[0089] Subsequently, the P5-scale mapped feature map is input into the TransformerEncoderBlock module for global feature encoding. The TransformerEncoderBlock module has a multi-head self-attention mechanism with 8 heads and a feedforward network dimension of 512, outputting a P5-scale global semantic feature map with a spatial size of 20×20 and 128 channels.

[0090] The P5-scale global semantic feature map is processed by the third ConvNormLayer_fuse module and upsampled by a factor of two to obtain an upsampled feature map with a spatial size of 40×40 and 128 channels. The upsampled feature map is then concatenated with the P4-scale mapping feature map along the channel dimension to obtain a P4-scale stitched feature map with a spatial size of 40×40 and 256 channels.

[0091] The P4-scale stitched feature map is input into the first RepNCSPELAN4 module for feature extraction and channel fusion to obtain a P4-scale basic fused feature map with a spatial size of 40×40 and 128 channels.

[0092] In the natural field setting of rice pests, the individual pests vary significantly in size, and the camouflage of the complex green background is quite noticeable. During conventional multi-scale feature fusion, fine-grained features such as the edges, textures, and local morphology of the pest targets are easily interfered with by background information. Therefore, this invention incorporates a channel-separated high-fidelity feature reconstruction module (CSHF module) in the feature path before the encoder output is sent to the target detection decoder, performing high-fidelity reconstruction of the fused features at the corresponding scale.

[0093] The CSHF module generates channel response weights based on the global channel description information of the input features, and uses these channel response weights and their complementary weights to perform complementary response modulation on the complete input features, forming high-response weighted features and low-response weighted features. Both the high-response weighted features and low-response weighted features retain all channels of the input features; they do not represent a fixed number of physical segments of the input features according to channel indices, nor do they represent spatial segmentation of the pest target area and the environmental background area.

[0094] Specifically, the base fused feature map output by the encoder is used as the input feature map. Input the CSHF module. For input features:

[0095]

[0096] in, Indicates the batch dimension. This indicates the number of channels for the input feature. and These represent the height and width of the input feature, respectively.

[0097] First, the input features are processed using adaptive global average pooling. Perform spatial dimension compression to obtain the channel description vector. :

[0098]

[0099] Among them, the In the nth sample Channel description values ​​for each channel Represented as:

[0100]

[0101] Number of intermediate channels in CSHF module Determined according to the following formula:

[0102]

[0103] Where r represents the channel compression ratio, and 8 represents the preset minimum number of intermediate channels.

[0104] The channel description vector The first channel mapping layer, the ReLU nonlinear activation layer, the second channel mapping layer, and the Sigmoid gated activation layer are input sequentially to generate channel response weights. :

[0105]

[0106] in, This represents the learnable parameters of the first channel mapping layer, which reduces the number of channels from... Compress to ; This represents the learnable parameters of the second channel mapping layer, which reduces the number of channels from... Restore to ; Represents the ReLU nonlinear activation function; This represents the Sigmoid-gated activation function. The channel response weights satisfy:

[0107]

[0108] According to the channel response weight and their complementary weights For input features Complementary response modulation was performed to obtain high-response weighted features. and low response weighted features :

[0109]

[0110]

[0111] in, This indicates that element-wise multiplication is performed after broadcasting the corresponding channel weights along the spatial dimension. The high-response weighted feature... and low response weighted features All input features are retained. One channel, and satisfying:

[0112]

[0113] Subsequently, the high-response weighted features were... The convolution kernel size is performed sequentially. High-response enhancement features are obtained by performing depthwise convolution, first batch normalization, and first SiLU activation. :

[0114]

[0115] The depthwise convolution has a stride of 1, a padding of 1, and both the number of input channels and the number of output channels are 1. The number of groups equals the number of input channels. ; This indicates the first batch normalization operation. This represents the first SiLU activation function. The high-response enhancement feature satisfies:

[0116]

[0117] The low response weighted features With high response enhancement features Perform element-wise addition and sequential convolution with kernel size of [size missing]. The low-response compensation features are obtained by performing ordinary convolution, a second batch normalization operation, and a second SiLU activation operation. :

[0118]

[0119] Among them, the The number of input channels and the number of output channels of convolution are both 1. ; This indicates the second batch normalization operation. This represents the second SiLU activation function. The low-response compensation feature satisfies:

[0120]

[0121] The high response enhancement feature and low response compensation characteristics Cascade along the channel dimension to obtain cascade features. :

[0122]

[0123] in:

[0124]

[0125] For the cascaded features The number of input channels is as follows The number of output channels is of The convolutional operation, the third batch normalization operation, and the third SiLU activation operation are combined to obtain the fused reconstructed features; then, the fused reconstructed features are compared with the input features. By summing the residuals, high-fidelity output characteristics can be obtained. :

[0126]

[0127] in, This indicates the third batch normalization operation. This represents the third SiLU activation function. The high-fidelity output features have the same spatial size and number of channels as the input features, i.e.:

[0128]

[0129] In this embodiment, the channel compression ratio r of the CSHF module is set to 4, therefore the number of intermediate channels is determined according to the following formula:

[0130]

[0131] Both the first and second channel mapping layers use convolutional kernels with a size of [size missing]. Furthermore, convolutions without bias terms are used. The non-linear activation function after the first channel mapping layer is the ReLU activation function, and the gated activation function after the second channel mapping layer is the Sigmoid activation function.

[0132] The high-response feature enhancement unit uses a convolution kernel size of A depthwise convolution with a stride of 1, padding of 1, and a group size equal to the number of input channels is performed, followed by BatchNorm2d batch normalization and SiLU activation. No bias term is set for the depthwise convolution.

[0133] The low-response information compensation unit uses the same number of input channels and output channels, and the convolution kernel size is... The ordinary convolution without bias term is followed by BatchNorm2d batch normalization and SiLU activation.

[0134] The fusion reconstruction unit uses an input channel count that is twice the number of input feature channels, an output channel count that is the same as the number of input feature channels, and a convolutional kernel size of [missing value]. The ordinary convolution without bias term is followed by BatchNorm2d batch normalization and SiLU activation.

[0135] In the P4 scale branch of this embodiment, the CSHF module has 128 input channels and 128 output channels, and its spatial size is [missing information]. Based on the channel compression ratio The number of intermediate channels is:

[0136]

[0137] Therefore, the first channel mapping layer compresses the number of channels from 128 to 32, and the second channel mapping layer restores the number of channels from 32 to 128. Since the number of input channels is the same as the number of output channels, the residual connections use identity mapping. The dimensions of the high-response weighted features, low-response weighted features, high-response enhancement features, low-response compensation features, and P4-scale high-fidelity output features are all:

[0138]

[0139] In the P5 scale branch of this embodiment, the CSHF module has 128 input and output channels, a spatial size of 20×20, and 32 intermediate channels. The dimensions of the P5 scale high-fidelity output features are:

[0140]

[0141] The high-fidelity features at the P4 scale output of the first CSHF module are used to construct the subsequent P5-scale fused features. The second CSHF module processes the basic fused features at the P5 scale using the same channel response weight generation, complementary response modulation, high-response local spatial enhancement, low-response information compensation, and fusion residual reconstruction process as the first CSHF module. The outputs of the two CSHF modules are respectively passed to the target detection decoding module to generate the category prediction results and bounding box prediction results for pest targets.

[0142] Subsequently, the P4-scale high-fidelity reconstructed features output by the first CSHF module are input into the SCDown module for double downsampling to obtain a P4 downsampled feature map with a spatial size of 20×20 and 128 channels.

[0143] The P4 downsampled feature map and the P5 scale transformation feature map output by the third ConvNormLayer_fuse module are concatenated along the channel dimension to obtain a P5 scale stitched feature map with a spatial size of 20×20 and 256 channels.

[0144] The P5-scale stitched feature map is input into the second RepNCSPELAN4 module for multi-branch feature extraction and channel fusion to obtain a P5-scale basic fused feature map with a spatial size of 20×20 and 128 channels.

[0145] The P5-scale basic fusion feature map is input into the second CSHF module and processed according to the same process as the first CSHF module, including channel description information extraction, channel response weight generation, complementary response modulation, high-response local spatial feature enhancement, low-response information compensation, and fusion residual reconstruction, to obtain a P5-scale high-fidelity reconstructed feature map with a spatial size of 20×20 and 128 channels.

[0146] The target detection decoder uses the DFINETransformer module based on the Transformer structure, and simultaneously receives the P4-scale high-fidelity reconstructed feature map output by the first CSHF module and the P5-scale high-fidelity reconstructed feature map output by the second CSHF module.

[0147] The P4-scale high-fidelity reconstructed feature map has a spatial size of 40×40 and 128 channels, corresponding to a feature step size of 16 relative to the input image; the P5-scale high-fidelity reconstructed feature map has a spatial size of 20×20 and 128 channels, corresponding to a feature step size of 32 relative to the input image.

[0148] In this embodiment, the hidden feature dimension of the DFINETransformer module is set to 128, the number of input feature layers is set to 2, the number of object detection decoding layers is set to 3, the feedforward network dimension is set to 512, and the number of attention sampling points at both the P4 and P5 scales is set to 6.

[0149] The DFINETransformer module establishes the correlation between target queries and P4-scale high-fidelity reconstruction features and P5-scale high-fidelity reconstruction features through multi-scale feature interaction and target query decoding, and outputs the category prediction results and bounding box prediction results of pest targets.

[0150] During the network training phase, a one-to-one bipartite graph matching method based on the Hungarian algorithm is adopted to establish a matching relationship between the set of predicted results output by the object detection decoder and the set of real annotations. Based on the matching relationship, the predicted results used to calculate the classification loss, bounding box regression loss, overlap loss, and localization distribution related loss are determined.

[0151] During the detection inference phase, without inputting a real labeled set, the pest category label, predicted bounding box coordinates, and corresponding confidence scores are obtained based on the category prediction results and bounding box prediction results output by the object detection decoder.

[0152] In step S4, the rice pest target detection dataset is used to train the rice pest end-to-end detection network.

[0153] During the training data loading phase, the input images in the training dataset are resized to 640×640 pixels, and the bounding box positions and sizes of the corresponding pest targets are simultaneously adjusted according to the image size transformation relationship to ensure that the resized bounding boxes correspond to the pest targets in the input images. Subsequently, the resized images are converted into the input data format required by the rice pest end-to-end detection network and input into the network in batches.

[0154] The AdamW optimization algorithm is used during training. The parameters of the AdamW optimizer... and Set them to 0.9 and 0.999 respectively, and set the global base weight decay coefficient to... .

[0155] A hierarchical learning rate strategy is adopted to set the parameters in the backbone network, excluding the normalization layer and bias parameters. The learning rate; the parameter settings for the normalization layers in the backbone network. The learning rate is set, and the corresponding weight decay coefficient is set to... The base learning rate for the remaining network parameters is set to... The normalization layer parameters and bias parameters in the encoder and decoder do not undergo weight decay.

[0156] In this embodiment, the maximum number of training rounds is set to 300. The training data transformation strategy is switched at the 4th, 148th, and 288th rounds of training, respectively. MixUp augmentation is enabled in the training phases corresponding to the 4th to 148th rounds. Data augmentation is stopped after the 288th round of training, and the final parameters are adjusted using training samples without data augmentation in the 289th to 300th rounds.

[0157] In addition to MixUp augmentation, training data can also be augmented with at least one of the following basic data augmentations: random flipping, scaling, or color transformation, depending on the training data loading configuration.

[0158] The network training employs a joint loss function to optimize the parameters of the rice pest end-to-end detection network. This joint loss function includes variable focus classification loss, bounding box L1 regression loss, generalized intersection-over-union loss, fine-grained localization loss, and decoupled distillation focus loss, expressed as:

[0159]

[0160] in, This represents the total loss during network training; This represents the Variable Focus Loss (VFL), used to constrain the difference between the predicted pest target category and the true category label; This represents the bounding box L1 regression loss, used to constrain the positional and dimensional deviations between the predicted and actual bounding box parameters. This represents the generalized intersection-union loss, used to constrain the overlap between the predicted bounding box and the ground truth bounding box; This represents the fine-grained localization loss (FGL), which is used to constrain the localization distribution of the bounding box. This represents the Decoupled Distillation Focal Loss (DDF), which is used to constrain the self-distillation of the localization distribution between different decoding layers.

[0161] In this embodiment, the basic weights of each loss term are set as follows:

[0162]

[0163] Therefore, the joint loss function is expressed as:

[0164]

[0165] The class balance parameter of the variable focus classification loss Set to 0.75, adjustment parameter for easy and difficult samples. Set to 2.0; upper limit parameter for bounding box positioning distribution Set it to 32.

[0166] The intersection-over-union ratio (IoU) between the predicted bounding box and the true bounding box is expressed as:

[0167]

[0168] The generalized intersection-union-ratio is expressed as:

[0169]

[0170] The corresponding generalized cross-union ratio loss is expressed as:

[0171]

[0172] in, This represents the area of ​​the intersection region between the predicted bounding box and the ground truth bounding box. This represents the area of ​​the union region between the predicted bounding box and the ground truth bounding box. This represents the area of ​​the smallest closure rectangle that simultaneously encloses both the predicted and actual bounding boxes.

[0173] During the training phase of the detection network, the Hungarian matching algorithm is used to establish a one-to-one matching relationship between the predicted result set output by the object detection decoding module and the ground truth annotation set. The matching cost includes classification matching cost, bounding box L1 distance cost, and generalized intersection-union (CIU) cost, expressed as:

[0174]

[0175] In this study, the class balance parameter for the classification matching cost was set to 0.25, and the difficulty sample adjustment parameter was set to 2.0. The class balance parameter used in the matching phase is independent of the class balance parameter of the variable focus classification loss in the joint loss function.

[0176] The backend processing device in step S5 includes an edge computing device, a server, or a mobile terminal with model inference capabilities. The backend processing device includes a memory, a processor, and a detection program stored in the memory and executable by the processor. The detection program loads the rice pest detection model trained and validated in step S4, receives field images acquired in real-time by a rice pest image acquisition device or pre-stored, and performs image resizing, data format conversion, network forward computation, and detection result parsing on the input images. The detection program outputs rice pest identification results including pest category labels, predicted bounding box coordinates, and corresponding confidence levels. The confidence level represents the reliability of the detection model's prediction of the corresponding prediction result; its specific value range and calculation method are determined by the output format of the target detection decoder.

Claims

1. A method for image acquisition and high-fidelity reconstruction detection of rice pests, characterized in that, Includes the following steps: S1. Acquire field images containing information about rice plants, pest targets, and natural environmental background using a rice pest image acquisition device; S2. The collected rice pest images are filtered, labeled, and the dataset is constructed to obtain a rice pest target detection dataset for model training. S3. Construct an end-to-end detection network for rice pests. The detection network includes a feature extraction backbone network, a multi-scale feature fusion module, a channel-separated high-fidelity feature reconstruction module, and a target detection decoding module. The multi-scale feature fusion module is used to fuse features of different scales output by the feature extraction backbone network, and a channel-separated high-fidelity feature reconstruction module, referred to as the CSHF module, is set at the multi-scale fused feature output end. The CSHF module generates channel response weights based on the global description information of each channel of the input feature, and generates complementary weights based on the complementary values ​​of the channel response weights. It then uses the channel response weights and the complementary weights to perform complementary response modulation on the input feature with a full number of channels, forming a high-response weighted feature and a low-response weighted feature with the same number of channels as the input feature. Local spatial feature enhancement is performed on the high-response weighted feature to obtain a high-response enhanced feature. The low-response weighted feature and the high-response enhanced feature are added element-wise, and a low-response compensation feature is obtained through channel mapping. Finally, the high-response enhanced feature and the low-response compensation feature are concatenated along the channel dimension, and a high-fidelity reconstructed feature is obtained through channel fusion and residual reconstruction. S4. Use the dataset constructed in step S2 to train the detection network to obtain a rice pest detection model. S5. Deploy the trained rice pest detection model to the backend processing equipment, perform inference calculations on the collected rice field images, and output pest category labels, predicted bounding box coordinates, and corresponding confidence scores.

2. The method for rice pest image acquisition and high-fidelity reconstruction detection according to claim 1, characterized in that: The rice pest image acquisition device adopts a fixed rod acquisition structure, including a fixed support module, an image acquisition module, an auxiliary lighting module, an attitude adjustment module, a distance detection module, and a data transmission module. The fixed support module includes a fixed base, a vertical support rod, and mounting connectors. The fixed base is fixedly set at a preset monitoring position in the paddy field. The vertical support rod is installed on the fixed base, and the mounting connectors are set on the vertical support rods for installing and supporting the attitude adjustment module, image acquisition module, auxiliary lighting module, distance detection module, and data transmission module. The attitude adjustment module includes a horizontal rotating base, a pitch support, a first drive motor, a second drive motor, a motor driver, and an attitude controller. The horizontal rotating base is rotatably mounted on the fixed support module. The first drive motor is driven by the horizontal rotating base and is used to drive the horizontal rotating base to rotate around a first vertical rotation axis to adjust the azimuth angle of the image acquisition module. The pitch support is rotatably mounted on the horizontal rotating base. The second drive motor is driven by the pitch support and is used to drive the pitch support to rotate around a second horizontal rotation axis to adjust the pitch angle of the image acquisition module. The image acquisition module is fixedly mounted on the pitch support and rotates with the horizontal rotating seat and the pitch support. It is used to acquire field images containing information on rice plants, pest targets, and natural environmental background. The attitude controller is used to receive attitude control commands including the target azimuth angle and the target pitch angle, control the rotation of the first drive motor according to the first angle deviation between the current azimuth angle and the target azimuth angle, and control the rotation of the second drive motor according to the second angle deviation between the current pitch angle and the target pitch angle, until the first angle deviation and the second angle deviation are respectively within the corresponding preset allowable error range, so that the rice target area to be monitored is located within the preset shooting field of view of the image acquisition module. The auxiliary lighting module is located around the image acquisition module and the lighting direction is towards the preset shooting field of view. It is used to provide supplementary lighting for the rice target area and adjust the lighting on-state or lighting brightness according to the lighting control command to reduce the impact of natural light changes on the brightness and clarity of the acquired image. The distance detection module is located near the image acquisition module, and the detection direction of the distance detection module is consistent with or approximately parallel to the shooting direction of the image acquisition module. It is used to detect the spatial distance between the image acquisition module and the rice target area, and send the distance detection result to the attitude controller or back-end processing device. The attitude controller or back-end processing device is used to compare the distance detection result with a preset imaging distance range; when the distance detection result is within the preset imaging distance range, the image acquisition module is allowed to perform image acquisition; when the distance detection result exceeds the preset imaging distance range, an installation position adjustment prompt, an imaging parameter adjustment prompt, or a shooting posture check prompt is output, and the current image acquisition is paused. The data transmission module is communicatively connected to the image acquisition module, distance detection module, and attitude adjustment module, respectively, and is used to receive the field image and the acquisition time, distance detection result, azimuth angle, and pitch angle information corresponding to the field image, and transmit the field image and its corresponding information to the back-end processing device or data storage device.

3. The method for rice pest image acquisition and high-fidelity reconstruction detection according to claim 1, characterized in that: The method for constructing the rice pest target detection dataset in step S2 includes: S21. Screen the acquired field images and remove invalid images whose image quality does not meet the requirements; S22. Label the pest targets in the filtered images with category and bounding box location; S23. Generate object detection label files, and divide the labeled images and their corresponding object detection label files into training datasets and validation datasets according to a preset ratio.

4. The method for rice pest image acquisition and high-fidelity reconstruction detection according to claim 1, characterized in that: The multi-scale feature fusion module includes at least two scale feature branches, and the CSHF module is set at the output position of different scale feature branches respectively. The CSHF module includes a channel response evaluation unit, a high response feature enhancement unit, a low response information compensation unit, and a fusion reconstruction unit. The channel response evaluation unit is used to perform global average pooling on the input features to obtain channel description information corresponding to each channel of the input features, and to generate channel response weights based on the channel description information. The high-response feature enhancement unit is used to weight the input features according to the channel response weights to obtain high-response weighted features, and to enhance the local spatial features of the high-response weighted features through deep convolution, normalization and feature activation to obtain high-response enhanced features. The low response information compensation unit is used to weight the input features according to the complementary weights of the channel response weights to obtain low response weighted features, add the low response weighted features to the high response enhancement features element by element, and obtain low response compensation features through channel mapping, normalization and feature activation. The fusion and reconstruction unit is used to cascade and fuse the high-response enhancement features and low-response compensation features along the channel dimension, and perform residual fusion with the channel fusion result and the input features to generate high-fidelity output features.

5. The method for rice pest image acquisition and high-fidelity reconstruction detection according to claim 4, characterized in that: The characteristic response modulation and reconstruction process of the CSHF module includes: For input features: in, Indicates the batch dimension. This indicates the number of channels for the input feature. and These represent the height and width of the input feature, respectively; the spatial dimension of the input feature is compressed using global average pooling to obtain the channel description vector: Among them, the In the nth sample The channel description values ​​for each channel are: number of intermediate channels Determined according to the following formula: Where r represents the channel compression ratio; Based on the channel description vector, channel response weights are generated sequentially through first channel mapping, nonlinear activation, second channel mapping, and gated activation: in, This indicates that the number of channels will be changed from... Compress to The learnable parameters of the first channel mapping, This indicates that the number of channels will be changed from... Restore to The learnable parameters of the second channel mapping, Represents the ReLU nonlinear activation function; Let represent the Sigmoid-gated activation function, and: Based on the channel response weights and their complementary weights, complementary response modulation is performed on the input features to obtain high-response weighted features and low-response weighted features respectively: in, This indicates an element-wise multiplication performed after broadcasting the channel weights along the spatial dimension. Both the high-response weighted features and the low-response weighted features retain all of the input features. One channel, and satisfying: The high-response weighted features are sequentially convolved with kernels of size [size missing]. Depthwise convolution, the first batch normalization operation, and the first SiLU activation operation are used to obtain high-response enhanced features: The depthwise convolution has a stride of 1, a padding of 1, and both the number of input channels and the number of output channels are 1. And the number of groups is equal to ; This indicates the first batch normalization operation. This represents the first SiLU activation function; The low-response weighted features and high-response enhancement features are added element-wise, and then the input and output channels are kept constant. of Convolution, a second batch normalization operation, and a second SiLU activation operation are used to obtain low-response compensation features. in, This indicates the second batch normalization operation. This represents the second SiLU activation function; The high-response enhancement feature and low-response compensation feature are cascaded along the channel dimension: in: The cascaded features are sequentially processed with the following number of input channels: The number of output channels is of The convolutional, third batch normalization, and third SiLU activation operations are combined, and the resulting fused features are residually fused with the input features to obtain high-fidelity output features. in, This indicates the third batch normalization operation. This represents the third SiLU activation function; The high-fidelity output feature has the same spatial size and number of channels as the input feature, that is: and the high-fidelity output feature It is then passed to the subsequent feature fusion module or target detection decoding module.

6. The method for rice pest image acquisition and high-fidelity reconstruction detection according to claim 1, characterized in that: The target detection decoding module adopts an end-to-end target detection decoder based on the Transformer structure; The target detection decoder receives high-fidelity features at different scales after processing by the CSHF module, establishes the association between the target query and the high-fidelity features at different scales through attention calculation, and generates the category prediction result and bounding box prediction result of the pest target. During the network training phase, a one-to-one bipartite graph matching method is used to establish a matching relationship between the predicted result set and the real annotation set, and the predicted results used to calculate the classification loss, bounding box regression loss, overlap loss and localization distribution related loss are determined based on the matching relationship. During the detection inference phase, the category label, predicted bounding box coordinates, and corresponding confidence scores of the pest targets are obtained based on the output of the target detection decoder.

7. The method for rice pest image acquisition and high-fidelity reconstruction detection according to claim 1, characterized in that: The back-end processing device includes a memory, a processor, and a detection program; The detection program is stored in the memory and can be executed by the processor; The detection program is used to load the trained rice pest detection model, perform feature extraction, feature reconstruction, and target detection inference on the collected field images, and output detection results containing pest category labels, bounding box coordinates, and confidence information.

Images