Paddy field weed detection method, device and equipment based on unmanned aerial vehicle low-altitude remote sensing and medium

Abstract

The invention relates to a paddy field weed detection method, device and equipment based on unmanned aerial vehicle low-altitude remote sensing and a medium, and the method comprises the steps: inputting a paddy field low-altitude remote sensing image obtained by an unmanned aerial vehicle platform, and extracting a multi-scale feature map containing shallow texture information and deep semantic information at the same time; inputting into a cavity space pyramid pooling module for processing to obtain a feature map set; inputting the feature map set into a feature refining module, and performing iterative feature refining and result prediction through a cascade residual decoding and gating guide attention module; and inputting the prediction result graph and the high-resolution feature graph in the set into a feature fusion module, performing feature fusion and generating a prediction response graph, and restoring the prediction response graph to the spatial resolution consistent with the original input image through up-sampling to obtain a final detection result graph. According to the method, the representation of the target area can be continuously improved on the basis of initial rough positioning, so that the detection accuracy of weak and camouflage weed targets is remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of agricultural remote sensing and computer vision technology, and in particular to a method, device, electronic equipment and computer-readable storage medium for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles. Background Technology

[0002] Weeds in paddy fields are one of the major biological stresses affecting rice yield and quality. While traditional large-scale chemical weeding methods have certain operational efficiency, they easily lead to problems such as excessive pesticide use, environmental pollution, and increased production costs. With the development of smart agriculture and precision agriculture, there is an urgent need to obtain spatial distribution information of weeds in the field through refined sensing methods to support precision weeding operations.

[0003] In recent years, UAV low-altitude remote sensing technology has been widely used for monitoring weeds in farmland due to its advantages such as high acquisition efficiency and wide coverage. However, in rice paddy scenarios, weeds typically have characteristics such as small target scale, similar texture and shape to rice, and easy occlusion, presenting obvious characteristics of weak and camouflaged targets, which brings great difficulties to the automatic identification of weeds in UAV images.

[0004] In low-altitude remote sensing of rice paddy scenarios by UAVs, existing deep learning-based weed detection methods still have shortcomings in complex rice paddy backgrounds. Due to the small scale of weed targets and their high similarity to the texture of crop backgrounds, existing weed detection and segmentation methods are prone to problems such as missed or false detection of weak weed targets, easy interference of feature responses by backgrounds, and unclear boundaries of prediction results under complex background conditions, making it difficult to meet the actual needs of fine weed identification in rice paddies. Summary of the Invention

[0005] The purpose of this invention is to overcome the aforementioned problems. This invention provides a method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles (UAVs), a corresponding device, electronic equipment, and a computer-readable storage medium.

[0006] To address the aforementioned technical problems, the present invention proposes the following technical solution: To achieve one of the objectives of this invention, a method for detecting weeds in paddy fields based on low-altitude remote sensing from unmanned aerial vehicles (UAVs) is proposed, comprising the following specific steps: S10. Input the low-altitude remote sensing image of the paddy field obtained by the UAV platform, preprocess it, and then input it into the feature extraction network for multi-scale feature extraction: use the hierarchical visual VMamba based on the state space model as the backbone network to extract multi-scale feature maps that contain both shallow texture information and deep semantic information, forming a multi-scale feature pyramid. S20. The multi-scale feature maps output from the backbone network are sequentially input into the hollow spatial pyramid pooling module for context enhancement and channel alignment, resulting in a feature map set with unified channel dimensions and incorporating multi-scale context information. ; S30, Set the feature map set The input is fed into the feature refinement module for iterative feature refinement and result prediction. In the first round of iterations, the feature set... Decoding features are obtained through cascaded residual decoding, and result prediction is performed based on these decoding features to generate a coarse prediction result for the initial weed area. Starting from the second iteration, a gated guidance attention module is introduced when performing cascaded residual decoding. The decoding features of the previous level are used as the spatial prior guidance information of the current level. For the first level, the prediction results of the previous round are used as the spatial prior guidance information. Based on spatial prior guidance information, the feature weights of the feature map of the current level are modulated, and then decoded by a cascaded residual feature decoder to obtain the decoded features of the current level. After decoding each level in turn, the final decoded features are obtained. The final decoded features are predicted to obtain the prediction result map of the current iteration round. After multiple iterations, the final prediction result is obtained. S40. Obtain the final prediction result map and feature map set. The high-resolution feature map is input to the feature fusion module. After feature fusion, the fused feature is obtained. Based on the fused feature, the prediction response map is generated and then the spatial resolution is restored to the same as that of the original input image through upsampling operation to obtain the final detection result map.

[0007] Preferably, the preprocessing of the original input image includes: S101. Use bilinear interpolation algorithm to uniformly resample all original input images to a fixed resolution; S102. Normalize each channel of the image, and perform standardization operation on each channel based on the preset channel mean and standard deviation, so that the input data meets the requirements of the network calculation for the consistency of numerical range and distribution, and complete the preprocessing.

[0008] Preferably, the specific steps for inputting the multi-scale feature maps output by the backbone network into the hollow spatial pyramid pooling module for context enhancement processing and channel alignment include: S201. Process the multi-scale feature map individually in sequence; S202. For each feature map, it is processed in parallel through a 1×1 convolution branch, three dilated convolution branches with different dilation rates, and a global context branch to complete the feature map context enhancement processing, and the processing results of each branch are spliced ​​together in the channel dimension. S203. Through the projection layer, the stitched features are uniformly mapped to a fixed channel dimension to obtain a feature map enhanced by multi-scale context. S204. After processing the multi-scale feature maps sequentially, a set of feature maps with multi-scale context enhancement is obtained. .

[0009] Preferably, the specific steps for obtaining the decoding features of the current level in the second and subsequent iterations include: S301. Use the decoding features of the previous level as the spatial prior guidance information of the current level. For the first level, use the prediction results of the previous round as the spatial prior guidance information. S302. Generate the gating signal of the current level based on the spatial prior guidance information, concatenate the gating signal with the feature map of the current level in the channel dimension, and generate a position-related gating weight map through batch normalization, convolution transformation and nonlinear activation function. S303. Adjust the feature representation of the current layer using residual gating modulation based on the gating weight map, and integrate the features through the output convolutional layer to obtain the gating enhanced feature map. S304. The gated enhanced feature map is decoded by a cascaded residual feature decoder to obtain the decoded features of the current level.

[0010] Furthermore, the cascaded residual feature decoder includes multiple cascaded residual channel attention blocks and convolutional layers; the cascaded residual feature decoder is configured with input channels and output channels of corresponding channel dimensions according to different decoding levels.

[0011] Furthermore, the processing steps of the cascaded residual feature decoder during decoding include: First, the feature map is refined in depth through several residual channel attention blocks; Then, the refined feature map is input into the convolutional layer for feature integration and channel mapping to generate high-frequency residual features relative to the input of the cascaded residual feature decoder. Finally, through global residual connection, the high-frequency residual feature is added to the initial input feature of the cascaded residual feature decoder to obtain the decoding feature of this stage of the decoding module.

[0012] Preferably, the processing steps within the feature fusion module include: S401, Set up multi-scale feature maps The medium-to-high resolution feature map is downsampled to align its spatial scale with the final prediction result map; S402. The final prediction result map and the aligned high-resolution feature map are concatenated along the channel dimension to obtain the fused input feature. S403. The fused input features are input into the hollow space pyramid pooling module for joint modeling. By introducing receptive field information of different scales in parallel, multi-scale context modeling and information integration are performed on the fused input features to obtain fused features.

[0013] A paddy field weed detection device based on low-altitude remote sensing by an unmanned aerial vehicle (UAV) is provided to suit another objective of the present invention, comprising: The feature extraction module is set to take low-altitude remote sensing images of rice paddies acquired by the UAV platform as input, preprocess them, and then input them into the feature extraction network for multi-scale feature extraction: the hierarchical visual VMamba based on the state space model is used as the backbone network to extract multi-scale feature maps that simultaneously contain shallow texture information and deep semantic information, forming a multi-scale feature pyramid. The feature processing module is configured to sequentially input the multi-scale feature maps output from the backbone network into the hollow spatial pyramid pooling module for context enhancement and channel alignment, resulting in a feature map set with uniform channel dimensions and incorporating multi-scale contextual information. ; The feature iterative prediction module is configured to use the feature map set The input is fed into the feature refinement module for iterative feature refinement and result prediction: In the first iteration, the feature set is... Decoding features are obtained through cascaded residual decoding. Based on these features, result prediction is performed to generate a coarse prediction result for the initial weed area. Starting from the second iteration, a gated guided attention module is introduced during cascaded residual decoding. The decoding features of the previous level are used as the spatial prior guidance information for the current level. For the first level, the prediction result of the previous round is used as the spatial prior guidance information. The feature weights of the feature map of the current level are modulated according to the spatial prior guidance information, and then decoded through the cascaded residual feature decoder to obtain the decoding features of the current level. After decoding each level in sequence, the final decoding features are obtained. Result prediction is performed on the final decoding features to obtain the prediction result map of the current iteration. After multiple iterations, the final prediction result is obtained. The feature fusion and prediction module is configured to obtain the final prediction result map and feature map set. The high-resolution feature map is input to the feature fusion module. After feature fusion, the fused feature is obtained. Based on the fused feature, the prediction response map is generated and then the spatial resolution is restored to the same as that of the original input image through upsampling operation to obtain the final detection result map.

[0014] An electronic device provided for another purpose of the present invention includes a central processing unit and a memory, wherein the central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the rice paddy weed detection method based on UAV low-altitude remote sensing of the present invention.

[0015] A computer-readable storage medium is provided to suit another objective of the present invention, which stores, in the form of computer-readable instructions, a computer program implemented according to the method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles, which, when invoked by a computer, executes the steps included in the corresponding method.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention introduces a multi-round iterative feature refinement mechanism from coarse to fine, and corrects the prediction deviation round by round through cyclic feedback, so that the model can continuously improve the representation of the target area on the basis of the initial coarse positioning, thereby significantly improving the detection accuracy of weak and disguised weed targets.

[0017] 2. In the iterative refinement process, this invention introduces a gated attention module, which uses the prediction results obtained in the previous iteration as spatial guidance information to adaptively modulate the feature response of the current iteration. This mechanism can effectively enhance the feature expression of potential weed areas, while suppressing interference from similar backgrounds such as water surface reflection and rice leaf texture, thereby improving the model's discrimination robustness in complex field environments and reducing the probability of false detection.

[0018] 3. This invention employs VMamba, a hierarchical visual backbone network based on a state-space model, which maintains linear computational complexity while possessing global context modeling capabilities. Compared with traditional convolutional networks, which are limited by local receptive fields, and Transformer structures, which have high computational overhead under high-resolution input, this invention can process high-resolution rice paddy remote sensing images acquired by UAVs more efficiently and effectively capture long-distance spatial dependencies under limited computational resources.

[0019] 4. This invention addresses the problem of spatial detail loss caused by deep features during multi-level downsampling. In the iterative decoding stage, this invention adopts a top-down cascaded residual decoding structure and introduces high-resolution detail features to fuse with the iterative results in the final prediction stage. This strategy effectively restores the edge contours and geometric structures of weed targets while ensuring global semantic consistency, resulting in a significant improvement in both regional integrity and boundary clarity of the final detection results. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart illustrating the method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles in an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall network architecture of the paddy field weed detection model; Figure 3 This is a schematic diagram of the gating and attention-guided module structure. Figure 4 This is a schematic diagram of the principle of the paddy field weed detection device based on low-altitude remote sensing by UAV in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the computer device in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of the cascaded residual feature decoder in an embodiment of the present invention. Detailed Implementation

[0022] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0023] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0024] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0025] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDAs (Personal Digital Assistants) that may include radio frequency receivers, pagers, internet / intranet access, web browsers, notebooks, calendars, and / or GPS (Global Positioning System) receivers; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0026] The hardware referred to by the terms "server," "client," and "service node" in this invention is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0027] It should be noted that the concept of "server" used in this invention can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method of this invention.

[0028] Unless otherwise specified, one or more technical features of this invention can be deployed on a server and accessed by a client remotely calling the online service interface provided by the server, or they can be directly deployed and run on a client for access.

[0029] Unless explicitly specified, the neural network models referenced or potentially referenced in this invention can be deployed on a remote server and invoked remotely on a client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on a client, the corresponding intelligence can be acquired through transfer learning to reduce the requirements on client hardware resources and avoid excessive consumption of client hardware resources.

[0030] Unless otherwise specified, all data involved in this invention may be stored remotely on a server or on a local terminal device, as long as it is suitable for being accessed by the technical solution of this invention.

[0031] Those skilled in the art will understand that although the various methods of the present invention are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this invention are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood as equivalent.

[0032] Unless otherwise expressly stated, the various embodiments disclosed in this invention can be combined in various ways to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this invention and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0033] like Figure 2As shown, in some embodiments, the present invention proposes a paddy field weed detection model for implementing a method for detecting paddy field weeds based on low-altitude remote sensing from unmanned aerial vehicles. The model includes: a hierarchical visual backbone network VMamba, a feature refinement module, and a feature aggregation fusion (FAF) module. The feature refinement module includes a gated guided attention (GGA) module and a cascaded residual feature decoder (RFD). The feature aggregation module includes an atrous spatial pyramid pooling (ASPP) module.

[0034] like Figure 1 As shown, the present invention proposes a method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles, which includes the following specific steps: S10. Input the low-altitude remote sensing image of the paddy field obtained by the UAV platform, preprocess it, and then input it into the feature extraction network for multi-scale feature extraction: use the hierarchical visual VMamba based on the state space model as the backbone network to extract multi-scale feature maps that contain both shallow texture information and deep semantic information, forming a multi-scale feature pyramid.

[0035] In some embodiments, the present invention uses low-altitude remote sensing images of rice paddies acquired by an unmanned aerial vehicle (UAV) platform as input data. The remote sensing images are visible light RGB images. To ensure the effectiveness of subsequent detection and processing, the acquired raw images are preprocessed. The preprocessing of the raw input images includes: S101. Given that the original images have high resolution and may have different scales, a bilinear interpolation algorithm is used to uniformly resample all original input images to a fixed resolution.

[0036] S102. Normalize each channel of the image, that is, first convert the pixel value into a floating-point representation in the range of [0,1], and then perform standardization operation on each channel based on the preset channel mean and standard deviation, so that the input data meets the requirements of the network calculation for the consistency of numerical range and distribution, and complete the preprocessing.

[0037] In addition, during the model training phase, in order to suppress overfitting and improve the model's generalization ability, an online data augmentation strategy is introduced. Different flight perspectives are simulated by performing geometric transformations such as random horizontal flipping and random cropping, and brightness, contrast and sharpness are randomly adjusted to cope with changes in lighting.

[0038] The preprocessed low-altitude remote sensing RGB images of rice paddies from UAVs are input into the feature extraction network. This invention employs VMamba, a hierarchical visual backbone network based on a State Space Model (SSM), as the feature encoder for multi-scale feature extraction from the input rice paddy remote sensing images. The VMamba backbone network, through state space modeling, effectively models long-range dependencies while maintaining low computational complexity, making it suitable for weed detection and segmentation tasks in complex backgrounds within high-resolution UAV remote sensing images.

[0039] In some embodiments, the VMamba backbone network adopts a phased hierarchical structure design. During the feature extraction process, by merging and downsampling spatial location features, the spatial sampling density of the feature map is gradually reduced, thereby achieving a gradual reduction in spatial resolution and improving the semantic abstraction level of the features, ultimately constructing a multi-scale feature pyramid representation.

[0040] The features output from different stages have significant differences in semantic level and spatial resolution: the features output from shallow stages have high spatial resolution and fewer channels, and can retain rich texture, edge and local geometric structure information; the features output from deep stages have lower spatial resolution and higher channel dimension, and can represent a wider range of contextual information and contain more abstract global semantic representations.

[0041] ; in, This represents the high spatial resolution features output from the shallow stage, with a downsampling factor of 4 and a spatial resolution of 1 / 4 of the original input image, preserving rich texture and geometric details; The features are mid-level features, with a downsampling factor of 8 and a spatial resolution of 1 / 8 of the original input image; For the second-deepest features, the downsampling factor is 16, and the spatial resolution is 1 / 16 of the original input image; This indicates low spatial resolution, high semantic features output from the deep stage, with a downsampling factor of 32.

[0042] S20. The multi-scale feature maps output from the backbone network are sequentially input into the hollow spatial pyramid pooling module for context enhancement and channel alignment, resulting in a feature map set with unified channel dimensions and incorporating multi-scale context information. .

[0043] To further enhance the expressive power of multi-scale features for weed targets of different sizes and improve the model's ability to distinguish weak weed targets in complex paddy field backgrounds, this invention sets up the multi-scale feature maps output by the backbone network. The data is then sequentially input into the Atrous Spatial Pyramid Pooling (ASPP) module for context enhancement processing.

[0044] In some embodiments, the ASPP module employs a multi-branch parallel structure, simultaneously introducing context modeling capabilities at different scales for features at the same level. Specifically, the ASPP module includes a 1×1 convolutional branch, three 3×3 dilated convolutional branches with different dilation rates, and a global context branch.

[0045] The specific steps for inputting the multi-scale feature maps output by the backbone network into the hollow spatial pyramid pooling module for context enhancement and channel alignment include: S201. Process the multi-scale feature map individually in sequence.

[0046] S202. For each feature map, it is processed in parallel through a 1×1 convolution branch, three 3×3 dilated convolution branches with different dilation rates, and a global context branch to complete the feature map context enhancement processing, and the processing results of each branch are concatenated in the channel dimension.

[0047] The 1×1 convolutional branch is used to preserve local details of the original features; The porosity of the three dilated convolution branches is respectively set as follows: This is used to introduce multi-scale receptive field information from near to far while keeping the kernel size constant, thereby enhancing the coverage of weed patches of different sizes. Meanwhile, to ensure that the spatial resolution of the output feature map remains consistent with the input, each branch is equipped with zero padding equal to the hole rate during the convolution operation. The global context branch first performs global average pooling on the feature map to obtain image-level global semantic information, then adjusts the channels through 1×1 convolution, and finally restores it to the input size through bilinear interpolation upsampling to supplement global semantic constraints and reduce false detections.

[0048] To ensure effective feature fusion across parallel branches, all branches in the ASPP module are designed to maintain the same spatial resolution as the input features during the output stage. Subsequently, the features output from each branch are concatenated along the channel dimension.

[0049] S203. Through the projection layer, the stitched features are uniformly mapped to a fixed channel dimension to obtain a feature map enhanced by multi-scale context.

[0050] In some embodiments, considering the difference in the number of channels among features at different levels of the backbone network, direct subsequent processing would lead to an imbalance in computational load. Therefore, this invention introduces a projection layer consisting of a 1×1 convolution, a normalization layer, and an activation function at the end of the ASPP module to uniformly map the concatenated features to a fixed channel dimension Cbase (where Cbase=32). The above multi-scale context enhancement and channel alignment process can be represented as: ; in, This represents the feature output by the i-th stage of the backbone network; This represents a 1×1 convolution branch; This represents the dilated convolution branch with a dilation rate of r; Represents a global pooling branch; Concat represents a channel-level concatenation operation; The projection mapping function represents channel dimensionality reduction and feature fusion; That is, what is ultimately obtained.

[0051] S204. After processing the multi-scale feature maps sequentially, a set of feature maps with unified channel dimensions and incorporating multi-scale context information is obtained, which has been enhanced with multi-scale context. .

[0052] This feature map set As input to the subsequent gate-guided iterative feature refinement process, it provides a standardized and stable multi-scale feature foundation for the gradual refinement modeling of weed areas.

[0053] S30, Set the feature map set The input is fed into the feature refinement module for iterative feature refinement and result prediction. The iterative feature refinement employs a progressive detection strategy from coarse to fine, while maintaining the multi-scale feature set after context enhancement. Under the premise of no change, the prediction results generated in the previous iteration are used as spatial prior guiding information and introduced into the feature modeling process of the current iteration, enabling the model to continuously correct omissions and misjudgments in the initial predictions during multiple iterations. The iterative process is based on the iteration number k (where, Based on the differences between the two stages, it is divided into the initial estimation stage and the feedback refinement stage.

[0054] In the first iteration ( k In the case of =1), no prior information for prediction has been obtained yet, therefore no gating guidance mechanism is introduced. Instead, the feature set is... Decoding features are obtained through cascaded residual decoding, and result prediction is performed based on these decoding features to generate a coarse prediction result for the initial weed area.

[0055] When performing cascaded residual decoding, a cascaded residual feature decoder (RFD) is used to process the multi-scale feature set. Perform top-down, layer-by-layer decoding and reconstruction.

[0056] In some embodiments, each RFD module consists of two cascaded Residual Channel Attention Blocks (RCABs) and a convolutional layer at the end, and introduces global residual connections to facilitate gradient propagation and feature reuse. The RCAB unit is the basic computational unit for feature refinement.

[0057] The cascaded residual feature decoder has input and output channels with corresponding channel dimensions depending on the decoding level.

[0058] like Figure 6 As shown, the processing steps of the cascaded residual feature decoder during decoding include: First, the feature map is refined in depth through several residual channel attention blocks.

[0059] The residual channel attention block RCAB incorporates a local residual learning structure, and the specific processing flow is as follows: A. Perform residual feature extraction, and perform nonlinear feature transformation and local texture extraction on the input feature map through a double convolution path.

[0060] The input feature map undergoes nonlinear feature transformation and local texture extraction through a two-layer convolutional path consisting of a convolutional layer, an activation function, and another convolutional layer.

[0061] B. Perform channel attention enhancement to generate weight coefficients for each channel. Multiply these weight coefficients with the input feature map channel by channel to obtain the attention-enhanced feature map.

[0062] To adaptively select key feature channels, the transformed features are fed into a channel attention module (CALayer). This module compresses the two-dimensional spatial information into one-dimensional channel descriptors through global average pooling; then, it uses a bottleneck structure of dimensionality reduction convolution-activation function-dimensionality increase convolution-Sigmoid activation to generate weight coefficients for each channel; finally, these weights are multiplied channel-by-channel by the input feature map to obtain the attention-enhanced feature map. This process can automatically assign higher response weights to channels related to weed targets based on global context information, while suppressing interference from irrelevant channels such as rice background.

[0063] C. Perform local residual connections by adding the attention-enhanced feature map to the original input feature map to obtain the local residual learning structure.

[0064] The features enhanced by channel attention are added to the original input of the RCAB unit to form a local residual learning structure, which allows each unit to learn only a small amount of feature correction (i.e., residual), effectively avoiding the gradient vanishing problem in deep networks.

[0065] Then, the refined feature map is input into the convolutional layer for feature integration and channel mapping to generate high-frequency residual features relative to the input of the cascaded residual feature decoder. Finally, through global residual connection, the high-frequency residual feature is added to the initial input feature of the cascaded residual feature decoder to obtain the decoding feature of this stage of the decoding module.

[0066] The set of multi-scale feature maps The decoding process for cascaded residual decoding is as follows: Level 1 Decoding: (Cbase number of channels) Input to the first-level RFD module. This module extracts features while maintaining the channel dimension and outputs the first-level decoded features. (The number of channels is Cbase) The process can be represented as: ; Second-level decoding: The features decoded at the first level... After performing a 2x bilinear interpolation upsampling, its spatial dimensions are restored to the same level as... Consistent. Then with the next deeper features. The splicing is performed along the channel dimension. Because... For Cbase channel, The concatenated feature dimension is increased to 2×Cbase channels. This is then input into the second-level RFD module to output the second-level decoded features. (2×Cbase channels).

[0067] ; Third-level decoding: Decoding the features from the second-level decoding... After performing a 2x bilinear interpolation upsampling, its spatial dimensions are restored to the same level as... Consistent with shallow features. The stitching is performed along the channel dimension. At this point, For 2×Cbase channels, The concatenated feature dimensions are further accumulated to 3×Cbase channels. This is then input into the third-level RFD module, which outputs the third-level decoded features. (3×Cbase channels) ; in, This indicates a bilinear interpolation upsampling operation. This indicates a feature concatenation operation performed at the channel level. The mapping function representing the i-th level residual feature decoding module (where...) It should be noted that although the network structures of each RFD module are similar, the number of their input and output channels are independently configured as i×Cbase to adapt to the dimensionality changes after the feature accumulation at each level. This is the final decoded feature obtained in this round of iteration.

[0068] Through final decoding features Methods for predicting outcomes include: It is then fed into the convolutional mapping layer. Because... The features are formed by multi-level cascading and concatenation. They have a high dimensionality in the channel dimension (specifically 3×Cbase). Therefore, the convolutional mapping layer uses 3×3 convolution to aggregate local neighborhood features in the spatial dimension, and uses the setting of the number of convolution kernels to compress and map the feature channels from 3×Cbase back to Cbase, thereby removing redundant information and refining key semantic features to obtain the coarse prediction result P1 of the first iteration.

[0069] ; The result It contains the core semantic distribution information of the weed area and will serve as a spatial prior guiding signal for the next iteration.

[0070] Starting from the second iteration (k>1), a gated guided attention (GGA) module is introduced when performing cascaded residual decoding. The decoding features of the previous level are used as the spatial prior guidance information of the current level. For the first level, the prediction results of the previous round are used as the spatial prior guidance information.

[0071] Based on spatial prior guidance information, the feature weights of the feature map of the current level are modulated, and then decoded by a cascaded residual feature decoder to obtain the decoded features of the current level. After decoding each level in turn, the final decoded features are obtained.

[0072] The specific steps for obtaining the decoding features of the current level in the second and subsequent iterations include: S301. Use the decoding features of the previous level as the spatial prior guidance information of the current level. For the first level, use the prediction results of the previous round as the spatial prior guidance information.

[0073] For the first-level deep features with the richest semantics The model uses the coarse prediction result P generated in the previous iteration. k-1 As a priori for the global space. Given P k-1 It is generated based on shallow decoding features, and... There are scale differences, therefore P needs to be adjusted first. k-1 The input is fed into a gated convolutional layer. Its spatial resolution is downsampled to the same level using a 1×1 convolution and a 4x bilinear interpolation downsampling operation. Consistent and compressed into single-channel features, thereby generating a matched gating signal. .

[0074] For subsequent hierarchical features ( , To enhance top-down semantic transmission, the output features of the previous level residual decoder are directly selected as guides. For example, when processing sub-deep features... At that time, the first-level decoding output is used. As prior information, first, a 1×1 convolution is performed... Compressed to a single channel, then subjected to 2x bilinear interpolation upsampling to make its spatial dimensions consistent with... Alignment, thereby generating a gating signal ( The processing logic is the same.

[0075] S302, such as Figure 3 As shown, a gating signal for the current level is generated based on spatial prior guidance information. The gating signal is then concatenated with the feature map of the current level in the channel dimension. Finally, a position-related gating weight map is generated through batch normalization, convolution transformation, and nonlinear activation function.

[0076] The gate signal generated above With current level features The input is fed into the gating-guided attention module. In this module, the input is first... and The data is concatenated along the channel dimension, and features are fused and mapped through batch normalization, convolutional transformation, and ReLU activation. Finally, a position-dependent gated weight map is generated using the Sigmoid activation function. The calculation process can be expressed as follows: ; in, This represents the gated weight graph of the i-th level obtained in the k-th iteration. This represents the convolution operation. This represents the Sigmoid non-linear activation function. This represents the context-enhanced feature at the i-th level in the feature set. Indicates channel splicing. This represents the gating signal for the i-th level.

[0077] S303. Adjust the feature representation of the current layer using residual gating modulation based on the gating weight map, and integrate the features through the output convolutional layer to obtain the gating enhanced feature map. After obtaining the gated weight map, the model uses residual gated modulation to adjust the feature representation of the current round, and integrates the features through a 3×3 convolutional layer at the output. The process can be represented as follows: ; in, This represents an output feature integration convolution operator consisting of 3×3 convolutional layers, used for inter-channel information fusion and smoothing of gated and modulated features. This indicates element-wise multiplication. This represents the feature map of the i-th level of gated enhancement obtained in the k-th iteration. The residual term ( The introduction of +1) ensures the complete preservation of the original feature information during the modulation process, while realizing adaptive feature reweighting based on prediction feedback.

[0078] S304. The gated enhanced feature map is decoded using a cascaded residual feature decoder to obtain the decoded features of the current level. These decoded features of the current level serve as prior information for the next level.

[0079] Subsequently, a top-down, step-by-step decoding and reconstruction is performed using a cascaded residual decoder with the same structure as the first iteration, and the feature map is enhanced by a gated attention module, i.e., through P... k-1 → → → → → → The cascaded decoding path yields the first... k The final decoded features of the round iteration (Number of channels is 3×C) base ).

[0080] For the final decoded features Perform result prediction to obtain the prediction result map for the current iteration round. Following the same prediction generation process as the first round, finally decode the features. The input is fed into the convolutional mapping layer to generate a coarse prediction result P for the weed region in the corresponding round. k After multiple iterations, the final prediction result is obtained.

[0081] Through the aforementioned gated, multi-round iterative feature refinement process, the model can gradually correct prediction biases and structural omissions based on the initial coarse predictions, allowing the spatial distribution, morphological structure, and boundary positions of the weed area to continuously stabilize in multiple iterations. T The coarse prediction result P obtained from rounds of iteration T (in T This represents the total number of iterations in the iterative refinement process, i.e. k =1,2,…, T This provides a reliable semantic localization foundation for the subsequent final feature fusion and prediction generation stages.

[0082] S40. Obtain the final prediction result graph P. T and feature map set High-resolution feature maps in The input is fed into the feature fusion module, where feature fusion is performed to obtain fused features. Based on these fused features, a predicted response map is generated, and the spatial resolution is restored to the same level as the original input image through an upsampling operation to obtain the final detection result map.

[0083] The processing steps within the feature fusion module include: S401, Set up multi-scale feature maps Medium to high resolution feature maps Perform a downsampling operation to make its spatial scale match the final predicted result P. T Alignment.

[0084] Select the coarse prediction P obtained from the last iteration T As a basis for global semantic localization, high-resolution features from the shallow output of the backbone network are also introduced. As a source of supplementary spatial details. It retains rich local geometric structure and edge contour information, which can effectively compensate for the loss of spatial details caused by deep features during multi-level downsampling.

[0085] To achieve effective alignment between features at different scales, the high-resolution features are first... Perform a 2x bilinear interpolation downsampling operation to make its spatial resolution the same as P. T Maintaining consistency can be represented as follows: ; in, This indicates a downsampling operation. These are high-resolution features after scale alignment.

[0086] S402, plot the final prediction result P T and aligned high-resolution feature maps The fused input features are obtained by concatenating them along the channel dimension.

[0087] The final prediction result is shown in Figure P. T Aligned high-resolution features Concatenation is performed along the channel dimension, since both are C... base The channels, when spliced ​​together, form a 2×C base The fused input features constitute the fused input features. The process can be represented as follows: ; S403. The fused input features are input into the hollow space pyramid pooling module for joint modeling. By introducing receptive field information of different scales in parallel, multi-scale context modeling and information integration are performed on the fused input features to obtain fused features.

[0088] The fused input features The data is fed into a dilated spatial pyramid pooling module for joint modeling, using the ASPP structure as the fusion operator. ASPP introduces dilated convolutions with different dilation rates in parallel, capturing multi-scale contextual information without reducing resolution, thereby achieving deep semantics (from P). T ) and shallow details (from The deep integration of ) can be represented as follows: ; in, This represents a feature fusion operator based on a hollow spatial pyramid pooling structure, used for multi-scale contextual modeling and information integration of coarse predictive semantic features and high-resolution detail features.

[0089] After feature fusion is completed, the fused features Furthermore, by using 1×1 convolution to compress the feature channels from high dimension to a single channel, a pixel-level predicted response map is generated; and by using an 8x bilinear interpolation upsampling operation, the resolution of the predicted map is restored from 1 / 8 to the spatial resolution consistent with the original input image, resulting in the final weed detection result. The process can be represented as follows: ; in, This represents a 1×1 convolution prediction operator. This indicates an eight-fold upsampling operation.

[0090] Through the aforementioned final feature fusion and prediction generation strategy, this invention effectively introduces high-resolution spatial detail information while maintaining the global semantic discrimination capability obtained in the iterative stage, enabling fine-tuning of the boundaries and morphological structures of weed targets. Compared with the coarse prediction results generated in the iterative stage, the final output weed detection results show significant improvements in regional integrity, boundary clarity, and background suppression capabilities, making it more suitable for the refined weed identification needs in low-altitude remote sensing of paddy fields using UAVs.

[0091] like Figure 4 As shown, a paddy field weed detection device based on low-altitude remote sensing from a UAV, provided to suit another objective of the present invention, includes a feature extraction module 1100, a feature processing module 1200, a feature iteration prediction module 1300, and a feature fusion and prediction module 1400. The feature extraction module 1100 is configured to input low-altitude remote sensing images of paddy fields acquired by a UAV platform, preprocess them, and then input them into a feature extraction network for multi-scale feature extraction: using a hierarchical visual VMamba based on a state-space model as the backbone network, it extracts multi-scale feature maps that simultaneously contain shallow texture information and deep semantic information, forming a multi-scale feature pyramid. The feature processing module 1200 is configured to sequentially input the multi-scale feature maps output from the backbone network into a hollow spatial pyramid pooling module for context enhancement processing and channel alignment, resulting in a feature map set with unified channel dimensions and fused multi-scale context information. The feature iterative prediction module 1300 is configured to use the feature map set... The input is fed into the feature refinement module for iterative feature refinement and result prediction: In the first iteration, the feature set is... Decoding features are obtained through cascaded residual decoding. Based on these features, result prediction is performed to generate a coarse prediction result for the initial weed area. Starting from the second iteration, a gated guided attention module is introduced during cascaded residual decoding. The decoding features of the previous level are used as the spatial prior guidance information for the current level. For the first level, the prediction result of the previous round is used as the spatial prior guidance information. The feature weights of the feature map of the current level are modulated according to the spatial prior guidance information, and then decoded through the cascaded residual feature decoder to obtain the decoding features of the current level. After decoding each level in sequence, the final decoding features are obtained. Result prediction is performed on the final decoding features to obtain the prediction result map of the current iteration round. After multiple iterations, the final prediction result is obtained. The feature fusion and prediction module 1400 is set to obtain the final prediction result map and feature map set. The high-resolution feature map is input to the feature fusion module. After feature fusion, the fused feature is obtained. Based on the fused feature, the prediction response map is generated and then the spatial resolution is restored to the same as that of the original input image through upsampling operation to obtain the final detection result map.

[0092] Based on any embodiment of the present invention, such as Figure 5 As shown, another embodiment of the present invention also provides an electronic device, which can be implemented by a computer device, such as... Figure 5 The diagram shows the internal structure of a computer device. This computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When executed by the processor, the computer-readable instructions enable the processor to implement a method for detecting weeds in paddy fields based on low-altitude remote sensing from a UAV. The processor provides computing and control capabilities, supporting the operation of the entire computer device. The memory stores computer-readable instructions, which, when executed by the processor, enable the processor to execute the method for detecting weeds in paddy fields based on low-altitude remote sensing from a UAV. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0093] In this embodiment, the processor is used to execute... Figure 4 The specific functions of each module are defined within the device, and the memory stores the program code and various data required to execute these modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules in the UAV-based low-altitude remote sensing paddy field weed detection device of the present invention. The server can call the server's program code and data to execute the functions of all modules.

[0094] The present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the rice paddy weed detection method based on UAV low-altitude remote sensing as described in any embodiment of the present invention.

[0095] The present invention also provides a computer program product, including a computer program / instructions, which, when executed by one or more processors, implement the steps of the rice paddy weed detection method based on UAV low-altitude remote sensing as described in any embodiment of the present invention.

[0096] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of the present invention can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0097] The above description is only a partial embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles (UAVs), comprising the following specific steps: S10. Input the low-altitude remote sensing image of the paddy field obtained by the UAV platform, preprocess it, and then input it into the feature extraction network for multi-scale feature extraction: use the hierarchical visual VMamba based on the state space model as the backbone network to extract multi-scale feature maps that contain both shallow texture information and deep semantic information, forming a multi-scale feature pyramid. S20. The multi-scale feature maps output from the backbone network are sequentially input into the hollow spatial pyramid pooling module for context enhancement and channel alignment, resulting in a feature map set with unified channel dimensions and incorporating multi-scale context information. ; S30, Set the feature map set The input is fed into the feature refinement module for iterative feature refinement and result prediction. In the first round of iterations, the feature set... Decoding features are obtained through cascaded residual decoding, and result prediction is performed based on these decoding features to generate a coarse prediction result for the initial weed area. Starting from the second iteration, a gated guidance attention module is introduced when performing cascaded residual decoding. The decoding features of the previous level are used as the spatial prior guidance information of the current level. For the first level, the prediction results of the previous round are used as the spatial prior guidance information. Based on spatial prior guidance information, the feature weights of the feature map of the current level are modulated, and then decoded by a cascaded residual feature decoder to obtain the decoded features of the current level. After decoding each level in turn, the final decoded features are obtained. The final decoded features are predicted to obtain the prediction result map of the current iteration round. After multiple iterations, the final prediction result is obtained. S40. Obtain the final prediction result map and feature map set. The high-resolution feature map is input to the feature fusion module. After feature fusion, the fused feature is obtained. Based on the fused feature, the prediction response map is generated and then the spatial resolution is restored to the same as that of the original input image through upsampling operation to obtain the final detection result map.

2. The method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles according to claim 1, characterized in that, The preprocessing of the original input image includes: S101. Use bilinear interpolation algorithm to uniformly resample all original input images to a fixed resolution; S102. Normalize each channel of the image, and perform standardization operation on each channel based on the preset channel mean and standard deviation, so that the input data meets the requirements of the network calculation for the consistency of numerical range and distribution, and complete the preprocessing.

3. The method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles according to claim 1, characterized in that, The specific steps for inputting the multi-scale feature maps output by the backbone network into the hollow spatial pyramid pooling module for context enhancement and channel alignment include: S201. Process the multi-scale feature map individually in sequence; S202. For each feature map, it is processed in parallel through a 1×1 convolution branch, three dilated convolution branches with different dilation rates, and a global context branch to complete the feature map context enhancement processing, and the processing results of each branch are spliced ​​together in the channel dimension. S203. Through the projection layer, the stitched features are uniformly mapped to a fixed channel dimension to obtain a feature map enhanced by multi-scale context. S204. After processing the multi-scale feature maps sequentially, a set of feature maps with multi-scale context enhancement is obtained. .

4. The method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles according to claim 1, characterized in that, The specific steps for obtaining the decoding features of the current level in the second and subsequent iterations include: S301. Use the decoding features of the previous level as the spatial prior guidance information of the current level. For the first level, use the prediction results of the previous round as the spatial prior guidance information. S302. Generate the gating signal of the current level based on the spatial prior guidance information, concatenate the gating signal with the feature map of the current level in the channel dimension, and generate a position-related gating weight map through batch normalization, convolution transformation and nonlinear activation function. S303. Adjust the feature representation of the current layer using residual gating modulation based on the gating weight map, and integrate the features through the output convolutional layer to obtain the gating enhanced feature map. S304. The gated enhanced feature map is decoded by a cascaded residual feature decoder to obtain the decoded features of the current level.

5. The method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles according to claim 4, characterized in that, The cascaded residual feature decoder includes multiple cascaded residual channel attention blocks and convolutional layers; the cascaded residual feature decoder has input channels and output channels of corresponding channel dimensions according to different decoding levels.

6. The method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles according to claim 5, characterized in that, The processing steps of the cascaded residual feature decoder during decoding include: First, the feature map is refined in depth through several residual channel attention blocks; Then, the refined feature map is input into the convolutional layer for feature integration and channel mapping to generate high-frequency residual features relative to the input of the cascaded residual feature decoder. Finally, through global residual connection, the high-frequency residual feature is added to the initial input feature of the cascaded residual feature decoder to obtain the decoding feature of this stage of the decoding module.

7. The method for detecting weeds in paddy fields based on low-altitude remote sensing by unmanned aerial vehicles according to claim 1, characterized in that, The processing steps within the feature fusion module include: S401, Set up multi-scale feature maps The medium-to-high resolution feature map is downsampled to align its spatial scale with the final prediction result map; S402. The final prediction result map and the aligned high-resolution feature map are concatenated along the channel dimension to obtain the fused input feature. S403. The fused input features are input into the hollow space pyramid pooling module for joint modeling. By introducing receptive field information of different scales in parallel, multi-scale context modeling and information integration are performed on the fused input features to obtain fused features.

8. A paddy field weed detection device based on low-altitude remote sensing by unmanned aerial vehicles, characterized in that, include: The feature extraction module is set to take low-altitude remote sensing images of rice paddies acquired by the UAV platform as input, preprocess them, and then input them into the feature extraction network for multi-scale feature extraction: the hierarchical visual VMamba based on the state space model is used as the backbone network to extract multi-scale feature maps that simultaneously contain shallow texture information and deep semantic information, forming a multi-scale feature pyramid. The feature processing module is configured to sequentially input the multi-scale feature maps output from the backbone network into the hollow spatial pyramid pooling module for context enhancement and channel alignment, resulting in a feature map set with uniform channel dimensions and incorporating multi-scale contextual information. ; The feature iterative prediction module is configured to use the feature map set The input is fed into the feature refinement module for iterative feature refinement and result prediction: In the first iteration, the feature set is... Decoding features are obtained through cascaded residual decoding, and result prediction is performed based on these decoding features to generate a coarse prediction result for the initial weed area. Starting from the second iteration, a gated guidance attention module is introduced when performing cascaded residual decoding. The decoding features of the previous level are used as the spatial prior guidance information of the current level. For the first level, the prediction results of the previous round are used as the spatial prior guidance information. Based on spatial prior guidance information, the feature weights of the feature map of the current level are modulated, and then decoded by a cascaded residual feature decoder to obtain the decoded features of the current level. After decoding each level in turn, the final decoded features are obtained. The final decoded features are predicted to obtain the prediction result map of the current iteration round. After multiple iterations, the final prediction result is obtained. The feature fusion and prediction module is configured to obtain the final prediction result map and feature map set. The high-resolution feature map is input to the feature fusion module. After feature fusion, the fused feature is obtained. Based on the fused feature, the prediction response map is generated and then the spatial resolution is restored to the same as that of the original input image through upsampling operation to obtain the final detection result map.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.

Images