A lightweight weed detection model and detection method for a low-power weeding robot

Abstract

The application provides a lightweight weed detection model and a detection method for a low-power consumption weeding robot, and belongs to the fields of agricultural engineering and computer vision technology. The detection model takes images collected by a visual sensor as input, and provides output including a weed category, a bounding box (bbox) and confidence (confidence). In feature extraction, a PAM module adopts a convolutional neural network-based architecture, and three masks are inserted in a small amount of convolution calculation to reduce the calculation amount. In the detection process, a FOG module further proposes a region suggestion on a feature map. Through the output of a deep model, linear or nonlinear transformation is applied to map the coordinates in the picture to the real world. Finally, a weeding device completes a weeding process.

Description

Technical Field

[0001] This invention belongs to the fields of agricultural engineering and computer vision technology, specifically, it relates to a lightweight weed detection model and detection method for low-power weeding robots. Background Technology

[0002] Deep learning-based visual object detection has achieved success on high-computing devices, but its performance on edge devices, which are frequently used in smart agriculture, is not outstanding. Specifically, existing model architectures and optimization methods are not suitable for low-power edge devices, while agricultural tasks such as weed detection require high accuracy, short inference latency, and low cost.

[0003] Mathematical models based on hyperspectral equipment or existing artificial neural networks require high human and material resources and perform poorly in unstable farm environments. While automatic parameter tuning tools exist for existing model optimization methods, their parameter search space is enormous, making model compression and optimization a significant waste of resources. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a lightweight weed detection model and method for low-power weeding robots, balancing cost and model accuracy, and optimizing model size, performance, and inference speed. While reducing the size and power consumption of the original model, it also improves the detection frame rate.

[0005] This invention is achieved through the following technical solution:

[0006] A lightweight weed detection model for low-power weeding robots:

[0007] The detection model includes a PAM module, a FOG module, a visual sensor, and a weeding device;

[0008] The detection model takes the image acquired by the visual sensor as input and provides outputs including weed category, bounding box (bbox), and confidence. In feature extraction, the PAM module adopts a convolutional neural network-based architecture, interspersing three types of masks in a small number of convolutional calculations to reduce computational load. During the detection process, the FOG module further proposes region suggestions on the feature map, and applies linear or nonlinear transformations to map the coordinates in the image to the real world through the output of the deep model. Finally, the weeding process is completed by the weeding device.

[0009] Furthermore, the PAM module uses three types of masked PAM. mul PAM add and PAM dotFeature extraction is achieved by performing element-wise matrix multiplication, matrix addition, and matrix dot product operations on the feature map, thereby extracting important location information.

[0010] The PAM mul PAM add and PAM dot The position of the mask in the model depends on the size of the input feature map and the computational complexity; the three masks are assigned to different stages and sorted by computational complexity and the complexity of the input feature map.

[0011] PAM mul In the early stages of model training, the distribution of the mask is unstable, so we only need to assign weights to different positions and choose to place it in the first position.

[0012] PAM add The computational complexity of the mask is lower than that of PAM. mul However, it also changes the original image data, prompting sparse processing during initialization to improve computational efficiency and preserve the original input information as much as possible;

[0013] PAM dot Masking uses spatial projection to achieve multi-scale weed detection adaptability. The feature map size is small, the spatial information is highly integrated, and the feature distribution of the weed image is complete and regular. Therefore, it is placed last.

[0014] Furthermore, PAM mul The mask is used for element-wise multiplication at corresponding positions. By performing element-wise matrix multiplication on the feature map, the feature at each position is multiplied by the weight of the corresponding position, thereby emphasizing important positional information.

[0015] PAM add The mask uses matrix addition to extract region weights in the input feature map, adding the feature at each location to the corresponding bias, enabling the model to learn features that are of more interest to specific locations;

[0016] PAM dot The mask uses a dot product matrix and spatial projection to achieve multi-scale weed detection adaptability. By calculating the similarity between the feature map and itself, the attention distribution of each location to other locations is obtained.

[0017] Furthermore, through PAM mul and PAM add After optimizing the feature map, PAM is used. dot To complete the final projection in the target space;

[0018] To distinguish between different variables, S represents the input image, S featureS′ represents a feature map in the general sense, while S′ feature The feature map representing the original model before PAM optimization; f θ This represents a parameterized mapping, where θ represents the set of parameters of f;

[0019] Obtain a suitable set of parameters θ to minimize f θ (S) and S′ feature The difference between them, where in the model where PAM is involved in the optimization, the output feature map is determined by f. θ (S) indicates that the generation of feature maps can be described as a projection problem, that is, projection mapping in the latent space of the embedded vectors. The optimal projection matrix is ​​iteratively obtained through model training to adapt to weed detection, based on orthogonal projection, as shown below.

[0020] f θ (S)=(A(A T A) -1 A T )S

[0021] Where A represents a matrix whose column vectors are basis vectors. To prevent computational complexity and the problem of some matrices being non-invertible, the matrix inversion process A is... -1 Replace with finding the pseudo-inverse A + PAM dot It played a role with f θ The same function is to spatially map the input image or feature map, update the parameters during each batch of training, and find the optimal mapping method.

[0022] Furthermore, the FOG module recovers global information through two dot products, solving the problem of insufficient global information in PAM. The FOG module can integrate global information into PAM without parameters; the global information is fully integrated into PAM by alternating dot product calculations through matrix transpose.

[0023] Furthermore, the detection model also includes loss function calculation, which includes two loss functions; the overall loss is calculated by performing a weighted linear combination of the two loss functions.

[0024] The first method calculates the loss value by measuring the consistency between the bounding box (bbox) and the confidence score and the label. The confidence score loss is used to ensure that the model's classification of the target and the background is accurate; the bounding box loss is used to ensure that the model's bounding box predictions are consistent with the true bounding boxes, thereby guiding the model parameter updates.

[0025] The second method is used to train the PAM module and ensure PAM dot The output maintains good properties of idempotency and symmetry, therefore a design was made. Loss, where A2 = A((A T A) + A T ) 2 A H =A((A T A) + A T ) T , A1=A((A T A) + A T ) 1 ;

[0026]

[0027] To ensure that A has a certain degree of orthogonality, an orthogonality loss is set, namely... Where A[i] represents the i-th channel of the trainable matrix A;

[0028]

[0029] Furthermore, considering that weeds often exhibit clustered distribution, regional salience information was explicitly added to the loss statement. Among them, Indicates bias;

[0030]

[0031]

[0032]

[0033]

[0034] and Let R1, R2, R3, R4 represent the mean and variance of the regions, respectively. R1, R2, R3, R4 are neighboring regions of the same size as the target region R. l，0 R0 represents the l-th target region, RegionNums is the number of target regions in the feature map, β and β′ are trainable, representing the weight of each target region, and k is an adaptive constant.

[0035] Training process for a lightweight weed detection model for low-power weeding robots:

[0036] The training process of the model includes:

[0037] S1, Data preparation stage: The input images are grouped according to their aspect ratio, and three types of PAM masks are initialized;

[0038] S2. First round of training: Using the dataset from S1, the first round of training is conducted through channel filtering and ratio filtering.

[0039] Based on the Red of each channel, the models are sorted. Considering the architectural characteristics of deep learning models, namely that the output of the previous layer directly affects the input of the next layer, a proportional filtering strategy is adopted. That is, taking into account the three PAM masks, the same proportion is selected to be removed according to the channel Red sorting results of each mask, so as to carry out secondary training.

[0040] S3. Due to the direct reduction of channels, the size of the model and the complexity of the algorithm are reduced on a large scale, and the model is optimized and updated through parameter inheritance; the model parameters trained in the first round are used for the second round of training.

[0041] A lightweight weed detection method for low-power weeding robots:

[0042] The method specifically includes the following steps:

[0043] Step 1. Input Image Acquisition: Use a vision sensor to acquire images that need to be detected for weeds;

[0044] Step 2. Image preprocessing: Preprocess the acquired images, including denoising, adjusting brightness and contrast, to improve the quality of the input images for the model;

[0045] Step 3. Use the trained model to detect weeds in the preprocessed image; the PAM module performs feature extraction, and the FOG module further extracts region proposals to ensure coverage of all weed areas;

[0046] Step 4. The model output includes the weed category, bounding box (bbox), and confidence score.

[0047] Step 5. Control of weeding device: The information of the detected weed area is transmitted to the weeding device. The coordinates in the image are mapped to the real world through linear or nonlinear transformation. The weeding device performs the corresponding weed removal operation according to the detection results.

[0048] An electronic device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the above method.

[0049] A computer-readable storage medium for storing computer instructions that, when executed by a processor, implement the steps of the above-described method.

[0050] Beneficial effects of the invention

[0051] Compared to other models, the PAM-FOG of this invention can run on intelligent weeding robots supported by edge devices, achieving higher accuracy and frame rate. This invention effectively balances model size, performance, and inference speed, reducing the original model size by nearly 50%, power consumption by 26%, and improving the detection frame rate by 40%. The effectiveness of the model architecture and optimization methods is demonstrated; the PAM-FOG network and corresponding optimization methods are compatible with edge devices, providing a reference for the future development of deep learning in smart agriculture. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the actual deployment and testing of the weed detection depth model of the present invention;

[0053] Figure 2 A schematic diagram of the PAM module;

[0054] Figure 3 This is a schematic diagram of the FOG module;

[0055] Figure 4 This is the overall architecture of the model. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Combination Figures 1 to 4 .

[0058] A lightweight weed detection model for low-power weeding robots:

[0059] The detection model includes a PAM module, a FOG module, a visual sensor, and a weeding device.

[0060] The detection model takes images acquired by a visual sensor as input and provides outputs including weed category, bounding box (bbox), and confidence level. In upstream feature extraction, a convolutional neural network-based architecture is employed, interspersed with three types of PAM module masks within a small number of convolutional calculations to reduce computational load. In the downstream detection process (model head), YOLO architecture is incorporated to further propose region suggestions on the feature map. Through the output of the deep model, linear or nonlinear transformations are applied to map the coordinates in the image to the real world. Finally, the weeding process is completed using relevant mechanical devices. Figure 1 As shown.

[0061] PAM module:

[0062] The PAM (Position Attention Module) is a position attention module. To prevent the model from becoming too verbose due to stacked convolutional layers during feature extraction, it uses three types of masked PAM. mul PAM add and PAM dot Perform element-wise matrix multiplication, matrix addition, and matrix dot product operations on the feature map, such as... Figure 2 As shown, feature extraction is achieved, thereby extracting important location information. During the feature extraction process, three PAM masks are used to simplify the feature extraction calculation by interspersing and integrating CNN modules.

[0063] Figure 4 The model's overall architecture and the mask insertion positions are shown. A small number of convolutional layers are interspersed among the three PAM masks for feature extraction. During the second training, targeted optimization is performed based on the data distribution characteristics of the masks.

[0064] The PAM mul PAM add The design concept comes from statistical experiments on the distribution of weed data, while PAM dot This is based on projection mappings in mathematical equations. Their position in the model depends on the size of the input feature map and the computational complexity;

[0065] Placing three masks consecutively does not necessarily improve performance. Three mask positions were considered.

[0066] First, in the shallow layers of the model, the input size is relatively large, and applying three masks consecutively will put too much burden on edge devices, especially when real-time performance is limited.

[0067] Secondly, applying downsampling can lead to instability. In the early stages of model training, strong perturbation of the input data often causes a sharp drop in model performance, making it easy to fail to converge.

[0068] Third, if the three masks are placed at the end of the network, updating and integrating them with gradients may cause the mask values ​​to become too similar, thus reducing their diversity and effectiveness. Therefore, we choose to assign the three masks to different stages, ranking them by computational complexity and the complexity of the input feature maps, resulting in different insertion positions of these masks in the model.

[0069] The PAMmul (Multiplicative Attention) mask is used for element-wise multiplication at corresponding positions. In the early stages of model training, the mask distribution is unstable; therefore, weights are simply assigned to different positions, and the first position is chosen. By performing element-wise matrix multiplication on the feature maps, the features at each position are multiplied by the corresponding weight, thus emphasizing important positional information. This allows the model to focus on important features at specific positions during training.

[0070] PAM add The (Additive Attention) mask uses matrix addition to extract region weights in the input feature map, adding the feature at each location to the corresponding bias. This helps introduce positional biases, enabling the model to learn features that are of greater interest to specific locations; its computational complexity is lower than PAM. mul However, this will also change the original image data, which prompts us to perform sparse processing on it during initialization to improve computational efficiency and preserve the original input information as much as possible, placing it in the middle;

[0071] PAM dot The mask employs a dot-matrix multiplication, which is crucial in PAM-FOG. It uses spatial projection to achieve multi-scale weed detection adaptability by calculating the similarity between the feature map and itself, thus obtaining the attention distribution of each location to other locations. This self-attention mechanism allows the model to capture global dependencies between features, rather than being limited to local information. Given the previous downsampling, the feature map size is small, spatial information is highly integrated, and the feature distribution of the weed image is complete and regular, which is one of the reasons why it is placed last.

[0072] Through PAM mul and PAM add After optimizing the feature map, PAM is used. aot The next step is to complete the final projection in the target space. The goal is to find a mapping that ensures that, given the same input, the values ​​of the output feature map are not significantly different from those of the original model.

[0073] To distinguish between different variables, S represents the input image, S feature S′ represents a feature map in the general sense, while S′ feature The feature map representing the original model before PAM optimization; f θ This represents a parameterized mapping, where θ represents the set of parameters of f;

[0074] The goal is to obtain a suitable set of parameters θ to minimize f. θ (S) and S′f eatureThe difference between them, where in the model where PAM is involved in the optimization, the output feature map is determined by f. θ (S) indicates that the generation of feature maps can be described as a projection problem, that is, projection mapping in the latent space of the embedded vectors. The optimal projection matrix is ​​iteratively obtained through model training to adapt to weed detection, based on orthogonal projection, as shown below.

[0075] f θ (S)=(A(A T A) -1 A T )S

[0076] Where A represents a matrix whose column vectors are basis vectors. To prevent computational complexity and the problem of some matrices being non-invertible, the matrix inversion process A is... -1 Replace with finding the pseudo-inverse A + PAM dot It played a role with f e The same function is to spatially map the input image or feature map, update the parameters during each batch of training, and find the optimal mapping method.

[0077] FOG module (Feature Overlap Guidance):

[0078] The FOG module is a feature-guided module that recovers global information through two dot products, solving the problem of insufficient global information in PAM. The FOG module can integrate global information into PAM without requiring parameters.

[0079] While the three PAM masks mentioned above can effectively extract features, they do not explicitly integrate global information. This limitation becomes particularly apparent when distinguishing between small targets and non-targets. Specifically, in agriculture, weeds and crops are often distributed in clusters and contain similar local semantic information, which may lead to most operations producing similar results.

[0080] To address the complexities of limited global information and self-interest in PAM, FOG (Forward-Looking Generics) has also been proposed, such as... Figure 3 As shown, it only requires two dot products to recover a certain amount of global information. This is a module that can integrate global information without parameters. It can complete the full fusion of global information through alternating dot products of matrix transpose, and then easily integrate it into PAM.

[0081] Identification of overlapping regions: The FOG module first identifies overlapping regions in the target feature map, that is, the areas where the target parts overlap each other.

[0082] Global Information Acquisition: To address the lack of global information in PAM, the FOG module uses two dot product operations to acquire global information. Introducing global information into the feature representation through dot product helps the network better understand the overall context.

[0083] Fusion of global and local information: Through two dot product operations, the FOG module merges global and local information. This helps to address the problem of insufficient global information that may arise in PAM due to an overemphasis on local information.

[0084] Integration into PAM: The FOG module is designed to integrate the fused global information into PAM without requiring additional parameters. This provides more global context information without introducing additional complexity, which helps improve object detection performance.

[0085] The detection model also includes loss function calculation, including two types of loss functions;

[0086] The first method is consistent with the YOLO series and is used to train the object detection head in the detection model. The loss value is calculated by calculating the consistency between the bounding box (bbox) and the confidence and the label. The confidence loss is used to ensure that the model classifies the object and the background accurately; the bounding box loss is used to ensure that the model's bounding box prediction is consistent with the true bounding box, thereby guiding the model parameter update.

[0087] The second type of constraint is consistent with the three PAM designs and is used to train the PAM module to ensure that the PAM... dot The output maintains good properties of idempotency and symmetry, therefore a design was made. Loss, where A2 = A((A T A) + A T ) 2 A H =A((A T A) + A T ) T , A1=A((A T A) + A T ) 1 This loss forces PAM to satisfy certain mathematical constraints when calculating features, thereby improving the model's ability to process global information.

[0088]

[0089] To ensure that A has a certain degree of orthogonality, an orthogonality loss is set, namely... Where A[i] represents the i-th channel of the trainable matrix A. This loss forces the model to learn a better feature representation, which helps improve generalization performance.

[0090]

[0091] Furthermore, considering that weeds often exhibit clustered distribution, regional salience information was explicitly added to the loss statement. Among them, This represents the bias. Through this loss, the model is guided to focus on regions of clustered weeds in the image, thereby improving its ability to detect such clusters.

[0092]

[0093]

[0094]

[0095]

[0096] Specifically, using feature maps (C feature H feature W feature Taking ) as an example, we hope that it can be spatially distributed (H) feature W feature The distribution of weeds in clusters is highlighted, thus amplifying the average difference between the target area and neighboring areas. Simultaneously, to ensure each area is unique, the variance of each area is amplified in the loss calculation. and Let R1, R2, R3, R4 represent the mean and variance of the regions, respectively. R1, R2, R3, R4 are neighboring regions of the same size as the target region R. l，0 Let R0 represent the l-th target region, RegionNums be the number of target regions in the feature map, and β and β′ be trainable, representing the weights of each target region. k is an adaptive constant.

[0097] The overall loss is calculated by performing a simple weighted linear combination of these loss functions.

[0098] Training process for a lightweight weed detection model for low-power weeding robots;

[0099] The training process of the model includes:

[0100] S1, Data preparation stage: The input images are grouped according to their aspect ratio, and three types of PAM masks are initialized;

[0101] With (C) feature Hfeature W feature For example, PAM can be selectively used. mul Set it to an all-one matrix, for PAM add Set it to a 0 matrix, and set PAM dot Set it to a diagonal identity matrix.

[0102] S2. First round of training: Using the dataset from S1, the first round of training is conducted through channel filtering and ratio filtering.

[0103] The first round of model training is conducted, including the object detection head, PAM module, etc.; channel redundancy is determined based on the dot product operation between matrices.

[0104] After the first round of training, channel redundancy is filtered by comparing redundancy along dimension C. Channel redundancy is determined based on dot product operations between matrices. Channels are sorted according to their Red values. Considering the architectural characteristics of deep learning models—that the output of the previous layer directly affects the input of the next—a proportional filtering strategy is implemented. This balances three PAMs (Positive Aspect Ratios), selecting the same proportion for each mask's channel Red ranking result to reduce reddening during secondary training. i This indicates the redundancy of the i-th channel of the current mask. This represents a very small constant.

[0105]

[0106] S3. Due to the direct reduction of channels, the size of the model and the complexity of the algorithm are reduced on a large scale, and the model is optimized and updated through parameter inheritance; the model parameters trained in the first round are used for the second round of training.

[0107] In the second training iteration, the direct reduction of channels resulted in a significant reduction in both model size and algorithm complexity. Since the parameters were inherited from the model trained in the first iteration, this accelerated convergence and facilitated model optimization. Experiments showed that with a 50% channel reduction, the model size decreased from 9MB to 4MB, while accuracy remained almost unchanged.

[0108] This invention proposes an optimization scheme for the PAM-FOG model. After the first training is completed, only a small amount of computation is needed to roughly estimate the redundancy of each channel in the model. During the model optimization process, the three proposed PAM masks are used to select the optimal channel index, and secondary model reconstruction training is performed only for these channels. The parameters are inherited from the first training results, thereby successfully reducing the model size and shortening the inference time.

[0109] A lightweight weed detection method for low-power weeding robots:

[0110] The method specifically includes the following steps:

[0111] Step 1. Input Image Acquisition: Use a vision sensor to acquire images that need to be detected for weeds;

[0112] Step 2. Image preprocessing: Preprocess the acquired images, including denoising, adjusting brightness and contrast, to improve the quality of the input images for the model;

[0113] Step 3. Use the trained model to detect weeds in the preprocessed image; the PAM module performs feature extraction, and the FOG module further extracts region proposals to ensure coverage of all weed areas;

[0114] Step 4. The model output includes the weed category, bounding box (bbox), and confidence score.

[0115] Step 5. Control of weeding device: The information of the detected weed area is transmitted to the weeding device. The coordinates in the image are mapped to the real world through linear or nonlinear transformation. The weeding device performs the corresponding weed removal operation according to the detection results.

[0116] In testing, this invention used the PAM-FOG model to perform video streaming and multi-scene image tests in real-world environments and datasets. Initially, the original PAM-FOG network achieved a speed of 12 frames per second with an average power consumption of 2182 mW. However, after channel selection, the inference speed of the NANO device reached 17 frames per second, while the power consumption was only 1608 mW, even during a 30-minute video streaming test in a real-world scenario. Compared to other models, the network selected by PAM-FOG significantly reduces power consumption and computational requirements in real-world environments. In contrast, models commonly used in agricultural detection tasks may exceed the processing capabilities of GPUs / CPUs. The results show that the model of this invention can accurately detect monocotyledonous and dicotyledonous weeds, as well as large and small weeds. It performs efficiently in complex scenarios and is well-suited for weeding tasks on edge devices.

[0117] This invention compares the computational resource overhead and accuracy of models. The PAM-FOG network achieves a balance between model size, performance, and inference speed. Specifically, the optimized model reduces the number of parameters by 50%, power consumption by nearly 26%, and frame rate by 40%. The PAM-FOG network and its corresponding optimization methods are compatible with edge devices, providing a reference for the application of deep learning algorithms in agriculture.

[0118] An electronic device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the above method.

[0119] A computer-readable storage medium for storing computer instructions that, when executed by a processor, implement the steps of the above-described method.

[0120] The memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory of the methods described in this invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0121] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means such as coaxial cable, optical fiber, digital subscriber line, DSL, or wireless means such as infrared, wireless, microwave, etc. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium such as a floppy disk, hard disk, magnetic tape; an optical medium such as a high-density digital video disc, DVD; or a semiconductor medium such as a solid-state disk, SSD, etc.

[0122] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0123] It should be noted that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as execution by a hardware decoding processor, or as execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0124] The foregoing has provided a detailed description of a lightweight weed detection model and method for low-power weeding robots proposed in this invention, and has elucidated the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A lightweight weed detection method for low-power weeding robots, characterized in that: The method includes a lightweight weed detection model for low-power weeding robots; The detection model includes a PAM module, a FOG module, a visual sensor, and a weeding device; The detection model takes the images collected by the visual sensor as input and provides outputs including weed category, bounding box (bbox), and confidence level. In feature extraction, the PAM module adopts a convolutional neural network-based architecture, interspersing three types of masks in a small number of convolutional calculations to reduce the amount of computation. The PAM module uses three masks. , and Feature extraction is achieved by performing element-wise matrix multiplication, matrix addition, and matrix dot product operations on the feature map, thereby extracting important location information. The , and The position of the mask in the model depends on the size of the input feature map and the computational complexity; the three masks are assigned to different stages and sorted by computational complexity and the complexity of the input feature map. In the early stages of model training, the distribution of the mask is unstable, so we only need to assign weights to different positions and choose to place it in the first position. The computational complexity of the mask is lower than However, it also changes the original image data, prompting it to be sparsified during initialization to improve computational efficiency and preserve the original input information as much as possible; Masking uses spatial projection to achieve multi-scale weed detection adaptability. The feature map size is small, the spatial information is highly integrated, and the feature distribution of the weed image is complete and regular. Therefore, it is placed last. During the detection process, the FOG module further proposes region suggestions on the feature map. Through the output of the deep model, linear or nonlinear transformations are applied to map the coordinates in the image to the real world. Finally, the weeding process is completed by the weeding device. The FOG module recovers global information through two dot products, solving the problem of insufficient global information in PAM. The FOG module can integrate global information into PAM without parameters. The global information is fully integrated into PAM by alternating dot product calculations through matrix transpose. The method specifically includes the following steps: Step 1. Input Image Acquisition: Acquire images of the area to be inspected for weeds using a vision sensor; Step 2. Image preprocessing: Preprocess the acquired images, including denoising, adjusting brightness and contrast, to improve the quality of the input images for the model; Step 3. Use the trained model to detect weeds in the preprocessed image; the PAM module performs feature extraction, and the FOG module further extracts region proposals to ensure coverage of all weed areas; Step 4. The model output includes the weed category, bounding box (bbox), and confidence score. Step 5. Weeding device control: The information of the detected weed area is transmitted to the weeding device. The coordinates in the image are mapped to the real world through linear or nonlinear transformation. The weeding device performs the corresponding weed removal operation according to the detection results.

2. The detection method according to claim 1, characterized in that: Masks are used for element-wise multiplication at corresponding positions. By performing element-wise matrix multiplication on the feature map, the feature at each position is multiplied by the weight at the corresponding position, thereby emphasizing important positional information. The mask uses matrix addition to extract region weights in the input feature map, adding the feature at each location to the corresponding bias, enabling the model to learn features that are of more interest to specific locations; The mask uses a dot-matrix multiplication and spatial projection to achieve multi-scale weed detection adaptability. By calculating the similarity between the feature map and itself, the attention distribution of each location to other locations is obtained.

3. The detection method according to claim 2, characterized in that: pass and After optimizing the feature map, through To complete the final projection in the target space; To distinguish different variables, Indicates the input image. This represents a feature map in the general sense, while The feature map representing the original model without PAM optimization; Represents a parameterized mapping, where express The parameter set; Obtain a suitable set of parameters To minimize and The difference between them, where in the model where PAM is involved in the optimization, the output feature map is determined by... The generation of feature maps can be described as a projection problem, that is, projecting and mapping in the latent space of the embedded vectors. The optimal projection matrix is ​​obtained through model training to adapt to weed detection, based on orthogonal projection, as shown below. in This represents a matrix whose column vectors are basis vectors. To prevent computational complexity and the problem of some matrices being non-invertible, the matrix inversion process is described below. Replace with finding the pseudo-inverse , It played a role with The same function is to spatially map the input image or feature map, update the parameters during each batch of training, and find the optimal mapping method.

4. The detection method according to claim 3, characterized in that: The detection model also includes loss function calculation, which includes two types of loss functions; the overall loss is calculated by weighted linear combination of the two loss functions. The first method calculates the loss value by measuring the consistency between the bounding box (bbox) and the confidence score with the label. The confidence loss is used to ensure that the model's classification of the target and the background is accurate; the bounding box loss is used to ensure that the model's bounding box predictions are consistent with the true bounding boxes, thereby guiding the model parameter updates. The second method is used to train the PAM module, ensuring... The output maintains good properties of idempotency and symmetry, therefore a design was made. Losses, among which ; To ensure It possesses a certain degree of orthogonality, and an orthogonality loss is set, i.e. ,in Represents a trainable matrix The One channel; Furthermore, considering that weeds often exhibit clustered distribution, regional salience information was explicitly added to the loss statement. Among them, Indicates bias; and These represent the mean and variance within the region, respectively. Is with the target area Adjacent areas of the same size Indicates the first Target areas , It is the number of target regions in the feature map. and It is trainable, representing the weights for each target region. It is an adaptive constant.

5. A training method for a lightweight weed detection method for low-power weeding robots, characterized in that: The training method is based on the lightweight weed detection method for low-power weeding robots as described in any one of claims 1 to 4; The training method includes: S1, Data preparation stage: The input images are grouped according to their aspect ratio, and three types of PAM masks are initialized; S2. First round of training: Using the dataset from S1, the first round of training is conducted through channel filtering and ratio filtering. According to each channel To rank the layers, considering the architectural characteristics of deep learning models—that is, the output of one layer directly affects the input of the next—a proportional selection strategy was implemented. This strategy takes into account three types of PAM masks and selects layers based on the channels of each mask. The sorted results are reduced by the same proportion for secondary training. S3. Due to the direct reduction of channels, the size of the model and the complexity of the algorithm are reduced significantly, and the model is optimized and updated through parameter inheritance; the model parameters from the first round of training are used for the second round of training.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 4.

Images