A weed detection method and system suitable for ARM architecture MCU

Abstract

The application discloses a kind of weeds detection methods suitable for ARM architecture MCU, comprising the following steps: S1, construct light weight SSD model: replace VGG network in SSD model with MobileNet V1 network, and add additional network layer and FPN structure to improve the limitation of MobileNet V1;S2, use dataset, and pre-process dataset;S3, using the pre-processed dataset, the improved light weight SSD model is trained and verified by TensorFlow framework;S4, using TensorFlow Lite to implement full integer quantization processing to the trained model;S5, after obtaining weed image, the light weight SSD model after quantization is applied to weed image recognition processing to determine position.The application constructs light weight SSD model, significantly improves the efficiency of weed detection, reduces the hardware requirement, while maintaining high detection accuracy, in addition, using full integer quantization processing makes the model easy to deploy and has wide applicability, better suitable for agricultural field.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection, and in particular to a weed detection method and system suitable for ARM architecture MCU. BACKGROUND

[0002] With the development of modern agricultural technology, precision agriculture has become an important means to improve crop yield and reduce pesticide use. Among them, weed detection and positioning is one of the key technologies to realize precise weeding. Traditional weed detection methods mainly rely on manual identification, which is not only inefficient, but also easily affected by subjective factors, resulting in low recognition accuracy.

[0003] In recent years, the development of deep learning technology has brought revolutionary changes to the field of image recognition. In particular, convolutional neural networks (CNN) have shown excellent performance in image classification, target detection and other aspects. SSD (Single Shot MultiBox Detector) is a popular target detection algorithm, which is widely used in various scenarios due to its fast speed and high detection accuracy. However, the traditional SSD model is difficult to run in real time on embedded devices with limited computing resources due to its large amount of calculation and numerous parameters, which to some extent limits its application in actual agricultural scenarios. At the same time, most of the existing weed detection methods are based on YOLO, and the deployment of YOLO requires high-performance edge devices, while weed detection in embedded devices, especially ARM architecture MCU, is still in the blank field. In addition, existing lightweight models often require expensive hardware resources during training and deployment, which increases the difficulty and cost of deployment in actual applications. Therefore, developing a lightweight SSD model that can run efficiently on embedded devices is of great significance for weed detection in precision agriculture. SUMMARY

[0004] To solve the above technical problems, the present application provides a weed detection method and system suitable for ARM architecture MCU.

[0005] To solve the above technical problems, the technical solution of the present application is: a weed detection method suitable for ARM architecture MCU, comprising the following steps:

[0006] S1, constructing a lightweight SSD model: replacing the VGG network in the SSD model with a MobileNet V1 network, and adding additional depth separable convolution layers, feature processing layers and feature pyramid structures to improve the limitations of MobileNet V1;

[0007] S2, using a data set and preprocessing the data set;

[0008] S3, using the pre-processed data set, training and verifying the improved lightweight SSD model through the TensorFlow framework;

[0009] S4, using TensorFlow Lite to implement full integer quantization processing on the trained model;

[0010] S5, after obtaining the weed image, applying the quantized lightweight SSD model to identify and process the weed image to determine the position.

[0011] As a preferred technical solution, step S1 specifically includes the following steps:

[0012] S11, replace the VGG network in the SSD model with the MobileNet V1 network, specifically: in the network architecture of MobileNet V1, first introduced is a standard convolutional layer, denoted as Conv1, which uses a 3x3 convolution kernel and sets the stride to 2 to achieve spatial downsampling. The image input to Conv1 has a resolution of 256x256 and 3 color channels. After processing by this layer, the number of channels of the output feature map increases to 8. Subsequently, the network is composed of a series of depth separable convolutional layers, numbered Conv_dw_1 to Conv_dw_13. Each depth separable convolutional layer is composed of a depth convolution and a pointwise convolution (1x1 convolution) two sub-layers. The stride of these convolutional layers is designed in an alternating pattern, i.e., some layers use a stride of 1, while others use a stride of 2, to gradually reduce the size of the feature map. Finally, at the end of the network, the Conv_dw_13 layer outputs a feature map with 256 channels and a size of 8x8.

[0013] S12, add four additional depth separable convolutional layers after the MobileNet V1 network and remove the regular convolutional layers in the SSD model, specifically: the first and third layers use a 3x3 convolution kernel to perform depth convolution on the input with a stride of 2, then fuse channel information through 256 1x1 pointwise convolutions and apply L2 regularization. The second and fourth layers use a 3x3 convolution kernel to perform spatial depth convolution on the input with a stride of 1 (each input channel is independently convolved), then fuse channel information through 256 1x1 pointwise convolutions and apply L2 regularization.

[0014] As a preferred technical solution, step S2 specifically includes the following steps:

[0015] S21, normalize the pixel values of the images in the data set to the range of -1 to 1;

[0016] S22, integrate the image size to adapt to a specific size while keeping the aspect ratio unchanged, and use the nearest neighbor interpolation method.

[0017] S23, convert the original image into RGB format.

[0018] As a preferred technical solution, the identification processing in step S5 includes the following steps:

[0019] S51, extracting a first stage feature map from the 5th deep separable convolution layer of the MobileNet V1 network;

[0020] S52, extracting a second stage feature map from the 11th deep separable convolution layer of the MobileNet V1 network;

[0021] S53, extracting a third stage feature map from the 13th deep separable convolution layer of the MobileNet V1 network;

[0022] S54, extracting a fourth stage feature map from the second layer of additional deep separable convolution layer;

[0023] S55, extracting a fifth stage feature map from the fourth layer of additional deep separable convolution layer;

[0024] S56, performing global average pooling operation on the fifth stage feature map and reshaping to form a sixth stage feature map;

[0025] S57, all stage feature maps are subjected to single layer convolution processing and batch normalization;

[0026] S58, applying FPN network to the processed first stage feature map to the sixth stage feature map, that is, each feature map layer receives the sampling result of the previous layer and is added with the current layer feature map element by element, so as to realize feature fusion, so that the high-level semantic information can be combined with the low-level detail information;

[0027] S59, the detection head processes the fused feature map obtained in step S58 to generate anchor frame, classification prediction and bounding box prediction;

[0028] S510, performing non-maximum suppression on the data output by the detection head in step S59 to obtain the final identification position.

[0029] As a preferred technical solution, a weed detection system suitable for ARM architecture MCU includes:

[0030] The data preprocessing module performs standardization, image format conversion and image compression on the collected image information, so that the image information is adapted to the input standard of the model;

[0031] The model inference module uses a micro control unit to run a lightweight SSD model;

[0032] The data post-processing module decodes and non-maximum suppression on the output of the model to obtain weed information.

[0033] As a preferred technical solution, it further comprises:

[0034] The data acquisition module adopts a camera to capture real-time image information.

[0035] The information display module displays the original image and the processed image information through a screen.

[0036] The model conversion module converts the model into a format that can run on an embedded device using CMSIS-NN.

[0037] Due to the adoption of the above technical solutions, the present application has the following advantages:

[0038] I. The core of MobileNet V1 is deep separable convolution. Although this convolution can greatly reduce the parameter quantity and calculation quantity, it has a significant advantage in calculation efficiency, but in some complex tasks, such as target detection, deep separable convolution cannot effectively extract high-level features, resulting in a decrease in detection performance. Therefore, MobileNet V1 network is not widely used in the prior art; the present application improves the limitations of MobileNet V1 network, such as adding additional deep separable convolution and FPN structure, and additionally processing the extracted feature map to improve the detection performance.

[0039] II. The present application significantly improves the efficiency of weed detection by constructing a lightweight SSD model, reduces the hardware requirements, and maintains high detection accuracy. In addition, the present application uses full integer quantization processing, making the model easy to deploy and having wide applicability, thereby promoting the development of precision agriculture and bringing an efficient and economical solution to the agricultural field, having significant social and economic benefits and application prospects. BRIEF DESCRIPTION OF DRAWINGS

[0040] The following drawings are only intended to illustrate and explain the present application and do not limit the scope of the present application. Among them:

[0041] Figure 1 is a structural schematic diagram of the lightweight SSD model of the present application;

[0042] Figure 2 is the precision-recall curve and AP of the Carpetweed class;

[0043] Figure 3 is the precision-recall curve and AP of the Cutleaf class;

[0044] Figure 4 is the precision-recall curve and AP of the Eclip ta class;

[0045] Figure 5 is the precision-recall curve and AP for Goosegrass class;

[0046] Figure 6 is the precision-recall curve and AP for MorningGlory class;

[0047] Figure 7 is the precision-recall curve and AP for PricklySida class;

[0048] Figure 8 is the precision-recall curve and AP for PalmerAmaranth class;

[0049] Figure 9 is the precision-recall curve and AP for Purslane class;

[0050] Figure 10 is the precision-recall curve and AP for Ragweed class;

[0051] Figure 11 is the precision-recall curve and AP for Sicklepod class;

[0052] Figure 12 is the precision-recall curve and AP for SpottedSpurge class;

[0053] Figure 13 is the precision-recall curve and AP for Waterhemp class;

[0054] Figure 14 is the precision-recall curve and AP for Weed class;

[0055] Figure 15 is the inference time test result of stm32 series development board; DETAILED DESCRIPTION

[0056] The application will be further described below in conjunction with the drawings and examples. In the following detailed description, only certain exemplary embodiments of the application are described through the illustration. It goes without saying that those skilled in the art can realize that the described embodiments can be modified in various ways without departing from the spirit and scope of the application. Therefore, the drawings and description are essentially illustrative, not for limiting the protection scope of the claims.

[0057] A weed detection method suitable for ARM architecture MCU, comprising the following steps:

[0058] S1. Build a lightweight SSD model: Replace the VGG network in the SSD model with the MobileNet V1 network, and add additional depthwise separable convolutional layers, feature processing layers, and feature pyramid structures to improve the limitations of MobileNet V1.

[0059] S2. Use the dataset and preprocess the dataset;

[0060] S3. Using the preprocessed dataset, the improved lightweight SSD model is trained and verified using the TensorFlow framework.

[0061] S4. Use TensorFlow Lite to perform full integer quantization on the trained model.

[0062] S5. After obtaining the weed image, the quantized lightweight SSD model is applied to perform recognition processing on the weed image to determine the location.

[0063] Step S1 is to perform lightweight improvement on the SSD model in the prior art, specifically including the following steps:

[0064] S11. The VGG network has two major shortcomings: 1. Huge number of parameters: The VGG network contains a large number of parameters. For example, VGG-16 has about 138 million parameters. This causes the model to require a large amount of storage space and a large amount of computing resources during training and inference. 2. Low computational efficiency: Due to the large number of parameters, the computational efficiency of the VGG model is relatively low. This is mainly because a large number of fully connected layers are used in the model, which are very time-consuming to calculate. These two defects mean that it cannot be deployed on embedded devices with strictly limited resources. Therefore, in order to lightweight the SSD model to meet the MCU deployment conditions, this application will replace the VGG network and replace the VGG network in the SSD model with the MobileNet V1 network in the table below.

[0065] Table 1 MobileNet V1 model network structure

[0066]

[0067] Specifically, in the network architecture of MobileNet V1, firstly introduced is a standard convolutional layer, denoted as Conv1, which adopts a 3x3 convolution kernel and is set to a stride of 2 to realize spatial down-sampling. The image input to Conv1 has a resolution of 256x256 and 3 color channels. After processing by this layer, the number of channels of the output feature map is increased to 8. Subsequently, the network is composed of a series of depth separable convolutional layers, numbered as Conv_dw_1 to Conv_dw_13. Each depth separable convolutional layer is composed of two sub-layers, i.e., a depth convolution and a point-wise convolution (1x1 convolution). The stride of these convolutional layers is designed in an alternating mode, i.e., part of the layers adopt a stride of 1, while the other layers adopt a stride of 2, so as to gradually reduce the size of the feature map. Finally, at the end of the network, the Conv_dw_13 layer outputs a feature map with 256 channels and a size of 8x8.

[0068] S12, four additional depth separable convolutional layers are added after the MobileNet V1 network, and the regular convolutional layers in the SSD model are removed.

[0069] Table 2 Structure of additional depth separable convolutional layers

[0070]

[0071] Among them, the first layer and the third layer use a 3x3 convolution kernel to perform depth convolution on the input with a stride of 2, and then fuse channel information through 256 1x1 point-wise convolutions and apply L2 regularization. The second layer and the fourth layer use a 3x3 convolution kernel to perform spatial depth convolution on the input with a stride of 1 (each input channel is independently convolved), and then fuse channel information through 256 1x1 point-wise convolutions and apply L2 regularization.

[0072] In this embodiment, the Cotton-Weed-12-Class dataset is used for training. The Cotton-Weed-12-Class dataset is a large database widely used for target detection and image labeling, and belongs to the prior art, which will not be described here. Of course, the Cotton-Weed-12-Class dataset can also be replaced by other weed datasets or self-made datasets.

[0073] Before step S2, first optimize the annotation file in the data set, and divide it into two categories: the first category is to maintain the original annotation unchanged, which is defined as the first data set, and the second category is to merge the original 12 weed categories in the data set into one category, which is defined as the second data set. The classification of the first data set and the second data set is used to verify the effect of the model. Since the original data set has 12 types of weeds, if the annotation is not processed, the model will recognize what type of weed it is, but this approach will increase the model's calculation and resource occupation. If the purpose is only to identify weeds without distinguishing species, then the second data set after merging the categories is given to the model for training, which can greatly reduce the model's calculation and resource occupation, which can be reflected by the hardware resource occupation of the lightweight SSD model in Table 4.

[0074] The preprocessing in step S2 specifically includes the following steps:

[0075] S21, image standardization processing is performed on the images in the data set, and the pixel values of the images in the data set are adjusted to the standardized interval of -1 to 1 by scaling 1 / 127.5 and offsetting to -1;

[0076] S22, the nearest neighbor interpolation technique is used to reduce the image size to 256x256x3, while ensuring that the scaled image maintains the aspect ratio of the original image;

[0077] S23, the original image is converted to RGB format.

[0078] Preferably, the parameters of the model in step S3 include three categories: image augmentation, training hyperparameters, and post-processing parameters. Image preprocessing and image augmentation are processing of the data set, training hyperparameters are for the model, which affect the recognition effect of the model, and post-processing parameters do not participate in model training, but are only called when the model is evaluated, for example, when calculating mAP. The training parameters of the model are as follows:

[0079] A, image augmentation

[0080] S31: the random rotation angle is set to 30 degrees;

[0081] S32: the shear angle is set to 15 degrees;

[0082] S33: the translation is set to 0.1;

[0083] S34: the vertical flip is set to 0.5;

[0084] S35: the horizontal flip is set to 0.2;

[0085] S36: the Gaussian blur is set to 3.0;

[0086] S37: Linear contrast is set to a random value between 0.75 and 1.5;

[0087] B, training hyperparameters

[0088] S38: Use MobileNet V1 model, alpha coefficient is 0.25, input is (256, 256, 3);

[0089] S39: Batch size is set to 64;

[0090] S310: The number of training rounds is set to 1000;

[0091] S311: Use Adam optimizer, learning rate is set to 0.001;

[0092] S312: Use ReduceLROnPlateau callback function to monitor validation loss, learning rate decay factor is 0.1, patience is 20 cycles, and learning rate lower limit is 0.000001;

[0093] S313: Use EarlyStopping callback function, patience is 40 cycles;

[0094] C, post-processing parameters

[0095] S314: Confidence threshold is set to 0.6;

[0096] S315: Non-maximum suppression (NMS) threshold is set to 0.5;

[0097] S316: IoU evaluation threshold is set to 0.4;

[0098] S317: Maximum number of detection boxes is set to 10.

[0099] Preferably, the model is post-trained quantization using TensorFlow Lite Converter, quantizing the input to uint8, keeping the output as float, and using per-channel quantization strategy to optimize performance.

[0100] Referring to Figure 1 After obtaining the weed image, the quantized lightweight SSD model is applied to identify and process the weed image to determine the position. The steps of the identification process are as follows:

[0101] S51, extract the first stage feature map from the 5th deep separable convolution layer of the MobileNet V1 network, corresponding to Figure 1mobilenetV1 Up to Conv_DW_5 and the feature map labeled 32x32x64, mobilenetV1 Up to Conv_DW_5 refers to the 5th depthwise separable convolution layer in the mobilenetV1 network.

[0102] S52, extracting the second stage feature map from the 11th depthwise separable convolution layer of the MobileNet V1 network, corresponding to Figure 1 mobilenetV1 Up to Conv_DW_11 and the feature map labeled 16x16x128, mobilenetV1 Up to Conv_DW_11 refers to the 11th depthwise separable convolution layer in the mobilenetV1 network.

[0103] S53, extracting the third stage feature map from the 13th depthwise separable convolution layer of the MobileNet V1 network, corresponding to Figure 1 mobilenetV1 Up to Conv_DW_13 and the feature map labeled 8x8x256, mobilenetV1 Up to Conv_DW_13 refers to the 13th depthwise separable convolution layer in the mobilenetV1 network.

[0104] S54, extracting the fourth stage feature map from the second layer of additional depthwise separable convolution layers, corresponding to Figure 1 extra Up to Conv_DW_2 and the feature map labeled 4x4x256, extra Up to Conv_DW_2 refers to the second depthwise separable convolution layer in the extra convolution layers.

[0105] S55, extracting the fifth stage feature map from the fourth layer of additional depthwise separable convolution layers, corresponding to Figure 1 extra Up to Conv_DW_4 and the feature map labeled 2x2x256, extra Up to Conv_DW_4 refers to the fourth depthwise separable convolution layer in the extra convolution layers.

[0106] S56, performing a global average pooling operation on the fifth stage feature map and reshaping to form a sixth stage feature map, corresponding to Figure 1 GAP&Reshape and the feature map labeled 1x1x256, GAP refers to global average pooling, and Reshape refers to reshaping.

[0107] S57, single-layer convolution processing and batch normalization are performed on all stage feature maps, the single-layer convolution is a 1*1 point convolution layer, which reduces the channel number of the input feature map to 32 while keeping the size of the feature map unchanged, the weight is initialized using He normal distribution, and L2 regularization is applied, corresponding to Figure 1 Conv&BN under each feature map, Conv refers to a single-layer convolution, and BN refers to batch normalization;

[0108] S58, the FPN network is applied to the processed first-stage feature map to the sixth-stage feature map, that is, each feature map layer receives the sampling result of the previous layer (using bilinear interpolation) and performs element-wise addition (Add) with the feature map of the current layer, thereby realizing feature fusion, so that high-level semantic information can be combined with low-level detailed information;

[0109] S59, the detection head, that is, Figure 1 The Detector&Classifier processes the fused feature map obtained in step S58 to generate anchor boxes, classification predictions, and bounding box predictions;

[0110] S510, non-maximum suppression (NMS) is performed on the data output by the detection head in step S59 to obtain the final recognition position.

[0111] A weed detection system suitable for ARM architecture MCU, comprising a data preprocessing module, a model inference module, a data post-processing module, a data acquisition module, an information display module, and a model conversion module.

[0112] The data preprocessing module performs standardization, image format conversion and image compression on the collected image information, so that the image information adapts to the input standard of the model; in the data preprocessing module, the image format conversion refers to converting RGB565 format image information into RGB888 format information, and using a nearest neighbor image compression algorithm to compress the image to 256*256*3 size.

[0113] The model inference module uses a micro control unit to run a lightweight SSD model, in the model inference module, the micro control unit uses an ARM architecture MCU;

[0114] The data post-processing module decodes and performs non-maximum suppression on the output of the model to obtain weed information;

[0115] The data acquisition module uses a camera to capture real-time image information, and uses an ov2640 camera to collect RGB565 format image information;

[0116] The information display module displays the original image and the processed image information on the screen;

[0117] The model conversion module converts the model into a format that can run on an embedded device using CMSIS-NN.

[0118] To verify the performance of the quantized lightweight SSD model in the application, the ssd_mobilenet_v2_fpnlite model is selected for comparison. The two models have different alpha coefficients, and the rest of the training parameters and image preprocessing are the same. The alpha coefficient of the ssd_mobilenet_v2_fpnlite model is 0.35. The average precision (AP) is tested using the first and second data sets, respectively.

[0119] As shown in Figures 2 to 13 , the average precision of the model in the first data set is evaluated by the precision-recall curve. In the detection of various weeds, the model achieves high scores, fully demonstrating the effectiveness and reliability of the model in identifying different weed categories.

[0120] Table 3: Precision of each class in the quantized SSD model and ssd_mobilenet_v2_fpnlite model in the first data set

[0121]

[0122]

[0123] The data in Table 3 shows that the average precision (mAP) of the quantized model in the application is higher than that of the ssd_mobilenet_v2_fpnlite model in all categories.

[0124] As shown in Figure 14 , the average precision of the model in the second data set is evaluated by the precision-recall curve. In the detection of unified and merged weed categories, the model achieves higher scores, while the AP of the ssd_mobilenet_v2_fpnlite model is 88.19, fully demonstrating the effectiveness and reliability of the model in identifying weeds.

[0125] To verify the credibility and effectiveness of the above-mentioned weed detection system based on a lightweight SSD model, an experimental process is designed for verification:

[0126] To verify the hardware resource occupation of the micro control unit running the lightweight SSD model in the application, MACC, Activation RAM and Weights FLASH are used as hardware resource evaluation indexes, MACC refers to the multiplication and accumulation operation in the neural network, represents the model complexity and calculation resource consumption, Activation RAM represents the memory size occupied during model inference, Weights FLASH represents the model weight memory size, and the ssd_mobilenet_v2_fpnlite model is selected for comparison.

[0127] Table 4: Hardware resource occupation of the lightweight SSD model

[0128]

[0129] Table 5: Hardware resource occupation of the ssd_mobilenet_v2_fpnlite model

[0130]

[0131] The ARM architecture MCU with high performance generally has a RAM size of 512K or 1M and a FLASH size of 1M or 2M, and the occupied resources of the model are less than the hardware resource limit, so it is suitable for deploying inference on a resource-limited embedded platform. The data in Table 4 and Table 5 show that the hardware resource occupation of the lightweight SSD model in the application is more suitable for deploying inference on a resource-limited embedded platform compared with the ssd_mobilenet_v2_fpnlite model.

[0132] To improve the lightweight degree of the model, according to the data shown in Table 4: Hardware resource occupation of the lightweight SSD model, the SSD model after merging processing has reduced hardware resource occupation. The optimized model can adapt to the ARM architecture micro controller unit (MCU) with 512K RAM, ensuring normal operation under this hardware condition.

[0133] To verify the effectiveness of the weed detection system based on the lightweight SSD model in the application, the model is deployed in an ARM architecture MCU, specifically STM32H743XIT6, and the CubeAI model verification tool is used to determine whether the model is effective. The model is independently tested for L2 relative error. The test data set uses random tensors, and the experiment is repeated three times.

[0134] Table 6: System model verification

[0135]

[0136] The data in Table 6 shows that the deployed model meets the accuracy requirements, and all L2 relative errors are controlled below 0.01, which is consistent with the data used to build the model. In this embodiment, the independent test of L2 relative error is used to test whether the model deployed in the actual device is consistent with the model on the computer. They all input the same random tensor of 256x256x3 size, and then compare the outputs of the two, measure the L2 relative error, and the L2 relative error meets the requirements, indicating that their outputs are highly consistent, and it can be said that the deployed model meets the accuracy requirements.

[0137] To verify the inference performance of the embedded crowd counting system based on the dual-model architecture in the present application, the model is deployed in various ARM architecture MCUs, specifically STM32 series development boards, including: STM32H7S78-DK with a running frequency of 600Mhz, STM32H735G-DK with a running frequency of 550Mhz, NUCLEO-H743ZI2 with a running frequency of 480Mhz, STM32H747I-DISCO with a running frequency of 400MHz, STM32H7B3I-DK with a running frequency of 280MHz, STM32H573I-DK with a running frequency of 250MHz, STM32F769I-DISCO with a running frequency of 216MHz, STM32F746G-DISCO with a running frequency of 216MHz, B-U585I-IOT02A with a running frequency of 160MHz, STM32F469I-DISCO with a running frequency of 180MHz, and STM32L4R9I-DISCO with a running frequency of 120MHz, and the model inference time is tested, and the results are shown in Figure 15 The experimental results show that the inference time is mainly affected by the performance of the MCU, among which the STM32H735G-DK exhibits the fastest inference speed, and it only takes 188.9 milliseconds to complete the inference of a frame of data. The inference performance experiment of the system in this embodiment is to verify the feasibility of the system, and also reflects the size of the model's calculation amount from the side. The same input size of 256x256x3 random tensor is used in the independent test of L2 relative error, and the time from input to output given by the MCU is the inference time of the model. The larger the calculation amount of the model, the slower the time required for inference of a frame of data. 188.9 milliseconds is already at an excellent level for target detection algorithms in MCU inference.

[0138] This paper proposes a weed detection method for ARM-based microcontrollers (MCUs), involving a target detection algorithm. Typically, target detection algorithms must be deployed on devices with high computing power, such as GPUs or NPUs. However, the lightweight SSD model weed detection method proposed in this paper can be effectively deployed on ARM-based microcontroller units (MCUs), significantly reducing deployment costs and enabling efficient operation on devices with lower computing power.

[0139] To verify the low-power consumption characteristics of the weed detection method based on the lightweight SSD model for ARM architecture MCU in the present invention, the model was deployed in an ARM architecture MCU, specifically STM32H743XIH6, with an operating frequency of 480 MHz. An ov2640 camera was used to collect data, and an LCD screen with model SPI200M1-240*320 was used to display information. A USB tester was used to test the working system composed of the above components, including the working voltage, working current, and system power consumption.

[0140] Table 7 Test results

[0141]

[0142] The data in Table 7 show that the weed detection system based on the lightweight SSD model has lower power consumption.

[0143] The above examples demonstrate that the weed detection method and system proposed in the present invention, which is applicable to ARM-based MCUs, has the advantages of fast computation speed and low resource consumption due to the lightweight design of the network structure. Furthermore, due to the use of embedded devices, the system has the advantages of low cost and low power consumption.

[0144] The lightweight SSD model proposed in the present application not only focuses on reducing model parameters, but more importantly, it can adapt to the microcontroller unit (MCU) of ARM architecture, ensuring efficient operation under strict resource constraints, which is the purpose of the lightweight model. Since the original SSD model and the ssd_mobilenet_v2_fpnlite model cannot be deployed on the MCU of ARM architecture due to high resource requirements, the applicability of the model of the present application in this field is particularly prominent. Compared with the lightweight model ssd_mobilenet_v2_fpnlite released by the Google team, the model of the present application has significant advantages after training and quantization on the Cotton-Weed-12-Class public data set: while maintaining an accuracy improvement of 1.45%, the computational complexity is reduced by 66.7%, the RAM occupancy is reduced by 57.5%, and the Flash occupancy is reduced by 41.9%. This shows that the model of the present application is superior to ssd_mobilenet_v2_fpnlite in terms of computational efficiency and resource occupancy, and also has an improvement in detection accuracy.

[0145] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above embodiments, and the above embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A weed detection method suitable for ARM architecture MCU, characterized in that: The following steps are involved: S1. Build a lightweight SSD model: Replace the VGG network in the SSD model with the MobileNet V1 network, and add additional depthwise separable convolutional layers, feature processing layers, and feature pyramid structures. Step S1 includes: S11. Replace the VGG network in the SSD model with the MobileNet V1 network. Specifically, in the network architecture of MobileNet V1, a standard convolutional layer is first introduced, denoted as Conv1. This convolutional layer uses a 3×3 convolution kernel and sets the stride to 2 to achieve spatial downsampling. The image input to Conv1 has a resolution of 256×256 and 3 color channels. After processing by this layer, the number of channels of the output feature map increases to 8; Subsequently, the network consists of a series of depthwise separable convolutional layers, numbered Conv_dw_1 to Conv_dw_13. Each depthwise separable convolutional layer consists of two sub-layers: depthwise convolution and pointwise convolution. A series of depthwise separable convolutional layers are alternately set with a stride of 1 or 2 to gradually reduce the size of the feature map; Finally, at the end of the network, the Conv_dw_13 layer outputs a feature map of 256 channels with a size of 8×8; S12. Add four additional depthwise separable convolutional layers after MobileNet V1 and remove the regular convolutional layers in the SSD model. Specifically, the first and third layers use 3×3 convolution kernels with a stride of 2 to perform depthwise convolution on the input, followed by 256 1×1 pointwise convolutions to fuse channel information and apply L2 regularization. The second and fourth layers use 3×3 convolution kernels with a stride of 1 to perform spatial depthwise convolution on the input, followed by 256 1×1 pointwise convolutions to fuse channel information and apply L2 regularization. S2. Use the dataset and preprocess the dataset; S3. Using the preprocessed dataset, the improved lightweight SSD model is trained and verified using the TensorFlow framework. S4. Use TensorFlow Lite to perform full integer quantization on the trained model. S5. After obtaining the weed image, the quantized lightweight SSD model is applied to identify the weed image to determine the location. Step S5 includes: S51, extract the first stage feature map from the 5th depth-wise separable convolutional layer of the MobileNet V1 network; S52, extract the second stage feature map from the 11th depth-wise separable convolutional layer of the MobileNet V1 network; S53, extract the third stage feature map from the 13th depth-wise separable convolutional layer of the MobileNet V1 network; S54, extracting the fourth stage feature map from the second layer of additional depth-separable convolutional layer; S55, extract the fifth stage feature map from the fourth layer of additional depth-separable convolutional layer; S56, performing a global average pooling operation on the fifth stage feature map, and reshaping it to form a sixth stage feature map; S57, all stage feature maps are processed by single-layer convolution and batch normalization; S58, applying the FPN network to the processed first-stage feature maps to the sixth-stage feature maps, that is, each feature layer receives the sampling result of the previous layer and adds it element by element with the feature map of the current layer, thereby achieving feature fusion, so that high-level semantic information can be combined with low-level detail information; S59, the detection head processes the fused feature map obtained in step S58 to generate anchor boxes, classification predictions, and bounding box predictions; S510, performing non-maximum suppression on the data output by the detection head in step S59 to obtain the final recognition position.

2. A weed detection method suitable for ARM architecture MCU according to claim 1, characterized in that: Step S2 includes: S21, normalizing the pixel values ​​of the images in the dataset to a range of -1 to 1; S22, integrate the image size to fit a specific size while keeping the aspect ratio unchanged and use the nearest neighbor interpolation method; S23. Convert the original image into RGB format.

3. A weed detection system applicable to an ARM architecture MCU, configured to implement the weed detection method applicable to an ARM architecture MCU as claimed in claim 1 or 2, comprising: The data preprocessing module performs standardization, image format conversion and image compression on the collected image information to make the image information adapt to the input standard of the model; Model inference module, which uses a microcontroller unit to run a lightweight SSD model; The data post-processing module decodes and performs non-maximum suppression on the model output to obtain weed information.

4. A weed detection system suitable for ARM architecture MCU as claimed in claim 3, characterized in that: Also includes: Data acquisition module, using a camera to capture real-time image information; Information display module, which displays the original image and processed image information on the screen; The model conversion module uses CMSIS-NN to convert the model into a format that can run on embedded devices.

Images