Weed detection method and system suitable for ARM architecture MCU

Abstract

The invention discloses a weed detection method suitable for an ARM architecture MCU, and the method comprises the following steps: S1, constructing a lightweight SSD model: replacing a VGG network in the SSD model with a MobileNet V1 network, and adding an additional network layer and an FPN structure to improve the limitation of the MobileNet V1; s2, using the data set, and preprocessing the data set; s3, the preprocessed data set is utilized, and the improved lightweight SSD model is trained and verified through a TensorFlow framework; s4, adopting TensorFlow Lite to carry out full-integer quantization processing on the trained model, and adopting the TensorFlow Lite to carry out full-integer quantization processing on the trained model; and S5, after obtaining a weed image, performing identification processing on the weed image by using the quantized lightweight SSD model so as to determine a position. According to the method, by constructing the lightweight SSD model, the efficiency of weed detection is remarkably improved, the hardware requirement is reduced, meanwhile, high detection precision is kept, and in addition, full-integer quantification processing is adopted, so that the model is easy to deploy, has wide applicability and is well suitable for the agricultural field.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a weed detection method and system suitable for an ARM architecture MCU. Background Art

[0002] With the development of modern agricultural technology, precision agriculture has become an important means to increase crop yields and reduce the use of pesticides. Among them, weed detection and positioning is one of the key technologies to achieve precise weed control. Traditional weed detection methods mainly rely on manual identification, which is not only inefficient but also easily affected by subjective factors, resulting in low identification accuracy.

[0003] In recent years, the development of deep learning technology has brought revolutionary changes to the field of image recognition. In particular, convolutional neural networks (CNNs) have shown excellent performance in image classification and target detection. SSD (Single Shot MultiBox Detector) is a popular target detection algorithm that is widely used in various scenarios due to its fast speed and high detection accuracy. However, due to the large amount of computation and numerous parameters, the traditional SSD model is difficult to run in real time on embedded devices with limited computing resources, which to a certain extent limits its application in actual agricultural scenarios. At the same time, most of the existing technologies are based on YOLO weed detection methods, and the decentralized deployment of YOLO requires high-performance edge devices, while weed detection is still in a blank field in embedded devices, especially ARM architecture MCUs. In addition, the existing lightweight models often require expensive hardware resources during training and deployment, which increases the difficulty and cost of deployment in practical applications. Therefore, developing a lightweight SSD model that can run efficiently on embedded devices is of great significance for realizing weed detection in precision agriculture. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a weed detection method and system suitable for ARM architecture MCU.

[0005] To solve the above technical problems, the technical solution of the present invention is: a weed detection method suitable for ARM architecture MCU, comprising the following steps:

[0006] S1. Build a lightweight SSD model: Replace the VGG network in the SSD model with the MobileNet V1 network, and add additional depth-separable convolutional layers, feature processing layers, and feature pyramid structures to improve the limitations of MobileNet V1;

[0007] S2, use the data set and preprocess the data set;

[0008] S3. Using the preprocessed data set, the improved lightweight SSD model is trained and verified through the TensorFlow framework;

[0009] S4. Use TensorFlow Lite to perform full integer quantization on the trained model.

[0010] S5. After obtaining the weed image, the quantized lightweight SSD model is applied to perform recognition processing on the weed image to determine the position.

[0011] As a preferred technical solution, step S1 specifically includes the following steps:

[0012] S11. Replace the VGG network in the SSD model with the MobileNet V1 network. Specifically, in the network architecture of MobileNet V1, a standard convolution layer is first introduced, denoted as Conv1. This layer uses a 3×3 convolution kernel and sets the step size to 2 to achieve spatial downsampling. The image input to Conv1 has a resolution of 256×256 and 3 color channels. After being processed by this layer, the number of channels of the output feature map increases to 8. Subsequently, the network consists of a series of depthwise separable convolutional layers, numbered Conv_dw_1 to Conv_dw_13. Each depthwise separable convolutional layer consists of two sublayers: depthwise convolution and pointwise convolution (1×1 convolution). The step sizes of these convolutional layers are designed to be alternating, that is, some layers use a step size of 1, while other layers use a step size of 2, so as to gradually reduce the size of the feature map. Finally, at the end of the network, the Conv_dw_13 layer outputs a feature map of 256 channels with a size of 8×8.

[0013] S12. Add four additional depthwise separable convolutional layers after the MobileNet V1 network, and remove the regular convolutional layers in the SSD model. Specifically: the first and third layers use 3×3 convolution kernels to perform depthwise convolution on the input with a stride of 2, and then fuse the channel information through 256 1×1 point-by-point convolutions and apply L2 regularization. The second and fourth layers use 3×3 convolution kernels to perform spatial depthwise convolution on the input with a stride of 1 (each input channel is convolved independently), and then fuse the channel information through 256 1×1 point-by-point convolutions and apply L2 regularization.

[0014] As a preferred technical solution, the preprocessing in step S2 specifically includes the following steps:

[0015] S21, normalizing the pixel values ​​of the images in the data set to a range of -1 to 1;

[0016] S22, integrate the image size to fit a specific size while keeping the aspect ratio unchanged and use the nearest neighbor interpolation method;

[0017] S23. Convert the original image into RGB format.

[0018] As a preferred technical solution, the identification process in step S5 includes the following steps:

[0019] S51, extract the first stage feature map from the 5th depth-separable convolutional layer of the MobileNet V1 network;

[0020] S52, extract the second stage feature map from the 11th depth-separable convolutional layer of the MobileNet V1 network;

[0021] S53, extract the third stage feature map from the 13th depth-separable convolutional layer of the MobileNet V1 network;

[0022] S54, extracting the fourth stage feature map from the second layer of additional depth-separable convolutional layer;

[0023] S55, extracting the fifth stage feature map from the fourth additional depth-separable convolutional layer;

[0024] S56, performing a global average pooling operation on the fifth stage feature map, and reshaping it to form a sixth stage feature map;

[0025] S57, all stage feature maps are processed by single-layer convolution and batch normalization;

[0026] S58, applying the FPN network to the processed first-stage feature maps to the sixth-stage feature maps, that is, each feature layer receives the sampling result of the previous layer and adds it element by element with the feature map of the current layer, thereby realizing feature fusion, so that high-level semantic information can be combined with low-level detail information;

[0027] S59, the detection head processes the fused feature map obtained in step S58 to generate an anchor box, a classification prediction, and a bounding box prediction;

[0028] S510, performing non-maximum suppression on the data output by the detection head in step S59 to obtain the final recognition position.

[0029] As a preferred technical solution, a weed detection system suitable for ARM architecture MCU includes:

[0030] The data preprocessing module performs standardization, image format conversion and image compression on the collected image information so that the image information adapts to the input standard of the model;

[0031] Model inference module, which uses a microcontroller unit to run a lightweight SSD model;

[0032] The data post-processing module decodes and suppresses the model output to obtain weed information.

[0033] As a preferred technical solution, it also includes:

[0034] The data acquisition module uses a camera to capture real-time image information;

[0035] An information display module displays the original image and processed image information on the screen;

[0036] The model conversion module uses CMSIS-NN to convert the model into a format that can run on embedded devices.

[0037] Due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0038] 1. The core of MobileNet V1 is the depthwise separable convolution. Although this convolution can greatly reduce the number of parameters and the amount of calculation, and has a significant advantage in computational efficiency, in some complex tasks, such as target detection, the depthwise separable convolution cannot effectively extract high-level features, resulting in a decrease in detection performance. Therefore, the MobileNet V1 network is not widely used in the prior art; the present invention improves the limitations of the MobileNet V1 network, such as adding additional depthwise separable convolution and FPN structures, and additionally processing the extracted feature maps to improve detection performance;

[0039] 2. The present invention significantly improves the efficiency of weed detection and reduces hardware requirements while maintaining high detection accuracy by constructing a lightweight SSD model. In addition, the present invention adopts full-integer quantization processing, which makes the model easy to deploy and has wide applicability, thereby promoting the development of precision agriculture and bringing an efficient and economical solution to the agricultural field, with significant social and economic benefits and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The following drawings are intended only to illustrate and explain the present invention, and are not intended to limit the scope of the present invention.

[0041] Figure 1 It is a structural schematic diagram of the lightweight SSD model of the present invention;

[0042] Figure 2 It is the precision-recall curve and AP of the Carpetweed class;

[0043] Figure 3 It is the precision-recall curve and AP of Cutleaf class;

[0044] Figure 4 It is the precision-recall curve and AP of the Ecl ipta class;

[0045] Figure 5 It is the precision-recall curve and AP of the Goosegrass class;

[0046] Figure 6 It is the precision-recall curve and AP of MorningGlory class;

[0047] Figure 7 It is the precision-recall curve and AP of the PricklySida class;

[0048] Figure 8 It is the PalmerAmaranth class precision-recall curve and AP;

[0049] Fig. 9 It is the precision-recall curve and AP of the Purslane class;

[0050] Fig.10 It is the precision-recall curve and AP of the Ragweed class;

[0051] Fig.11 It is the Sicklepod class precision-recall curve and AP;

[0052] Fig.12 It is the precision-recall curve and AP of SpottedSpurge class;

[0053] Fig.13 It is the Waterhemp class precision-recall curve and AP;

[0054] Fig.14 It is the precision-recall curve and AP of the Weed class;

[0055] Fig.15 This is the inference time test result of the stm32 series development board; DETAILED DESCRIPTION

[0056] The present invention is further described below in conjunction with the accompanying drawings and examples. In the following detailed description, certain exemplary embodiments of the present invention are described only by way of illustration. Needless to say, those of ordinary skill in the art will recognize that the described embodiments may be modified in various ways without departing from the spirit and scope of the present invention. Therefore, the drawings and description are illustrative in nature and are not intended to limit the scope of protection of the claims.

[0057] A weed detection method applicable to an ARM architecture MCU comprises the following steps:

[0058] S1. Build a lightweight SSD model: Replace the VGG network in the SSD model with the MobileNet V1 network, and add additional depth-separable convolutional layers, feature processing layers, and feature pyramid structures to improve the limitations of MobileNet V1;

[0059] S2, use the data set and preprocess the data set;

[0060] S3. Using the preprocessed data set, the improved lightweight SSD model is trained and verified through the TensorFlow framework;

[0061] S4. Use TensorFlow Lite to perform full integer quantization on the trained model.

[0062] S5. After obtaining the weed image, the quantized lightweight SSD model is applied to perform recognition processing on the weed image to determine the position.

[0063] Step S1 is to make a lightweight improvement on the SSD model in the prior art, which specifically includes the following steps:

[0064] S11. The VGG network has two major shortcomings: 1. The number of parameters is huge: the VGG network contains a large number of parameters. For example, VGG-16 has about 138 million parameters. This causes the model to require a large amount of storage space and a large amount of computing resources during training and inference. 2. Low computational efficiency: Due to the large number of parameters, the computational efficiency of the VGG model is relatively low. This is mainly because a large number of fully connected layers are used in the model, which are very time-consuming to calculate. These two defects mean that it cannot be deployed on embedded devices with strictly limited resources. Therefore, in order to lightweight the SSD model to meet the MCU deployment conditions, this application replaces the VGG network and replaces the VGG network in the SSD model with the MobileNet V1 network in the following table.

[0065] Table 1 MobileNet V1 model network structure

[0066]

[0067] Specifically, in the network architecture of MobileNet V1, a standard convolution layer is first introduced, denoted as Conv1. This layer uses a 3×3 convolution kernel and sets the step size to 2 to achieve spatial downsampling. The image input to Conv1 has a resolution of 256×256 and 3 color channels. After processing by this layer, the number of channels of the output feature map increases to 8; subsequently, the network consists of a series of depthwise separable convolutional layers, numbered Conv_dw_1 to Conv_dw_13. Each depthwise separable convolutional layer consists of two sub-layers: depthwise convolution and pointwise convolution (1×1 convolution). The step size of these convolutional layers is designed to be an alternating mode, that is, some layers use a step size of 1, while other layers use a step size of 2, so as to gradually reduce the size of the feature map; finally, at the end of the network, the Conv_dw_13 layer outputs a feature map of 256 channels with a size of 8×8.

[0068] S12. Add four additional depth-wise separable convolutional layers after the MobileNet V1 network and remove the regular convolutional layers in the SSD model.

[0069] Table 2 Additional depth-separable convolutional layer structure

[0070]

[0071] Among them, the first and third layers use 3×3 convolution kernels to perform depth convolution on the input with a stride of 2, and then fuse the channel information through 256 1×1 point-by-point convolutions and apply L2 regularization. The second and fourth layers use 3×3 convolution kernels to perform spatial depth convolution on the input with a stride of 1 (each input channel is convolved independently), and then fuse the channel information through 256 1×1 point-by-point convolutions and apply L2 regularization.

[0072] In this embodiment, the Cotton-Weed-12-Class dataset is used for training. The Cotton-Weed-12-Class dataset is a large database widely used for target detection and image annotation, which belongs to the prior art and will not be described in detail here. Of course, the Cotton-Weed-12-Class dataset can also be replaced with other weed datasets or self-made datasets.

[0073] Before the preprocessing of step S2, the annotation files in the data set are first optimized and divided into two categories: the first category is to maintain the original annotations unchanged, which is defined as the first category of data sets, and the second category is to unify the original 12 weed categories in the data set into one category, which is defined as the second category of data sets. The classification of the first category of data sets and the second category of data sets is used to verify the effect of the model. Since the original data set has 12 types of weeds, if the annotations are not processed, the model will identify what type of weed it is, but this approach will increase the computational workload and resource usage of the model. If the purpose is only to identify weeds without distinguishing between types, then the second category of data sets after merging the types will be submitted to the model for training, which will greatly reduce the model's computational workload and resource usage, which can be reflected in the hardware resource usage of the lightweight SSD model in Table 4.

[0074] The preprocessing in step S2 specifically includes the following steps:

[0075] S21, performing image standardization processing on the images in the data set, adjusting the image pixel values ​​in the data set to a standardized interval of -1 to 1 by scaling the image by 1 / 127.5 and offsetting the image to -1;

[0076] S22, using the nearest neighbor interpolation technique to reduce the image size to 256×256×3, while ensuring that the scaled image maintains the aspect ratio of the original image;

[0077] S23. Convert the original image into RGB format.

[0078] Preferably, the parameters of the training model in step S3 include three categories: image augmentation, training hyperparameters and post-processing parameters. Image preprocessing and image augmentation are processing of the data set, while training hyperparameters are for the model and affect the recognition effect of the model. Post-processing parameters do not participate in model training and are only called when the model is evaluated, for example, when calculating mAP. The training parameters of the model are as follows:

[0079] A. Image Augmentation

[0080] S31: The random rotation angle is set to 30 degrees;

[0081] S32: The shear angle is set to 15 degrees;

[0082] S33: translation is set to 0.1;

[0083] S34: vertical flip is set to 0.5;

[0084] S35: horizontal flip is set to 0.2;

[0085] S36: Gaussian blur set to 3.0;

[0086] S37: linear contrast set to random values ​​between 0.75 and 1.5;

[0087] B. Training Hyperparameters

[0088] S38: Use the MobileNet V1 model, alpha coefficient is 0.25, and input is (256, 256, 3);

[0089] S39: Batch size is set to 64;

[0090] S310: The number of training rounds is set to 1000;

[0091] S311: Use Adam optimizer and set the learning rate to 0.001;

[0092] S312: Use the ReduceLROnPlateau callback function to monitor the validation loss, with a learning rate decay factor of 0.1, a tolerance of 20 cycles, and a learning rate lower limit of 0.000001;

[0093] S313: Use the EarlyStopping callback function with a tolerance of 40 cycles;

[0094] C. Post-processing parameters

[0095] S314: The confidence threshold is set to 0.6;

[0096] S315: The non-maximum suppression (NMS) threshold is set to 0.5;

[0097] S316: The IoU evaluation threshold is set to 0.4;

[0098] S317: The maximum number of detection frames is set to 10.

[0099] Preferably, use TensorFlow Lite Converter to perform post-training quantization on the model, quantize the input to uint8, keep the output as floating point, and adopt a per-channel quantization strategy to optimize performance.

[0100] See also Figure 1 , S5, after obtaining the weed image, the quantized lightweight SSD model is applied to perform recognition processing on the weed image to determine the location. The steps of recognition processing are as follows:

[0101] S51, extract the first stage feature map from the fifth depth-separable convolutional layer of the MobileNet V1 network, corresponding to Figure 1In mobilenetV1 Up to Conv_DW_5 and the feature map marked as 32×32×64, mobilenetV1 Up toConv_DW_5 refers to the 5th depthwise separable convolutional layer in the mobilenetV1 network.

[0102] S52, extract the second stage feature map from the 11th depth-separable convolutional layer of the MobileNet V1 network, corresponding to Figure 1 In mobilenetV1 Up to Conv_DW_11 and the feature map marked as 16×16×128, mobilenetV1 Upto Conv_DW_11 refers to the 11th depthwise separable convolutional layer in the mobilenetV1 network.

[0103] S53, extract the third-stage feature map from the 13th depth-separable convolutional layer of the MobileNet V1 network, corresponding to Figure 1 In mobilenetV1 Up to Conv_DW_13 and the feature map marked as 8×8×256, mobilenetV1 Up toConv_DW_13 refers to the 13th depthwise separable convolutional layer in the mobilenetV1 network.

[0104] S54, extract the fourth stage feature map from the second layer of additional depth separable convolutional layer, corresponding to Figure 1 In the figure, extra Up to Conv_DW_2 and the feature map marked as 4x4x256, extra Up to Conv_DW_2 refers to the second depth-wise separable convolutional layer in the extra convolutional layer.

[0105] S55, extract the fifth stage feature map from the fourth layer of additional depth separable convolutional layer, corresponding to Figure 1 In the figure, extra Up to Conv_DW_4 and the feature map marked as 2×2×256, extra Up to Conv_DW_4 refers to the fourth depth-wise separable convolutional layer in the extra convolutional layer.

[0106] S56, perform a global average pooling operation on the fifth stage feature map, and reshape it to form a sixth stage feature map, corresponding to Figure 1 In GAP&Reshape and the feature map marked as 1×1×256, GAP refers to global average pooling and Reshape refers to reshaping.

[0107] S57, perform single-layer convolution and batch normalization on all stage feature maps. The single-layer convolution is a 1×1 point-by-point convolution layer, which reduces the number of channels of the input feature map to 32 while keeping the size of the feature map unchanged. The weights are initialized using the He normal distribution and L2 regularization is applied, corresponding to Figure 1 Conv&BN below each feature map, Conv refers to a single layer of convolution, and BN refers to batch normalization;

[0108] S58, applying the FPN network to the processed first-stage feature maps to the sixth-stage feature maps, that is, each feature layer receives the sampling result of the previous layer (using bilinear interpolation) and adds it element by element with the feature map of the current layer, thereby realizing feature fusion, so that high-level semantic information can be combined with low-level detail information;

[0109] S59, detection head Figure 1 The fused feature map obtained in the Detector & Classifier processing step S58 is used to generate anchor boxes, classification predictions, and bounding box predictions;

[0110] S510, performing non-maximum suppression (NMS) on the data output by the detection head in step S59 to obtain the final recognition position.

[0111] A weed detection system suitable for ARM architecture MCU includes a data preprocessing module, a model reasoning module, a data post-processing module, a data acquisition module, an information display module and a model conversion module.

[0112] The data preprocessing module performs standardization, image format conversion and image compression on the collected image information, so that the image information adapts to the input standard of the model; in the data preprocessing module, the image format conversion refers to converting the RGB565 format image information into RGB888 format information, and using the nearest neighbor image compression algorithm to compress the image to a size of 256×256×3.

[0113] The model inference module uses a microcontroller unit to run a lightweight SSD model. In the model inference module, the microcontroller unit uses an ARM architecture MCU.

[0114] The data post-processing module decodes and suppresses the model output to obtain weed information;

[0115] The data acquisition module uses a camera to capture real-time image information and uses the ov2640 camera to collect RGB565 format image information;

[0116] The information display module displays the original image and processed image information through the screen;

[0117] The model conversion module utilizes CMSIS-NN to convert the model into a format that can run on embedded devices.

[0118] In order to verify the performance of the quantized lightweight SSD model in the present invention, the ssd_mobilenet_v2_fpnl ite model is selected for comparison. Except for the α coefficient, the other training parameters and image preprocessing of the two models are the same. The α coefficient of the ssd_mobilenet_v2_fpnl ite model is 0.35. The first and second data sets are used to test the average precision, i.e., AP, respectively.

[0119] like Figures 2 to 13 As shown, the average accuracy of the model of the present invention in the first category of data sets is evaluated by the precision-recall curve. In the detection of various types of weeds, the model achieves relatively high scores, which fully proves the effectiveness and reliability of the model in identifying different types of weeds.

[0120] Table 3 Accuracy of various types of the first dataset in the quantized SSD model and ssd_mobilenet_v2_fpnlite model

[0121]

[0122]

[0123] The data in Table 3 reflect that the average precision (mAP) of the quantization model of the present invention on all categories is higher than that of the ssd_mobilenet_v2_fpnl ite model;

[0124] like Fig.14 As shown, the average accuracy of the model of the present invention in the second type of data set is evaluated by the precision-recall curve. In the detection of the unified merged weeds, the model of the present invention achieves a higher score, while the AP of the ssd_mobilenet_v2_fpnl ite model is 88.19, which fully proves the effectiveness and reliability of the model in identifying weeds.

[0125] In order to verify the credibility and effectiveness of the above-mentioned weed detection system based on the lightweight SSD model, an experimental process is designed for verification:

[0126] In order to verify the hardware resource occupancy of the microcontroller unit in the present invention running the lightweight SSD model, MACC, Activation RAM, and Weights FLASH are used as hardware resource evaluation indicators. MACC refers to the multiplication and accumulation operations in the neural network, indicating the model complexity and computing resource consumption. Activation RAM indicates the memory size occupied during model inference, and Weights FLASH indicates the memory size occupied by the model weights. The ssd_mobilenet_v2_fpnl ite model is selected for comparison.

[0127] Table 4 Hardware resource usage of lightweight SSD model

[0128]

[0129] Table 5 Hardware resource usage of ssd_mobilenet_v2_fpnlite model

[0130]

[0131] The RAM size of a high-performance ARM architecture MCU is generally 512K or 1M, and the FLASH size is 1M or 2M. The occupied resources of the model are less than the hardware resource limit, so it is suitable for deploying reasoning on resource-constrained embedded platforms. The data in Tables 4 and 5 show that the hardware resource occupation of the lightweight SSD model of the present invention is more suitable for deploying reasoning on resource-constrained embedded platforms than the ssd_mobilenet_v2_fpnl ite model.

[0132] In order to improve the lightweight degree of the model, according to the data shown in Table 4, the hardware resource occupation of the lightweight SSD model, the SSD model after merging has reduced the hardware resource occupation. The optimized model can adapt to the ARM architecture microcontroller unit (MCU) with 512KB RAM, ensuring normal operation under this hardware condition.

[0133] To verify the effectiveness of the weed detection system based on the lightweight SSD model in the present invention, the model was deployed in an ARM architecture MCU, specifically STM32H743XIT6, and the CubeAI model verification tool was used to determine whether the model was effective. The model was independently tested for L2 relative error. The test data set used random tensors, and the experiment was repeated three times.

[0134] Table 6 System model verification

[0135]

[0136] The data in Table 6 reflect that the deployed model meets the accuracy requirements, and all L2 relative errors are controlled below 0.01, which is consistent with the data used to build the model. The independent test of L2 relative error in this embodiment is used to test whether the model deployed on the actual device is consistent with the model on the computer. They both input the same random tensor of size 256×256×3, and then compare the outputs of the two to measure the L2 relative error. The L2 relative error meets the requirements, indicating that their outputs are highly consistent, and it can also be said that the deployed model meets the accuracy requirements.

[0137] In order to verify the reasoning performance of the embedded people counting system based on the dual model architecture in the present invention, the model is deployed in a variety of ARM architecture MCUs, specifically STM32 series development boards, including: STM32H7S78-DK operating frequency is 600Mhz, STM32H735G-DK operating frequency is 550Mhz, NUCLEO-H743ZI2 operating frequency is 480Mhz, STM32H747I-DISCO operating frequency is 400MHz, STM32H7B3I-DK operating frequency is 400MHz, and STM32H7S78-DK operating frequency is 600Mhz. The frequency is 280MHz, STM32H573I-DK is 250MHz, STM32F769I-DISCO is 216MHz, STM32F746G-DISCO is 216MHz, B-U585I-IOT02A is 160MHz, STM32F469I-DISCO is 180MHz, STM32L4R9I-DISCO is 120MHz, and the model inference time is tested. The results are as follows Fig.15 As shown in the figure, the solid points represent the reasoning time of the first type of data set, and the hollow points represent the reasoning time of the second type of data set. The experimental results show that the reasoning time is mainly affected by the performance of the MCU, among which the STM32H735G-DK shows the fastest reasoning speed, and it only takes 188.9 milliseconds to complete the reasoning of a frame of data. The reasoning performance experiment of the system of this embodiment is to verify the feasibility of the system, and it can also reflect the size of the model's calculation amount from the side. The same as the independent test of the L2 relative error, the input size is a random tensor of 256×256×3, and the calculation from the input to the MCU giving the output is the reasoning time of the model. The larger the calculation amount of the model, the slower the time required to reason a frame of data. 188.9 milliseconds is already at an excellent level for the target detection algorithm to be reasoned in the MCU.

[0138] The present invention proposes a weed detection method suitable for ARM architecture MCU, which involves a target detection algorithm. Generally, the target detection algorithm needs to be deployed on a device with higher computing power, such as a GPU or NPU. However, the lightweight SSD model weed detection method proposed in the present invention can be effectively deployed on an ARM architecture microcontroller unit (MCU), thereby greatly reducing the deployment cost and achieving efficient operation on devices with lower computing power.

[0139] In order to verify the low power consumption characteristics of the weed detection method based on the lightweight SSD model suitable for ARM architecture MCU in the present invention, the model is deployed in the ARM architecture MCU, specifically STM32H743XIH6, with an operating frequency of 480Mhz, an ov2640 camera is used to collect data, and an LCD screen is used to display information, model SPI200M1-240*320. A USB tester is used to test the working system composed of the above components, including the working voltage, working current, and system power consumption.

[0140] Table 7 Test results

[0141]

[0142] The data in Table 7 show that the weed detection system based on the lightweight SSD model has lower power consumption.

[0143] It can be seen from the verification of the above embodiments that the weed detection method and system suitable for ARM architecture MCU proposed in the present invention have the advantages of fast calculation speed and less resource occupation due to the lightweight design of the network structure; and the system has the advantages of low cost and low power consumption due to the use of embedded devices.

[0144] The lightweight SSD model proposed in this application not only focuses on reducing model parameters, but more importantly, it can adapt to the ARM architecture microcontroller unit (MCU) to ensure efficient operation under strict resource constraints, which is also the purpose of lightweighting this application. Since the original SSD model and the ssd_mobilenet_v2_fpnl ite model cannot be deployed on the ARM architecture MCU due to excessive resource requirements, the applicability of the model of this application in this field is particularly prominent. Compared with the lightweight model ssd_mobilenet_v2_fpnl ite released by the Google team, the model of this application and the ssd_mobilenet_v2_fpnl ite model showed significant advantages after training and quantization on the Cotton-Weed-12-Class public data set: while maintaining an accuracy improvement of 1.45%, the amount of calculation was reduced by 66.7%, the RAM occupancy was reduced by 57.5%, and the Flash occupancy was reduced by 41.9%. This shows that the model of this application is superior to ssd_mobilenet_v2_fpnl ite in terms of computational efficiency and resource usage, and also improves detection accuracy.

[0145] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A weed detection method suitable for ARM architecture MCU, characterized in that: The following steps are involved: S1. Build a lightweight SSD model: Replace the VGG network in the SSD model with the MobileNet V1 network, and add additional depth-separable convolutional layers, feature processing layers, and feature pyramid structures to improve the limitations of MobileNet V1; S2, use the data set and preprocess the data set; S3. Using the preprocessed data set, the improved lightweight SSD model is trained and verified through the TensorFlow framework; S4. Use TensorFlow Lite to perform full integer quantization on the trained model. S5. After obtaining the weed image, the quantized lightweight SSD model is applied to perform recognition processing on the weed image to determine the position.

2. A weed detection method suitable for ARM architecture MCU as claimed in claim 1, characterized in that: Step S1 specifically includes the following steps: S11. Replace the VGG network in the SSD model with the MobileNet V1 network. Specifically, in the network architecture of MobileNet V1, a standard convolution layer is first introduced, denoted as Conv1. This layer uses a 3×3 convolution kernel and sets the step size to 2 to achieve spatial downsampling. The image input to Conv1 has a resolution of 256×256 and 3 color channels. After being processed by this layer, the number of channels of the output feature map increases to 8. Subsequently, the network consists of a series of depthwise separable convolutional layers, numbered Conv_dw_1 to Conv_dw_13. Each depthwise separable convolutional layer consists of two sublayers: depthwise convolution and pointwise convolution (1×1 convolution). The step sizes of these convolutional layers are designed to be alternating, that is, some layers use a step size of 1, while other layers use a step size of 2, so as to gradually reduce the size of the feature map. Finally, at the end of the network, the Conv_dw_13 layer outputs a feature map of 256 channels with a size of 8×8. S12. Add four additional depthwise separable convolutional layers after the MobileNet V1 network, and remove the regular convolutional layers in the SSD model. Specifically: the first and third layers use 3×3 convolution kernels to perform depthwise convolution on the input with a stride of 2, and then fuse the channel information through 256 1×1 point-by-point convolutions and apply L2 regularization. The second and fourth layers use 3×3 convolution kernels to perform spatial depthwise convolution on the input with a stride of 1 (each input channel is convolved independently), and then fuse the channel information through 256 1×1 point-by-point convolutions and apply L2 regularization.

3. A weed detection method suitable for ARM architecture MCU as claimed in claim 1, characterized in that: The preprocessing in step S2 specifically includes the following steps: S21, normalizing the pixel values ​​of the images in the data set to a range of -1 to 1; S22, integrate the image size to fit a specific size while keeping the aspect ratio unchanged and use the nearest neighbor interpolation method; S23. Convert the original image into RGB format.

4. A weed detection method suitable for ARM architecture MCU as claimed in claim 1, characterized in that: The identification process in step S5 includes the following steps: S51, extract the first stage feature map from the 5th depth-separable convolutional layer of the MobileNet V1 network; S52, extract the second stage feature map from the 11th depth-separable convolutional layer of the MobileNet V1 network; S53, extract the third stage feature map from the 13th depth-separable convolutional layer of the MobileNet V1 network; S54, extracting the fourth stage feature map from the second layer of additional depth-separable convolutional layer; S55, extracting the fifth stage feature map from the fourth additional depth-separable convolutional layer; S56, performing a global average pooling operation on the fifth stage feature map, and reshaping it to form a sixth stage feature map; S57, all stage feature maps are processed by single-layer convolution and batch normalization; S58, applying the FPN network to the processed first-stage feature maps to the sixth-stage feature maps, that is, each feature layer receives the sampling result of the previous layer and adds it element by element with the feature map of the current layer, thereby realizing feature fusion, so that high-level semantic information can be combined with low-level detail information; S59, the detection head processes the fused feature map obtained in step S58 to generate an anchor box, a classification prediction, and a bounding box prediction; S510, performing non-maximum suppression on the data output by the detection head in step S59 to obtain the final recognition position.

5. A weed detection system applicable to an ARM architecture MCU, used to implement a weed detection method applicable to an ARM architecture MCU as claimed in any one of claims 1 to 4, comprising: The data preprocessing module performs standardization, image format conversion and image compression on the collected image information so that the image information adapts to the input standard of the model; Model inference module, which uses a microcontroller unit to run a lightweight SSD model; The data post-processing module decodes and suppresses the model output to obtain weed information.

6. A weed detection system suitable for ARM architecture MCU as claimed in claim 5, characterized in that: Also includes: The data acquisition module uses a camera to capture real-time image information; An information display module displays the original image and processed image information on the screen; The model conversion module uses CMSIS-NN to convert the model into a format that can run on embedded devices.

Images