Alfalfa weed detection method and system based on lightweight equilibrium feature fusion
By deploying a fine-grained alfalfa weed detection model on cloud servers and image processing devices, combining data enhancement and loss function optimization, the problem of alfalfa and broadleaf weed recognition in the existing technology is solved, and the detection accuracy and efficiency are improved. Human-machine precise weeding provides support.
Patent Information
- Application Number
- CN202411018118.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-26
AI Technical Summary
Existing weed identification methods are difficult to accurately classify and count alfalfa and broadleaf weeds, especially in alfalfa plots with similar colors and appearances. They cannot support drone precise dispensing and spraying, resulting in low weeding efficiency and accuracy.
Using alfalfa weed detection method based on lightweight balanced feature fusion, by deploying a fine-grained alfalfa weed detection model on cloud servers and image processing devices, using image preprocessing, backbone network feature extraction layer, and lightweight hierarchical multipathing The balanced feature fusion layer and detection head are identified, and the model parameters are optimized by combining data enhancement and loss function to achieve real-time detection.
It improves the classification detection and regional statistical accuracy of alfalfa weeds, provides a theoretical basis for the precise proportion of herbicides, enhances the detection capabilities of alfalfa and multiple types of weeds, and is suitable for the deployment of resource-constrained equipment.
Smart Images

Figure BDA0004966347480000051 
Figure BDA0004966347480000053 
Figure BDA0004966347480000061
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision target detection, and in particular to an alfalfa weed detection method and system based on lightweight balanced feature fusion. Background Art
[0002] Alfalfa is a perennial leguminous forage grass that was cultivated the earliest, with the largest planting area and the most planting countries in the world. It is also the main forage grass cultivated in my country. It has the characteristics of high yield, good quality, strong adaptability and rich nutrition. It can not only ensure the protein source of livestock and poultry, but also use the advantages of nitrogen fixation to improve soil structure and increase soil nutrients. It plays an important role in many fields such as food security, environment, climate change, and carbon cycle. As of 2023, the planting area of commercial alfalfa in my country is about 6.59 million mu, and the dry matter output that can be harvested is about 3.5 to 3.8 million tons, including 1.7 to 1.75 million tons of alfalfa hay and 1.6 to 1.8 million tons of alfalfa silage.
[0003] Competition from weeds is one of the main hazards affecting alfalfa yield and quality, and has a great impact on animal husbandry, breeding, and the economic income of growers. In the farmland ecosystem, weeds not only compete with alfalfa for resources such as light, nutrients, and space, resulting in reduced alfalfa yields, but also provide an environment and conditions for pests to survive and reproduce, inducing alfalfa diseases and insect pests.
[0004] In order to effectively control weeds, large-scale alfalfa farms currently rely mainly on drones for weeding operations. This method improves the efficiency and coverage of weeding through automated drug dispensing and drone spraying. Accurately counting the number, proportion and distribution of weeds is the key to accurate drug dispensing and spraying by drones. Therefore, the process of identifying, counting and locating weeds is very important. Many existing weed identification methods are mainly used to distinguish between crops and weeds. Due to the significant differences between crops and weeds, the recognition accuracy is relatively high. However, there are many types of weeds in alfalfa plots. Alfalfa and some broadleaf weeds have great similarities in color and appearance characteristics, making weed identification more difficult. It is difficult to accurately classify and count weed types and areas using existing image recognition technology, and it is impossible to support accurate drug dispensing and spraying by drones. Summary of the invention
[0005] In view of this, the present invention provides an alfalfa weed detection method and system based on lightweight balanced feature fusion, which can perform classification detection and regional statistics on alfalfa weeds, effectively improve recognition efficiency and accuracy, and provide a strong theoretical basis for the precise ratio of herbicides.
[0006] The technical solution adopted by the embodiment of the present invention to solve the technical problem is:
[0007] A method for detecting alfalfa weeds based on lightweight balanced feature fusion, comprising:
[0008] Step S1, deploying a lightweight hierarchical multi-path balanced feature fusion fine-grained alfalfa weed detection model on both the cloud server device and the image processing device, wherein the fine-grained alfalfa weed detection model includes an image preprocessing input layer, a backbone network feature extraction layer, a lightweight hierarchical multi-path balanced feature fusion layer, and a detection head;
[0009] Step S2, the cloud server device obtains an original image set, generates labels for the original images, and further enhances each of the original images to obtain an enhanced image, wherein the labels are description sequences of the locations of all weeds in the image and the weed categories corresponding to the weed locations; a data set is established using the original images and the enhanced images, and a training set, a validation set, and a test set are divided based on samples in the data set;
[0010] Step S3, the cloud server device trains the fine-grained alfalfa weed detection model using the training set, and optimizes the model parameter weights using a loss function to obtain an optimized model, verifies the optimized model using the verification set, and evaluates the model performance using the test set;
[0011] Step S4, the cloud server device transmits the weight parameters of the optimized model to the image processing device;
[0012] Step S5, the image processing device uses the weight parameters transmitted by the cloud server device to update the model, and uses the updated model to perform real-time detection of alfalfa weeds on the image collected in real time by the image acquisition device.
[0013] Preferably, step S2 comprises:
[0014] Step S21, classifying the elements in the original image set into a simple environment image set and a complex environment image set; wherein a complex environment image refers to an image in which the number of weeds blocked accounts for more than 30%, and a simple environment image refers to an image in which the number of weeds blocked accounts for no more than 30%;
[0015] Step S22, generating a label for each of the original images according to the pixel locations of the weeds in each image and the actual weed categories corresponding to the pixel locations;
[0016] Step S23, respectively expanding the elements in the simple environment image set and the complex environment image set, the expansion method is to enhance the original elements and place the obtained enhanced images in the set where the original elements are located, the enhancement processing includes five methods: horizontal flipping, vertical flipping, Gaussian blurring, brightness brightening, and brightness dimming. Accordingly, each of the original elements corresponds to five enhanced images with different effects, and the number of set elements is expanded to six times the original number;
[0017] Step S24, divide the simple environment image set into a first training set, a first verification set and a first test set in proportion, and divide the complex environment image set into a second training set, a second verification set and a second test set in proportion, merge the first training set and the second training set into the training set, merge the first verification set and the second verification set into the verification set, and merge the first test set and the second test set into the test set.
[0018] Preferably, the fine-grained alfalfa weed detection model detects two types of weeds, including broadleaf weeds and grass weeds;
[0019] Brightness brightening means brightening the image by 1.5 times, and brightness dimming means dimming the image by 0.5 times;
[0020] When the two data sets are divided into training sets, validation sets, and test sets, the division ratio of the simple environment data set is 6:3:1, and the division ratio of the complex environment data set is 7:2:1.
[0021] Preferably, the step S3 of training the fine-grained alfalfa weed detection model using the training set comprises:
[0022] Step S31, the image preprocessing input layer inputs the sample image in the training set, reshapes the sample image into an image X of 320 pixels×320 pixels and inputs it into the backbone network feature extraction layer;
[0023] Step S32, in the backbone network feature extraction layer, the image X is sequentially subjected to a fine-grained feature enhancement module, a feature extraction layer composed of an inverted residual with a step size of 1, a feature extraction layer composed of an inverted residual with a step size of 1 and two inverted residuals with a step size of 2, three feature extraction layers composed of an inverted residual with a step size of 2 and a MobileViT module, and four feature extraction layers composed of an inverted residual with a step size of 2, and down-sampled feature maps of 4 times, 8 times, 16 times, 32 times, 64 times, 128 times, 256 times and 512 times are obtained respectively, and a 16-fold feature map C4, a 32-fold feature map C5, a 64-fold feature map C6, a 128-fold feature map C7, a 256-fold feature map C8 and a 512-fold feature map C9 are taken and output to the lightweight hierarchical multipath equalization feature fusion layer;
[0024] Step S33, the lightweight hierarchical multipath equalization feature fusion layer uses five levels and six group convolutions to perform feature fusion on the six groups of feature maps respectively, and outputs the six groups of fused feature maps P4, P5, P6, P7, P8, and P9 output by the group convolution to the detection head module:
[0025] Step S331, the first level performs three upsampling on the feature map C9 to align the sizes of the feature maps of C8, C6 and C4 respectively, and directly splices them in the channel dimension, the second level performs two upsampling on the C8 feature map to align the sizes of the feature maps of C7 and C5 respectively, and directly splices them in the channel dimension, the third level performs two upsampling on the C7 feature map to align the sizes of the feature maps of C6 and C4 respectively, and directly splices them in the channel dimension, the fourth level performs one upsampling on the C6 feature map to align the size of the feature map of C5, and directly splices it in the channel dimension, the fifth level performs one upsampling on the C5 feature map to align the size of the feature map of C4, and directly splices it in the channel dimension;
[0026] Step S332, performing group convolution with a grouping number of 8 on each splicing result, unifying the number of all channels to 256, and outputting cross-channel fusion feature maps P4, P5, P6, P7, P8, and P9;
[0027] Step S34, the detection head module performs depthwise separable convolution on the fused feature maps P4, P5, P6, P7, and P8 respectively, and the regressor outputs the offset of the prediction box based on the feature map obtained by the depthwise separable convolution, and the classifier generates the weed type probability distribution of the corresponding prediction box based on the feature map obtained by the depthwise separable convolution; the fused feature map P9 is subjected to 1×1 standard convolution, and the regressor outputs the offset of the prediction box based on the feature map obtained by the 1×1 standard convolution, and the classifier generates the weed type probability distribution of the corresponding prediction box based on the feature map obtained by the 1×1 standard convolution;
[0028] The depthwise separable convolution operation is as follows: performing convolution operations on each channel of the feature fusion map respectively to obtain features of each channel, and then using point-by-point convolution to linearly combine local features of all channels to obtain the feature map obtained by the depthwise separable convolution;
[0029] Step S35, setting a priori boxes with different aspect ratios for each unit of each of the feature fusion images, and mapping them back to the sample image through the scaling ratio, to obtain a prediction result of the sample image, wherein the prediction result includes prediction box regression and prediction box selection target category;
[0030] Step S36, using the loss function to adjust the model weight: by calculating the cross entropy loss function and the SmoothL1 loss function, the model parameter weights are gradually corrected and optimized, wherein the expression of the cross entropy loss function is:
[0031]
[0032] In the formula, y i is the true label sequence of the i-th sample, is the probability sequence of the model's prediction results for the i-th sample, and N is the total number of samples in the training set;
[0033] The expression of SmoothL1 loss function is:
[0034]
[0035] Where x is the difference between the predicted value and the true value, and is the coordinate deviation and size deviation;
[0036] Step S37, after the model is trained using all samples in the training set, the trained model is verified using the verification set;
[0037] Step S38, using the test set to evaluate the model performance: the samples of the test set are input into the trained model in batches to perform predictions and obtain test prediction results, the test prediction results are sequentially screened by confidence and non-maximum suppression algorithms, and the model performance is evaluated using three indicators for the screened test prediction results:
[0038]
[0039] Where Precision is the accuracy; TP is the true positive class, which indicates the number of broadleaf weeds and grass weeds that are correctly detected by the model; FP is the false positive class, which indicates the number of places that are not actually weeds that the model mistakenly judges as broadleaf weeds or grass weeds; Recall is the recall rate; FN is the false negative class, which indicates the number of actual broadleaf weeds or grass weeds that are not detected by the model; mAP50 is the average of all the final classes, the average precision AP of the broadleaf weed class and the grass weed class i is the area under the Precision-Recall curve; n is the total number of weed species.
[0040] Preferably, the feature map sizes of feature maps C4, C5, C6, C7, C8, and C9 are 20×20, 10×10, 5×5, 3×3, 2×2, and 1×1, respectively;
[0041] The fine-grained feature enhancement module is composed of a 3×3 convolution and an ASPP module with a dilation rate of [1, 6, 12, 18] in sequence; the ASPP module calculates the convolution operation expression with a dilation rate of r as follows:
[0042]
[0043] In the formula, i is the position index of the current output element; r is the dilation rate, which indicates the distance between the convolution kernel elements; k is the index of the convolution kernel element, so i+r·k represents the sampling position in the input feature map; x[i+r·k] is the element extracted from the input feature map X, and ω[k] is the weight of the convolution kernel at position k; the sum of all convolution kernel elements is the output value at position i;
[0044] Feature map Y after ASPP module final The expression is:
[0045]
[0046] In the formula, represents the output feature map of 4 dilated convolutions; GAP(X) represents the feature map after the global average pooling operation; Concat represents concatenating all the above output feature maps in the channel dimension; W represents the weight of 1×1 convolution; σ represents the ReLU activation function;
[0047] The inverted residual structure with a step size of 1 is divided into four stages, the first stage is a 1×1 convolution layer to increase the dimension of the input, the second stage is a 3×3 deep convolution layer and the activation function Relu6, the third stage is a 1×1 convolution layer, and the fourth stage concatenates the input feature map and the feature map output by the third stage on the feature channel; the inverted residual structure with a step size of 2 is divided into three stages, the first stage is a 1×1 convolution layer to increase the dimension of the input, the second stage is a 3×3 deep convolution layer and the activation function Relu6, and the third stage is a 1×1 convolution layer;
[0048] The MobileViT module includes a local feature extraction unit, a global feature extraction unit, a 1×1 convolution layer, a splicing unit, and a 3×3 convolution layer in sequence. The local feature extraction unit is composed of a 3×3 convolution layer and a 1×1 convolution layer, and the global feature extraction unit is composed of an expansion unit, L Transformers, and a folding unit in sequence. After the original image is input into the MobileViT module, local feature extraction processing, global feature extraction processing, and 1×1 convolution processing are performed in sequence. The obtained feature map is spliced with the original image in the channel dimension and then output after passing through a 3×3 convolution layer.
[0049] Optimally, the model hyperparameters are set as follows: input image size 320 pixels × 320 pixels, width factor 1.0, number of categories 2, initial learning rate 0.001, momentum 0.9, weight decay 4e -5, minimum learning rate 0.00001, training rounds 1000, warm-up rounds 1, optimizer, SGD, learning rate change strategy is cosine annealing, batch size is 16; in the detection head module, a priori boxes with different aspect ratios are set for each unit of the feature map, and mapped back to the original image through the scaling ratio. The six sets of feature map sizes used are 20×20, 10×10, 5×5, 3×3, 2×2, and 1×1, with a total of 539 units. The number of prior boxes set for each unit is 6, and there are 3234 prior boxes for all units.
[0050] Preferably, the step S332 uses bilinear interpolation to perform upsampling.
[0051] Preferably, the image acquisition device is a drone, and after step S5, the method further includes:
[0052] Step S6, the drone performs a preset route flight mission, collects real-time images and sends them to the image processing device and the cloud server device; Step S7, the image processing device receives the real-time images, uses the optimized model to perform real-time alfalfa weed recognition on the real-time images, and obtains recognition results;
[0053] Step S8, the cloud server device receives the real-time image, uses the real-time image to establish an updated data set, regularly optimizes the model according to the updated data set, and provides the updated model to the image processing device.
[0054] Preferably, the step S8 of periodically optimizing the model according to the updated data set and providing the updated model to the image processing device includes:
[0055] Step S81, obtaining the updated data set according to step S2, training a model based on the updated training set, updating the parameter weights of the model, and calculating the mAP50 of the updated model;
[0056] Step S82: If the mAP50 of the updated model is greater than the mAP50 of the model currently used by the image processing device, the weight parameters of the updated model are passed to the image processing device for weight update; if the mAP50 of the updated model is not better than the mAP50 of the model currently used by the image processing device, the update data is not passed to the image processing device.
[0057] The present invention provides an alfalfa weed detection system based on lightweight balanced feature fusion, comprising a cloud server device, an image processing device, and an image acquisition device, which are used to execute the above-mentioned method.
[0058] It can be seen from the above technical scheme that the alfalfa weed detection method and system based on lightweight balanced feature fusion provided by the embodiment of the present invention deploys a fine-grained alfalfa weed detection model with lightweight hierarchical multi-path balanced feature fusion on both the cloud server device and the image processing device; the cloud server device obtains an original image set, generates labels for the original images, and enhances each original image to obtain an enhanced image; a data set is established using the original images and the enhanced images, and a training set, a validation set and a test set are divided based on the samples in the data set; the cloud server device trains the fine-grained alfalfa weed detection model using the training set, verifies it using the validation set, and evaluates the model performance using the test set; the cloud server device transmits the weight parameters of the optimized model to the image processing device; the image processing device updates the model using the weight parameters transmitted by the cloud server device, and performs real-time detection of alfalfa weeds on the images collected in real time. The lightweight hierarchical multi-path balanced feature fusion fine-grained alfalfa weed detection model of the present invention can perform classification detection and regional statistics on alfalfa weeds, improve the accuracy of detection results, effectively improve recognition efficiency and precision, provide a strong theoretical basis for the precise proportion of herbicides, improve the detection ability of alfalfa and various types of weeds, and also provide a reference and possibility for the deployment of the model in resource-constrained devices (such as drones, mechanical vehicles and smart phones). BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Figure 1 Schematic diagram of the network structure of the fine-grained alfalfa weed detection model with lightweight hierarchical multi-path balanced feature fusion.
[0060] Figure 2 This is a structural diagram of the ASPP module.
[0061] Figure 3 Schematic diagram of the MV2 module (inverted residual) structure.
[0062] Figure 4 This is a schematic diagram of the MobileViTBlock module structure.
[0063] Figure 5 Schematic diagram of the depth-wise separable convolution structure.
[0064] Figure 6 Schematic diagram of the structure of alfalfa weed detection system based on lightweight balanced feature fusion. DETAILED DESCRIPTION
[0065] The technical scheme and technical effects of the present invention are further elaborated in detail below in conjunction with the accompanying drawings of the present invention.
[0066] The present invention provides a method and system for detecting alfalfa weeds based on lightweight balanced feature fusion, such as Figure 1As shown in the figure, the backbone network of the system is a feature extraction network, which is used to enhance the fine-grained features of various weeds and alfalfa and extract the multi-scale features of weeds; the neck is a feature fusion network, which performs hierarchical multi-path fusion of the feature maps extracted by the backbone network, with the aim of transmitting deep features back to the shallow layer and enhancing the representation ability of the shallow feature map; finally, the fused feature map is passed to the detection head for classification and regression prediction.
[0067] Image preprocessing and dataset construction: The input of the model is an annotated weed image. In order to be more suitable for use in resource-constrained devices such as edge devices and mobile devices, the model will resize the input image to 320 pixels × 320 pixels. In addition, in order to enhance the robustness and generalization ability of the model, the original image is enhanced by flipping, filtering, and brightness adjustment.
[0068] Construct a lightweight hierarchical multi-path balanced feature fusion weed detection model:
[0069] Backbone network: The backbone network consists of the MobileViT network integrated with Atrous Spatial Pyramid Pooling (ASPP) and additional convolutional layers. Figure 2 As shown in the figure, the ASPP module is introduced to extract features at different scales using dilated convolutions with different dilation rates. The feature maps obtained by a global average pooling layer are concatenated in the channel dimension, and finally the features are further fused through a 1×1 convolution layer. This module is mainly used to enhance the fine-grained features of alfalfa and weeds, while effectively reducing the loss of information during downsampling. The MobileViT network is mainly composed of some MV2 modules and MobileViT modules with different step sizes. Figure 3 As shown in the figure, the MV2 module uses a depth-separable convolution and inverted residual structure to balance computational efficiency and model performance, which can reduce the amount of computation and parameters while still enhancing the performance of the model in resource-constrained devices. Figure 4 As shown in the figure, the MobileViT module effectively combines the local features extracted by the convolutional neural network (CNN) and the global features extracted by the vision transformer (ViT), capturing global dependencies and improving the flexibility of the module while maintaining a low computational cost. In the additional convolutional layer part, all MV2 modules with a stride of 2 are used for feature extraction.
[0070] Neck network: The neck network uses the lightweight Hierarchical Multi-Path Fusion Net (HMPF Net) proposed in this invention to fuse the six sets of feature maps extracted by the backbone network. All layers are upsampled using bilinear interpolation and spliced in the channel dimension. All spliced feature maps are then subjected to a group convolution to achieve cross-channel feature fusion and unify the number of output channels to 256.
[0071] Detecting the head: Detecting the head uses depth-wise separable convolution to replace some standard convolutions, such as Figure 5 As shown in the figure, first, each channel of the input feature map is convolved separately using a deep convolution to extract local features. Then, the features of each channel are linearly combined using a point-by-point convolution (1x1 convolution) to capture cross-channel dependencies. It is worth noting that the last layer of feature maps still uses a 1×1 standard convolution. The final prediction layer consists of a classifier and a regressor. The classifier uses a sigmoid function to generate a probability distribution, and the regressor outputs the offset of the bounding box.
[0072] Lightweight hierarchical multi-path balanced feature fusion weed detection system reference Figure 6 As shown, it has an image acquisition component, a storage component, an image preprocessing component, a model reasoning component, a positioning component and a data transmission component, etc.; the image acquisition component can use a drone to collect images of alfalfa and various weeds during the flight of the drone; the storage component is used to store all images collected by the drone in a flight collection mission and their corresponding coordinate positions; the image preprocessing component is used to enhance the contrast between the target and the background of the image collected by the drone during the flight, and reshape the image size to 320×320, and finally send the processed image to the model reasoning component; the model reasoning component is used to detect the image after preprocessing, count the types and quantities of weeds contained in the current image, and cooperate with the positioning component to mark the approximate range of the weeds, and generate the corresponding number, proportion and distribution of various weeds after the end of a flight mission; the positioning component is used to mark the geographic coordinate position of the image acquisition module each time it takes an image; the data transmission component mainly has two functions. The first part is to upload the images in the storage component to the cloud server after a flight mission and realize the communication between the drone and the operator. The second part is to transmit the detection report of the inference engine to the operator after a flight mission, provide accurate data for pesticide ratio, upload the backup to the cloud server and receive its updated weight.
[0073] The alfalfa weed detection method based on lightweight balanced feature fusion provided by the present invention comprises the following steps:
[0074] Step S1, deploying a lightweight hierarchical multi-path balanced feature fusion fine-grained alfalfa weed detection model on both the cloud server device and the image processing device, wherein the fine-grained alfalfa weed detection model includes an image preprocessing input layer, a backbone network feature extraction layer, a lightweight hierarchical multi-path balanced feature fusion layer, and a detection head;
[0075] Step S2, the cloud server device obtains the original image set, generates labels for the original images, and further enhances each original image to obtain an enhanced image, wherein the label is a description sequence of the locations of all weeds in the image and the weed categories corresponding to the weed locations; a data set is established using the original images and the enhanced images, and a training set, a validation set, and a test set are divided based on the samples in the data set;
[0076] Step S3, the cloud server device trains the fine-grained alfalfa weed detection model using the training set, and optimizes the model parameter weights using the loss function to obtain an optimized model, verifies the optimized model using the validation set, and evaluates the model performance using the test set;
[0077] Step S4, the cloud server device transmits the weight parameters of the optimized model to the image processing device;
[0078] Step S5, the image processing device uses the weight parameters transmitted by the cloud server device to update the model, and uses the updated model to perform real-time detection of alfalfa weeds on the images collected in real time by the image acquisition device.
[0079] The specific implementation of step S2 includes:
[0080] Step S21, classifying the elements in the original image set into a simple environment image set and a complex environment image set; wherein the complex environment image refers to an image in which the number of weeds blocked accounts for more than 30%, and the simple environment image refers to an image in which the number of weeds blocked accounts for no more than 30%;
[0081] Step S22, generating a label for each original image according to the pixel locations of the weeds in each image and the actual weed categories corresponding to the pixel locations;
[0082] Step S23, respectively expanding the elements in the simple environment image set and the complex environment image set, the expansion method is to enhance the original elements and place the obtained enhanced images in the set where the original elements are located, the enhancement processing includes five methods: horizontal flip, vertical flip, Gaussian blur, brightness brightening, and brightness dimming. Brightness brightening refers to brightening the image by 1.5 times, and brightness dimming refers to dimming the image by 0.5 times. Accordingly, each original element corresponds to five enhanced images with different effects, and the number of set elements is expanded to six times the original number;
[0083] Step S24, divide the simple environment image set into a first training set, a first verification set and a first test set in proportion, and divide the complex environment image set into a second training set, a second verification set and a second test set in proportion, merge the first training set and the second training set into a training set, merge the first verification set and the second verification set into a verification set, and merge the first test set and the second test set into a test set.
[0084] There are two types of weeds detected by the fine-grained alfalfa weed detection model, including broadleaf weeds and grass weeds. In the alfalfa plot, 919 alfalfa weed images were collected at different angles and lighting conditions using a mobile phone, with a resolution of 3024 pixels × 4032 pixels. In order to improve the recognition ability of the model on small-scale weeds, these original images were divided into four parts along the width and height midpoints, and 1040 weed images with a resolution of 1512 × 2016 pixels were retained after screening. Next, in order to further expand the dataset, these images were subjected to data enhancement operations such as horizontal flipping, vertical flipping, Gaussian blurring, and brightness adjustment, and the number of images was finally expanded to 6 times the original. Finally, LabelImg software was used to annotate the various types of weeds in the image and save them in Pascal VOC format. From the expanded dataset, the images were classified into 2640 simple environment datasets and 3600 complex environment datasets according to the number and occlusion of weeds. The ratios of the training set, validation set, and test set are as follows: 6:3:1 for simple environment data sets and 7:2:1 for complex environment data sets.
[0085] Then the feature map will pass through a five-layer structure consisting of an MV2 module and a MobileViT module, but the step size of the first layer of the MV2 module is 1, so there are only four downsamplings, and the downsampled feature maps of 4 times, 8 times, 16 times, and 32 times are obtained respectively. However, in order to meet the detection requirements of weeds of other sizes, the present invention splices an additional 4 layers of MV2 modules with a step size of 2 at the end of its five-layer structure, and continues to downsample to obtain 64 times, 128 times, 256 times, and 512 times downsampled feature maps, and finally takes the 16 times, 32 times, 64 times, 128 times, 256 times, and 512 times feature maps and inputs them into the neck.
[0086] The specific implementation of step S3 using the training set to train the fine-grained alfalfa weed detection model includes:
[0087] Step S31, the image preprocessing input layer inputs the sample image in the training set, reshapes the sample image into an image X of 320 pixels × 320 pixels and inputs it into the backbone network feature extraction layer;
[0088] Step S32, in the backbone network feature extraction layer, the image X passes through a fine-grained feature enhancement module, a feature extraction layer composed of an inverted residual with a step size of 1, a feature extraction layer composed of an inverted residual with a step size of 1 and two inverted residuals with a step size of 2, 3 feature extraction layers composed of an inverted residual with a step size of 2 and a MobileViT module, and 4 feature extraction layers composed of an inverted residual with a step size of 2, and obtains 4 times, 8 times, 16 times, 32 times, 64 times, 128 times, 256 times and 512 times down-sampled feature maps respectively, and takes a 16-fold feature map C4, a 32-fold feature map C5, a 64-fold feature map C6, a 128-fold feature map C7, a 256-fold feature map C8 and a 512-fold feature map C9, and outputs them to a lightweight hierarchical multipath equalization feature fusion layer;
[0089] The fine-grained feature enhancement module consists of a 3×3 convolution and an ASPP module with a dilation rate of [1, 6, 12, 18] (1×1 convolution, 3×3 convolution with a dilation rate of 6, 3×3 convolution with a dilation rate of 12, 3×3 convolution with a dilation rate of 18, image global average pooling processing, and 1×1 convolution); the ASPP module calculates the convolution operation expression with a dilation rate of r as follows:
[0090] y[i]=∑ k x[i+r·k]ω[k] (1)
[0091] In the formula, i is the position index of the current output element; r is the dilation rate, which indicates the distance between the convolution kernel elements; k is the index of the convolution kernel element, so i+r·k represents the sampling position in the input feature map; x[i+r·k] is the element extracted from the input feature map X, and ω[k] is the weight of the convolution kernel at position k; the sum of all convolution kernel elements is the output value at position i;
[0092] Feature map Y after ASPP module final The expression is:
[0093]
[0094] In the formula, represents the output feature map of 4 dilated convolutions; GAP(X) represents the feature map after the global average pooling operation; Concat represents concatenating all the above output feature maps in the channel dimension; W represents the weight of 1×1 convolution; σ represents the ReLU activation function;
[0095] The inverted residual structure with a stride of 1 is divided into four stages. The first stage is a 1×1 convolution layer to increase the dimension of the input, the second stage is a 3×3 deep convolution layer and the activation function Relu6, the third stage is a 1×1 convolution layer, and the fourth stage concatenates the input feature map and the feature map output by the third stage on the feature channel; the inverted residual structure with a stride of 2 is divided into three stages. The first stage is a 1×1 convolution layer to increase the dimension of the input, the second stage is a 3×3 deep convolution layer and the activation function Relu6, and the third stage is a 1×1 convolution layer;
[0096] The MobileViT module includes a local feature extraction unit, a global feature extraction unit, a 1×1 convolution layer, a splicing unit, and a 3×3 convolution layer in sequence. The local feature extraction unit is composed of a 3×3 convolution layer and a 1×1 convolution layer, and the global feature extraction unit is composed of an expansion unit, L Transformers, and a folding unit in sequence. After the original image is input into the MobileViT module, it is processed by local feature extraction, global feature extraction, and 1×1 convolution in sequence. The obtained feature map is spliced with the original image in the channel dimension and then output after a 3×3 convolution layer.
[0097] Step S33, the lightweight hierarchical multipath equalization feature fusion layer uses five levels and six group convolutions to fuse six groups of feature maps respectively, and outputs the six groups of fused feature maps P4, P5, P6, P7, P8, and P9 output by the group convolution to the detection head module:
[0098] Step S331, using bilinear interpolation method, the first level performs three upsampling on the feature map C9 to align with the feature map sizes of C8, C6 and C4 respectively, and directly splices on the channel dimension, the second level performs two upsampling on the C8 feature map to align with the feature map sizes of C7 and C5 respectively, and directly splices on the channel dimension, the third level performs two upsampling on the C7 feature map to align with the feature map sizes of C6 and C4 respectively, and directly splices on the channel dimension, the fourth level performs one upsampling on the C6 feature map to align with the feature map size of C5, and directly splices on the channel dimension, the fifth level performs one upsampling on the C5 feature map to align with the feature map size of C4, and directly splices on the channel dimension; all subsequent splicing operations are performed on the new feature map formed after the previous splicing operation is completed;
[0099] Step S332, performing group convolution with a group number of 8 on each splicing result, unifying the number of all channels to 256, completing the cross-channel fusion of the feature map, and outputting the cross-channel fusion feature maps P4, P5, P6, P7, P8, and P9;
[0100] Step S34, the detection head module performs depth-separable convolution on the fused feature maps P4, P5, P6, P7, and P8 respectively, and the regressor outputs the offset of the prediction box (coordinates) based on the feature map obtained by the depth-separable convolution, and the classifier generates the weed type probability distribution of the corresponding prediction box based on the feature map obtained by the depth-separable convolution; the fused feature map P9 is subjected to 1×1 standard convolution, and the regressor outputs the offset of the prediction box (coordinates) based on the feature map obtained by the 1×1 standard convolution, and the classifier generates the weed type probability distribution of the corresponding prediction box based on the feature map obtained by the 1×1 standard convolution; (wherein, the classifier uses the sigmoid function to generate the probability distribution);
[0101] The depthwise separable convolution operation is as follows: each channel of the feature fusion map is convolved separately to obtain the features of each channel, and then the local features of all channels are linearly combined by point-by-point convolution to obtain the feature map obtained by the depthwise separable convolution.
[0102] Step S35, setting a priori boxes with different aspect ratios for each unit of each feature fusion map, and mapping it back to the sample image through the scaling ratio, and obtaining the prediction result of the sample image, the prediction result includes prediction box regression and prediction box selection target category; (wherein, the six sets of feature map sizes used are 20×20, 10×10, 5×5, 3×3, 2×2, 1×1, with a total of 539 units, the number of prior boxes set for each unit is 6, and all units have a total of 3234 prior boxes)
[0103] Step S36, using the loss function to adjust the model weight: input the processed image and label into the initialized detection model to obtain the predicted box and category corresponding to the real box, and gradually correct and optimize the model parameter weights by calculating the cross entropy loss function and the SmoothL1 loss function, where the expression of the cross entropy loss function is:
[0104]
[0105] In the formula, y i is the true label sequence of the i-th sample, is the probability sequence of the model's prediction results for the i-th sample, and N is the total number of samples in the training set; the cross entropy loss function measures the degree of mismatch between the probability distribution of the model output and the true label distribution. The smaller the loss value, the more accurate the model's prediction.
[0106] The expression of Smooth L1 loss function is:
[0107]
[0108] Where x is the difference between the predicted value and the true value, which is the coordinate deviation and the size deviation; for an input x, when |x|<1, the loss function is L2 loss, that is, 0.5·x 2 ; When |x|≥1, the loss function is L1 loss, that is, |x|-0.5. This calculation method ensures that the position error of each prediction box is smoothed to reduce the impact of outliers on the training process, while focusing on the position error of positive samples. (Coordinate deviation, the horizontal and vertical coordinate values of the upper left corner of the prediction box minus the horizontal and vertical coordinate values of the real box, including 2 types of horizontal and vertical coordinate values; size deviation, the width and height of the prediction box minus the width and height of the real box, including 2 types of width and height);
[0109] Step S37, after the model is trained using all samples in the training set, the trained model is verified using the verification set;
[0110] Step S38, using the test set to evaluate the model performance: the samples of the test set are input into the trained model in batches for prediction, and the test prediction results are obtained. The test prediction results are screened for confidence in turn, and a confidence threshold (0.5) is set. The test prediction results above the confidence threshold are retained, and the retained test prediction results are screened by the non-maximum suppression algorithm (NMS) (setting IoU = 0.5, screening the prediction boxes exceeding 0.5, and retaining the prediction boxes with the highest confidence). Based on the test prediction results and labels, three indicators are used to evaluate the model performance:
[0111]
[0112]
[0113]
[0114] Where Precision is the accuracy; TP is the true positive class, which indicates the number of broadleaf weeds and grass weeds that are correctly detected by the model; FP is the false positive class, which indicates the number of places that are not actually weeds that the model mistakenly judges as broadleaf weeds or grass weeds; Recall is the recall rate; FN is the false negative class, which indicates the number of actual broadleaf weeds or grass weeds that are not detected by the model; mAP50 is the average of all the final classes, the average precision AP of the broadleaf weed class and the grass weed class i is the area under the Precision-Recall curve (Precision data is used as the Y-axis, Recall data is used as the X-axis, and the Precision-Recall curve is a line graph); n is the total number of weed species.
[0115]
[0116] In formula (6), t represents different IoU thresholds, Recall(t) represents the recall rate under different thresholds, and T is the number of thresholds.
[0117]
[0118] In the above four formulas, T infer represents the total inference time, N represents the number of inferences, and the division is the inference time of a single image; the frame rate (FPS) is the reciprocal of the sum of image preprocessing, single inference time and non-maximum suppression time; the total number of parameters of the model is the cumulative sum of the parameters of each layer of the model, and the total number of floating-point operations of the model is the same.
[0119] Among the above evaluation indicators, the larger the mAP50, AR and FPS, the better, while the smaller the other indicators, the better.
[0120] In the splicing process of step S33, all subsequent splicing operations are performed on the new feature map formed after the previous splicing operation is completed; the advantages of this design are:
[0121] (1) Using the concatenation operation (Concat) for feature fusion effectively avoids the amount of calculation introduced by addition or multiplication;
[0122] (2) Through hierarchical design, it can effectively ensure that the feature maps closer to the lower layer can integrate more feature maps from the higher layer, effectively avoiding the additional parameters caused by the introduction of weights;
[0123] (3) By transferring feature maps through multiple paths, high-level features can be directly transferred to low-level features, effectively avoiding the problem of time-consuming and information loss in the layer-by-layer feature transfer;
[0124] The use of grouped convolution to fuse the concatenated feature maps across channels effectively solves the problem of the huge number of parameters caused by the one-to-one correspondence between convolution kernels and channels. At the same time, by setting different numbers of groups, the adaptability and flexibility of the model in different environments can be improved.
[0125] Further, refer to Figure 6 As shown, in the system, the image acquisition device may be a drone with an image acquisition function. After step S5, the following steps are also included:
[0126] Step S6, the drone performs a preset route flight mission, collects real-time images and sends them to an image processing device and a cloud server device; Step S7, the image processing device receives the real-time images, uses the optimized model to perform real-time alfalfa weed recognition on the real-time images, and obtains recognition results;
[0127] Step S8: The cloud server device receives the real-time image, uses the real-time image to establish an updated data set, regularly optimizes the model based on the updated data set, and provides the updated model to the image processing device. Specifically:
[0128] Step S81, obtaining an updated data set according to step S2, training a model based on the updated training set, updating the parameter weights of the model, and calculating the mAP50 of the updated model;
[0129] Step S82: If the mAP50 of the updated model is greater than the mAP50 of the model currently used by the image processing device, the weight parameters of the updated model are passed to the image processing device for weight update; if the mAP50 of the updated model is not better than the mAP50 of the model currently used by the image processing device, the update data is not passed to the image processing device.
[0130] In the alfalfa weed detection system, when the drone performs a flight collection mission, the image acquisition component uses vertical ground shooting to shoot images according to the set rate (flight rate and shooting frequency), locates the coordinate position of each image, and puts the collected images and coordinate information into the storage component; the inference engine reads the images and coordinate information in the storage component, puts the pre-processed images into the model inference component for inference, and counts the types and quantities of weeds in the current image. After the flight mission is completed, the model inference component generates a detection report, including: the number, proportion and distribution of various types of weeds, and then transmits the detection results to the cloud server and the operation end through wireless transmission for backup and guidance of the next step of drug dispensing. After the flight mission is completed, the drone uploads the collected images to the cloud server through wireless transmission. At the same time, the cloud server enters the new feature learning stage. After obtaining the new weight, it returns to the inference engine through wireless transmission to complete the weight update.
[0131] Training settings: In the training process of deep learning models, reasonable hyperparameter settings are crucial. These hyperparameters include learning rate, batch size, optimizer selection, regularization parameters, etc., which have a significant impact on the convergence speed and final performance of the model. By optimizing these hyperparameters, the training effect and accuracy of the model can be effectively improved.
[0132] The hyperparameter settings in this embodiment are shown in Table 1:
[0133]
[0134]
[0135] Table 1
[0136] The present invention uses a backbone network fused with an ASPP module to enhance fine-grained features and extract multi-scale feature maps, so that the model can simultaneously focus on weeds of different scales, reducing information loss during downsampling, thereby improving the expression of multi-scale features. This provides a strong theoretical basis for the precise proportion of herbicides.
[0137] The present invention proposes a lightweight hierarchical multi-path balanced feature fusion network, which integrates features of each level as much as possible while ensuring the lightweight of the model, solves the problem of feature isolation in the traditional SSDLite model, improves the detection capability of alfalfa and various types of weeds, and also enhances the feasibility of the model deployment in drones.
[0138] In summary, the similarity between alfalfa and some broadleaf weeds, the diversity of weed sizes, and the incompleteness of features caused by occlusion and connection are all difficulties faced in fine-grained weed detection. Through the detection method of the present invention, it is possible to improve the detection accuracy while maintaining a low parameter amount and floating-point operation number, effectively taking into account both detection accuracy and real-time performance, thereby efficiently and scientifically managing alfalfa weeding work and ensuring the yield and quality of alfalfa. In addition, the present invention also provides a complete alfalfa weed detection system, which fully integrates the powerful advantages of drones and deep learning models, greatly reduces labor costs, and also ensures the accuracy and efficiency of alfalfa weed detection, and provides theoretical support and practical feasibility for the precise configuration of pesticides.
[0139] What is disclosed above is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. A person skilled in the art can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.
Claims
1. A method for detecting alfalfa weeds based on lightweight balanced feature fusion. It is characterized in that include: Step S1, deploying a lightweight hierarchical multi-path balanced feature fusion fine-grained alfalfa weed detection model on both the cloud server device and the image processing device, wherein the fine-grained alfalfa weed detection model includes an image preprocessing input layer, a backbone network feature extraction layer, a lightweight hierarchical multi-path balanced feature fusion layer, and a detection head; Step S2, the cloud server device obtains an original image set, generates labels for the original images, and further enhances each of the original images to obtain an enhanced image, wherein the labels are description sequences of the locations of all weeds in the image and the weed categories corresponding to the weed locations; a data set is established using the original images and the enhanced images, and a training set, a validation set, and a test set are divided based on samples in the data set; Step S3, the cloud server device trains the fine-grained alfalfa weed detection model using the training set, and optimizes the model parameter weights using a loss function to obtain an optimized model, verifies the optimized model using the verification set, and evaluates the model performance using the test set; Step S4, the cloud server device transmits the weight parameters of the optimized model to the image processing device; Step S5, the image processing device uses the weight parameters transmitted by the cloud server device to update the model, and uses the updated model to perform real-time detection of alfalfa weeds on the image collected in real time by the image acquisition device.
2. The alfalfa weed detection method based on lightweight balanced feature fusion as claimed in claim 1, It is characterized in that Step S2 includes: Step S21, classifying the elements in the original image set into a simple environment image set and a complex environment image set; wherein a complex environment image refers to an image in which the number of weeds blocked accounts for more than 30%, and a simple environment image refers to an image in which the number of weeds blocked accounts for no more than 30%; Step S22, generating a label for each of the original images according to the pixel locations of the weeds in each image and the actual weed categories corresponding to the pixel locations; Step S23, respectively expanding the elements in the simple environment image set and the complex environment image set, the expansion method is to enhance the original elements and place the obtained enhanced images in the set where the original elements are located, the enhancement processing includes five methods: horizontal flipping, vertical flipping, Gaussian blurring, brightness brightening, and brightness dimming. Accordingly, each of the original elements corresponds to five enhanced images with different effects, and the number of set elements is expanded to six times the original number; Step S24, divide the simple environment image set into a first training set, a first verification set and a first test set in proportion, and divide the complex environment image set into a second training set, a second verification set and a second test set in proportion, merge the first training set and the second training set into the training set, merge the first verification set and the second verification set into the verification set, and merge the first test set and the second test set into the test set.
3. The alfalfa weed detection method based on lightweight balanced feature fusion as claimed in claim 2, Features: There are two types of weeds detected by the fine-grained alfalfa weed detection model, including broadleaf weeds and grass weeds; Brightness brightening means brightening the image by 1.5 times, and brightness dimming means dimming the image by 0.5 times; When the two data sets are divided into training sets, validation sets, and test sets, the division ratio of the simple environment data set is 6:3:1, and the division ratio of the complex environment data set is 7:2:
1.
4. The alfalfa weed detection method based on lightweight balanced feature fusion as claimed in claim 3, It is characterized in that The step S3 uses the training set to train the fine-grained alfalfa weed detection model, including: Step S31, the image preprocessing input layer inputs the sample image in the training set, reshapes the sample image into an image X of 320 pixels×320 pixels and inputs it into the backbone network feature extraction layer; Step S32, in the backbone network feature extraction layer, the image X is sequentially passed through a fine-grained feature enhancement module, a feature extraction layer consisting of an inverted residual with a step size of 1, a feature extraction layer consisting of an inverted residual with a step size of 1 and two inverted residuals with a step size of 2, three feature extraction layers consisting of an inverted residual with a step size of 2 and a MobileViT module, and four feature extraction layers consisting of an inverted residual with a step size of 2, to obtain downsampled feature maps of 4 times, 8 times, 16 times, 32 times, 64 times, 128 times, 256 times and 512 times, respectively, and the 16-fold feature map C is taken. 4 , 32 times the feature map C 5 , 64 times the feature map C 6 , 128 times the feature map C 7 , 256 times the feature map C 8 and 512 times the feature map C 9 Output to the lightweight hierarchical multipath equalization feature fusion layer; Step S33, the lightweight hierarchical multipath equalization feature fusion layer uses five levels and six group convolutions to perform feature fusion on the six groups of feature maps respectively, and the six groups of fused feature maps P output by the group convolution are combined into a single group. 4 , P 5 , P 6 , P 7 , P 8 , P 9 Output to the detection head module: Step S331: First level feature map C 9 Perform upsampling three times and C 8 , C 6 and C 4 The feature map sizes of C are aligned and directly concatenated in the channel dimension. 8 The feature map is upsampled twice and C 7 and C 5 The feature map sizes of C are aligned and directly concatenated in the channel dimension. 7 The feature map is upsampled twice and C 6 and C 4 The feature map sizes are aligned and directly concatenated in the channel dimension. The fourth level is C 6 The feature map is upsampled once and C 5 The feature map sizes are aligned and directly concatenated in the channel dimension. The fifth level is C 5 The feature map is upsampled once and C 4 The feature map sizes are aligned and directly concatenated in the channel dimension; Step S332, perform group convolution on each splicing result with a group number of 8, unify the number of all channels to 256, and output the cross-channel fusion feature map P 4 , P 5 , P 6 , P 7 , P 8 , P 9 ; Step S34: the detection head module detects the fusion feature map P 4 , P 5 , P 6 , P 7 , P 8 Depthwise separable convolution is performed respectively, and the regressor outputs the offset of the prediction box based on the feature map obtained by the depthwise separable convolution, and the classifier generates the probability distribution of the weed type corresponding to the prediction box based on the feature map obtained by the depthwise separable convolution; the fusion feature map P 9 Performing a 1×1 standard convolution, the regressor outputs an offset of the prediction box based on the feature map obtained by the 1×1 standard convolution, and the classifier generates a probability distribution of the weed type corresponding to the prediction box based on the feature map obtained by the 1×1 standard convolution; The depthwise separable convolution operation is as follows: performing convolution operations on each channel of the feature fusion map respectively to obtain features of each channel, and then using point-by-point convolution to linearly combine local features of all channels to obtain the feature map obtained by the depthwise separable convolution; Step S35, setting a priori boxes with different aspect ratios for each unit of each of the feature fusion images, and mapping them back to the sample image through the scaling ratio, to obtain a prediction result of the sample image, wherein the prediction result includes prediction box regression and prediction box selection target category; Step S36, using the loss function to adjust the model weight: by calculating the cross entropy loss function and the SmoothL1 loss function, the model parameter weights are gradually corrected and optimized, wherein the expression of the cross entropy loss function is: In the formula, y i is the true label sequence of the oth sample, is the probability sequence of the model's prediction results for the i-th sample, and N is the total number of samples in the training set; The expression of SmoothL1 loss function is: Where x is the difference between the predicted value and the true value, and is the coordinate deviation and size deviation; Step S37, after the model is trained using all samples in the training set, the trained model is verified using the verification set; Step S38, using the test set to evaluate the model performance: the samples of the test set are input into the trained model in batches to perform predictions and obtain test prediction results, the test prediction results are sequentially screened by confidence and non-maximum suppression algorithms, and the model performance is evaluated using three indicators for the screened test prediction results: Where Precision is the accuracy; TP is the true positive class, which indicates the number of broadleaf weeds and grass weeds that are correctly detected by the model; FP is the false positive class, which indicates the number of places that are not actually weeds that the model mistakenly judges as broadleaf weeds or grass weeds; Recall is the recall rate; FN is the false negative class, which indicates the number of actual broadleaf weeds or grass weeds that are not detected by the model; mAP50 is the average of all the final classes, the average precision AP of the broadleaf weed class and the grass weed class i is the area under the Precision-Recall curve; n is the total number of weed species.
5. The alfalfa weed detection method based on lightweight balanced feature fusion as claimed in claim 4, Features: Feature map C 4 , C 5 , C 6 , C 7 , C 8 , C 9 The feature map sizes are 20×20, 10×10, 5×5, 3×3, 2×2, and 1×1 respectively; The fine-grained feature enhancement module is composed of a 3×3 convolution and an ASPP module with a dilation rate of [1, 6, 12, 18] in sequence; the ASPP module calculates the convolution operation expression with a dilation rate of r as follows: In the formula, i is the position index of the current output element; r is the dilation rate, which indicates the distance between the convolution kernel elements; k is the index of the convolution kernel element, so i+r·k represents the sampling position in the input feature map; x[i+r·k] is the element extracted from the input feature map X, and ω[k] is the weight of the convolution kernel at position k; the sum of all convolution kernel elements is the output value at position i; Feature map Y after ASPP module final The expression is: In the formula, represents the output feature map of 4 dilated convolutions; GAP(X) represents the feature map after the global average pooling operation; Concat represents concatenating all the above output feature maps in the channel dimension; W represents the weight of 1×1 convolution; σ represents the ReLU activation function; The inverted residual structure with a step size of 1 is divided into four stages, the first stage is a 1×1 convolution layer to increase the dimension of the input, the second stage is a 3×3 deep convolution layer and the activation function Relu6, the third stage is a 1×1 convolution layer, and the fourth stage concatenates the input feature map and the feature map output by the third stage on the feature channel; the inverted residual structure with a step size of 2 is divided into three stages, the first stage is a 1×1 convolution layer to increase the dimension of the input, the second stage is a 3×3 deep convolution layer and the activation function Relu6, and the third stage is a 1×1 convolution layer; The MobileViT module includes a local feature extraction unit, a global feature extraction unit, a 1×1 convolution layer, a splicing unit, and a 3×3 convolution layer in sequence. The local feature extraction unit is composed of a 3×3 convolution layer and a 1×1 convolution layer, and the global feature extraction unit is composed of an expansion unit, L Transformers, and a folding unit in sequence. After the original image is input into the MobileViT module, local feature extraction processing, global feature extraction processing, and 1×1 convolution processing are performed in sequence. The obtained feature map is spliced with the original image in the channel dimension and then output after passing through a 3×3 convolution layer.
6. The alfalfa weed detection method based on lightweight balanced feature fusion as claimed in claim 5, It is characterized in that The model hyperparameters are set as follows: input image size 320 pixels × 320 pixels, width factor 1.0, number of categories 2, initial learning rate 0.001, momentum 0.9, weight decay 4e -5 , minimum learning rate 0.00001, training rounds 1000, warm-up rounds 1, optimizer, SGD, learning rate change strategy is cosine annealing, batch size is 16; in the detection head module, a priori boxes with different aspect ratios are set for each unit of the feature map, and mapped back to the original image through the scaling ratio. The six sets of feature map sizes used are 20×20, 10×10, 5×5, 3×3, 2×2, and 1×1, with a total of 539 units. The number of prior boxes set for each unit is 6, and there are 3234 prior boxes for all units.
7. The alfalfa weed detection method based on lightweight balanced feature fusion as claimed in claim 6, It is characterized in that The step S332 uses bilinear interpolation to perform upsampling.
8. The alfalfa weed detection method based on lightweight balanced feature fusion as claimed in claim 7, It is characterized in that The image acquisition device is a drone, and after step S5, the method further includes: In step S6, the drone performs a flight mission along a preset route, collects real-time images and sends them to the image processing device and the cloud server device; in step S7, the image processing device receives the real-time images, uses the optimized model to perform real-time alfalfa weed recognition on the real-time images, and obtains recognition results; in step S8, the cloud server device receives the real-time images, uses the real-time images to establish an updated data set, regularly optimizes the model according to the updated data set, and provides the updated model to the image processing device.
9. The alfalfa weed detection method based on lightweight balanced feature fusion as claimed in claim 8, It is characterized in that The step S8 of regularly optimizing the model according to the updated data set and providing the updated model to the image processing device includes: Step S81, obtaining the updated data set according to step S2, training a model based on the updated training set, updating the parameter weights of the model, and calculating the mAP50 of the updated model; Step S82: If the mAP50 of the updated model is greater than the mAP50 of the model currently used by the image processing device, the weight parameters of the updated model are passed to the image processing device for weight update; if the mAP50 of the updated model is not better than the mAP50 of the model currently used by the image processing device, the update data is not passed to the image processing device.
10. An alfalfa weed detection system based on lightweight balanced feature fusion, It is characterized in that It comprises a cloud server device, an image processing device, and an image acquisition device, and is used to execute the method described in any one of claims 1 to 9.
