Weed detection method and device used in multiple scenes

Abstract

The invention discloses a weed detection method and device used in multiple scenes, and the method comprises the steps: collecting and marking weed data in different scenes, constructing a data set, and then carrying out the data set evaluation; a re-parameterized large convolution kernel feature extraction-fusion module, a spatial pyramid pooling network, a DySample up-sampler and a general efficient layer aggregation network are combined, and a hoisting parameterized feature extraction detection network is established; and detecting weeds in multiple scenes based on the re-parameterized feature extraction detection network. The device comprises a processor and a memory. According to the method, the weed detection performance can be improved, the size of the model can be reduced, and the field deployment detection effect of weed detection edge equipment in intelligent agriculture is remarkable.

Description

Technical Field

[0001] The present invention relates to the field of weed detection, and in particular to a weed detection method and device for multiple scenarios. Background Art

[0002] With the development of agricultural modernization, precision agriculture technology has played an important role in increasing crop yields and reducing resource waste. In this process, intelligent weed control technology has gradually become one of the important research areas. Through early identification and control of weeds in the field, farmers can significantly reduce the use of pesticides, thereby reducing environmental pollution and improving economic benefits. However, how to achieve efficient and accurate weed detection in complex field environments is still a difficult problem that needs to be solved.

[0003] Depending on the time of emergence, density and type of weeds, weeds can cause total grain yield losses if not controlled. Therefore, selective spraying equipment to reduce the use of herbicides has been widely studied. Such equipment must be based on accurate perception of target information to achieve unmanned precision weeding, and one of the key issues to be solved is how to achieve accurate detection of multiple weeds in the field.

[0004] Considering the actual scenarios of farmland weed detection, the existing weed detection algorithms lack high accuracy in detecting different types of weeds in multiple scenarios. At the same time, their large and complex backbone networks limit their application in weed monitoring terminals with weak computing power. Summary of the invention

[0005] The present invention provides a method and device for weed detection in multiple scenarios. The present invention adopts a re-parameterized large convolution kernel feature extraction network model (RLCFE-Net) for weed detection in multiple scenarios, which can not only improve the performance of weed detection, but also reduce the model size. It has a significant detection effect on the field deployment of weed detection edge devices in smart agriculture, as described below:

[0006] In a first aspect, a weed detection method for multiple scenarios is provided, the method comprising:

[0007] Collect and label weed data in different scenarios, build datasets, and then evaluate the datasets;

[0008] The re-parameterized large convolution kernel feature extraction-fusion module, spatial pyramid pooling network, DySample upsampler and general efficient layer aggregation network are combined to establish a re-parameterized feature extraction detection network;

[0009] Weed detection in multiple scenarios based on re-parameterized feature extraction detection network.

[0010] Wherein, the re-parameterized feature extraction-network is:

[0011] One stage of CSPELAN consists of a partial tensor block and a partial transition layer. In the partial tensor block, the feature map of the base layer in the stage is divided into two parts by the channel x0 = [x0′, x0″]; between x0′ and x0″, the former is directly connected to the end of the stage, and the latter will pass through the tensor block. All the steps involved in the partial transition layer are as follows:

[0012] First, the output of the tensor layer, [x0", x1, x2, x3], will go through the transition layer, and the output of the transition layer x T It will be concatenated with x0′ and go through another transition layer to generate the output x U ; The equations for feedforward transfer and weight update are respectively;

[0013] X K =W K *[x0″,x1,x2],

[0014] X T =W T *[x0″,x1,x2,x3],

[0015] X U =W U *[x0′,x T ],

[0016] Among them, * represents the convolution operator, [x0,x1,x2,x3] represents the connection x0,x1,x2,x3, W K ,W T ,W U and X K ,X T ,X U are the weights and outputs of the K, T and U tensor layers respectively;

[0017] W K ′=f K (W K ,{g0″,g1,g2}),

[0018] W T ′=f T (W T ,{g0″,g1,g2,g3}),

[0019] W U ′=f U (W U ,{g0′,g T}),

[0020] Among them, f K ,f T ,f Uis the weight update function of the K, T and U tensor layers, g1,g2,g3,g T Represents the gradient propagated to the 1st, 2nd, 3rd, and Tth tensor layers.

[0021] Among them, the re-parameterized large convolution kernel feature extraction-fusion module is:

[0022] Original convolution kernel: W∈R k×k ,

[0023] After inserting zero terms: W'∈R ((k-1)r+1)×((k-1)r+1) ,

[0024] This is achieved by transposing a convolution with a stride of r: W = conv_transpose2d(W, I, stride = r).

[0025] The spatial pyramid pooling network comprises an average pooling layer and a maximum pooling layer which are operated twice in series and then in parallel.

[0026] Wherein, the DySample upsampler is:

[0027] Given a feature map of size C×H1×W1, and a sample set S of size 2×H2×W2, where the first dimension of 2 represents the x and y coordinates, the grid_sample function resamples the assumed bilinear interpolation X using the locations in S to generate X' of size C×H2×W2, defined as:

[0028] X'=grid_sample(X,S),

[0029] Given an upsampling factor s and a feature map X of size C×H×W, use a linear layer with C and 2s input and output channels respectively. 2 , the generated size is 2s 2 ×H×W offset O; reshaped into 2×sH×sW by the pixel shuffle module, the sampling set S is the sum of the offset O and the original sampling grid G, that is:

[0030] O=linear(X),

[0031] S=G+O,

[0032] Through grid_sample and sampling set S, an upsampled feature map X' of size 2×sH×sW is generated.

[0033] Wherein, the method further comprises:

[0034] Testing using edge device deployment and graphical user interface to simulate field deployment.

[0035] In a second aspect, a weed detection device for multiple scenarios includes: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute any one of the methods described in the first aspect.

[0036] In a third aspect, a computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes any one of the methods described in the first aspect.

[0037] The beneficial effects of the technical solution provided by the present invention are:

[0038] 1. The present invention proposes a feature extraction-fusion module (DRBNCSPELAN4) to avoid information loss when processing input data after feature extraction and spatial transformation; the present invention designs a new spatial pyramid pooling (ASPPFELAN) network, which introduces an average pooling layer to comprehensively extract the features of weed texture and background compared with the previous single maximum pooling layer of the same type; the present invention introduces a DySample upsampler to construct an upsampler from the perspective of point sampling, which is more resource-saving and efficient;

[0039] 2. The RLCFE-Net network model for weed detection in multiple scenarios adopted by the present invention is evaluated on three data sets. The overall performance surpasses the existing one-stage target detector and the functional model of the same type. At the same time, the model can effectively improve the average detection accuracy and reduce the model complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a weed detection method for multiple scenarios;

[0041] Figure 2 It is a schematic diagram of the operation process of RLCFE-Net;

[0042] Figure 3 A schematic diagram of a re-parameterized large convolution kernel feature extraction network for weed detection in multiple scenarios;

[0043] Figure 4 This is a schematic diagram showing six representative weed data in different scenarios in the self-made dataset;

[0044] Figure 5 This is the structural diagram of a new feature extraction and fusion module DRBNCSPELAN4;

[0045] Figure 6 This is the structural diagram of a new spatial pyramid pooling module ASPPELAN;

[0046] Figure 7 The structural diagram of the introduced dynamic point sampling DySample;

[0047] Figure 8 Renderings of the deployment for both models.

[0048] Among them, (a) is a schematic diagram of the edge device (Raspberry) deployment; (b) is a schematic diagram of the development of the graphical user interface (GUI).

[0049] Table 1 is an example of the comparison of training evaluation indicators on public datasets and self-made datasets;

[0050] Table 2 is an example of the comparison of training evaluation indicators of different functional modules under the self-made dataset. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0052] In consideration of the actual scenarios of farmland weed detection, the existing weed detection algorithms lack high accuracy in detecting different types of weeds in multiple scenarios. At the same time, their large and complex backbone networks limit their application in weed monitoring terminals with weak computing power. Therefore, an embodiment of the present invention proposes a re-parameterized large convolution kernel feature extraction network model (RLCFE-Net) for weed detection in multiple scenarios, which can not only improve the performance of weed detection, but also reduce the model size, and has a significant detection effect on the field deployment of weed detection edge devices in smart agriculture.

[0053] Example 1

[0054] like Figure 1 As shown, a weed detection method for multiple scenarios provided by an embodiment of the present invention is used. In this method, a re-parameterized large convolution kernel feature extraction network model is used. Figure 2 As the overall structure diagram of this model, the model includes the following steps:

[0055] Step 101: Collect and label weed data in different scenarios, build a data set, and then evaluate the data set;

[0056] Step 102: Propose a new re-parameterized large convolution kernel feature extraction-fusion module (DRBNCSPELAN4), which reduces the loss of a large amount of information when processing input data after feature extraction and spatial transformation, so that significant feature information can be retained, and further increases the amount of data for feature extraction;

[0057] Step 103: A new spatial pyramid pooling (ASPPFELAN) network is designed. Compared with the previous single maximum pooling layer of the same type, the average pooling layer is introduced to comprehensively extract the features of weed texture and background;

[0058] Step 104: Introduce the DySample upsampler and construct the upsampler from the perspective of point sampling. This method is more resource-saving and efficient.

[0059] Step 105: Combine DRBNCSPELAN4, ASPPFELAN, DySample and GELAN to establish a re-parameterized feature extraction detection network (RLCFE-Net) with higher detection performance and less complexity, and evaluate the model on the public datasets CottonWeedDet12 (5648), 4WEED DATASET (618) and the self-made dataset Six weeds (2118), so that it can better meet the needs of efficient and accurate detection in actual scenarios.

[0060] The embodiment of the present invention further includes: using edge device (Raspberry) deployment and graphical user interface (GUI) to simulate field deployment testing to further verify the accuracy of weed detection.

[0061] In summary, the embodiment of the present invention designs a heavy parameterized weed detection network that occupies a small amount of edge device resources of an intelligent weeding robot in smart agriculture through the above steps 101 to 105, and improves the detection accuracy while reducing the network complexity.

[0062] Example 2

[0063] The scheme in Example 1 is further introduced below, as described below:

[0064] Step 201: Collect and label weed data in different scenarios, build a data set, and then evaluate the data set;

[0065] Step 202: Propose a new re-parameterized large convolution kernel feature extraction-fusion module (DRBNCSPELAN4), which reduces the loss of a large amount of information when processing input data after feature extraction and spatial transformation, so that significant feature information can be retained, and further increases the amount of data for feature extraction;

[0066] The embodiment of the present invention designs DRBNCSPELAN4 which takes into account light weight, inference speed and accuracy. DRBNCSPELAN4 can be split into DRBN-CSP-ELAN4, where DRBN is used for heavy parameterized calculation, CSP is used for gradient enrichment, and GELAN is used for efficient feature aggregation.

[0067] 1. Cross Stage Partial and Efficient Layer Aggregation Networks (CSPELAN):

[0068] Figure 3 Figure (f) in Figure 5 shows the architecture of one stage of the proposed CSPELAN (Cross-Stage Local and Efficient Layer Aggregation Network). One stage of CSPELAN consists of a partial tensor block and a partial transition layer. In the partial tensor block, the feature map of the base layer in the stage is divided into two parts by the channel x0 = [x0′, x0″]. Between x0′ and x0″, the former is directly connected to the end of the stage, and the latter will pass through the tensor block. All the steps involved in the partial transition layer are as follows: First, the output of the tensor layer, [x0", x1, x2, x3] will go through the transition layer. Second, the output x0 of the transition layer T It will be concatenated with x0′ and go through another transition layer, and then generate the output x U The equations for the feedforward pass and weight update of CSPELAN are shown in Equations (1) and (2), respectively.

[0069]

[0070] Among them, * represents the convolution operator, [x0,x1,x2,x3] represents the connection x0,x1,x2,x3, W K ,W T ,W U and X K ,X T ,X U are the weights and outputs of the K, T and U tensor layers respectively.

[0071] If back propagation is used to update the weights, the weight update equation can be written as:

[0072]

[0073] Among them, f K ,f T ,f U is the weight update function of the K, T and U tensor layers, g1,g2,g3,g T Represents the gradient propagated to the 1st, 2nd, 3rd, and Tth tensor layers. It can be observed that the weights of different tensor layers are updated by reusing the gradient information after the stage. This will prevent different tensor layers from repeatedly learning repeated gradient information.

[0074] In the embodiment of the present invention, it can be seen that the gradient from the tensor layer is integrated separately. On the other hand, the feature map x0' that has not passed through the tensor layer is also integrated separately. For the gradient information of updating the weight, both sides do not contain the repeated gradient information belonging to the other side.

[0075] In general, the CSPELAN proposed in the embodiment of the present invention retains the advantages of ELAN feature aggregation, but at the same time prevents excessive repeated gradient information by truncating the gradient flow. This idea is achieved by designing a hierarchical feature fusion strategy and is used in some transition layers.

[0076] 2. Dilated Reparam Block (DRB)

[0077] The overall architecture of DRBNCSPELAN4 is as follows Figure 5 As shown in Figure (a) in the figure. Since the capability of ELAN, which originally only uses convolutional layer stacking, is extended to the new architecture GELAN that can use any computing block, the embodiment of the present invention selects Dilated Reparam Block as the computing block after repeated experimental analysis. Among them, Figure 5 The case with four parallel layers shown in (d) is represented by K=9, r=(1,2,3,4), k=(5,5,3,3); from the perspective of parameters, such an expansion layer is equivalent to a non-expanded convolution layer with a larger sparse kernel, so the entire block can be equivalently converted to a single large kernel convolution. This example shows K=9, and embodiments of the present invention can use more expansion layers to obtain a larger K.

[0078] In practice, large kernel convolutions are used with parallel small kernel convolutions, which help capture small-scale patterns during training. Their outputs are added after two respective batch normalization (BN) layers, as shown in Figure 5 Figure (d) in the figure. After training, using the structural reparameterization method, the embodiment of the present invention merges the BN layer into the convolution layer. Ignoring the input pixels in the dilated convolution is equivalent to inserting additional zero terms into the convolution kernel. Therefore, the dilated convolution layer with a small convolution kernel can be equivalently converted to a non-dilated (i.e., r=1) layer with a sparse larger kernel. The need to capture such patterns is exactly matched with the mechanism of dilated convolution - from the perspective of the sliding window, the dilated convolution layer with a dilation rate of r scans the input channel to capture the spatial pattern, where each pixel of interest is r-1 pixels away from its neighbors. k is the kernel size of the dilated layer, and by inserting zero terms, the kernel size of the corresponding non-dilated layer will be (k-1)r+1.

[0079] Original convolution kernel: W∈R k×k , (3)

[0080] After inserting zero terms: W'∈R((k-1)r+1)×((k-1)r+1) , (4)

[0081] This is achieved by transposing the convolution with a stride of r: W = conv_transpose2d(W, I, stride = r). (5)

[0082] The equivalence can be easily verified - given any W∈R k×k and any input channels, a convolution with W and dilation rate r always produces the same result as a non-dilated convolution with W. In the common case where the output shape is equal to the input shape, i.e. the padding of the former is (k-1) / 2, note that the padding of the latter should be (k-1)r / 2, since the size of the equivalent sparse kernel is (k-1)r+1.

[0083] Based on this equivalent transformation, the embodiment of the present invention uses an expanded Reparam block, which uses a non-expanded small convolution kernel and multiple expanded small convolution kernel layers to enhance the non-expanded large convolution kernel conv layer. Its hyper parameters include the large kernel as K, the parallel convolution kernel as k, and the expansion rate as r. The case with four parallel layers ( Figure 5 (d)) is represented by K=9, r=(1,2,3,4), k=(5,5,3,3). For larger K, embodiments of the present invention can use more dilation layers with larger kernel sizes or dilation rates. The kernel size and dilation rate of parallel branches are flexible, and the only constraint is (k-1)r+1≤K. For example, when K=13 (the default setting in the experiments of embodiments of the present invention), embodiments of the present invention use five layers of k=(5,7,3,3,3), r=(1,2,3,4,5), so the equivalent kernel sizes will be (5,13,7,9,11) respectively. In order to convert the DialtedReparam Block into a large kernel convolutional layer for inference, embodiments of the present invention first merge each BN into the previous convolutional layer, use the single-channel conversion to multi-channel case to convert each layer with dilation r>1, and add all generated kernels with appropriate padding zeros. For example, Figure 5 In (d), the layer with k=3, r=3 is converted to a sparse 7×7 kernel and added to a 9×9 kernel with one pixel zero padding on each side.

[0084] Step 203: a new spatial pyramid pooling (ASPPFELAN) network is designed. Compared with the single maximum pooling layer of the same type in the past, the embodiment of the present invention introduces an average pooling layer to comprehensively extract the features of weed texture and background;

[0085] As the depth of CNN increases, the size of the receptive field becomes larger. Due to the limited size of the input image, feature extraction is repetitive on large receptive fields. Therefore, the SPPELAN module is used to add corresponding modules by fusing feature maps of different receptive fields to eliminate this problem. This module combines local and global features to maximize the expressive power of feature maps, expand the receptive field of the backbone network, and separate the most important contextual features for dimensional object detection. In order to integrate features of receptive fields of different scales, SPPELAN uses multiple maximum pooling operations in series and then in parallel, which is more sensitive to texture feature information. However, the maximum pooling operation fails to capture background information and causes information loss, resulting in inaccurate detection.

[0086] In order to solve this problem, the embodiment of the present invention proposes an ASPPELAN module, which replaces the original three maximum pooling layers that are first serial and then parallel, with an average pooling layer and a maximum pooling layer that are first serial and then parallel. Compared with a single maximum pooling layer, the introduction of the average pooling layer comprehensively extracts the features of the weed texture and background, so that the detection accuracy is significantly improved without much change in the module scale, thus forming ASPPELAN. Figure 6 As shown in Figure 2. Although both operations expand the network receptive field, the maximum pooling significantly reduces the spatial resolution, resulting in the loss of feature information. In contrast, the average pooling layer, which is more sensitive to background feature information, enriches the extracted features by capturing the background information of the detection target with less loss of resolution. The output features are then combined at the same level to enhance the feature representation. Therefore, this module improves the learning ability of the network while maintaining a fast detection speed and minimizing the increase in computational cost.

[0087] Step 204: DySample upsampler is introduced, and the upsampler is constructed from the perspective of point sampling. This method is more resource-saving and efficient;

[0088] Based on the built-in functions of PyTorch, the embodiment of the present invention first provides a simple implementation to demonstrate the feasibility of dynamic upsampling based on sampling ( Figure 7 (as shown in Figure (a)).

[0089] The input features, upsampled features, generated offsets, and original grid are represented by X, X', O, and G, respectively. (a) The sampling set is generated by the sampling point generator, and the input features are resampled by the grid sampling function. Figure 7As shown in (b) of Figure 1, the sampled set is the sum of the generated offset and the original grid position. The top box shows the version with a "static range factor", where the offset is generated through a linear layer. The bottom depicts the version with a "dynamic range factor", where the range factor is first generated and then used to modulate the offset, and 'σ' represents the sigmoid function.

[0090] Grid sampling: Given a feature map of size C×H1×W1, and a sample set S of size 2×H2×W2, where the first dimension of 2 represents the x and y coordinates. The grid_sample function resamples the assumed bilinear interpolation X using the locations in S to generate X' of size C×H2×W2. This process is defined as:

[0091] X' = ​​grid_sample(X,S), (6)

[0092] Simple implementation: Given an upsampling scale factor s and a feature map X of size C×H×W, the present embodiment uses a linear layer whose input and output channels are C and 2s respectively. 2 , to generate a size of 2s 2 ×H×W offset O. Then, it is reshaped into 2×sH×sW by the Pixel Shuffling module. Then, the sampling set S is the sum of the offset O and the original sampling grid G, that is:

[0093] O=linear(X), (7)

[0094] S=G+O, (8)

[0095] In which, the reshape operation is omitted. Finally, through grid_sample and the sampling set S, an upsampled feature map X' of size 2×sH×sW can be generated, as shown in equation (6).

[0096] Step 205: DRBNCSPELAN4, ASPPFELAN and DySample are combined with GELAN to establish a re-parameterized feature extraction detection network (RLCFE-Net) with higher detection performance and less complexity, and the model is evaluated on the public datasets CottonWeedDet12 (5648), 4WEED DATASET (618) and the self-made dataset Six weeds (2118) to better meet the needs of efficient and accurate detection in actual scenarios.

[0097] Step 206: Two modes of model deployment, namely edge device (Raspberry) deployment and graphical user interface (GUI) development.

[0098] Deployment on edge devices (Raspberry Pi): Since the purpose of this study is to adapt the models to edge devices, some evaluations were performed on them to help determine evaluation metrics such as mAP. For testing purposes, a field that was not captured in the training set was selected and the video was taken as the prediction source. The video selected a translation transformation to create a scene where the agricultural robot moves on the field and crosses the weeds. The video was modified to match the 640×* adaptive resolution of the camera, because the hardware movement is based on accurate pixel calculations. Finally, it was loaded into the code and four models were built and tested. Figure 8 (a) shows the two model parameters during detection using Raspberry Pi, i.e., image and video detection results.

[0099] Graphical User Interface (GUI): A GUI application developed using Python and PySide6 to increase the flexibility of users in deploying detection models ( Figure 8 (b)). The application allows the user to select a single model from multiple options and perform multiple tasks such as single image or video multiple weed detection and counting. In this case, the main task with RLCAFE-Net is to detect pixel data and send it to the Arduino for hardware control. In addition, it also provides additional functionality such as counting the number of different weeds detected, which can prove beneficial in monitoring the effectiveness of this weed control system after multiple trips to the field. During testing, the application was edited to use a pre-built video containing weed images, models to detect and utilize the hardware. This was done to study how the system would perform in a real-world scenario, such as in a cotton field. The application extracts a frame from the video, similar to reading a frame from a camera, and performs the detection task. In addition, several parameters were selected for detection to modify the localization process of the application. Once the frame was cropped as required, the confidence threshold for the detection was set to 70% accuracy, allowing the model to localize weeds with high similarity. Since the model had a mAP@50 of 97.7%, it was appropriate to keep the confidence value at 0.7.

[0100] To evaluate the object detection model, the detection performance in this study is measured by the number of detection parameters (Params), gigaflop operations per second (GFLOPs), model size (MB), model stability (F1-core), frames per second (Fps), average detection accuracy mAP (0.5) and mAP (0.5:0.95).

[0101] Example 3

[0102] Combine the following Figure 3-Figure 8 The feasibility of the schemes in Examples 1 and 2 is verified as follows:

[0103] Figure 3The architecture of the proposed RLCFE-Net is mainly to improve the performance of GELAN weed detection and reduce the complexity of the model. A GELAN-based re-parameterized weed feature extraction network is proposed.

[0104] Figure 4 This is a self-made dataset of weed images in different scenes, which mainly includes real image data in six common scenes in real farmland, namely weak light, strong light, seedling stage, flowering stage, fruiting stage, and multiple targets of the same and different types. Multiple scenes make the composition of the dataset complex and comprehensive, because it fully considers the various interference factors that exist in different natural environments where weeding machinery is operated with edge devices.

[0105] Figure 5 This is the structural diagram of a new feature extraction and fusion module DRBNCSPELAN4. This module avoids information loss when processing input data after feature extraction and spatial transformation.

[0106] Figure 6 For the new spatial pyramid pooling (ASPPFELAN) network designed, compared with the previous single maximum pooling layer of the same type, we introduced the average pooling layer to comprehensively extract the features of weed texture and background.

[0107] Figure 7 The DySample upsampler is introduced to build the upsampler from the perspective of point sampling, which is more resource-saving and efficient.

[0108] Figure 8 Deployment renderings for both models, namely edge device (Raspberry) deployment and graphical user interface (GUI) development.

[0109] Table 1 shows an example of the training results and parameter comparison of RLCFE-Net, a heavily parameterized large convolution kernel feature extraction network for weed detection in multiple scenarios, on public and self-made datasets.

[0110] Table 2 is an example of the comparison of training evaluation indicators of different functional modules under the self-made dataset.

[0111] Table 1

[0112]

[0113] 1. In the model column, bold indicates the network model proposed in this study; in the evaluation indicators, bold indicates the best performance. 2. Underline indicates the second performance.

[0114] Table 2

[0115]

[0116] An embodiment of the present invention constructs a medium-sized weed dataset named SixWeed, which contains 2118 real-world weed images; the dataset presents a full-cycle weed dataset under different lighting and growth stages, namely, weak light, strong light, seedling stage, flowering stage, fruiting stage, and multiple targets of the same and different types; the complexity and comprehensiveness of the dataset composition take into account various interference factors existing in different natural environments where edge devices operate. The evaluation of the dataset is mainly carried out on two large and small public datasets on the same model, and then the feasibility of the dataset is verified. As shown in Table 1, the embodiment of the present invention adopts the open source GELAN model to verify the dataset, and achieves a detection performance of 96.6% mAP(50) on the large public CottonWeedDet12(5648) dataset, and 69.6% mAP(50) on the small public 4WEED DATASET(618) dataset. The medium-sized (Sixweed) dataset built by the embodiment of the present invention achieves a detection accuracy of 96.35%, which is not much different from the average detection accuracy of the large CottonWeedDet12(5648) dataset, which is twice as high. It can be seen that the construction of the dataset of the embodiment of the present invention is relatively successful.

[0117] The embodiment of the present invention designs a hierarchical feature fusion strategy to truncate the gradient flow to prevent excessive repeated gradient information, thereby ensuring better feature aggregation characteristics.

[0118] On the self-built data set, the DRBNCSPELAN4 proposed in the embodiment of the present invention is 11.32%, 40.31% and 11.08% smaller in Params (M), FLOPs (G) and Size (MB) than the fusion module RepNCSPELAN4 in the baseline model (GELAN), and 0.934% and 0.458% higher in mAP (50) and mAP (50:95) than RepNCSPELAN4. The specific experimental results are shown in Table 2. The experimental comparison further verifies that the superiority of the DRBNCSPELAN4 module is reflected in the reduction of information loss when processing input data after feature extraction and spatial transformation; it can improve the performance of the model while reducing the amount of calculation, which is particularly effective for image detection tasks.

[0119] The embodiment of the present invention introduces an average pooling layer to comprehensively extract the features of weed texture and background. The embodiment of the present invention proposes ASPPFELAN, which is 1.24% and 1.26% higher in mAP(50) and mAP(50:95) than the spatial pyramid pooling module in the baseline model (GELAN) with a reasonable increase in indicators such as computational complexity and model weight and a slight decrease in F1-score and FPS. The specific experimental results are shown in Table 2.

[0120] The embodiment of the present invention introduces the DySample upsampler, and constructs an upsampler from the perspective of point sampling. This method is more resource-saving and efficient. Compared with Upsample in the baseline model (GELAN), mAP(50) and mAP(50:95) are 1.14% and 0.57% higher. The specific experimental results are shown in Table 2. DySample avoids dynamic convolution and constructs an upsampler from the perspective of point sampling. This method is more resource-efficient.

[0121] In step 205, DRBNCSPELAN4, ASPPFELAN and DySample are combined with GELAN to establish a re-parameterized feature extraction detection network (RLCFE-Net) with higher detection performance and less complexity, and the model is evaluated on the public datasets CottonWeedDet12 (5648), 4WEED DATASET (618) and the self-made dataset Six weeds (2118). As shown in Table 1, RLCFE-Net improves mAP(50) and mAP(50:90) by 3.49%, 1.45% mAP(50) and 6.27%, 1.03% mAP(50:90) compared with YOLOv10l and GELANc, respectively; and reduces FLOPs, Params and Size by 51.17% and 3.03% respectively compared with YOLOv10l and GELANc. 9.82% (FLOPs), 10.24%, 9.06% (Params) and 10.15%, 8.75% (Size); compared with YOLOv10l and GELANc, the F1-score (model stability) and FPS (f / s) are improved by 3.27%, 0.63% (F1-score) and 54.60%, 29.52% (f / s), respectively. It can be seen that the comprehensive test performance of RLCFE-Net is significantly superior.

[0122] In step 206, Figure 8 Figure (a) shows the effect of Raspberry Pi in image and video detection. Two frames are randomly selected for image detection. In order to control the variable, the number of frames for image detection is recorded for video detection, which achieves good detection results. Figure 8 Figure (b) shows the GUI test effect developed using Python and PySide6. From the screenshot, it can be seen that this weed detection user interface can not only identify weeds accurately, but also count them, and it also has good results in multiple types of dense weeds.

[0123] Example 4

[0124] A weed detection device for multiple scenarios, the device comprising: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute the following method steps in embodiment 1:

[0125] Collect and label weed data in different scenarios, build datasets, and then evaluate the datasets;

[0126] The re-parameterized large convolution kernel feature extraction-fusion module, spatial pyramid pooling network, DySample upsampler and general efficient layer aggregation network are combined to establish a re-parameterized feature extraction detection network;

[0127] Weed detection in multiple scenarios based on re-parameterized feature extraction detection network.

[0128] Among them, the reparameterized feature extraction-network is:

[0129] One stage of CSPELAN consists of a partial tensor block and a partial transition layer. In the partial tensor block, the feature map of the base layer in the stage is divided into two parts by the channel x0 = [x0′, x0″]; between x0′ and x0″, the former is directly connected to the end of the stage, and the latter will pass through the tensor block. All the steps involved in the partial transition layer are as follows:

[0130] First, the output of the tensor layer, [x0", x1, x2, x3], will go through the transition layer, and the output of the transition layer x T It will be concatenated with x0′ and go through another transition layer to generate the output x U ; The equations for feedforward transfer and weight update are respectively;

[0131] X K =W K *[x0″,x1,x2],

[0132] X T =W T *[x0″,x1,x2,x3],

[0133] X U =W U *[x0′,x T ],

[0134] Among them, * represents the convolution operator, [x0,x1,x2,x3] represents the connection x0,x1,x2,x3, W K ,W T ,W U and X K ,X T ,X U are the weights and outputs of the K, T and U tensor layers respectively;

[0135] W K ′=f K (W K ,{g0″,g1,g2}),

[0136] W T ′=f T (W T ,{g0″,g1,g2,g3}),

[0137] W U ′=f U (W U ,{g0′,g T}),

[0138] Among them, f K ,f T ,f U is the weight update function of the K, T and U tensor layers, g1,g2,g3,g T Represents the gradient propagated to the 1st, 2nd, 3rd, and Tth tensor layers.

[0139] Among them, the re-parameterized large convolution kernel feature extraction-fusion module is:

[0140] Original convolution kernel: W∈R k×k ,

[0141] After inserting zero terms: W'∈R ((k-1)r+1)×((k-1)r+1) ,

[0142] This is achieved by transposing a convolution with a stride of r: W = conv_transpose2d(W, I, stride = r).

[0143] Among them, the spatial pyramid pooling network is: an average pooling layer and a maximum pooling layer are operated twice in series and then in parallel.

[0144] Among them, the DySample upsampler is:

[0145] Given a feature map of size C×H1×W1, and a sample set S of size 2×H2×W2, where the first dimension of 2 represents the x and y coordinates, the grid_sample function resamples the assumed bilinear interpolation X using the locations in S to generate X' of size C×H2×W2, defined as:

[0146] X'=grid_sample(X,S),

[0147] Given an upsampling factor s and a feature map X of size C×H×W, use a linear layer with C and 2s input and output channels respectively.2 , the generated size is 2s 2 ×H×W offset O; reshaped into 2×sH×sW by the pixel shuffle module, the sampling set S is the sum of the offset O and the original sampling grid G, that is:

[0148] O=linear(X),

[0149] S=G+O,

[0150] Through grid_sample and sampling set S, an upsampled feature map X' of size 2×sH×sW is generated.

[0151] It should be pointed out here that the device description in the above embodiment corresponds to the method description in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0152] The execution subjects of the above-mentioned processor and memory can be devices with computing functions such as computers, single-chip microcomputers, and microcontrollers. In specific implementation, the embodiments of the present invention do not limit the execution subjects and are selected according to the needs of actual applications.

[0153] The data signal is transmitted between the memory and the processor via a bus, which is not described in detail in the embodiment of the present invention.

[0154] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, the storage medium includes a stored program, and when the program is running, the device where the storage medium is located is controlled to execute the method steps in the above embodiment.

[0155] The computer-readable storage medium includes but is not limited to a flash memory, a hard disk, a solid-state drive, and the like.

[0156] It should be pointed out here that the description of the readable storage medium in the above embodiment corresponds to the description of the method in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0157] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present invention are generated.

[0158] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be accessed by the computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium or a semiconductor medium, etc.

[0159] Unless otherwise specified, the models of the components in the embodiments of the present invention are not limited, and any device that can perform the above functions may be used.

[0160] Those skilled in the art will appreciate that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A weed detection method for multiple scenarios, characterized in that: The method comprises: Collect and label weed data in different scenarios, build datasets, and then evaluate the datasets; The re-parameterized large convolution kernel feature extraction-fusion module, spatial pyramid pooling network, DySample upsampler and general efficient layer aggregation network are combined to establish a re-parameterized feature extraction detection network; Weed detection in multiple scenarios based on re-parameterized feature extraction detection network.

2. A weed detection method for multiple scenarios according to claim 1, characterized in that: The re-parameterized feature extraction-network is: One stage of CSPELAN consists of a partial tensor block and a partial transition layer. In the partial tensor block, the feature map of the base layer in the stage is divided into two parts by the channel x0 = [x0′, x0″]; between x0′ and x0″, the former is directly connected to the end of the stage, and the latter will pass through the tensor block. All the steps involved in the partial transition layer are as follows: First, the output of the tensor layer, [x0", x1, x2, x3], will go through the transition layer, and the output of the transition layer x T It will be concatenated with x0′ and go through another transition layer to generate the output x U ; The equations for feedforward transfer and weight update are respectively; X K =W K *[x0′′,x1,x2], X T =W T *[x0′′,x1,x2,x3], X U =W U *[x0′,x T ], Among them, * represents the convolution operator, [x0,x1,x2,x3] represents the connection x0,x1,x2,x3, W K ,W T ,W U and X K ,X T ,X U are the weights and outputs of the K, T and U tensor layers respectively; W K ′=f K (W K ,{g0″,g1,g2}), W T ′=f T (W T ,{g0″,g1,g2,g3}), W U ′=f U (W U ,{g0′,g T }), Among them, f K ,f T ,f U is the weight update function of the K, T and U tensor layers, g1,g2,g3,g T Represents the gradient propagated to the 1st, 2nd, 3rd, and Tth tensor layers.

3. The weed detection method for multiple scenarios according to claim 1, characterized in that: The re-parameterized large convolution kernel feature extraction-fusion module is: Original convolution kernel: W∈R k×k , After inserting zero terms: W'∈R ((k-1)r+1)×((k-1)r+1) , This is achieved by transposing a convolution with a stride of r: W = conv_transpose2d(W, I, stride = r).

4. The weed detection method for multiple scenarios according to claim 1, characterized in that: The spatial pyramid pooling network is: an average pooling layer and a maximum pooling layer are operated twice in series and then in parallel.

5. The weed detection method for multiple scenarios according to claim 1, characterized in that: The DySample upsampler is: Given a feature map of size C×H1×W1, and a sample set S of size 2×H2×W2, where the first dimension of 2 represents the x and y coordinates, the grid_sample function resamples the assumed bilinear interpolation X using the locations in S to generate X' of size C×H2×W2, defined as: X'=grid_sample(X,S), Given an upsampling factor s and a feature map X of size C×H×W, use a linear layer with C and 2s input and output channels respectively. 2 , the generated size is 2s 2 ×H×W offset O; reshaped into 2×sH×sW by the pixel shuffle module, the sampling set S is the sum of the offset O and the original sampling grid G, that is: O=linear(X) S=G+O Through grid_sample and sampling set S, an upsampled feature map X' of size 2×sH×sW is generated.

6. The weed detection method for multiple scenarios according to claim 1, characterized in that: The method also includes: using edge device deployment and a graphical user interface to simulate field deployment testing.

7. A weed detection device for multiple scenarios, characterized in that: The device comprises: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute any one of the methods of claims 1-6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is enabled to perform the method according to any one of claims 1 to 6.

Images