Method and device for detecting weeds in fence and medium

Abstract

The embodiment of the invention provides a method and equipment for detecting weeds in a fence and a medium. The method comprises the following steps: receiving a to-be-identified image and a shooting position which are sent by shooting equipment and are obtained by shooting a to-be-identified fence; obtaining a target historical image shot at a shooting position from a plurality of historical images obtained in advance, and performing feature alignment on the to-be-recognized image and the target historical image to obtain an aligned to-be-recognized image; performing foreground detection on the aligned to-be-recognized image by adopting a background model corresponding to the shooting position to obtain a foreground region; and carrying out weed detection on the foreground region by adopting a weed detection model to obtain a detection result. The method is used for achieving the effect of improving the detection accuracy and robustness of the weeds in the fence.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to a method, device, and medium for detecting weeds within a fence. Background Art

[0002] The safe operation of substation capacitor banks is affected by a variety of environmental factors, among which weed growth in fenced areas is a significant concern. Weeds can not only obstruct equipment and hinder inspections, but can also pose a fire hazard and, in severe cases, even disrupt equipment operation. Therefore, ensuring weed-free fenced areas is a critical task in substation operations and maintenance, and any weeds found must be removed immediately.

[0003] In the existing technology, weeds in newly collected images are directly detected using a weed detection model. However, the accuracy of the detection results obtained using this method is greatly affected by background fluctuations. In complex environments, there are often problems of false detection and missed detection of weeds. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, and medium for detecting weeds within a fence, so as to improve the accuracy and robustness of detecting weeds within a fence.

[0005] In a first aspect, embodiments of the present application provide a method for detecting weeds within a fence, including:

[0006] Receive the image to be identified and the shooting position obtained by shooting the fence to be identified sent by the shooting device;

[0007] Acquire a target historical image captured at the shooting position from a plurality of previously acquired historical images, and perform feature alignment between the image to be identified and the target historical image to obtain an aligned image to be identified;

[0008] Performing foreground detection on the aligned image to be identified using a background model corresponding to the shooting position to obtain a foreground area, wherein the background model is a model for foreground and background detection obtained by training a Gaussian mixture model based on multiple historical images shot at the shooting position;

[0009] A weed detection model is used to detect weeds in the foreground area to obtain a detection result. The weed detection model is a model for detecting weeds in an image obtained by pre-training a machine learning model.

[0010] In a second aspect, an embodiment of the present application provides a weed detection device within a fence, comprising:

[0011] The first acquiring unit is configured to receive an image to be identified and a shooting position obtained by shooting the fence to be identified and sent by a shooting device;

[0012] a first alignment unit, configured to obtain a target historical image captured at the shooting position from a plurality of pre-acquired historical images, and perform feature alignment between the image to be identified and the target historical image to obtain an aligned image to be identified;

[0013] a first detection unit, configured to perform foreground detection on the aligned image to be identified using a background model corresponding to the shooting position to obtain a foreground area, wherein the background model is a model for foreground and background detection obtained by training a Gaussian mixture model based on multiple historical images shot at the shooting position;

[0014] The second detection unit is configured to detect weeds in the foreground area using a weed detection model to obtain a detection result. The weed detection model is a model for detecting weeds in an image obtained by pre-training a machine learning model.

[0015] In a third aspect, an embodiment of the present application provides a computer device, including: a memory, a processor;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0020] The methods, devices, and media for detecting weeds within a fence provided by embodiments of the present application first receive an image to be identified and a shooting location of a fence to be identified, sent by a shooting device. Then, a target historical image captured at the shooting location is obtained from multiple previously acquired historical images. Feature alignment is performed on the image to be identified and the target historical images to obtain an aligned image to be identified. Foreground detection is then performed on the aligned image to be identified using a background model corresponding to the shooting location to obtain a foreground area. Finally, weed detection is performed on the foreground area using a weed detection model to obtain a detection result. Based on the pre-acquired background model corresponding to the shooting location of the image to be identified, the foreground area is extracted from the image to be identified, and weeds in the foreground model are detected using the weed detection model. This effectively eliminates interference from background fluctuations (e.g., lighting, background shaking, etc.) on the weed detection results, thereby improving the accuracy and robustness of weed detection within the fence. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] Figure 1 A schematic diagram of the flow of the method for detecting weeds in a fence provided in Example 1 of the present application;

[0023] Figure 2 A schematic diagram of the structures of the C3k module and the C3k_GD module provided in Example 2 of the present application;

[0024] Figure 3 Schematic diagram of the structure of the Ghost Bottleneck submodule and the improved Ghost Bottleneck submodule provided in Example 2 of the present application;

[0025] Figure 4 This is a flow chart of a method for detecting weeds in a fence provided in Example 3 of the present application;

[0026] Figure 5 This is a schematic diagram of the structure of the weed detection device in the fence provided in the fourth embodiment of the present application;

[0027] Figure 6 This is a schematic diagram of the structure of a weed detection device in a fence provided in Example 5 of the present application;

[0028] Figure 7 This is a structural diagram of the computer device provided in Example 6 of the present application.

[0029] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0031] In response to the technical problems in the above-mentioned background technology, the inventors, while studying a method for detecting weeds within the fence of a substation capacitor bank, discovered that by separating the foreground and background in a newly acquired image and only detecting whether weeds exist in the foreground area extracted from the image, it is possible to effectively eliminate the interference of background fluctuations on the accuracy of weed detection.

[0032] Based on the technical concept of the aforementioned inventors, this application provides a method for detecting weeds within a fence. The method first receives an image of a fence to be identified and the shooting location of the fence to be identified, transmitted by a camera. Then, a target historical image captured at the shooting location is obtained from multiple previously acquired historical images. Feature alignment is performed on the image to be identified with the target historical images to obtain an aligned image to be identified. Foreground detection is then performed on the aligned image to be identified using a background model corresponding to the shooting location to obtain a foreground area. Finally, weed detection is performed on the foreground area using a weed detection model to obtain a detection result. Based on the pre-acquired background model corresponding to the shooting location of the image to be identified, the foreground area is extracted from the image to be identified, and weeds are detected within the foreground model using the weed detection model. This method effectively eliminates interference from background fluctuations (such as lighting and background sway) on weed detection results, thereby improving the accuracy and robustness of weed detection within fences.

[0033] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0034] Figure 1 This is a flow chart of the method for detecting weeds in a fence provided in Example 1 of the present application, as shown in FIG. Figure 1As shown, the method for detecting weeds in a fence provided in this embodiment includes:

[0035] S101: Receive an image to be identified and a shooting position obtained by shooting a fence to be identified and sent by a shooting device.

[0036] In this step, in order to detect whether there are weeds in the current fence, it is necessary to obtain the image to be identified obtained by the shooting device. In order to clearly determine the location of the fence in the image to be identified, it is also necessary to obtain the shooting location of the image to be identified.

[0037] It should be understood that the image to be identified taken by the shooting device should include the entire fence area.

[0038] In addition, since weeds are not allowed to appear inside the fence of the substation capacitor bank, the detection of weeds inside the fence in this solution generally refers to the detection of whether there are weeds inside the fence of the substation capacitor bank.

[0039] In a specific implementation, a drone may be used as a shooting device, and the image to be identified and the shooting position of the fence to be identified, which are received from the shooting device, include:

[0040] Receive an image to be identified and a hovering position obtained by shooting the fence to be identified sent by the drone, wherein the shooting position includes the hovering position.

[0041] Specifically, a drone inspection route can be pre-planned based on the geographic location of each fence. This route includes the drone's hovering position at each fence. During the inspection, the drone follows the pre-planned route and captures an image of the fence to be identified at each hovering position. The captured image and the drone's current hovering position are then transmitted. The image captured by the drone at the hovering position must include the entire fence area.

[0042] In practical applications, the camera carried by the drone may be a high-definition red, green, and blue (RBG) camera.

[0043] In practical applications, in order to obtain clearer and more accurate images, the drone can shoot the fence to be identified directly below it perpendicular to the ground to obtain a bird's-eye view of the fence to be identified.

[0044] Optionally, the drone may be equipped with a Global Positioning System (GPS), or a GPS and carrier phase real-time kinematic (RTK) system to determine the shooting position of the drone.

[0045] In this implementation, it should be understood that if a fixed camera is used as the camera, due to its fixed viewing angle, it will be difficult for the fixed camera to cover every corner of the fence area, resulting in weeds in blind spots not being detected, posing a safety hazard. Therefore, compared to fixed cameras, this implementation, based on drone inspections, can achieve full coverage of the fence to be identified, significantly improving the reliability of weed detection within the fence.

[0046] S102 , obtaining a target historical image captured at a shooting position from a plurality of pre-acquired historical images, and performing feature alignment between the image to be identified and the target historical image to obtain an aligned image to be identified.

[0047] In this step, first, it is necessary to obtain any image taken at the shooting position from the multiple historical images acquired in advance as the target historical image; in order to eliminate the differences between the image to be identified and the target historical image due to factors such as different feature scales, shooting angles, lighting, and rotation angles, it is necessary to adopt a feature alignment method, using the target historical image as a reference, to perform feature alignment on the image to be identified and the target historical image to obtain the aligned image to be identified.

[0048] It should be understood that the background model used to extract the foreground area of ​​the aligned image to be identified is trained using multiple historical images taken at the same location. Therefore, aligning the image to be identified with the target historical image is equivalent to aligning each pixel in the image to be identified with each pixel in the background model.

[0049] Among them, the feature alignment method can be a feature transformation algorithm commonly used in the field of computer vision, such as the Scale-Invariant Feature Transform (SIFT) algorithm, the Speeded Up Robust Features (SURF) algorithm, the Oriented FAST and Rotated BRIEF (ORB) algorithm, etc.

[0050] In a specific implementation, the following method may be used to perform feature alignment on the image to be identified and the target historical image to obtain the aligned image to be identified:

[0051] Based on the SIFT algorithm, the descriptors of the key points of the image to be identified and the descriptors of the key points of the target historical image are obtained respectively, and based on the descriptors of the key points of the image to be identified and the descriptors of the key points of the target historical image, the features of the image to be identified and the target historical image are aligned to obtain the aligned image to be identified.

[0052] The following steps 1-7 will illustrate the specific implementation method of the aligned image to be identified based on the SIFT algorithm:

[0053] Step 1: Construct Gaussian scale spaces for the image to be identified and the target historical image respectively to obtain the Difference of Gaussian (DoG) pyramid corresponding to the image to be identified and the DoG pyramid corresponding to the target historical image.

[0054] Specifically, for each pixel in the image to be identified (or the target historical image), first, according to the pixel value and Gaussian kernel function of the pixel, the Gaussian blur result of the pixel at multiple scales is obtained to construct a Gaussian pyramid. The calculation formula is as follows:

[0055]

[0056]

[0057] in, Represents the coordinates in the image to be identified (or the target historical image) The pixel value of the pixel at ; It is the standard deviation of the preset Gaussian kernel, which is used to control the degree of blur. The larger the standard deviation, the stronger the smoothing effect. is a two-dimensional Gaussian kernel function; Represents the coordinates in the image to be identified (or target historical image) Gaussian blur result of the pixel at when the scale is σ.

[0058] Then, the difference between the Gaussian blur results of adjacent scales is calculated to construct the DoG pyramid. The calculation formula is as follows:

[0059]

[0060] in, is the scale multiplication factor, usually taken as , Represents the coordinates in the image to be identified (or target historical image) The pixel point at the scale is Gaussian difference results when .

[0061] Step 2: perform key point detection and positioning on the image to be identified and the target historical image respectively to obtain the key points in the image to be identified and the key points in the target historical image.

[0062] Specifically, first, in the DoG pyramid, for each pixel in the image to be identified (or the target historical image), compare the pixel with the 8 neighboring pixels in the current scale and the 18 neighboring pixels in the two adjacent scales above and below. If the pixel value of the pixel is greater than or less than the pixel values ​​of all neighboring points, the pixel is considered to be a local extreme point and is confirmed as a candidate key point.

[0063] Then, for each candidate key point of the image to be identified (or the target historical image), the sub-pixel positioning of the candidate key point is performed based on the Taylor expansion, and whether the candidate key point is an edge point is judged based on the Hessian matrix, and the position of the candidate key point that is not an edge point is filtered as the position of the key point.

[0064] Specifically, for each candidate key point, Perform Taylor expansion and accurately adjust the positions of candidate key points by taking derivatives;

[0065] At the same time, for each candidate key point, it is necessary to calculate the Hessian matrix corresponding to the candidate key point and the principal curvature of the Hessian matrix. Based on whether the principal curvature meets the restriction conditions, it is judged whether the candidate key point is an edge point; the edge points in the candidate key points are eliminated, and the remaining points are determined as key points. The Hessian matrix is ​​calculated using the following formula: ,in, 、 、 They are Second-order partial derivatives with respect to x and y.

[0066] The constraint condition is expressed as follows: ,in, is a matrix traces is a matrix The determinant value of ; r is the preset curvature threshold, for example r=10.

[0067] Step 3: Determine the direction of each key point in the image to be identified and the direction of each key point in the target historical image.

[0068] Specifically, for each key point in the image to be identified (or the target historical image), first, calculate the gradient distribution of the neighborhood where the key point is located , the calculation formula is as follows:

[0069]

[0070] Then, the gradient direction of the key point is divided into 36 directional histogram units, and the gradient direction of each pixel in the neighborhood of the key point is counted, and its amplitude is accumulated to the corresponding histogram unit to obtain the gradient histogram of the key point; the peak of the gradient histogram is determined as the main direction of the key point, and the secondary peak direction in the gradient direction distribution histogram with an amplitude exceeding 80% of the main peak is determined as the secondary direction of the key point; according to the main direction and secondary direction of the key point, the direction of the key point is obtained.

[0071] Step 4: Generate a descriptor for each key point in the image to be identified and a descriptor for each key point in the target historical image.

[0072] Specifically, for each key point in the image to be identified (or the target historical image), the neighborhood of the key point is divided into a 4×4 sub-region grid, and the gradient histograms in 8 directions are counted in each sub-region to obtain a 128-dimensional feature vector, which is the descriptor of the key point.

[0073] Optionally, in order to eliminate the impact of scale changes, the descriptor of the key point can be normalized by L2 norm to obtain the normalized descriptor of the key point: ,in, Descriptors representing key points; Represents the normalized descriptor of the keypoint.

[0074] Step 5: Calculate the distance between the descriptors of the key points in the target historical image and the image to be identified (usually using the Euclidean distance), find the best matching key point in the target historical image for each key point in the image to be identified, and obtain multiple sets of matching point pairs.

[0075] Optionally, for any key point in the image to be identified , the nearest neighbor ratio method can be used to determine whether the matching point pairs are reliable, and unreliable matching point pairs can be eliminated from multiple sets of matching point pairs to obtain multiple sets of matching point pairs after screening. Each matching pair includes a key point in the image to be identified and the key point in the target historical image that best matches the key point. The calculation formula of the nearest neighbor ratio method is as follows: ,in, and They are respectively the descriptors of the key points of the target history image and Descriptors of the nearest and next nearest neighbor keypoints, To experimentally verify the effective threshold, for example, .

[0076] Step 6: Based on the matching point pairs, the Random Sample Consensus (RANSAC) algorithm is used to estimate the optimal homography matrix.

[0077] Specifically, the optimal homography matrix It can be determined using the following formula: ,in, represents the homography matrix; Represents the key points in the image to be identified; Indicates the image to be identified after alignment The corresponding points; ρ is the Huber loss function, which is used to reduce the influence of noise and is defined as: ,in, It is usually set to 3, and the number of RANSAC iterations is generally 2000.

[0078] Step 7: Transform the image to be identified according to the optimal homography matrix to obtain the aligned image to be identified.

[0079] Optionally, after obtaining the aligned image to be identified, the alignment accuracy can be evaluated by calculating the root mean square error between the key point of the image to be identified in each matching point pair and the point corresponding to the key point in the aligned image to be identified. The root mean square error can be expressed as follows: ,in, Indicates the The key points of the image to be identified in the matching point pairs; Indicates the key points in the image to be identified after alignment corresponding points; Indicates the number of matching point pairs.

[0080] The method provided by this implementation is based on the SIFT algorithm to obtain an aligned image to be identified that is aligned with the target historical image features, eliminating differences between the image to be identified and the target historical image caused by factors such as feature scale, shooting angle, lighting, and rotation angle, thereby improving the accuracy of the foreground area of ​​the image to be identified that is subsequently extracted based on the background model.

[0081] S103. Use the background model corresponding to the shooting position to perform foreground detection on the aligned image to be identified to obtain a foreground area, wherein the background model is a model for foreground and background detection obtained by training a Gaussian mixture model based on multiple historical images shot at the shooting position.

[0082] In this step, it is necessary to use a background model corresponding to the shooting position of the image to be identified, perform foreground detection on the aligned image to be identified, and extract the foreground area corresponding to the image to be identified.

[0083] Among them, for each shooting position (equivalent to each fence area), the multiple historical images used to train the background model corresponding to the shooting position are multiple images obtained by shooting the entire fence area at the same shooting position as the image to be identified at different time stages.

[0084] It should be understood that multiple historical images are taken at different time stages in order to capture images of the fence area under different background fluctuations to ensure the comprehensiveness of the background model; the same shooting position refers to the same latitude, longitude and altitude of the shooting. Taking multiple historical images and images to be identified at the same shooting position can facilitate the establishment of the background model and facilitate the feature alignment of the image to be identified with the target historical image.

[0085] In a specific implementation, before using the background model corresponding to the shooting position to perform foreground detection on the aligned image to be identified, the following steps 1-3 may be used to obtain the background model corresponding to the shooting position:

[0086] Step 1: For each shooting location, obtain multiple historical images shot at the shooting location.

[0087] It should be understood that in this solution, a background model needs to be established for each shooting location (equivalent to each fence area), and each background model is trained using multiple historical images shot at the shooting location.

[0088] Step 2: Perform feature alignment on multiple historical images to obtain multiple aligned historical images.

[0089] In this step, in order to eliminate the differences between the multiple historical images taken at the shooting location due to different factors such as feature scale, shooting angle, lighting, rotation angle, etc., in order to improve the detection accuracy of the background model, it is necessary to adopt a feature alignment method, select any historical image as the reference image, and align the features of other historical images with the reference image to obtain multiple aligned historical images.

[0090] The feature alignment method may be a feature transformation algorithm commonly used in the field of computer vision, such as the SIFT algorithm, the SURF algorithm, and the ORB algorithm.

[0091] Furthermore, in a specific implementation, the following method can be used to perform feature alignment on multiple historical images to obtain multiple aligned historical images:

[0092] Based on the SIFT algorithm, the descriptors of the key points of each historical image in multiple historical images are obtained respectively, and based on the descriptors of the key points of each historical image, the features of the multiple historical images are aligned to obtain multiple aligned historical images.

[0093] In a specific implementation, the specific step of aligning features of other historical images with the reference image is similar to the method of aligning features of the image to be identified with the target historical image in step S102, and will not be repeated here.

[0094] This implementation provides a method that uses the SIFT algorithm to generate multiple aligned historical images, which are then used to train a background model corresponding to the shooting locations of these images. By training the background model using multiple historical images that eliminate differences caused by factors such as feature scale, shooting angle, lighting, and rotation angle, the accuracy of background model detection can be improved.

[0095] Step 3: Train the Gaussian mixture model based on multiple aligned historical images to obtain the background model corresponding to the shooting location.

[0096] In this step, it is necessary to establish K Gaussian distributions for each pixel point based on the initially set parameters to obtain an initial Gaussian mixture model; multiple aligned historical images are used as a multi-frame image sequence and input into the initial Gaussian mixture model frame by frame to update the parameters in the Gaussian mixture model until the last aligned historical image is input into the background model and the update is completed to obtain the background model.

[0097] For example, the parameters initially set may include: the initial weight of each Gaussian distribution of the i-th pixel is , the initial mean is, for example, a value obtained by random sampling, and the initial covariance matrix is ,in, is the initial variance, which can be 30. is the identity matrix.

[0098] Specifically, after the t-th frame image is input into the Gaussian mixture model and the update is completed, the probability distribution of each pixel in the Gaussian mixture model is as follows:

[0099]

[0100] In the above formula, Represents the pixel value of pixel x in the t-th frame image, They respectively represent the weight, mean, and covariance matrix of the i-th Gaussian distribution of pixel x in the Gaussian mixture model after the t-th frame image is input into the Gaussian mixture model and the update is completed; n represents the dimension of the pixel point, which usually depends on the color space used.

[0101] Among them, after the t-th frame image is input into the Gaussian mixture model, the method of updating the parameters in the Gaussian mixture model is as follows:

[0102] For each pixel in the t-th frame image, determine whether the pixel value matches any existing Gaussian distribution in the Gaussian mixture model based on the pixel value. If it matches any Gaussian distribution successfully, update the weight of the Gaussian distribution; if it does not match any Gaussian distribution successfully, generate a new Gaussian distribution and replace the Gaussian distribution with the smallest weight.

[0103] The corresponding calculation formulas in the above update method are as follows ①-③:

[0104] ① Determine the pixel value of pixel x The formula for whether it matches the i-th Gaussian distribution is as follows: ,in, is a preset threshold, for example, 2.5, .

[0105] If the pixel value of pixel x in frame t is If the mean of the ith Gaussian distribution corresponding to the pixel point x at the t-1th frame satisfies the above formula, then the pixel value is considered to be Matches with the i-th Gaussian distribution; otherwise, does not match.

[0106] ② When a match is successful with any Gaussian distribution and the weight of the Gaussian distribution is updated, the calculation formula for the weight of the updated Gaussian distribution is as follows (assuming that the match is successful with the i-th Gaussian distribution): ,in, is the learning rate, for example 5; is the matching mark. When it matches the i-th Gaussian distribution, .

[0107] ③ When no Gaussian distribution is successfully matched, and a new Gaussian distribution is generated to replace the Gaussian distribution with the smallest weight, the calculation formulas for the weight, mean, and covariance matrix of the replaced Gaussian distribution are as follows (assuming that the Gaussian distribution with the smallest weight is the i-th Gaussian distribution corresponding to the pixel): .

[0108] Optionally, to further improve the accuracy and adaptability of the trained background modeling, a dynamic Gaussian component optimization mechanism can be introduced. The details are as follows:

[0109] During the training of the Gaussian mixture model, if a Gaussian distribution is not matched for a long time, the weight of the Gaussian distribution will be gradually reduced, and when the weight of the Gaussian distribution is lower than the preset threshold, the Gaussian distribution will be automatically eliminated, thereby avoiding the interference of invalid Gaussian distribution on the detection accuracy of the background model.

[0110] At the same time, in the process of training the Gaussian mixture model, the number of Gaussian components K can be dynamically adjusted according to the complexity of the current scene and the distribution of pixel values, so that the background model can not only describe the multimodal pixel distribution but also maintain high computational efficiency.

[0111] It should be understood that the above-mentioned Gaussian component optimization mechanism adaptively updates the background model by monitoring the matching frequency and statistical performance of each Gaussian distribution, thereby improving the robustness of the model to illumination changes and background dynamic changes.

[0112] The method in steps 1-3 provided in this implementation method obtains multiple historical images taken at each shooting location; aligns the features of the multiple historical images to obtain multiple aligned historical images; and trains a Gaussian mixture model based on the multiple aligned historical images to successfully obtain the background model corresponding to each shooting location. In addition, by introducing a Gaussian component optimization mechanism, the detection accuracy and robustness of the background model are further improved.

[0113] In a specific implementation, since the background model includes multiple Gaussian distributions corresponding to each pixel and the weight of each Gaussian distribution, the background model corresponding to the shooting position is used to perform foreground detection on the aligned image to be identified to obtain the foreground area. This can be achieved by using the following methods in steps 1-3:

[0114] Step 1: For each pixel point in the aligned image to be identified, obtain a target number of Gaussian distributions with the largest weight value from the multiple Gaussian distributions corresponding to the pixel point, wherein the target number is determined according to a preset weight accumulation threshold and the weight of each Gaussian distribution in the multiple Gaussian distributions.

[0115] In this solution, it should be understood that for each pixel point in the aligned image to be identified, it is necessary to judge whether the pixel point belongs to the foreground or the background based on the multiple Gaussian distributions corresponding to the pixel point in the background model, and finally, the foreground area corresponding to the aligned image to be identified is obtained based on the multiple pixels judged to belong to the foreground.

[0116] In this step, each pixel in the background model corresponds to multiple Gaussian distributions, and each Gaussian distribution has a weight. For each pixel in the aligned image to be identified, the target number of Gaussian distributions with the largest weights is obtained from the multiple Gaussian distributions corresponding to that pixel in the background model.

[0117] In a specific implementation, the method for determining the number of targets includes: for each pixel in the aligned image to be identified, first sorting multiple Gaussian distributions corresponding to the pixel in the background model from low to high according to weight, and obtaining the top H Gaussian distributions with the highest weights that satisfy the following formula from the multiple Gaussian distributions: , in the above formula, H represents the number of selected Gaussian distributions; Indicates the preset weight accumulation threshold. The preset weight accumulation threshold is a value less than 1, and this application does not limit the value of the preset number. For example, the preset weight accumulation threshold can be 0.7, 0.8, 0.9, etc.

[0118] Step 2: Match the pixel value of the pixel point with the target number of Gaussian distributions respectively; if the pixel point matches any Gaussian distribution, the pixel point is determined to be a background pixel point; if the pixel point does not match the target number of Gaussian distributions, the pixel point is determined to be a foreground pixel point.

[0119] In this step, for each pixel point in the aligned image to be identified, the pixel value of the pixel point needs to be matched with the target number of Gaussian distributions respectively. If the pixel point matches any Gaussian distribution, the pixel point is determined to be a background pixel point; if the pixel point does not match any Gaussian distribution in the target number of Gaussian distributions, the pixel point is determined to be a foreground pixel point.

[0120] In a specific implementation, the following formula can be used to determine whether the pixel value of a pixel point matches each Gaussian distribution in the target number of Gaussian distributions: , in the above formula, Represents the pixel value of any pixel in the aligned image to be identified; Represents the mean of the mth Gaussian distribution in the target number Gaussian distribution corresponding to the pixel; is a preset threshold, for example, 2.5; ,in, Represents the covariance matrix of the mth Gaussian distribution in the target number of Gaussian distributions corresponding to the pixel point. If the pixel value of the pixel point If the mean of the mth Gaussian distribution among the target number of Gaussian distributions corresponding to the pixel point satisfies the above formula, the pixel value is considered to match the Gaussian distribution; otherwise, it is not matched.

[0121] Step 3: Based on all the aligned foreground pixels in the image to be identified, a foreground area is obtained.

[0122] In this step, the determined foreground pixels are combined to obtain the foreground area corresponding to the aligned image to be identified.

[0123] In this implementation, for each pixel point in the aligned image to be identified in the above steps 1-3, a target number of Gaussian distributions with the largest weight value are obtained from multiple Gaussian distributions corresponding to the pixel point; the pixel value of the pixel point is matched with the target number of Gaussian distributions respectively; if the pixel point matches any Gaussian distribution, the pixel point is determined to be a background pixel point; if the pixel point does not match any of the target number of Gaussian distributions, the pixel point is determined to be a foreground pixel point; finally, based on all the foreground pixels in the aligned image to be identified, the foreground area is obtained, and the foreground area is successfully extracted from the aligned image to be identified.

[0124] Optionally, in a specific implementation, after obtaining the foreground area, the parameters of the background model can be updated using the aligned image to be identified. The specific parameter updating method is similar to the method for training a Gaussian mixture model based on multiple aligned historical images to obtain a background model corresponding to the shooting position, where each aligned historical image is input into the Gaussian mixture model and the parameters of the Gaussian mixture model are updated. This method will not be further described here.

[0125] This implementation uses the methods in steps 1-3 above to update the background model using the newly acquired aligned image to be identified, further improving the accuracy and adaptability of the background model, improving the accuracy of the detected foreground area, and thus improving the accuracy of weed detection in the image to be identified.

[0126] S104: Detect weeds in the foreground area using a weed detection model to obtain a detection result, wherein the weed detection model is a model for detecting weeds in an image obtained by pre-training a machine learning model.

[0127] In this step, a pre-trained weed detection model is used to detect whether weeds are present in the foreground area, and a detection result is obtained to detect weeds within the fence. The detection result indicates whether weeds are present in the foreground area.

[0128] The machine learning model can be a commonly used model in the field of image recognition. This application does not restrict the selection of machine learning models. For example, the machine learning model can be a convolutional neural network model, a support vector machine model, a random forest model, a YOLO (You Only Look Once) series model, a single shot multibox detector (SSD), etc.

[0129] In a specific implementation, the method for detecting weeds in a fence provided in this embodiment further includes: if the detection result indicates that weeds exist in the fence to be identified, pushing an alarm message, wherein the alarm message is used to indicate that weeds exist in the fence to be identified.

[0130] In a specific implementation, when the detection result indicates that there are weeds in the fence to be identified, an alarm message is pushed to the staff, where the alarm message may also include the location of the fence where the weeds exist and the time when the weed detection was performed; after receiving the alarm message, the staff clears the weeds in the fence according to the content of the alarm message to ensure the normal operation of the equipment in the weed fence.

[0131] Optionally, if the frequency of weed detection increases in any fenced area, the detection frequency of that fenced area may be increased. It should be understood that if the frequency of weed detection increases in any fenced area, it indicates that there may be a higher risk of weed growth or a faster weed growth rate in that area. Increasing the detection frequency of that fenced area can effectively prevent weeds from not being cleared in a timely manner, which could affect the normal operation of equipment within the fence.

[0132] Optionally, for any fenced area, the detection frequency of the fenced area may be adjusted based on historical detection data of the fenced area and environmental factors (eg, temperature, humidity, precipitation, etc.).

[0133] In this implementation, if detection results indicate the presence of weeds within a fence, an alert is sent, ensuring timely response from staff and preventing weed accumulation within the fence, thereby ensuring the normal operation of equipment within the fence. Furthermore, by adjusting the detection frequency of a fenced area based on the frequency of weed detection and environmental factors, timely weed removal is further ensured.

[0134] The method for detecting weeds within a fence provided in this embodiment first receives an image to be identified and the shooting location of a fence to be identified, sent by a camera. Then, a target historical image captured at the shooting location is obtained from multiple previously acquired historical images. Feature alignment is performed on the image to be identified and the target historical images to obtain an aligned image to be identified. Foreground detection is then performed on the aligned image to be identified using a background model corresponding to the shooting location to obtain a foreground area. Finally, weed detection is performed on the foreground area using a weed detection model to obtain a detection result. This method effectively eliminates interference from background fluctuations (such as lighting and background motion) on the weed detection results, thereby improving the accuracy and robustness of weed detection within the fence.

[0135] A second embodiment of the present application provides a method for detecting weeds within a fence. Based on the above embodiment, in the method for detecting weeds within a fence provided in this embodiment, the weed detection model is specifically a model for detecting weeds in an image obtained by training an improved YOLOv11 model.

[0136] Among them, the improved YOLOv11 (You Only Look Once version 11) model replaces the backbone network of the YOLOv11 model with the StarNet network, and replaces the C3k2 module in the neck of the YOLOv11 model with the C3k_GD (C3k_Ghost_Dynamic Convolution) module;

[0137] The C3k_GD module is obtained by replacing the bottleneck submodule in the C3k module with the improved Ghost Bottleneck submodule. The improved Ghost Bottleneck submodule is obtained by replacing the depthwise convolution (DWConv) in the GhostBottleneck submodule with dynamic convolution (DConv).

[0138] It's worth noting that the YOLOv11 model primarily consists of an input, a backbone network, a neck, and a detection head. The model's detection process involves the following: after an image is input to the backbone network, the backbone performs feature extraction, generating a low-resolution, high-dimensional feature map, which is then output to the neck. The neck then performs multi-scale feature fusion on this high-dimensional feature map to accommodate detection of objects of varying sizes, generating a multi-scale feature map, which is then output to the detection head. The detection head then performs bounding box regression, category prediction, and confidence calculation based on this multi-scale feature map, ultimately outputting the detection result.

[0139] In the improved YOLOv11 model provided in the second embodiment, the backbone network of the YOLOv11 model is first improved. This improvement is described in detail below:

[0140] The core of the StarNet network is the star operation, which significantly increases feature dimensionality through recursive stacking. Using StarNet as the backbone of the object detection model maintains computational efficiency while achieving richer and more expressive feature representations, enabling efficient and rapid identification of weeds under varying lighting and background conditions.

[0141] The specific structure of the StarNet network is as follows:

[0142] The StarNet network consists of a Convolutional-Batch-Normalization-ReLU (CBR) module and four Star Layer modules. Each CBR module includes a Convolutional (Conv) module with a 3×3 kernel size, a stride of 2, and 24 output channels, a Batch Normalization (BN) module, and a ReLU6 activation module. Each Star Layer module consists of a Space-to-Depth Convolution (SPD Conv) module and several Star Block modules.

[0143] In the structure of the StarNet network described above, the SPD Conv module in the Star Layer module performs convolution processing on each pixel in the feature map corresponding to the input image to complete downsampling and modify the number of embedded channels to obtain an intermediate feature map after channel dimensionality reduction.

[0144] Specifically, the SPD Conv module first divides the feature map into multiple sub-feature maps from the spatial dimension; then splices the above sub-feature maps in the channel dimension to generate an intermediate feature map; then, the point convolution layer in the SPD Conv module performs channel dimensionality reduction on the intermediate feature map to obtain the intermediate feature map after channel dimensionality reduction.

[0145] For example, if the image to be identified is input into the target detection model and processed by the CBR module, the feature map corresponding to the image to be identified is output. The size is , then the SPD Conv module will process the feature map as follows:

[0146] ① Divide the feature map into sub-feature maps, where The sub-feature graph is expressed using the following formula:

[0147]

[0148] Among them, the size of each sub-feature map is .

[0149] ②The above The sub-feature maps are spliced ​​in the channel dimension to generate an intermediate feature map .

[0150] Among them, the first intermediate feature map The size is .

[0151] ③Use The size is , a convolution operation is performed with a convolution kernel with a step size of 1 to achieve channel dimensionality reduction of the intermediate feature map, and the intermediate feature map after channel reduction is obtained.

[0152] Among them, the size of the intermediate feature map after the channel is reduced is .

[0153] In the above formula, It is a feature map The spatial dimensions, is 2, is the feature map The channel dimension, is the number of embedding channels of the intermediate feature map after channel reduction.

[0154] It should be understood that the use of the SPD Conv module in this structure can effectively circumvent the problem of loss of fine-grained information caused by downsampling using strided convolution in traditional convolutional neural networks. While ensuring the feature representation capability in the downsampling process, it more effectively retains the fine-grained feature information of the image, thereby improving the robustness of weed detection in different scenes and under different lighting conditions.

[0155] At the same time, in the structure of the above-mentioned StarNet network, in the Star Block module in the Star Layer module, the intermediate feature map after channel dimensionality reduction undergoes the first depth-wise separable convolution, channel number expansion, nonlinear activation, element-by-element multiplication, channel number adjustment, second depth-wise separable convolution and residual connection processing steps in sequence to finally generate a high-dimensional feature map.

[0156] Specifically, in each Star Block module, the intermediate feature map after the input channel dimensionality reduction is first processed by a 7×7 first depth-wise separable convolution module to extract the basic feature map; the obtained basic feature map is then passed to two 1×1 convolution layers in parallel to expand the number of channels to four times the original number. The output of the first 1×1 convolution layer is processed by the ReLU6 activation function and then element-wise multiplied with the output of the second 1×1 convolution layer to generate a higher-dimensional feature map; then, the generated higher-dimensional feature map is passed through a 1×1 convolution layer to obtain a higher-dimensional feature map with the number of channels restored to the original number of channels; then, the higher-dimensional feature map with the number of channels restored to the original number of channels is further extracted by a 7×7 second depth-wise separable convolution module to obtain a high-order feature map; finally, the obtained high-order feature map is regularized by the drop_path layer to prevent overfitting, obtaining a regularized high-order feature map, and the obtained regularized high-order feature map is residually connected with the feature map originally input to the StarNet lightweight network to generate the final high-dimensional feature map.

[0157] Among them, the ReLU6 activation function is a linear activation function truncated between 0 and 6, which can be expressed by the following formula: , Represents the output of the first 1×1 convolutional layer.

[0158] Optionally, the path dropping probability of the drop_path layer in each Star Block module can be set to random. Based on this, the diversity and generalization ability of the YOYLv11 model can be enhanced, and the model's anti-interference ability to overfitting can be further improved.

[0159] In the improved YOLOv11 model provided in the second embodiment, the neck module of the YOLOv11 model is also improved. Figure 2 and Figure 3 Explain the structure of the C3k_GD module in the improved YOLOv11 model. Figure 2 This is a structural diagram of the C3k module and C3k_GD module provided in Example 2 of this application, as shown in FIG. Figure 2 As shown, the C3k_GD module provided in this application is obtained by replacing the Bottleneck submodule in the C3k module with the improved Ghost Bottleneck submodule. In addition, the structures of the C3k module and the C3k_GD module also include a Convolution (Conv) submodule and a Concatenate (Concat) submodule.

[0160] It's worth noting that, compared to the original C3k module in the YOLOv11 model's neck, the C3k_GD module in this application incorporates the Ghost Bottleneck submodule. By leveraging the Ghost Bottleneck submodule's ability to split the convolution operation into two steps, this not only effectively reduces computational complexity and achieves a lightweight improvement to the YOLOv11 model, but also maintains the effectiveness of the convolution module's feature extraction, focusing on generating more texture and edge features relevant to weed detection, improving the model's detection accuracy and making the improved YOLOv11 model suitable for tasks requiring efficient feature extraction, such as weed detection.

[0161] Among them, the operation process of the Ghost Bottleneck submodule is as follows:

[0162] In step 1, we aggregate the features of weeds from the input data using a traditional 1×1 convolution operation. This process focuses on feature extraction between channels and generates a feature map with fewer channels by compressing the number of channels.

[0163] It should be understood that the goal of this operation is to reduce the number of channels of the feature map, avoid the generation of redundant features, and ensure that key features are retained.

[0164] Step 2: Apply convolution operation to the feature map with fewer channels obtained in step 1 for further feature extraction.

[0165] It should be understood that this step further extracts texture and edge features related to weed detection through the convolution operation. These features are usually crucial for weed recognition and can help the model accurately locate the shape and edges of weeds.

[0166] Furthermore, in the C3k_GD module of this application, the Ghost Bottleneck submodule is also improved. Figure 3 This is a schematic diagram of the structure of the Ghost Bottleneck submodule and the improved Ghost Bottleneck submodule provided in Example 2 of this application, as shown in FIG. Figure 3 As shown, the improved Ghost Bottleneck submodule included in the C3k_GD module provided in this application is obtained by replacing DWConv in the original Ghost Bottleneck submodule with DConv. In addition, the structure of the Ghost Bottleneck submodule and the improved Ghost Bottleneck submodule also includes a Ghost Module.

[0167] It is worth noting that compared to the original Ghost Bottleneck submodule, the improved Ghost Bottleneck submodule provided in this application replaces DWonv with DConv. DConv dynamically adjusts the convolution kernel weights, enabling the improved Ghost Bottleneck submodule to adaptively adjust the convolution operation according to the input features. This not only significantly increases the number of parameters, allowing the model to learn more complex and richer feature representations, but also avoids an increase in the computing resources required by the model (for example, floating-point operations per second (FLOPS)).

[0168] Among them, DConv achieves more efficient feature expression by dynamically generating convolution kernels. Compared with DWConv, its improvements are as follows:

[0169] Assume that the input feature is , the weight feature is , in the original GhostBottleneck submodule DWConv, the convolution operation is represented as: ,in, is the output feature.

[0170] In the improved Ghost Bottleneck submodule DConv, weight features Replaced by a set of dynamically generated convolution kernels , through dynamic hyperparameters and convolution kernel A linear combination of , its convolution operation is expressed as: , ,

[0171] In the above formula, It is dynamic convolution kernels; is the input feature The relevant dynamic hyperparameters are dynamically generated through global average pooling, MLP network and SoftMax activation. The specific calculation formula is as follows: ,in, , are dynamically generated coefficients.

[0172] In the fence weed detection method provided in an embodiment of the present application, a model for detecting weeds in an image, obtained by training an improved YOLOv11 model, is used as the weed detection model. This improved YOLOv11 model not only replaces the YOLOv11 model's backbone network with the StarNet network, achieving a lightweight improvement to the YOLOv11 model, but also replaces the C3k module in the YOLOv11 model's neck with the C3k_GD module, which incorporates the Ghost Bottleneck submodule and DConv. This not only further improves the lightweight performance of the weed detection model but also effectively alleviates the performance bottleneck caused by the low FLOPs trap, significantly enhancing feature extraction and representation capabilities. This enables the model to run efficiently on resource-limited embedded devices while maintaining excellent detection accuracy, achieving a good balance between computational efficiency and detection precision, and thus providing reliable technical support for weed detection scenarios.

[0173] Figure 4 This is a flow chart of a method for detecting weeds within a fence provided in Example 3 of the present application. Based on the above embodiments, the method for detecting weeds within a fence provided in this embodiment further includes a method for obtaining a weed detection model before using the weed detection model to detect weeds in the foreground area and obtaining the detection results.

[0174] S201: Acquire multiple historical images, wherein each of the multiple historical images includes weeds within a fence.

[0175] In this step, it is necessary to obtain multiple historical images including weeds within the fence.

[0176] Optionally, in order to improve the detection accuracy of the weed detection model, the shooting locations of the multiple historical images can be the same as the shooting locations (latitude, longitude, altitude) of the images to be identified in the subsequent detection process.

[0177] S202: Construct an image dataset based on multiple historical images.

[0178] In this step, it is necessary to construct an image dataset based on the multiple historical images obtained.

[0179] In a specific implementation, constructing an image dataset based on multiple historical images includes the following steps 1-5:

[0180] Step 1: Use Gaussian filtering to denoise multiple historical images separately to obtain multiple denoised images.

[0181] In this step, Gaussian filtering is used to denoise multiple historical images respectively to obtain multiple denoised images to remove noise in the images.

[0182] In a specific implementation, Gaussian filtering is used to denoise multiple historical images separately, which may include the following steps ab:

[0183] Step a: Construct a Gaussian filter and obtain a Gaussian kernel: Based on the size of the historical image and the desired smoothing effect, select a Gaussian filter of an appropriate size (e.g., 3x3, 5x5, 7x7, etc.). Then, calculate the value of each element in the Gaussian filter based on the preset standard deviation σ to form the Gaussian kernel.

[0184] Step b: For each historical image, perform a convolution operation on it using a Gaussian filter. This involves convolving the image with the Gaussian kernel, performing a dot-wise multiplication of the Gaussian filter matrix with each pixel in the image and performing a weighted average. The result of the convolution operation is a new value for each pixel, which is the weighted average of the pixels in its surrounding neighborhood.

[0185] Optionally, since the Gaussian filter cannot completely cover the surrounding pixels, a boundary processing method can be used to extend the image boundary for each historical image. The boundary processing method can be a commonly used boundary processing method in the art, such as zero-filling method, copying edge pixels method, etc. This application does not limit the selection of boundary processing method.

[0186] Step 2: Use the CLAHE algorithm to perform illumination equalization on each denoised image to obtain multiple equalized images.

[0187] In this step, the CLAHE algorithm is used to perform illumination equalization on each denoised image to obtain multiple equalized images to reduce the impact of illumination changes on the quality of historical images.

[0188] In a specific implementation, the CLAHE algorithm is used to perform illumination equalization processing on each denoised image, which may include the following steps cj:

[0189] Step c, dividing the denoised image into multiple small areas: First, divide the denoised image into multiple small sub-blocks, usually in the size of 8×8 or 16×16.

[0190] Step d: For each sub-block, calculate the grayscale histogram of the sub-block, which is specifically achieved by the following formula: ,in, represents the histogram frequency of gray level i; represents the grayscale value of pixel p; Ω represents all pixels in the sub-block; represents the indicator function, .

[0191] Step e: Based on the histogram of the sub-block, calculate the cumulative distribution function of each gray level corresponding to the sub-block: ,in, represents the cumulative distribution function of gray level i, is the histogram frequency of gray level j.

[0192] Step f: clipping the grayscale in the histogram according to a preset contrast clipping threshold, which is achieved by the following formula: ,in, Is the contrast clipping threshold. If the histogram frequency of the gray level in the histogram exceeds , then cut it.

[0193] Step g: Exceed Total number of pixels Evenly distribute it to all gray levels to obtain the corrected histogram.

[0194] in, ;

[0195] The following formula is used for allocation: ,in, represents the histogram frequency of gray level i after correction; L represents the total number of gray levels in the histogram.

[0196] Step h: Calculate the corrected cumulative distribution function of each gray level corresponding to the sub-block: ,in, represents the modified cumulative distribution function of gray level i.

[0197] Step i: Based on the mapping function, the pixel value corresponding to each gray level is mapped to a new range, wherein the mapping formula is as follows: ,in, Represents the minimum value of the modified cumulative distribution function of all gray levels corresponding to the sub-block; Indicates the total number of pixels in the submodule; Represents the pixel value corresponding to the mapped gray level i.

[0198] It should be understood that the function of the above formula is to map the pixel value corresponding to the gray level i in the sub-block to the pixel value range of the new pixel. to maintain the contrast of local areas.

[0199] Step j: Use bilinear interpolation to smooth the transition areas between multiple sub-blocks.

[0200] It should be understood that since the image is divided into multiple sub-blocks, equalization alone will lead to blocky boundary artifacts. Bilinear interpolation (BI) is used to make the pixel values ​​between adjacent sub-blocks transition smoothly to ensure a natural visual effect.

[0201] This is achieved through the following formula:

[0202]

[0203] in, Indicates location The final pixel value of the pixel at ; , , , is the equalized pixel value of the nearest four sub-blocks; and are the normalized interpolation weights: ,in, are the coordinates of the adjacent sub-blocks.

[0204] Step 3: Label the weeds in the multiple equalized images to obtain multiple labeled images.

[0205] In this step, weeds in multiple equalized images need to be labeled to obtain multiple labeled images, so that during the process of training a weed detection model, the model can learn the characteristics of weeds.

[0206] In a specific implementation, a labeling system can be used to manually annotate the weeds in each equalized image using a minimum bounding rectangle (MBR) to generate a labeled dataset. This dataset is stored in the YOLO dataset's annotation format as a txt file and includes the labeled box information for each equalized image. This information includes the box's category and coordinates. The box's category is weeds, and the box's coordinates include the box's center coordinates, width, and height. This coordinate information is used to determine the relative position of the weeds in the image.

[0207] Step 4: Perform data enhancement processing on multiple labeled images to obtain multiple images.

[0208] In this step, a data enhancement method can be used to process multiple annotated images to increase the number of annotated images, obtain multiple images, and further expand the final image dataset.

[0209] Among them, the data enhancement method can be a data enhancement method commonly used in the field of image processing, and this application does not limit this.

[0210] Exemplary data enhancement methods may include at least one of: noise injection, brightness adjustment, random occlusion, rotation, translation, and flipping. The noise injection operation is intended to simulate sensor interference in real environments by adding random noise to the image; the brightness adjustment operation is intended to simulate scene changes under different light intensities by changing the image brightness; the random occlusion operation is intended to enhance the model's robustness to local feature loss by randomly adding occluded areas to the image; the rotation operation is intended to simulate observation environments from different perspectives by applying random angle transformations to the image, thereby improving the model's robustness; the translation operation is intended to enhance the model's adaptability to spatial position changes by randomly translating the image; and the mirror flipping operation is intended to enhance the symmetry and diversity of the data by flipping the image horizontally or vertically, enabling the model to adapt to weeds arranged in different directions and improving its generalization ability to targets.

[0211] Step 5: Build an image dataset based on multiple images.

[0212] In this step, the obtained multiple images need to be constructed into an image dataset.

[0213] This implementation uses the methods in steps 1-5 above to perform denoising, illumination balancing, and data augmentation on multiple historical images, effectively improving the quality of the final constructed image dataset. This ensures that the weed detection model trained on this image dataset has high detection accuracy and robustness.

[0214] S203. Replace the backbone network of the YOLOv11 model with the StarNet network, and replace the C3k module in the neck with the C3k_GD module to obtain an initial weed detection model.

[0215] In this step, you need to replace the backbone network of the YOLOv11 model with the StarNet network and replace the C3k module in the neck with the C3k_GD module to obtain the initial weed detection model.

[0216] The initial weed detection model is the improved YOLOv11 model in the previous embodiment. Its structure and implementation principle are consistent with those in the previous embodiment and will not be described again here.

[0217] S204: Train the initial weed detection model based on the image dataset to obtain a weed detection model.

[0218] In this step, the initial weed detection model needs to be trained based on the image dataset to obtain a weed detection model.

[0219] Optionally, the image dataset can be divided into a training set and a validation set according to a preset ratio. The training set is used to train the initial weed detection model to obtain a weed detection model; the validation set is used to test the performance of the trained weed detection model. The preset ratio can be determined empirically and is not limited in this application. For example, it can be 8:2.

[0220] The fence weed detection method provided in this embodiment obtains multiple historical images containing weeds within a fence and constructs an image dataset based on these images. The backbone network of the YOLOv11 model is then replaced with a StarNet network, and the C3k module in the neck is replaced with the C3k_GD module to obtain an initial weed detection model. Finally, the initial weed detection model is trained based on the image dataset to obtain a weed detection model. This method ensures that the trained weed detection model is lightweight, highly accurate, and robust. Furthermore, by performing denoising, illumination balancing, and data augmentation on the multiple historical images, the quality of the constructed image dataset is effectively improved, further ensuring that the weed detection model trained based on this image dataset has high detection accuracy and robustness.

[0221] Figure 5 This is a schematic diagram of the structure of the weed detection device in the fence provided in the fourth embodiment of the present application, as shown in FIG. Figure 5 As shown, the weed detection device 30 provided in this embodiment includes:

[0222] The first acquiring unit 301 is configured to receive an image to be identified and a shooting position obtained by shooting a fence to be identified and sent by a shooting device;

[0223] A first alignment unit 302 is configured to obtain a target historical image captured at the shooting position from a plurality of pre-acquired historical images, and perform feature alignment between the image to be identified and the target historical image to obtain an aligned image to be identified;

[0224] A first detection unit 303 is configured to perform foreground detection on the aligned image to be identified using a background model corresponding to the shooting location to obtain a foreground area, wherein the background model is a model for foreground and background detection obtained by training a Gaussian mixture model based on multiple historical images shot at the shooting location;

[0225] The second detection unit 304 is configured to detect weeds in the foreground area using a weed detection model to obtain a detection result. The weed detection model is a model for detecting weeds in an image obtained by pre-training a machine learning model.

[0226] The weed detection device 30 in the fence provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.

[0227] Figure 6 This is a schematic diagram of the structure of the weed detection device in the fence provided in Example 5 of the present application, as shown in FIG. Figure 6 As shown, based on the above embodiment, the weed detection device 30 provided in this embodiment further includes:

[0228] A second acquiring unit 305 is configured to acquire a plurality of historical images, each of the plurality of historical images including weeds within the fence;

[0229] A construction unit 306 is configured to construct an image dataset based on the plurality of historical images;

[0230] Establishing unit 307, for replacing the backbone network of the YOLOv11 model with the StarNet network, and replacing the C3k2 module in the neck with the C3k_GD module, to obtain an initial weed detection model;

[0231] The first training unit 308 is configured to train the initial weed detection model based on the image dataset to obtain the weed detection model.

[0232] The third acquisition unit 309 is configured to acquire, for each shooting location, a plurality of historical images shot at the shooting location;

[0233] A second alignment unit 310 is configured to perform feature alignment on the plurality of historical images to obtain a plurality of aligned historical images;

[0234] The second training unit 311 is configured to train a Gaussian mixture model based on the plurality of aligned historical images to obtain a background model corresponding to the shooting position.

[0235] The alarm unit 312 is configured to push an alarm message if the detection result indicates that weeds exist in the fence to be identified, where the alarm message indicates that weeds exist in the fence to be identified.

[0236] In one possible implementation, the weed detection model in the second detection unit 304 is specifically a model for detecting weeds in an image obtained by training an improved YOLOv11 model. The improved YOLOv11 model is obtained by replacing the backbone network of the YOLOv11 model with a StarNet network, and replacing the C3k module in the neck of the YOLOv11 model with a C3k_GD module. The C3k_GD module is obtained by replacing the Bottleneck submodule in the C3k module with an improved Ghost Bottleneck submodule, and the improved Ghost Bottleneck submodule is obtained by replacing the deep convolution DWConv in the Ghost Bottleneck submodule with the dynamic convolution DConv.

[0237] In one possible embodiment, the first alignment unit 302 is specifically used to: obtain the descriptors of the key points of the image to be identified and the descriptors of the key points of the target historical image based on the SIFT algorithm, and perform feature alignment on the image to be identified and the target historical image based on the descriptors of the key points of the image to be identified and the descriptors of the key points of the target historical image to obtain the aligned image to be identified.

[0238] In a possible implementation, the background model in the first detection unit 303 includes multiple Gaussian distributions corresponding to each pixel and a weight of each Gaussian distribution; accordingly, the first detection unit 303 is specifically configured to:

[0239] For each pixel in the aligned image to be identified, obtaining a target number of Gaussian distributions with the largest weight value from the multiple Gaussian distributions corresponding to the pixel, where the target number is determined based on a preset weight accumulation threshold and the weight of each Gaussian distribution in the multiple Gaussian distributions;

[0240] Matching the pixel value of the pixel point with the target number of Gaussian distributions respectively; if the pixel point matches any Gaussian distribution, determining that the pixel point is a background pixel point; if the pixel point does not match any of the target number of Gaussian distributions, determining that the pixel point is a foreground pixel point;

[0241] The foreground area is obtained based on all foreground pixels in the aligned image to be identified.

[0242] In a possible implementation, the construction unit 306 is specifically configured to:

[0243] De-noising the plurality of historical images using Gaussian filtering to obtain a plurality of de-noised images;

[0244] The CLAHE algorithm is used to perform illumination equalization on each denoised image to obtain multiple equalized images;

[0245] Marking weeds in the plurality of equalized images respectively to obtain a plurality of marked images;

[0246] Performing data augmentation processing on the multiple annotated images to obtain multiple images;

[0247] The image dataset is constructed according to the multiple images.

[0248] In a possible implementation manner, the second alignment unit 310 is specifically configured to:

[0249] Based on the SIFT algorithm, descriptors of key points of each of the multiple historical images are obtained respectively, and based on the descriptors of the key points of each historical image, feature alignment is performed on the multiple historical images to obtain multiple aligned historical images.

[0250] In a possible implementation, the first acquiring unit 301 is specifically configured to:

[0251] Receive an image to be identified and a hovering position obtained by photographing the fence to be identified sent by the drone, wherein the photographing position includes the hovering position.

[0252] The weed detection device 30 in the fence provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.

[0253] Figure 7 This is a schematic diagram of the structure of the computer device provided in Example 6 of this application. Figure 7 As shown, the computer device 40 provided in this embodiment includes: at least one processor 401 and a memory 402. Optionally, the computer device 40 further includes a communication component 403. The processor 401, the memory 402 and the communication component 403 are connected via a bus 404.

[0254] In a specific implementation process, at least one processor 401 executes the computer-executable instructions stored in the memory 402, so that the at least one processor 401 performs the above method.

[0255] The specific implementation process of the processor 401 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0256] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0257] Memory can include read-only memory and random access memory. The memory can be volatile or non-volatile, or can include both volatile and non-volatile memory. Non-volatile memory can include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), which is used as an external cache memory. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct RAM bus random access memory (DR RAM).

[0258] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0259] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0260] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0261] The readable storage medium can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as SRAM, EEPROM, EPROM, PROM, ROM, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0262] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.

Claims

1. A method for detecting weeds in a fence, characterized in that: include: Receive the image to be identified and the shooting position obtained by shooting the fence to be identified sent by the shooting device; Acquire a target historical image captured at the shooting position from a plurality of previously acquired historical images, and perform feature alignment between the image to be identified and the target historical image to obtain an aligned image to be identified; Performing foreground detection on the aligned image to be identified using a background model corresponding to the shooting position to obtain a foreground area, wherein the background model is a model for foreground and background detection obtained by training a Gaussian mixture model based on multiple historical images shot at the shooting position; A weed detection model is used to detect weeds in the foreground area to obtain a detection result. The weed detection model is a model for detecting weeds in an image obtained by pre-training a machine learning model.

2. The method according to claim 1, characterized in that The weed detection model is specifically a model for detecting weeds in an image obtained by training an improved YOLOv11 model; The improved YOLOv11 model replaces the backbone network of the YOLOv11 model with the StarNet network, and replaces the C3k module in the neck of the YOLOv11 model with the C3k_GD module; The C3k_GD module is obtained by replacing the Bottleneck submodule in the C3k module with the improved GhostBottleneck submodule, and the improved Ghost Bottleneck submodule is obtained by replacing the deep convolution DWConv in the GhostBottleneck submodule with the dynamic convolution DConv.

3. The method according to claim 1 or 2, characterized in that The step of aligning features of the image to be identified with the target historical image to obtain an aligned image to be identified includes: Based on the SIFT algorithm, the descriptors of the key points of the image to be identified and the descriptors of the key points of the target historical image are respectively obtained, and based on the descriptors of the key points of the image to be identified and the descriptors of the key points of the target historical image, the features of the image to be identified and the target historical image are aligned to obtain the aligned image to be identified.

4. The method according to claim 1 or 2, characterized in that The background model includes multiple Gaussian distributions corresponding to each pixel and the weight of each Gaussian distribution; Accordingly, the foreground detection is performed on the aligned image to be identified using the background model corresponding to the shooting position to obtain the foreground area, including: For each pixel in the aligned image to be identified, obtaining a target number of Gaussian distributions with the largest weight value from the multiple Gaussian distributions corresponding to the pixel, where the target number is determined based on a preset weight accumulation threshold and the weight of each Gaussian distribution in the multiple Gaussian distributions; Matching the pixel value of the pixel point with the target number of Gaussian distributions respectively; If the pixel point matches any Gaussian distribution, the pixel point is determined to be a background pixel point; If the pixel point does not match the target number of Gaussian distributions, then the pixel point is determined to be a foreground pixel point; The foreground area is obtained based on all foreground pixels in the aligned image to be identified.

5. The method according to claim 2, characterized in that Before detecting weeds in the foreground area using the weed detection model and obtaining a detection result, the method further includes: Acquire a plurality of historical images, each of the plurality of historical images including weeds within a fence; constructing an image dataset based on the plurality of historical images; The backbone network of the YOLOv11 model is replaced with the StarNet network, and the C3k module in the neck is replaced with the C3k_GD module to obtain the initial weed detection model; The initial weed detection model is trained based on the image dataset to obtain the weed detection model.

6. The method according to claim 5, characterized in that The constructing of an image dataset based on the plurality of historical images includes: De-noising the plurality of historical images using Gaussian filtering to obtain a plurality of de-noised images; The CLAHE algorithm is used to perform illumination equalization on each denoised image to obtain multiple equalized images; Marking weeds in the plurality of equalized images respectively to obtain a plurality of marked images; Performing data augmentation processing on the multiple annotated images to obtain multiple images; The image dataset is constructed based on the multiple images.

7. The method according to claim 1 or 2, characterized in that Before performing foreground detection on the aligned image to be identified using the background model corresponding to the shooting position to obtain the foreground area, the method further includes: For each shooting location, obtaining a plurality of historical images shot at the shooting location; Performing feature alignment on the multiple historical images to obtain multiple aligned historical images; A Gaussian mixture model is trained according to the multiple aligned historical images to obtain a background model corresponding to the shooting position.

8. The method according to claim 1 or 2, characterized in that The receiving device sends an image to be identified and a shooting position of the fence to be identified, including: Receive an image to be identified and a hovering position obtained by photographing the fence to be identified sent by the drone, wherein the photographing position includes the hovering position.

9. A computer device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

Images