Weed detection and growth point localization method and system based on shared features

Abstract

This invention relates to the field of agricultural intelligent equipment and machine vision technology, and discloses a method for weed detection and growth point localization based on shared features. The method constructs a collaborative dataset containing weed detection labels and growth point labels, utilizes a shared feature extraction network to extract multi-scale features, and sets detection task branches and growth point localization branches in a unified feature space to achieve joint modeling of weed detection and growth point localization. Then, adaptive positive sample selection is performed based on the joint cost between candidate sample points and labeled growth points, and joint training is conducted using detection loss, growth point localization classification loss, and growth point localization regression loss. During the inference stage, weed detection results and growth point localization results are output, and target point information is generated. This method can reduce redundant calculations and error cascading in staged processing, and improve the accuracy of growth point localization and the stability of target point output in complex farmland scenarios.

Description

Technical Field

[0001] This invention relates to the field of agricultural intelligent equipment and machine vision technology, specifically to a method and system for weed detection and growth point localization based on shared features, applicable to precision agriculture operations such as laser weeding. Background Technology

[0002] Laser weeding equipment needs to quickly, accurately, and stably identify field weed targets during continuous operation to provide a reliable basis for subsequent target confirmation, operation path planning, and laser strike control. Compared with traditional spraying or mechanical weeding, laser weeding has higher requirements for the spatial accuracy, response speed, and robustness of the sensing results, especially in situations with dense targets, overlapping plants, leaf shading, changing light, and complex backgrounds, where sensing errors can directly affect strike accuracy and operational effectiveness.

[0003] In this invention, "growth point" refers to a key part of a weed plant with a clear structural semantics, corresponding to the area in the image where the growth trend converges or the structure begins to extend, usually located at the center of the unfolded leaves or the extended branches; "center point" refers to the geometric center position calculated based on the target detection box or local region, often used as an approximate representation of the growth point; "target point" refers to the actual target position of the laser actuator, usually with the growth point or center point as the target.

[0004] Currently, existing visual perception solutions typically employ a phased approach to achieve weed target recognition and target point localization. Based on whether they rely on target detection results as prior information, they can be mainly divided into two categories: one type of method estimates the location based on the target detection results, that is, firstly, the target detection model obtains the weed target detection box information to determine the location, scale, and category of the weed, and then obtains the target point location based on the detection box through key point detection or center point calculation; the other type of method directly models the target point, using the growth point region as the detection object for training, and representing it by constructing a local detection box with a scale slightly larger than the growth point, and approximating the target point location with the center position of this local detection box during the inference stage.

[0005] Both of the above methods have certain limitations in practical applications. For methods based on target detection results, they rely on the detection bounding box as prior information, and the detection and point localization processes are independent, leading to significant computational redundancy, long processing chains, and cascading error propagation between different stages. For methods that directly model target points, although they do not explicitly rely on detection results, they lack effective modeling of the scale, category, and overall structure information of weed targets. In practical applications, they usually still need to be combined with the target detection process to achieve target confirmation and constraint. Therefore, they are essentially still unable to avoid the computational redundancy and error propagation problems caused by the separation of detection and point localization tasks.

[0006] Within the aforementioned phased framework, growth point localization is a key subtask. Its performance is not only limited by the model structure, but also further affected by the complex farmland environment and the distribution characteristics of training samples. In particular, existing methods have significant shortcomings in sample allocation under conditions of occlusion and uneven sample quality.

[0007] In weed grow point localization tasks, complex occlusion is one of the key factors affecting localization performance. When grow points are partially occluded by leaves, crops, or other weeds, existing methods typically define positive samples using fixed neighborhoods, fixed radii, or single spatial rules. These strategies fail to fully utilize the dynamic relationship between samples and target points, easily assigning positive samples to regions semantically unrelated or weakly related to the true grow point. This makes it difficult for the network to stably learn the mapping relationship between the "local visible region and the true grow point," resulting in a significant decrease in localization accuracy in occluded scenarios.

[0008] Therefore, the existing technology has at least the following shortcomings:

[0009] 1. At the structural level, target detection and growth point localization are usually achieved by staged splicing, lacking a collaborative modeling mechanism based on shared features. This makes it difficult to achieve joint optimization of detection and localization tasks in a unified feature space, while also resulting in large computational redundancy, lengthy processing links, and cascading error propagation.

[0010] 2. At the expression level, growth points have structural semantic features. They are approximated by the center of the detection box or the center of a local small box. Essentially, they replace the structural semantic points with the geometric center, which makes it difficult to accurately represent the real target position.

[0011] 3. At the learning level, in scenarios with occlusion, dense areas, and small targets, the positive sample allocation strategy based on fixed rules is difficult to reliably select effective samples that are semantically related to the growth point, resulting in a decrease in localization robustness and accuracy. Summary of the Invention

[0012] The purpose of this invention is to overcome the problems of computational redundancy, error cascading, and insufficient collaborative optimization caused by the phased splicing of detection and point localization tasks in existing technologies, and to provide a collaborative sensing method for weed detection and growth point localization based on shared features. This method collaboratively completes the weed detection and growth point localization tasks in a unified feature space, and improves the growth point localization performance in complex farmland scenarios, especially in occluded scenarios, through an adaptive positive sample selection mechanism.

[0013] To achieve the above objectives, this invention provides a method for weed detection and growth point localization based on shared features, comprising the following steps:

[0014] Step S1: Construct a collaborative dataset for weed detection and growth point localization;

[0015] Images of weeds in farmland are collected, filtered, cleaned, and standardized. Weed detection labels and growth point labels are labeled separately. A one-to-one correspondence between images, detection labels, and growth point labels is established, and training, validation, and test sets are divided.

[0016] Step S2: Construct a collaborative sensing network based on shared features;

[0017] A shared feature extraction network is constructed to map the input image into a multi-scale shared feature map. Based on the multi-scale shared feature map, a detection task branch and a growth point localization branch are constructed respectively. The detection task branch is used to output the location information, scale information and category information of the weed target, and the growth point localization branch is used to output the classification results and location regression results of the growth point candidate samples. A unified collaborative perception model is formed by the shared feature extraction network, the detection task branch and the growth point localization branch.

[0018] Step S3: Perform adaptive positive sample selection;

[0019] For each labeled growth point, the joint cost between the candidate sample point and the labeled growth point is calculated in the growth point localization branch, and a set of positive samples with high semantic relevance to the corresponding labeled growth point is selected from the candidate sample points based on the joint cost.

[0020] Step S4: Jointly train and optimize the collaborative perception model;

[0021] The collaborative perception model is jointly optimized and trained based on detection task loss, point localization classification loss, and point localization regression loss. In some implementations, geometric consistency loss can also be introduced to enhance the geometric constraint relationship between the predicted growth point and the target weed structure.

[0022] Step S5: Output the collaborative sensing results and use them to execute control;

[0023] The image of the farmland to be tested is input into the trained collaborative perception model, which outputs weed detection results and growth point localization results, and generates target point control information for the laser actuator based on the growth point localization results.

[0024] Furthermore, step S1 specifically includes:

[0025] S11. Collect images of farmland weeds under different lighting conditions, different degrees of shading, and different density distribution states;

[0026] S12. Filter, clean and resize the original images, and remove severely blurred, abnormally exposed and missing target images;

[0027] S13. Label the weed detection box and the coordinates of the weed growth point for each image;

[0028] S14. Establish the correspondence between image files, detection label files, and growth point label files;

[0029] S15. Divide the training set, validation set, and test set according to the preset ratio, and perform data augmentation preprocessing.

[0030] Furthermore, step S2 specifically includes:

[0031] S21. Construct a shared feature extraction network to extract shallow texture features, mid-level structural features and deep semantic features from the input image, and output a multi-scale shared feature map.

[0032] S22. Construct a detection task branch and output the bounding box of the weed target, category information and corresponding confidence score based on the multi-scale shared feature map;

[0033] S23. Construct a growth point localization branch, classify and regress candidate samples of growth points based on high-resolution shared feature maps, and output the classification results and location prediction results of candidate growth points.

[0034] S24. Construct a candidate sample point generation module to form a set of candidate sample points based on the feature map grid or sampling rules for subsequent positive sample selection.

[0035] Furthermore, in step S3, the joint cost between the candidate sample point and the labeled growth point is defined as:

[0036]

[0037] in, Indicates the first The candidate sample point and the first The joint cost among the labeled growth points; Indicates the first Coordinates of candidate sample points; Indicates by the first The coordinates of the growth point predicted from each candidate sample point; Indicates the first The coordinates of each growth point are marked; This represents the joint cost weighting coefficient, used to adjust the degree of influence of the original and predicted positions of candidate sample points on the positive sample selection results.

[0038] Furthermore, based on the joint cost, the candidate sample points corresponding to each labeled growth point are sorted, and the top sample points with the smallest joint cost are selected. The candidate sample points are taken as the set of positive samples corresponding to the labeled growth point, and are represented as follows:

[0039]

[0040] in, Indicates the relationship with the first The set of positive samples corresponding to each labeled growth point; This indicates selecting the top sample points from the candidate sample points with the minimum joint cost. 1 sample point.

[0041] Furthermore, the adaptive positive sample selection process described in step S3 simultaneously considers the original spatial distance between the candidate sample point and the labeled growth point, as well as the deviation between the predicted result of the candidate sample point and the labeled growth point. This enables more stable selection of effective positive samples that are semantically related to the real growth point in occluded, dense, and partially visible scenarios, thereby improving the training quality of the point localization task.

[0042] Furthermore, in step S4, the total loss function of the collaborative perception model is defined as:

[0043]

[0044] in, Indicates the total loss; This indicates a loss in the detection task; This represents the point localization and classification loss; This represents the point localization regression loss; This represents the geometric consistency loss; , , and These are the corresponding weight parameters.

[0045] Furthermore, the detection task loss Used to constrain the consistency between weed target detection results and labeled detection boxes; the point localization classification loss The point localization regression loss is used to constrain the consistency between the candidate sample classification results and the positive and negative sample labels. Used to constrain the deviation between the predicted growth point location and the labeled growth point location; the geometric consistency loss This is used to constrain the consistency of the geometric relationship between the predicted growth point and the target weed structure, thereby enhancing the stability and rationality of the positioning results.

[0046] Furthermore, in some implementations, when geometric consistency constraints are not introduced, let At this point, the total loss function degenerates into a weighted combination of the detection task loss and the point localization task loss.

[0047] Furthermore, in step S5, after inputting the farmland image to be tested into the trained collaborative perception model, the target bounding box of weeds, category information and corresponding growth point position can be output simultaneously; target point control information of the laser actuator is generated according to the growth point position, which is used for subsequent galvanometer control or laser strike execution.

[0048] Furthermore, in some implementations, when the growth point localization result meets the preset confidence condition, the growth point is used as the target point; when the growth point localization result does not meet the preset confidence condition, the center of the detection box or the center of a local area can be used as an approximate target point to improve the stability of the target point output in complex scenarios.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. By sharing a feature extraction network and a dual-task collaborative branch, weed detection and growth point localization are unified into the same feature space for joint modeling, reducing redundant calculations and error cascading in the phased splicing process;

[0051] 2. By simultaneously outputting weed detection results and growth point location results, the consistency between target confirmation and target point location is improved, which helps to reduce the difficulty of subsequent execution control.

[0052] 3. By adopting an adaptive positive sample selection strategy, the ability to select effective samples in occluded, dense, and small target scenarios is improved, thereby enhancing the robustness and accuracy of growth point localization.

[0053] 4. Through joint optimization training, the stability and reliability of collaborative sensing results in complex farmland environments have been improved, thereby enhancing the effectiveness of precision operations. Attached Figure Description

[0054] Figure 1 This is a schematic diagram illustrating the growth point localization results and collaborative output effects of existing point localization methods and the method of this invention in occluded scenarios. Figure 1 (a) is a schematic diagram of the growth point localization results of the existing point localization method in an occluded scene. Figure 1 (b) is a schematic diagram of the growth point localization results of the method of the present invention in the same scenario. Figure 1 (c) to Figure 1 (f) is a schematic diagram of the weed detection results, growth point location results and target point generation results output by the method of the present invention.

[0055] Figure 2 This is a flowchart of the overall process of the collaborative sensing method for weed detection and growth point localization of the present invention.

[0056] Figure 3 This is a schematic diagram of the adaptive positive sample selection mechanism of the present invention.

[0057] Figure 4 This is a schematic diagram of the feature extraction network structure in an embodiment of the present invention.

[0058] Figure 5 This is a schematic diagram of the detection task head structure in an embodiment of the present invention.

[0059] Figure 6 This is a schematic diagram of the point positioning task head structure in an embodiment of the present invention.

[0060] in, Figure 1 This is used to illustrate the comparison of localization results between existing methods and the method of the present invention in occluded scenarios, as well as the output effect of the method of the present invention. Figures 2 to 4 Used to illustrate the method flow and key improvement mechanisms of the present invention; Figure 5 and Figure 6 This is used to illustrate the network structure implementation in a preferred embodiment of the present invention. Detailed Implementation

[0061] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that these specific embodiments are only for explaining the present invention and are not intended to limit the scope of protection of the present invention. Without departing from the concept of the present invention, those skilled in the art can make equivalent substitutions or adjustments to the relevant steps, module structures, and parameters, all of which should fall within the scope of protection of the present invention.

[0062] This embodiment is described according to the technical solution steps in the invention content, and the overall correspondence is as follows: Figure 1 (a) and Figure 1 (b) Used to illustrate the comparison between existing point localization methods and the method of the present invention in occluded scenarios. Figure 1 (c) to Figure 1 (f) is used to illustrate the actual output effect of the method of the present invention; Figure 2 The method flow corresponding to steps S1 to S5, Figure 3 This corresponds to the adaptive positive sample selection mechanism in step S3. Figure 4 Corresponding to the shared feature extraction network in step S2, Figure 5 Corresponding to the detection task branch in step S2, Figure 6 This corresponds to the growth point location branch in step S2.

[0063] This embodiment provides a collaborative sensing method for weed detection and growth point localization based on shared features, such as... Figure 2 As shown, the specific steps include the following:

[0064] Step S1: Construct a collaborative dataset for weed detection and growth point localization

[0065] (1) Image acquisition: Use a professional camera to acquire images of weeds in farmland scenes. To ensure sample diversity, the acquisition should cover different light intensities (strong light, cloudy, weak light in the morning and evening), different degrees of occlusion (no occlusion, slight occlusion, heavy occlusion), different density distribution states (sparse, medium, dense) and different imaging angles (top view, side view).

[0066] (2) Image preprocessing: The original images are filtered, cleaned, and standardized to form trainable samples. Preferably, the preprocessing may include sample validity screening, image format standardization, and size and coordinate reference standardization to ensure the consistency between the subsequent labels and the images.

[0067] (3) Labeling: Each image is labeled with a weed detection label and a growth point label. The detection label is used to record the target category and target bounding box information, and the growth point label is used to record the pixel coordinate information of the growth point. Label conflicts are reduced through consistency verification.

[0068] Furthermore, to improve the consistency and operability of growth point labeling, the following rules are preferred for labeling:

[0069] Under unobstructed conditions, mark the growth point at the location where the growth trend converges in the plant structure or in the central area of ​​the unfolded leaves;

[0070] Under partial occlusion conditions, when the growth point region is still identifiable, the location of the growth point is inferred and labeled based on the visible structural information;

[0071] When the growth point is severely obscured or unidentifiable, the growth point can be approximated by combining the overall morphology of the plant and the geometric center of the corresponding detection frame, and marked as a difficult sample to avoid introducing erroneous supervision.

[0072] For weed targets with irregular shapes or multiple branching structures, when there are multiple candidate growth points, it is preferable to select the main node that is consistent with the main growth direction of the plant for labeling.

[0073] In scenarios where targets are densely distributed, it is necessary to ensure that each detected target corresponds to only one growth point annotation.

[0074] The above annotation rules are used to guide the generation process of growth point labels in order to improve annotation consistency and training data quality.

[0075] (4) Data organization structure: Establish a one-to-one correspondence between image files, detection label files, and growth point label files to form a unified data organization structure. For each image, the detection label file contains the bounding box information of all weed targets in the image, and the growth point label file contains the coordinate information of all growth points. This enables the target detection supervision information and point localization supervision information in the same image to be collaboratively correlated.

[0076] (5) Dataset Partitioning and Augmentation: The dataset is divided into training, validation, and test sets according to a preset ratio. During the training phase, data augmentation operations are performed on the training set images. In this embodiment, random horizontal flipping augmentation is preferred to improve the model's adaptability to changes in left and right poses. In addition, random brightness adjustment, contrast adjustment, blurring, and other enhancement strategies can be added as needed.

[0077] (6) Construction of collaborative training samples: Collaborative training samples are constructed by combining detection labels and growth point labels. Among them, detection labels are used to characterize the spatial range and category attributes of weed targets, and growth point labels are used to characterize key locations with structural semantics in weed plants, thereby providing a unified data foundation for the joint training of subsequent detection and point localization tasks.

[0078] Step S1 establishes a unified data organization for collaborative perception, enabling detection and point localization tasks to be jointly trained and uniformly evaluated within the same sample system.

[0079] Step S2: Construct a cooperative sensing network based on shared features

[0080] Step S2 corresponds to Figure 2 The process of "shared feature extraction - collaborative branch - detection branch processing - point localization branch processing - collaborative result fusion and constraint" is described in the article, while also referring to... Figure 4 , Figure 5 and Figure 6 The specific network structure is shown below. First, a unified shared feature extraction network is used to obtain multi-scale features. Then, based on the shared features, detection task branches and growth point localization branches are constructed separately, thereby achieving collaborative modeling of weed detection and growth point localization within the same feature space.

[0081] Step S21: Shared Feature Extraction Network

[0082] In this embodiment, the shared feature extraction network uses a combination of YOLO11n's Backbone and Neck as its basic structure, such as... Figure 4 As shown, the input image first undergoes hierarchical feature extraction through several convolutional modules and the C3k2 module, followed by high-level semantic enhancement through the SPPF and C2PSA modules. Subsequently, multi-scale feature fusion is achieved through upsampling, concatenation, and convolutional fusion structures in the neck network, ultimately outputting three shared feature maps P3, P4, and P5 with different resolutions. The P3 feature map has high spatial resolution and is suitable for representing fine-grained location information such as growth points; the P4 and P5 feature maps have stronger semantic expressive power and are suitable for representing the overall scale, category, and pose information of weed targets.

[0083] in, Figure 4 In this diagram, Conv represents the convolution module, used to perform convolution operations, normalization, and non-linear activation on the input features to extract local texture and semantic features; Concat represents the feature concatenation operation, used to concatenate feature maps of different scales or levels along the channel dimension to achieve the fusion of shallow spatial detail features and deep semantic features. C3k2 represents the convolutional feature extraction module used to enhance feature representation capabilities; SPPF represents the fast spatial pyramid pooling module, used to expand the receptive field and aggregate multi-scale contextual information; and C2PSA represents the attention enhancement module, used to improve the network's ability to represent features in key regions.

[0084] S22: Detection Task Branch

[0085] Please see Figure 5 This branch receives the three shared feature maps P3, P4, and P5 output from step S2, and sets up three parallel prediction subheadings at each scale:

[0086] The regression subheader is used to predict the distance distribution information of the target boundary (relative to preset anchor points or grid points).

[0087] The classification subheader is used to predict the probability score of each anchor point or grid point belonging to different weed categories;

[0088] Angle subheading, used to predict the angle of the detection box.

[0089] Subsequently, the decoding module decodes the distance distribution output by the regression subheading along with the preset anchor points and feature map stride information, and combines this with the angle mapping module to output the rotation box detection result for the detection box. The detection result includes at least the target center position, width and height information, category score, and angle information. This detection task branch is better suited to scenarios with irregular weed morphology, large pose variations, and dense target distribution.

[0090] S23: Growth point positioning branch

[0091] The structure of the growth point-localized branch is as follows: Figure 6As shown, the classification and location regression results of candidate growth point samples are output. In a preferred embodiment, the growth point localization branch preferentially receives the high-resolution shared feature map P3 to retain more spatial detail information. This branch may include a feature adaptation module, a candidate sample point generation module, a classification prediction module, and a location regression module. The feature adaptation module maps the shared features to a representation space more suitable for the point localization task; the candidate sample point generation module generates a fixed set of candidate sample points based on the shared feature map, each candidate sample point corresponding to a set of feature vectors, whose spatial location is predicted by the network based on the feature response and adaptively determined through parameter updates during model training; the classification prediction module determines whether each candidate sample point is related to a real growth point; and the location regression module predicts the coordinate offset between the candidate sample point and the real growth point. After fusing the initial coordinates of the candidate sample point with its predicted offset, the predicted coordinates of the candidate growth point can be obtained.

[0092] S24: End-to-end integration of collaborative sensing models

[0093] The shared feature extraction network, detection task branch, and growth point localization branch described above together constitute a unified, end-to-end trainable collaborative perception model. The input image undergoes only one forward propagation to simultaneously obtain detection results and growth point prediction results. Compared to existing staged stitching methods, in this embodiment, the detection and point localization tasks no longer construct independent feature extraction processes but are jointly modeled in a unified feature space, thereby reducing redundant computation and mitigating the risk of error cascading between different stages.

[0094] Step S3: Perform adaptive positive sample selection

[0095] Step S3 corresponds to Figure 2 The core idea of ​​the point positioning branch processing stage is as follows: Figure 3 As shown, during the training phase of the point localization task, positive samples are no longer selected using fixed neighborhoods, fixed radii, or single spatial rules. Instead, dynamic screening is performed based on the correlation between candidate sample points and real growth points to enhance sample effectiveness under occlusion, density, and local visibility conditions.

[0096] Specifically, for each labeled growth point First, a set of candidate sample points is generated in the growth point localization branch, and the initial coordinates of each candidate sample point are obtained. and its predicted growth point coordinates Then, calculate the first... The candidate sample point and the first The joint cost among the labeled growth points is preferably defined as:

[0097]

[0098] in, Indicates the cost of joint action; Indicates the first Coordinates of candidate sample points; Indicates by the first The coordinates of the growth point predicted from each candidate sample point; Indicates the first The coordinates of each growth point are marked; This represents the joint cost weighting coefficient, used to adjust the degree of influence of the original and predicted positions of candidate sample points on the positive sample selection results.

[0099] After calculating the joint cost between each candidate sample point and the labeled growth point, they are sorted, and the top candidates with the smallest joint cost are selected. Each candidate sample point is used as the set of positive samples corresponding to the labeled growth point. Preferably, the set of positive samples is represented as follows:

[0100]

[0101] in, Indicates the relationship with the first The set of positive samples corresponding to each labeled growth point This indicates selecting the top sample points from the candidate sample points with the minimum joint cost. 1 sample point.

[0102] Through the above adaptive positive sample selection process, the positive sample screening considers both the original spatial distance between the candidate sample point and the labeled growth point, as well as the deviation relationship between the prediction result of the candidate sample point and the labeled growth point. Therefore, compared with the traditional fixed rule positive sample allocation method, it can more stably screen out effective samples that are semantically related to the real growth point in occluded, dense and partially visible scenarios, thereby improving the training quality and growth point localization accuracy of the point localization task.

[0103] Step S4: Jointly train and optimize the collaborative perception model

[0104] Step S4 corresponds to Figure 2 The core of this approach lies in the collaborative result fusion and constraint stage, as well as the joint optimization stage of the model. Under a unified optimization objective, both the detection task branch and the growth point localization branch are simultaneously constrained, enabling them to achieve joint learning and collaborative convergence within a shared feature space.

[0105] In this embodiment, the collaborative perception model is jointly trained based on detection task loss, point localization classification loss, and point localization regression loss. Preferably, the total loss function is defined as:

[0106]

[0107] in, Indicates the total loss; This indicates a loss in the detection task; This represents the point localization and classification loss; This represents the point localization regression loss; This represents the geometric consistency loss; , , and These are the corresponding weight parameters.

[0108] Among them, the loss of detection tasks Used to constrain the consistency between weed detection results and labeled detection boxes; point localization classification loss. Used to constrain the consistency between candidate sample classification results and positive / negative sample labels; point localization regression loss. Used to constrain the deviation between the predicted growth point location and the labeled growth point location; geometric consistency loss. This constraint is used to constrain the geometric consistency between the predicted growth point and the detected target structure, thereby enhancing the stability and rationality of the output results. In some implementations, when geometric consistency constraints are not introduced, it can be set as follows: At this point, the total loss function degenerates into a weighted combination of the detection task loss and the point localization task loss. Preferably, the geometric consistency loss can be defined as:

[0109]

[0110] in, This indicates the number of effectively predicted growth points; Indicates the first One predicted growth point coordinate; This represents the weed detection box corresponding to that growth point; Represents the detection box The center coordinates; , These are the weighting coefficients.

[0111] in, To constrain the predicted growth point to be inside the detection box, it is defined as follows:

[0112]

[0113] in, Represents the detection box Points within a spatial range. When predicting growth points. Located in the corresponding detection box When inside the box, the constraint loss is 0; when predicting the growth point... Located in the corresponding detection box When outside the bounding box, the in-box constraint loss is the minimum distance from the predicted growth point to the detection box space.

[0114] During training, the dataset constructed in step S1 is input into the collaborative perception model constructed in step S2. Detection results and growth point prediction results are obtained through forward propagation. Then, the total loss mentioned above is used for backpropagation and parameter updates, enabling the shared feature extraction network, detection task branch, and growth point localization branch to be jointly optimized. Since the detection branch and the point localization branch share the feature extraction process, joint training can further enhance feature reuse efficiency and improve the semantic consistency between detection and localization results.

[0115] Step S5: Output the collaborative sensing results and use them to execute control.

[0116] Step S5 corresponds to Figure 2 The "output" stage in the middle, Figure 1 (c) to Figure 1 (f) shows a schematic diagram of the actual output effect of the method of the present invention in different scenarios.

[0117] After inputting the image of the farmland to be tested into the trained collaborative perception model, the detection results of weed targets and the corresponding growth point localization results can be output simultaneously. Preferably, the detection results include the bounding box of the weed target, category information, and corresponding confidence score; the growth point localization results include the candidate growth point locations and their confidence scores.

[0118] To generate target point control information that can be directly invoked by the laser actuator, this embodiment further fuses and constrains the detection results and growth point positioning results. Preferably, the candidate growth points output by the point positioning branch can be filtered using the detection frame spatial range, retaining only growth point results that fall within the detection target area or meet preset neighborhood constraints, thereby completing a one-to-one correspondence between weed targets and growth points. In some embodiments, geometric consistency checks can also be performed on candidate growth points in conjunction with the semantic information of the detection frame to further improve the stability of operation point generation.

[0119] In a preferred embodiment, when the growth point positioning result meets a preset confidence level, the growth point is used as the target point of the laser actuator; when the growth point positioning result does not meet the preset confidence level, an approximate target point can be output by combining the center point information of the corresponding weed target area, so as to improve the stability of the target point output in complex scenarios. Finally, control information for the laser actuator is generated based on the target point for subsequent laser strike execution.

[0120] Through the above steps, this embodiment achieves integrated weed detection, growth point localization, and target point generation within the same collaborative sensing framework. Compared to traditional staged splicing methods, this invention reduces redundant calculations and error cascading, improving the stability and reliability of target point output in complex farmland scenarios, thereby enhancing the execution effect of subsequent laser weeding operations.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art can still make various modifications, equivalent substitutions or variations to the technical solutions of the present invention without departing from the spirit and scope of the present invention.

Claims

1. A method for weed detection and growth point localization based on shared features, characterized in that, Includes the following steps: Step S1: Construct a collaborative dataset for weed detection and growth point localization, wherein each image in the dataset contains both weed detection labels and growth point labels; Step S2: Construct a collaborative perception network based on shared features. The collaborative perception network includes a shared feature extraction network, a detection task branch, and a growth point localization branch. The shared feature extraction network is used to map the input image into a multi-scale shared feature map. The detection task branch outputs the location information, scale information, and category information of the weed target based on the multi-scale shared feature map. The growth point localization branch outputs the classification results and location regression results of the growth point candidate samples based on the multi-scale shared feature map. Step S3: Perform adaptive positive sample selection. For each labeled growth point, calculate the joint cost between the candidate sample point and the labeled growth point in the growth point localization branch, and select a set of positive samples from the candidate sample points based on the joint cost. The joint cost includes both the spatial distance between the original position of the candidate sample point and the labeled growth point and the deviation between the predicted position of the candidate sample point and the labeled growth point. Step S4: Jointly train and optimize the collaborative perception model, at least based on the detection task loss, point localization classification loss, and point localization regression loss. Step S5: Input the farmland image to be tested into the trained collaborative perception model, and output the weed detection results and growth point positioning results simultaneously. Then, generate the target point control information of the laser actuator based on the growth point positioning results.

2. The shared feature based weed detection and growth point localization method according to claim 1, characterized in that, In step S3, the joint cost between the candidate sample point and the labeled growth point is defined as: in, Indicates the first The candidate sample point and the first The joint cost among the labeled growth points; Indicates the first Coordinates of candidate sample points; Indicates by the first The coordinates of the growth point predicted from each candidate sample point; Indicates the first The coordinates of each growth point are marked; This represents the joint cost weighting coefficient, used to adjust the degree of influence of the original and predicted positions of candidate sample points on the positive sample selection results.

3. The method of claim 2, wherein the shared feature-based weed detection and growth point localization method further comprises: In step S3, the candidate sample points corresponding to each labeled growth point are sorted according to the joint cost, and the top M candidate sample points with the smallest joint cost are selected as the positive sample set corresponding to that labeled growth point, as shown below: in, Indicates the relationship with the first The set of positive samples corresponding to each labeled growth point; This indicates selecting the top sample points from the candidate sample points with the minimum joint cost. 1 sample point.

4. The method of claim 1, wherein, In step S2, the shared feature extraction network outputs at least three shared feature maps of different resolutions, including a high-resolution feature map, a medium-resolution feature map, and a low-resolution feature map; the growth point localization branch preferentially receives the high-resolution feature map to preserve spatial detail information, and the detection task branch simultaneously receives the three shared feature maps of high-resolution, medium-resolution, and low-resolution feature maps to fuse multi-scale semantic information.

5. The method for weed detection and growth point localization based on shared features according to claim 4, characterized in that, The detection task branch sets a regression subheader, a classification subheader, and an angle subheader at each scale, and outputs the rotation box detection results through the decoding module. The detection results include at least the target center position, width and height information, category score, and angle information.

6. The shared feature based weed detection and growth point localization method of claim 4, wherein, The growth point positioning branch includes: The feature adaptation module is used to map the shared feature map to the feature space of the point localization task; The candidate sample point generation module is used to generate a set of candidate sample points based on the feature map grid. The classification and prediction module is used to determine whether each candidate sample point is related to the true growth point; The location regression module is used to predict the coordinate offset between candidate sample points and the actual growth point; The predicted growth point coordinates are obtained by adding the initial coordinates of the candidate sample points to their predicted coordinate offsets.

7. The shared feature based weed detection and growth point localization method of claim 1, wherein, In step S4, the total loss function of the collaborative perception model is defined as: in, Indicates the total loss; This indicates a loss in the detection task; This represents the point localization and classification loss; This represents the point localization regression loss; This represents the geometric consistency loss; , , and These are the corresponding weight parameters.

8. The shared feature based weed detection and growth point localization method according to claim 7, wherein, The detection task loss Used to constrain the consistency between weed target detection results and labeled detection boxes; the point localization classification loss The point localization regression loss is used to constrain the consistency between the candidate sample classification results and the positive and negative sample labels. Used to constrain the deviation between the predicted growth point location and the labeled growth point location; the geometric consistency loss This is used to constrain the geometric consistency between the predicted growth point and the weed target detection box, so that the predicted growth point is located inside the corresponding detection box or meets the preset spatial constraint conditions.

9. The method for weed detection and growth point localization based on shared features according to claim 7, characterized in that, When no geometric consistency constraint is introduced, let =0, the total loss function degenerates into a weighted combination of the detection task loss and the point localization task loss.

10. The shared feature based weed detection and growth point localization method of claim 1, wherein, In step S5, after inputting the farmland image to be tested into the trained collaborative perception model, the target bounding box of weeds, category information and corresponding growth point position are output synchronously; target point control information of the laser actuator is generated based on the growth point position. Specifically, when the growth point location result meets the preset confidence condition, the growth point is used as the target point; when the growth point location result does not meet the preset confidence condition, the center point of the corresponding weed target detection box or the center point of a local area is used as an approximate target point to improve the stability of the target point output in complex scenarios. The preset confidence condition is used to characterize the reliability of the growth point prediction result, including: the classification confidence of the candidate growth point is not lower than a preset threshold, the classification confidence is output by the classification prediction module of the point location task branch; the regression error between the predicted growth point position and its corresponding candidate sample point is not higher than a preset threshold, the regression error is calculated by the position regression result of the point location task branch; and the predicted growth point and the corresponding weed detection box meet the preset geometric consistency constraint, the geometric consistency constraint is used to limit the predicted growth point to be within the spatial range of the corresponding weed detection box, or the distance between the predicted growth point and the center of the detection box does not exceed a preset threshold.

11. The shared feature based weed detection and growth point localization method of claim 1, wherein, Step S1 specifically includes: S11. Collect images of farmland weeds under different lighting conditions, different degrees of shading, and different density distribution states; S12. Filter, clean, and resize the original images; S13. Mark the weed detection box and the coordinates of the weed growth point for each image, wherein the coordinates of the growth point correspond to the pixel position of the apical meristem or main growth node of the weed plant. S14. Establish the correspondence between image files, detection label files, and growth point label files; S15. Divide the training set, validation set, and test set according to the preset ratio, and perform data augmentation preprocessing.

12. A weed detection and growth point localization system based on shared features, characterized in that, For performing the method according to any one of claims 1 to 11, comprising: The data construction module is used to build a collaborative dataset that includes both weed detection tags and growth point tags; The collaborative sensing network module includes a shared feature extraction network, a detection task branch, and a growth point localization branch, which are used to synchronously output weed detection results and growth point localization results in a unified feature space; An adaptive positive sample selection module is used to dynamically filter positive samples of the growth point localization branch based on a joint cost, wherein the joint cost includes both a spatial distance term and a prediction bias term. The joint training optimization module is used to perform end-to-end joint training of the cooperative perception network based on at least detection task loss, point localization classification loss and point localization regression loss. The output control module is used to generate target point control information for the laser actuator based on the growth point positioning result, and to use the center point of the detection frame as a backup target point when the growth point positioning result is unreliable.

Images