Intelligent monitoring and weeding equipment for gramineous weeds in winter wheat field

Abstract

The invention relates to the technical field of agricultural intelligent equipment, in particular to intelligent monitoring and weeding equipment for gramineous weeds in a winter wheat field, and aims to solve the technical problem of high missing detection rate and false detection rate of a traditional visual identification method caused by similar forms of winter wheat and gramineous weeds in a seedling stage. The equipment comprises an autonomous mobile platform, a multi-mode sensing assembly, a data processing and decision-making unit and a targeted clearing execution mechanism. The multi-mode sensing assembly integrates a hyperspectral imaging module, a three-dimensional shape measurement module and a polarization characteristic analysis module, and pixel-level data registration is achieved in combination with the cooperative illumination system. The data processing unit performs high-confidence classification decision on the multi-source features through a hierarchical Bayesian fusion model; and the targeted clearing mechanism adopts a laser galvanometer system to carry out non-contact precise clearing on the identified weed growing points. Intelligent identification and efficient physical weeding of weeds in the winter wheat field are achieved, and the method has the advantages of being free of chemical residues and high in operation continuity.

Description

Technical Field

[0001] This invention relates to the field of intelligent agricultural equipment technology, and in particular to an intelligent monitoring and weeding device for grassy weeds in winter wheat fields. Background Technology

[0002] As one of the core crops for ensuring national food security, winter wheat occupies a pivotal position in my country's agricultural production system. Effective field management throughout the entire growth cycle of winter wheat is crucial for ensuring its final yield and quality, with weed control being particularly important. Among the many types of weeds, grasses (gramineous weeds) pose the most direct and intense competition to winter wheat due to their close biological classification and similar growth habits, competing for nutrients, water, light, and growing space. They severely threaten the growth and development of winter wheat and are one of the primary biological stressors leading to yield loss. Therefore, the precise and efficient monitoring and removal of grasses in winter wheat fields has always been a core issue that modern agricultural technology continues to focus on and strive to address.

[0003] To address this challenge, agricultural equipment technology has evolved from traditional manual and mechanical weeding to chemical weeding. Particularly in the field of chemical weeding, the development of automation and machine vision technologies has given rise to intelligent weeding equipment aimed at precision spraying. These devices typically incorporate image acquisition units (such as industrial cameras) and central processing systems. Their basic working principle involves acquiring farmland images through aerial photography or ground inspection, and then analyzing these images using image processing algorithms. Specifically, the core of this technology lies in utilizing the color differences between crops and soil within the visible spectrum. By setting specific color thresholds or constructing color indices such as supergreen features (ExG), the technology segmentes and extracts green vegetation areas, thereby controlling the spraying system to apply herbicides only to the identified vegetation areas. Compared to indiscriminate, full-coverage "blanket" spraying of the entire farmland, this target area identification-based approach reduces herbicide overuse, lowers production costs and environmental pollution, representing a significant advancement in farmland weed management technology at the time.

[0004] However, with the deepening of the precision agriculture concept and the increasingly stringent requirements for recognition accuracy and operational efficiency in application scenarios, the aforementioned target recognition technologies based on macroscopic color differences or simple morphological features have inherent limitations at the principle level. These limitations are gradually becoming apparent when dealing with the specific and complex symbiotic scenario of grassy weeds in winter wheat fields. The reason for this is that the effectiveness of these technologies highly depends on significant and stable visual feature differences between the target (weeds) and the background (soil) or non-target (crop). However, during the critical window for weed control in winter wheat from the seedling stage to the tillering stage, as plants belonging to the same grass family, weed seedlings and winter wheat seedlings exhibit extremely high consistency and similarity in macroscopic visual features such as plant type, leaf morphology, color, and texture. Their spectral reflectance curves in the visible light band highly overlap, making it almost impossible for traditional RGB color space-based segmentation algorithms to effectively distinguish between them. This inherent, high degree of morphological similarity directly leads to an irreconcilable technical contradiction in the practical application of existing technologies: if the judgment threshold of the identification algorithm is relaxed to ensure that no weeds are missed, the system is very likely to misclassify a large number of winter wheat seedlings as weeds, resulting in the accidental spraying of herbicides, causing phytotoxicity to crops, and directly affecting yield; conversely, if the judgment threshold is tightened to protect crops, a large number of weeds with very similar morphology to crops will inevitably be missed, allowing them to continue to grow and ultimately losing the best opportunity for early removal, creating a situation of "nurturing a tiger to cause trouble." This trade-off between the missed detection rate and the false detection rate constitutes the core technical bottleneck restricting the effectiveness of the application of existing technologies.

[0005] Furthermore, this limitation is not merely an algorithmic issue, but rather stems from the singular nature of its information acquisition methods. Traditional visual recognition systems acquire only two-dimensional planar information and limited spectral information, lacking the ability to perceive multi-dimensional information such as the fine structure and internal physiological state of plants. This fundamentally determines its inability to accurately identify species with highly similar morphologies. Therefore, simply optimizing image processing algorithms within the existing technological framework is insufficient to fundamentally overcome this bottleneck. Against this backdrop, how to transcend the traditional scope of macroscopic visual recognition and establish a technical system capable of high-precision, high-reliability real-time identification and differentiated processing of winter wheat and gramineous weeds during their early symbiotic stage, when their morphologies are highly similar, has become a key challenge and an urgent technical problem for those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent monitoring and weeding device for grassy weeds in winter wheat fields. It aims to solve the technical problem in the background art that the high similarity between winter wheat and grassy weeds in terms of macroscopic characteristics such as seedling morphology and color leads to an irreconcilable contradiction between the false detection rate and the missed detection rate of traditional visible light vision-based identification methods, thus making it impossible to achieve accurate and efficient weed removal.

[0007] To achieve the above-mentioned objectives, the present invention provides an intelligent monitoring and weeding device for grassy weeds in winter wheat fields, comprising: An autonomous mobile platform used to move along a preset path in winter wheat fields; A multimodal sensing assembly is fixed to the autonomous mobile platform and is used to synchronously acquire multidimensional, high-resolution composite feature data of a single plant in the field of view below it. The data processing and decision unit is electrically connected to the multimodal sensing assembly and is used to process the composite feature data in real time, identify the species of the target plant based on a preset fusion classification model, and generate a high-confidence classification decision instruction and the target three-dimensional spatial coordinates. The system also includes a targeted removal mechanism electrically connected to the data processing and decision-making unit, used to perform targeted, non-contact physical removal operations on plants identified as grass weeds according to the classification decision instructions and the target's three-dimensional spatial coordinates.

[0008] Furthermore, the multimodal sensing assembly includes a collaborative illumination system, a hyperspectral imaging module, a three-dimensional topography measurement module, and a polarization characteristic analysis module; The hyperspectral imaging module, the three-dimensional morphology measurement module, and the polarization characteristic analysis module are all rigidly mounted on a reference frame with a predetermined spatial position relationship to ensure that the field of view centers of each module coincide and the imaging plane is confocal, thereby achieving pixel-level data registration for the same target plant. The collaborative illumination system, integrated on the reference architecture, provides a stable and controlled active light source for the three modules. It includes: a broadband linear light source array for the hyperspectral imaging module, composed of multiple halogen lamps, which, after being shaped by a homogenizing lens group, forms a uniformly illuminated linear light band across the direction of the device's movement on the ground, with a spectral coverage range of 400 nm to 1000 nm; a structured light projection unit for the three-dimensional topography measurement module, employing a vertical-cavity surface-emitting laser (VCSEL) array with a center wavelength of 850 nm as the light source, projecting the laser beam into a structured light pattern containing pseudo-random speckle or de Bruin sequence encoding through a diffractive optical element (DOE). This infrared band selection avoids spectral interference to the data acquisition of the hyperspectral imaging module in the visible and near-infrared bands; and a polarization illumination unit for the polarization characteristic analysis module, consisting of a high-brightness white light-emitting diode (LED) surface light source and a linear polarizer covering it, with the transmission direction of the linear polarizer fixed.

[0009] In a preferred embodiment of the present invention, the hyperspectral imaging module is a pushbroom imaging system, which internally includes a front objective lens, a slit aperture, a collimating lens, a transmission grating beam splitter, and an array image sensor. The module observes the ground along the device's travel direction (i.e., the pushbroom direction), and its spectral detection range matches the spectral range of the broadband linear light source array of the cooperative illumination system, i.e., 400 nm to 1000 nm. The spectral resolution is set to be no less than 5 nm, thereby enabling the acquisition of spectral reflectance data of the target plant in hundreds of consecutive narrow bands, forming a hyperspectral data cube. The spatial resolution of the array image sensor is 1920 × 1080 pixels. By precisely controlling the travel speed of the autonomous moving platform and the frame rate of the sensor, the ground sampling distance is ensured to be equal to 1 mm in both the travel direction and the vertical travel direction.

[0010] Furthermore, the three-dimensional topography measurement module is constructed based on the principle of active binocular structured light, which includes the structured light projection unit and a pair of infrared image acquisition units symmetrically arranged on both sides of it; each infrared image acquisition unit includes an imaging lens equipped with an 850-nanometer narrowband filter and a global shutter complementary metal-oxide-semiconductor (CMOS) image sensor with a resolution of 1280×1024 pixels; the optical axes of the two infrared image acquisition units are parallel to each other, and their baseline distance is precisely set to 200 mm; by synchronously capturing plant surface images modulated by structured light patterns and using triangulation to solve the three-dimensional coordinates of spatial points, the module can reconstruct a three-dimensional point cloud model of the target plant with sub-millimeter precision, and its depth direction measurement accuracy reaches 0.2 mm.

[0011] In a preferred embodiment of the present invention, the polarization characteristic analysis module employs an image sensor with an integrated micro-polarizer array. Each 2×2 pixel unit of this sensor is covered with linearly polarized microlenses in four different directions: 0 degrees, 45 degrees, 90 degrees, and 135 degrees. Under the illumination of the polarization illumination unit, the module can simultaneously capture image information in four polarization directions in a single exposure, with an effective sensor resolution of 2048×1088 pixels. By calculating the intensity information of these four sub-images, the Stokes parameters (S0, S1, S2), degree of linear polarization (DoLP), and angle of polarization (AoLP) of the reflected light from the target plant surface can be calculated in real time, thereby characterizing non-morphological features such as the thickness and uniformity of the wax layer on the plant leaf surface, as well as surface micro-roughness.

[0012] Furthermore, the data processing and decision-making unit is an embedded system built into a ruggedized industrial computer casing. Its core hardware includes: an octa-core central processing unit (CPU) with a main frequency of not less than 3.5 GHz; a graphics processing unit (GPU) with no less than 2048 parallel processing cores and a video memory capacity of not less than 8 gigabytes, used to accelerate subsequent data-intensive computing tasks; 32 gigabytes of dual-channel error-correcting code memory (ECC RAM); and a data acquisition card connected to the multimodal sensing assembly via a CameraLink HS or CoaXPress high-speed data interface. The data processing and decision-making unit internally runs a complete set of preset data processing and classification decision-making algorithms.

[0013] As the core part of the technical solution of this invention, the data processing and classification decision algorithm specifically includes the following steps executed in sequence: Multimodal data preprocessing and registration are performed: For the received hyperspectral data cube, dark current noise subtraction, sensor response non-uniformity correction, and radiometric calibration based on standard whiteboard reference data are performed to convert it into absolute reflectance data; for binocular infrared image pairs, lens distortion correction is performed; for polarized images, bad pixel repair and noise filtering are performed; based on the pre-determined intrinsic and extrinsic parameters and rigid transformation matrix between each sensor, the hyperspectral data, 3D point cloud data, and polarization characteristic spectrum data are all projected onto the same world coordinate system to achieve pixel-level precise alignment in space, ensuring that the features extracted in each dimension all originate from the same physical location of the plant; Perform parallel extraction of multimodal feature vectors: Firstly, in the hyperspectral data dimension, for single-plant areas where vegetation segmentation has been completed, the average spectral reflectance curve is extracted as a primary feature. Simultaneously, a series of narrow-band vegetation indices are calculated, and specifically, the "Wheat-Weed Physiological Differentiation Index (WWSDI)," defined in this invention for enhancing the difference between winter wheat and gramineous weeds, is calculated. Its mathematical expression is as follows: ,in This index represents the reflectance at a center wavelength of X nanometers. The index was constructed by utilizing the subtle differences in reflectance between the two at the red edge (around 755 nm), the chlorophyll absorption valley (around 690 nm), and the absorption peaks of carotenoids and chlorophyll b (around 445 nm). Secondly, in the dimension of three-dimensional morphological data, the reconstructed plant point cloud model is analyzed to extract a set of quantitative parameters describing the micro-morphological structure. This set of parameters constitutes a three-dimensional feature vector. The specific parameters include: the maximum height of the plant, the equivalent diameter of the stem at 2 mm from the soil surface, the number of tillering nodes, the histogram of the average tilt angle distribution of leaves, and the standard deviation of the angle between the normal vector of all leaf surfaces and the line connecting its centroid. This standard deviation is used to quantify the degree of leaf curling. Third, in terms of polarization characteristic data, statistical features such as the mean DoLP, variance DoLP, and mean AoLP gradient magnitude within the plant region are extracted from the calculated linear polarization degree (DoLP) and polarization angle (AoLP) spectra. These features are directly related to the physical properties of the cuticle layer on the leaf surface.

[0014] Performing a hierarchical Bayesian fusion classification decision is the core discrimination mechanism of this invention, which consists of two levels: The first layer is the base classifier layer: three specialized base classifiers are trained and deployed for the feature vectors of the three different modalities mentioned above. Specifically, a one-dimensional convolutional neural network (1D-CNN) is used for classification of the hyperspectral curve composed of hundreds of band reflectance; a gradient boosting decision tree (GBDT) model is used for classification of the three-dimensional micromorphological feature vector composed of multiple discrete scalars; and a support vector machine (SVM) is used for classification of the statistical feature vector extracted from DoLP and AoLP maps. Each base classifier independently processes the input features and outputs a preliminary probability estimate of whether the target plant is winter wheat or a gramineous weed. The second layer is the fusion decision layer: the probability estimates output by the three base classifiers (e.g., each classifier outputs a two-dimensional vector [P(wheat), P(weed)]) are concatenated to form a higher-dimensional fusion feature vector. This fusion feature vector is input into a pre-trained Bayesian network classifier. This Bayesian network explicitly learns and encodes the conditional dependencies between evidence provided by different perceptual modalities, as well as the joint probability distribution between this evidence and the final plant category. Through probabilistic inference in the Bayesian network, the posterior probability of the target plant belonging to the grass family weeds is calculated, and a confidence score is given. The data processing and decision unit determines the target to be a weed and generates a clearing instruction only if and only if the posterior probability is greater than a preset decision threshold (e.g., 0.95). This hierarchical fusion architecture can effectively integrate the discriminative information of different physical dimensions, greatly improve the robustness and accuracy of classification decisions, and overcome the limitations of a single information source.

[0015] Once the data processing and decision-making unit generates a clearing command, the targeted clearing execution mechanism is activated. This mechanism includes a continuous-wave ytterbium-doped fiber laser with an output power of 100 watts and a center wavelength of 1070 nanometers; a two-dimensional high-speed galvanometer scanning system containing two reflective mirrors driven by servo motors for precisely controlling the exit angle of the laser beam; and a focusing lens group. The data processing and decision-making unit transmits the three-dimensional world coordinates of the target weed's growth point (i.e., meristem, whose three-dimensional coordinates have been precisely extracted from the three-dimensional morphology data) to the controller of the targeted removal actuator. The controller calculates the corresponding galvanometer deflection angle and drives the galvanometer system to precisely guide the focused laser beam to the growth point. The laser beam remains at the target point for a preset pulse time, which is set between 50 and 100 milliseconds, and the laser spot diameter is controlled at 1.5 millimeters. During this period, the high-energy-density laser beam can instantly heat the plant tissue in the target area to above the protein denaturation temperature, causing irreversible necrosis of its meristem, thereby achieving the purpose of physical removal. Moreover, this process leaves no chemical residue and has no impact on the surrounding soil or adjacent winter wheat plants.

[0016] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: During operation, the autonomous mobile platform of this invention moves at a constant speed of 0.5 m / s along the winter wheat planting row. The multimodal sensing assembly continuously scans and collects data from the crop row below. The data processing and decision-making unit processes the collected data stream in real time in a pipeline manner in the background, completing the entire process from data acquisition, preprocessing, feature extraction to fusion decision for each individual plant within the field of view. Once a weed is identified with high confidence, its three-dimensional coordinates are locked and transmitted to the targeted removal execution mechanism. Due to the high-speed response capability of the galvanometer system (angular velocity up to 500 radians / second), the laser removal action can be completed in a very short time. The entire closed-loop cycle of "identification-decision-execution" is controlled within 200 milliseconds, thereby ensuring the continuous and efficient operation capability of the equipment during its movement. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall structure of an intelligent monitoring and weeding device for grassy weeds in winter wheat fields according to the present invention; Figure 2 This is a system control block diagram of the device described in this invention; Figure 3 This is a schematic diagram of the structure of the multimodal sensing assembly described in this invention; Figure 4 This is a flowchart illustrating the data processing and classification decision-making method described in this invention; Figure 5 This is a schematic diagram illustrating the working principle of the targeted removal actuator described in this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] Reference Figure 1 and Figure 2 As shown, this invention discloses an intelligent monitoring and weeding device for grassy weeds in winter wheat fields. This device aims to achieve high-precision and high-efficiency automated targeted removal of grassy weeds that are macroscopically similar to winter wheat seedlings through deep fusion of multimodal sensing information and intelligent decision-making. In a specific embodiment, the overall architecture of the device includes an autonomous mobile platform, a multimodal sensing assembly fixed thereon, a data processing and decision-making unit responsible for data processing and decision-making, and a targeted removal execution mechanism for performing the removal operation. These four core components work closely together through an internal data bus and control link to form a complete "perception-decision-execution" closed-loop operating system.

[0020] Specifically, the autonomous mobile platform is a four-wheel drive, differential steering electric chassis equipped with a high-precision real-time dynamic (RTK-GNSS) positioning module and an inertial measurement unit (IMU). It can autonomously navigate between winter wheat planting rows with a tracking accuracy better than 2 centimeters, based on a pre-planned farmland operation path. The platform's designed travel speed is constant at 0.5 meters per second during operation to ensure that the subsequent multimodal sensing assembly can perform continuous, non-overlapping, and complete ground scanning at a fixed time reference.

[0021] The multimodal sensing assembly serves as the core information acquisition unit of this invention, and its overall structure is as follows: Figure 3 As shown, the assembly is rigidly mounted on a cantilever bracket at the front of the autonomous mobile platform, with its optical observation window maintained at a constant height of 0.8 meters above the ground. All internal sensing modules are precisely fixed to a CNC-machined, one-piece reference frame made of 6061-T6 aerospace-grade aluminum alloy. This frame, optimized through finite element analysis, possesses extremely high rigidity and thermal stability, and all internal surfaces undergo matte blackening anodizing treatment to minimize internal stray light reflection. This integrated rigid mounting design ensures that the relative spatial poses of the hyperspectral imaging module, the 3D topography measurement module, and the polarization characteristic analysis module remain constant under conditions of vibration and temperature changes during equipment operation. Before leaving the factory, multi-module synchronous imaging of a high-precision 3D calibration target has accurately determined and solidified the rigid transformation matrix between the coordinate systems of each module, thus providing a foundation for subsequent pixel-level multimodal data registration.

[0022] Furthermore, the multimodal sensing assembly integrates a collaborative illumination system. This system provides stable, uniform, and spectrally controlled active illumination to both active and passive sensing modules, thus completely eliminating the impact of variations in natural lighting conditions on data quality. This collaborative illumination system consists of three functionally independent light source units. One is a broadband linear light source array serving the hyperspectral imaging module. This array comprises 20 10-watt bromotungsten halogen lamps arranged closely along a straight line perpendicular to the device's direction of travel, with a total length of 1.2 meters. The light emitted by the lamps is shaped by a carefully designed set of cylindrical homogenizing lenses, forming a 1-meter-wide linear light band on the ground 0.8 meters from the light source, with a width of only 5 millimeters along the device's direction of travel. The illuminance uniformity within this light band is better than 97%. The spectral output range of this light source array is precisely controlled between 400 nanometers and 1000 nanometers, perfectly matching the detection range of the hyperspectral imaging module. The second component is the structured light projection unit serving the 3D topography measurement module. The core light-emitting element of this unit is a 10×10 array of vertical-cavity surface-emitting lasers (VCSELs) with a center wavelength of 850±5 nm and a total output power of 2 watts. The coherent laser beam emitted by the VCSEL array passes through a computer-generated diffractive optical element (DOE) to project a 1 m × 0.6 m structured light pattern containing tens of thousands of pseudo-random speckles onto the ground target area. The 850 nm near-infrared band was chosen because it is at the edge of the spectral detection range of the hyperspectral imaging module and far from the main absorption band of plant chlorophyll; therefore, the projected structured light pattern will not significantly contaminate the hyperspectral reflectance data. The third component is the polarization illumination unit serving the polarization characteristic analysis module. This unit consists of a 20 cm × 20 cm high-brightness, high-uniformity white LED backlight panel and a dichroic linear polarizer with high transmittance and an extinction ratio better than 1000:1 covering it. The illumination unit provides a large-area linearly polarized light field with a fixed polarization direction, which is used to actively detect the depolarization effect of the target plant surface on the incident polarized light.

[0023] In a preferred embodiment of the present invention, the hyperspectral imaging module employs a push-broom imaging system. Its optical system consists of a front objective lens with a focal length of 35 mm and an F-number of 2.8, a precision slit aperture with a width of 25 micrometers, a collimating lens for collimating light passing through the slit into parallel light, a transmission grating spectrometer using a volume phase holographic grating (VPHG) as the core spectroscopic element, and a scientific-grade CMOS image sensor with an array of 1920 × 1080 pixels. As the device travels along its path, the module continuously scans and images the ground. Its spectral detection range covers 400 nm to 1000 nm; through the spectroscopic effect of the grating, this range can be discretized into 512 consecutive narrow-band channels, thereby achieving an excellent spectral resolution of approximately 1.17 nm. By precisely synchronizing the 0.5 m / s travel speed of the autonomous mobile platform 1 with the 500 Hz frame rate set by the CMOS sensor, the ground pixel size can be ensured to be accurately 1 mm in both the travel direction and the vertical travel direction (slit direction), that is, a ground sampling distance of 1 mm × 1 mm (GSD) is achieved, providing a high spatial resolution data foundation for subsequent refined species identification.

[0024] Furthermore, the three-dimensional topography measurement module is constructed based on the principle of active binocular structured light triangulation. In addition to the aforementioned structured light projection unit, it also includes a pair of infrared image acquisition units symmetrically arranged on both sides of the projection unit. Each acquisition unit consists of a 16mm focal length megapixel-level industrial lens and a global shutter CMOS image sensor with a resolution of 1280×1024 pixels and a pixel size of 4.8 micrometers. A narrow-band bandpass filter with a center wavelength of 850 nm and a full width at half maximum (FWHM) of 10 nm is mounted at the front of the lens. Its function is to allow only the laser reflection signal modulated by the structured light pattern to pass through, while effectively filtering out interference from ambient light and other light sources in the co-illumination system. The optical axes of the two infrared image acquisition units are strictly parallel, and the physical distance between their optical centers, i.e., the baseline length, is precisely marked as 200.0 ± 0.1 mm. When the equipment is operating, the two acquisition units synchronously capture structured light speckle images modulated on the plant surface. Due to the difference in left and right viewing angles, there is parallax in the pixel coordinates of the same spatial point in the two images. By calculating the disparity value of each pixel using a subpixel-level stereo matching algorithm, and combining it with pre-calibrated camera intrinsic and extrinsic parameters and baseline length, the three-dimensional spatial coordinates of each pixel in the field of view can be accurately calculated using the principle of triangulation. This allows for the reconstruction of a three-dimensional point cloud model of the target plant surface with sub-millimeter precision, and the root mean square error of the measurement in the depth direction (Z-axis) is no higher than 0.2 mm.

[0025] In a preferred embodiment of the present invention, the polarization characteristic analysis module employs a technologically advanced image sensor with an integrated micro-polarizer array, specifically a Sony IMX250MZR. This sensor integrates a 2×2 superpixel unit composed of linearly polarized microlenses in four different directions (0°, 45°, 90°, and 135°) in the traditional Bayer color filter array position. Under linearly polarized light illumination from the polarization illumination unit 12, the sensor can simultaneously capture image information from four independent polarization channels in a single exposure. Its effective resolution is 2048×1088 pixels. The four sub-images acquired... The intensity information can be used to calculate the Stokes parameters of the entire field in real time. Based on these parameters, two key polarization physical quantities can be further derived: linear polarization degree (…). ) and polarization angle ( DoLP characterizes the proportion of linearly polarized components retained in reflected light and is closely related to the thickness, water content, and surface micro-roughness of the plant leaf surface; while AoLP reflects the polarization direction of reflected light and is related to the micro-geometric orientation of the leaf surface. This information provides a new physical dimension beyond morphology for distinguishing winter wheat from grass weeds.

[0026] The data processing and decision-making unit, the "brain" of the entire device, is integrated into a ruggedized industrial computer casing with IP67 protection rating to withstand harsh field operating environments. Its core hardware configuration includes: an Intel Core i7-11850HE eight-core, sixteen-thread processor with a base frequency of 3.5 GHz and a turbo boost up to 4.7 GHz; an NVIDIA RTX A4000 embedded graphics processor with 2560 CUDA cores and 8 gigabytes of GDDR6 memory, providing powerful parallel computing capabilities for subsequent deep learning model inference and point cloud processing; 32 gigabytes of dual-channel DDR4 ECC memory to ensure stability and reliability during data processing; and a data acquisition card supporting dual-channel CoaXPress CXP-12 high-speed interfaces for bottleneck-free reception of raw data streams from the three sensor modules in the multimodal sensing assembly. The unit runs a customized software system based on the Ubuntu 20.04 LTS operating system and applies the PREEMPT_RT real-time kernel patch, on which the core data processing and classification decision algorithm of the present invention is deployed.

[0027] Reference Figure 4The data processing and classification decision algorithm is a meticulously designed, serially executed data processing pipeline. When the raw data stream enters the decision unit, preprocessing and registration of the multimodal data are performed first. For the data cube output by the hyperspectral imaging module, the dark current image acquired at the same integration time is first subtracted to eliminate sensor dark noise. Then, it is divided by the pre-acquired flat-field image to correct sensor pixel response non-uniformity (PRNU) and lens vignetting. Finally, by dividing by the data obtained from synchronous imaging of a standard Lambertian white board, the original digital quantization value (DN) is converted into physically meaningful absolute reflectance data. For the binocular infrared image pairs output by the 3D topography measurement module, the camera intrinsic parameters and distortion coefficients (including radial and tangential distortion) obtained using the Zhang Zhengyou calibration method are used to perform distortion correction on the images. For the raw four-channel images output by the polarization characteristic analysis module, a median filtering algorithm is used to remove salt-and-pepper noise, and bad pixels are repaired through interpolation. After completing their respective preprocessing, based on the rigid transformation matrix calibrated at the factory, the hyperspectral reflectance cube, three-dimensional point cloud, and polarization characteristic maps (DoLP and AoLP maps) are all transformed into a unified world coordinate system with the optical center of the hyperspectral imaging module as the origin, achieving pixel-level precise alignment in space.

[0028] Next, the algorithm enters the parallel extraction stage of multimodal feature vectors. This stage operates on single plant data regions that have been spatially aligned and segmented from the soil background using a simple thresholding method. In the hyperspectral data dimension, the average spectral reflectance curve of all pixels within the plant region is first calculated; this is a high-dimensional feature containing 512 data points. In addition, a series of known narrow-band vegetation indices sensitive to vegetation physiological states are calculated, such as the Normalized Difference Vegetation Index (NDVI) and the Photochemical Reflectance Index (PRI). Specifically, this invention defines and calculates a "Wheat-Weed Physiological Differentiation Index (WWSDI)" specifically designed to enhance the subtle physiological differences between winter wheat and gramineous weeds; its calculation formula is as follows: ,in The index represents the reflectance at a center wavelength of X nanometers. The design principle of this index is that winter wheat and some grass weeds (such as Alopecurus aequalis) have statistically significant but small absolute differences in the steep slope position of the red edge region (around 755 nm), the depth of the main absorption valley of chlorophyll (around 690 nm), and the position of the absorption peak of carotenoids and chlorophyll b in the blue light band (around 445 nm). WWSDI amplifies this difference through nonlinear combination. In the dimension of three-dimensional morphological data, geometric analysis was performed on the reconstructed plant point cloud model to extract a set of quantitative parameters describing its micromorphology, including: the height of the highest point of the plant canopy above the ground; the equivalent stem diameter obtained by fitting a Hough transform at a height of 2 mm above the soil surface; the number of tillering nodes identified by DBSCAN clustering of the point cloud; the peak value and width of the histogram of the average leaf tilt angle distribution calculated by fitting a plane using the RANSAC algorithm after segmenting the leaf point cloud; and the standard deviation of the angle between the normal vector of all leaf surfaces and the line connecting it to its centroid. The latter serves as a quantitative indicator characterizing the degree of leaf curling or flattening. In the dimension of polarization characteristic data, the mean, variance, skewness, and kurtosis of the linear polarization degree (DoLP) and the mean gradient magnitude of the polarization angle (AoLP) within the plant region were extracted from the calculated DoLP and AoLP maps. These statistical features collectively depict the physical properties of the plant leaf surface.

[0029] Then, the algorithm executes the core discrimination mechanism of this invention—a hierarchical Bayesian fusion classification decision. This mechanism consists of two layers. The first layer is the base classifier layer, where three specialized base classifiers are trained and deployed for the feature vectors of the three modalities mentioned above. Specifically, for the hyperspectral average reflectance curve with a dimension as high as 512, a one-dimensional convolutional neural network (1D-CNN) containing three one-dimensional convolutional layers and two fully connected layers is used for classification. For the three-dimensional micromorphological feature vector composed of more than a dozen scalars, a gradient boosting decision tree (GBDT) model is used, which consists of 100 decision trees with a learning rate of 0.1. For the polarization feature vector composed of several statistical quantities, a support vector machine (SVM) configured with radial basis function (RBF) kernels is used for classification. These three base classifiers run independently and output preliminary probability estimates of whether the target plant is winter wheat or grass weed, in the form of a two-dimensional vector [P(wheat), P(weed)]. The second layer is the fusion decision layer, which concatenates the probability estimates (a total of 6 values) output by the three base classifiers into a 6-dimensional fusion feature vector. This vector is input into a Bayesian network classifier pre-trained on large-scale labeled data. The topology of this Bayesian network is optimized using a structure learning algorithm, enabling explicit modeling of the conditional dependencies between evidence provided by different perceptual modalities. For example, the probability of a specific spectral feature may be influenced by the plant's three-dimensional morphology (such as leaf age). By performing a confidence propagation-based probabilistic inference algorithm within this network, the posterior probability P(weed|evidence) of the target plant belonging to the Poaceae family is calculated, along with a confidence score. The data processing and decision unit sets a high-confidence decision threshold, for example, 0.95. Only when the calculated posterior probability is greater than 0.95 is the target ultimately determined to be a weed, generating a removal command and the 3D world coordinates of the weed's meristem (growing point). The 3D coordinates of the meristem are precisely extracted by locating the geometric center of the tillering node and the stem base in a 3D point cloud model.

[0030] Finally, refer to Figure 5Once the data processing and decision-making unit generates a removal command, the targeted removal execution mechanism is activated. This mechanism consists of a 100-watt continuous-wave ytterbium-doped fiber laser, a two-dimensional high-speed galvanometer scanning system, and an F-Theta focusing lens group. The laser generates a laser beam with a center wavelength of 1070 nanometers, a wavelength that can be efficiently absorbed by water in plant tissue. The laser beam first enters the two-dimensional high-speed galvanometer scanning system, which consists of two reflecting mirrors driven by high-precision servo motors, controlling the deflection of the X and Y axes respectively, with an angular response speed of up to 500 radians / second. The three-dimensional world coordinates of the target weed growth point transmitted from the decision-making unit are received by the controller of the execution mechanism, and the required target galvanometer deflection angle is calculated in real time through an inverse kinematics model. The servo motor drives the mirrors to precisely deflect to the specified angle, thereby guiding the laser beam in the correct direction. The guided laser beam then passes through the F-Theta focusing lens group, forming a high-energy-density focused spot with a diameter of 1.5 millimeters on its focal plane (i.e., the ground). The controller instructs the laser to emit a laser pulse lasting between 50 and 100 milliseconds. Within this brief period, the energy density accumulated at the focused spot is sufficient to instantly heat the plant tissue in the target area to over 100 degrees Celsius, causing protein denaturation and cell wall rupture, resulting in permanent and irreversible necrosis of the meristematic tissue, thus achieving physical removal. Due to the extremely small area of ​​effect and the extremely short duration, the thermal impact on the surrounding soil is negligible, and it causes no harm to nearby winter wheat plants, nor does it leave any chemical residues. The entire closed-loop operation cycle from identification to removal is strictly controlled within 200 milliseconds, ensuring that even at a travel speed of 0.5 meters per second, the equipment can continuously remove all weeds within its field of view without omission.

[0031] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A smart monitoring and weeding device for grassy weeds in winter wheat fields, characterized in that, include: An autonomous mobile platform used to move along a preset path in winter wheat fields; A multimodal sensing assembly is fixed to the autonomous mobile platform and is used to synchronously acquire multidimensional, high-resolution composite feature data of a single plant in the field of view below it. The data processing and decision unit is electrically connected to the multimodal sensing assembly and is used to process the composite feature data in real time, identify the species of the target plant based on a preset fusion classification model, and generate a high-confidence classification decision instruction and the target three-dimensional spatial coordinates. The system also includes a targeted removal mechanism electrically connected to the data processing and decision-making unit, used to perform targeted, non-contact physical removal operations on plants identified as grass weeds according to the classification decision instructions and the target's three-dimensional spatial coordinates.

2. The device according to claim 1, characterized in that, The multimodal sensing assembly includes: Baseline architecture; A collaborative lighting system, integrated on the reference architecture, is used to provide an active light source; A hyperspectral imaging module, rigidly mounted on the reference frame, is used to acquire hyperspectral data of the target plant under the illumination of the cooperative lighting system; A three-dimensional topography measurement module, rigidly mounted on the reference frame, is used to measure the three-dimensional point cloud model of the target plant under the illumination of the coordinated lighting system; and A polarization characteristic analysis module is rigidly mounted on the reference frame and is used to analyze the polarization characteristics of the light reflected from the surface of the target plant under the illumination of the coordinated lighting system. The hyperspectral imaging module, the three-dimensional morphology measurement module, and the polarization characteristic analysis module have a predetermined spatial relationship to achieve pixel-level data registration for the same target plant.

3. The device according to claim 2, characterized in that, The coordinated lighting system includes: A broadband linear light source array is used to provide illumination for the hyperspectral imaging module. The broadband linear light source array is composed of multiple halogen lamp beads. After being shaped by a uniform light lens group, it can form a linear light band with uniform illuminance that spans the direction of the equipment's movement on the ground. Its spectral coverage range is 400 nanometers to 1000 nanometers. The structured light projection unit is used to provide illumination for the three-dimensional topography measurement module. The structured light projection unit uses a vertical cavity surface emission laser array with a center wavelength of 850 nanometers as the light source and projects the laser beam into a structured light pattern containing pseudo-random speckle or de Bruin sequence encoding through diffractive optical elements. And a polarization illumination unit for providing illumination to the polarization characteristic analysis module, the polarization illumination unit consisting of a high-brightness white light-emitting diode surface light source and a linear polarizer covering it, the linear polarizer having a fixed transmission direction.

4. The device according to claim 2, characterized in that, The hyperspectral imaging module is a pushbroom imaging system, which includes a front objective lens, a slit aperture, a collimating lens, a transmission grating beam splitter, and an array image sensor. The hyperspectral imaging module observes the ground along the direction of the device's movement, with a spectral detection range of 400 nanometers to 1000 nanometers and a spectral resolution of no less than 5 nanometers. By controlling the movement speed of the autonomous mobile platform and the frame rate of the array image sensor, the ground sampling distance is ensured to be equal to 1 millimeter in both the direction of movement and the vertical direction of movement.

5. The device according to claim 2, characterized in that, The three-dimensional topography measurement module is constructed based on the principle of active binocular structured light. It includes the structured light projection unit and a pair of infrared image acquisition units symmetrically arranged on both sides of the structured light projection unit. Each infrared image acquisition unit includes an imaging lens with an 850-nanometer narrowband filter and a global shutter complementary metal-oxide-semiconductor image sensor. The optical axes of the two infrared image acquisition units are parallel to each other, and their baseline distance is set to 200 mm. By synchronously capturing plant surface images modulated by the structured light pattern and using triangulation to calculate the three-dimensional coordinates of spatial points, a three-dimensional point cloud model of the target plant with sub-millimeter precision can be reconstructed.

6. The device according to claim 2, characterized in that, The polarization characteristic analysis module employs an image sensor with an integrated micro-polarizer array. Each 2×2 pixel unit of this sensor is covered with linearly polarized microlenses in four different directions: 0 degrees, 45 degrees, 90 degrees, and 135 degrees. Under the illumination of the polarization illumination unit, the polarization characteristic analysis module simultaneously captures image information in four polarization directions in a single exposure and performs calculations based on the intensity information of the four images to calculate the Stokes parameter, degree of linear polarization, and polarization angle of the light reflected from the surface of the target plant in real time.

7. The device according to claim 1, characterized in that, The data processing and classification decision-making algorithm process running inside the data processing and decision-making unit specifically includes the following steps executed sequentially: Perform multimodal data preprocessing and registration: Perform preprocessing operations such as dark current subtraction, radiometric calibration, lens distortion correction, and noise filtering on the received hyperspectral data, binocular infrared image pairs, and polarization images. Then, based on the pre-determined rigid transformation matrix between each sensor, project each modal data onto the same world coordinate system to achieve pixel-level precise alignment in space. Parallel extraction of multimodal feature vectors: For aligned single plant data, multiple sets of feature vectors for species identification are extracted in parallel from three dimensions: hyperspectral data, three-dimensional morphology data, and polarization characteristic data. And perform hierarchical Bayesian fusion classification decision: input the multiple sets of feature vectors into a hierarchical classification model. The model first performs preliminary classification of each set of feature vectors through multiple parallel base classifiers to obtain preliminary probability estimates. All preliminary probability estimates are input into a top-level Bayesian network classifier for information fusion and final inference to calculate the posterior probability that the target plant belongs to the grass family weeds and generate the classification decision instruction.

8. The device according to claim 7, characterized in that, In the parallel extraction step of the multimodal feature vector, the extracted feature vector specifically includes: In the hyperspectral data dimension, the average spectral reflectance curve of the target plant area, multiple narrow-band vegetation indices, and a mathematical expression are extracted. The defined wheat-weed physiological differentiation index, in which This represents the reflectivity at a center wavelength of X nanometers; In the dimension of three-dimensional topographic data, the reconstructed plant point cloud model is analyzed, and a set of quantitative parameters describing the micro-morphological structure is extracted. These parameters include: the maximum height of the plant, the equivalent diameter of the stem at 2 mm from the soil surface, the number of tillering nodes, the histogram of the average tilt angle distribution of leaves, and the standard deviation of the angle between the surface normal vector of all leaves and the centroid of the line connecting them. Furthermore, in terms of polarization characteristic data, statistical features such as the mean value of linear polarization, the variance of linear polarization, and the mean value of polarization angle gradient magnitude are extracted from the calculated linear polarization degree spectrum and polarization angle spectrum within the plant area.

9. The device according to claim 7, characterized in that, In the hierarchical Bayesian fusion classification decision step: The base classifier layer includes: a one-dimensional convolutional neural network for processing the average spectral reflectance curve, a gradient boosting decision tree model for processing the quantification parameters of the microstructure, and a support vector machine for processing the statistical features of the polarization characteristics; each base classifier independently outputs a preliminary probability estimate of whether the target plant is winter wheat or a grass weed. In the fusion decision layer, the preliminary probability estimates output by the three base classifiers are concatenated to form a fusion feature vector, which is then input into the Bayesian network classifier. The Bayesian network classifier calculates the final posterior probability through probabilistic inference. The data processing and decision-making unit determines the target as weeds if and only if the posterior probability is greater than a preset decision threshold.

10. The device according to claim 1, characterized in that, The targeted removal actuator includes: A continuous-wave ytterbium-doped fiber laser with an output power of 100 watts and a center wavelength of 1070 nanometers. The two-dimensional high-speed galvanometer scanning system is used to calculate the corresponding galvanometer deflection angle based on the three-dimensional coordinates of the target weed growth point received from the data processing and decision-making unit, and drive the reflecting mirror to precisely control the emission angle of the laser beam. And a focusing lens assembly, used to focus the laser beam guided by the two-dimensional high-speed galvanometer scanning system onto the growth point; The laser applies a 50-100 millisecond laser pulse to the growth point to instantaneously heat the plant tissue, causing irreversible necrosis of its meristematic tissue.

Images