A Smart Weed Control System for Photovoltaic Panel Areas Based on Multimodal Sensing

Abstract

This invention discloses an intelligent weeding control system for photovoltaic panel areas based on multimodal perception, belonging to the field of weeding control technology. The system includes: a feature extraction module for acquiring multispectral image data of the photovoltaic panel area and obtaining vegetation analysis results; a collaborative control parameter determination module for path planning, generating weeding paths, performing dynamic energy allocation, and determining collaborative control parameters; and a control feedback module for providing control feedback to the controller in the photovoltaic panel area to execute intelligent weeding operations. This invention solves the technical problems of inaccurate weed identification, low weeding efficiency, and unreasonable energy allocation in existing technologies, which easily lead to poor weeding effects and energy waste. It achieves accurate identification and efficient removal of weeds in photovoltaic panel areas, effectively improving weeding efficiency and energy utilization efficiency.

Description

Technical Field

[0001] This invention relates to the field of weed control technology, specifically to an intelligent weed control system for photovoltaic panel areas based on multimodal perception. Background Technology

[0002] As the scale of photovoltaic power generation continues to expand, weed control in photovoltaic panel areas has become a critical task for ensuring power generation efficiency and equipment safety. Currently, manual weeding is inefficient and costly, making large-scale operations difficult. Mechanical weeding lacks automation, easily damages photovoltaic panels, and lacks comprehensive environmental perception and intelligent analysis, failing to provide targeted operations based on weed growth status and density, resulting in energy waste and poor weeding effectiveness. Furthermore, existing technologies lack dynamic optimization mechanisms for weeding path planning and energy allocation, making it difficult to meet the complex and ever-changing environmental requirements of photovoltaic panel areas.

[0003] Existing technologies suffer from problems such as inaccurate weed identification in photovoltaic panel areas, low weeding efficiency, and unreasonable energy distribution, which can easily lead to poor weeding results and energy waste. Summary of the Invention

[0004] This application provides an intelligent weeding control system for photovoltaic panel areas based on multimodal perception, which is used to address the technical problems in the prior art, such as inaccurate weed identification, low weeding efficiency, and unreasonable energy distribution in photovoltaic panel areas, which easily lead to poor weeding effect and energy waste.

[0005] In view of the above problems, this application provides an intelligent weed control system for photovoltaic panel areas based on multimodal perception.

[0006] This application provides an intelligent weed control system for photovoltaic panel areas based on multimodal sensing, the system comprising:

[0007] The feature extraction module is used to collect multispectral image data of the photovoltaic panel area, generate a multimodal sensing dataset, extract features from the multimodal sensing dataset, and obtain vegetation analysis results based on the sensing feature set. The collaborative control parameter determination module is used to perform path planning based on the vegetation analysis results, generate a weeding path, perform dynamic energy allocation according to the weeding path, and determine collaborative control parameters. The control feedback module is used to provide control feedback to the controller of the photovoltaic panel area according to the collaborative control parameters, update the collaborative control parameters according to the feedback results, and execute intelligent weeding operations in the photovoltaic panel area.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] The feature extraction module collects multispectral image data of the photovoltaic panel area, generates a multimodal sensing dataset, extracts features, and obtains vegetation analysis results. The collaborative control parameter determination module performs path planning based on the vegetation analysis results, generates weeding paths, performs dynamic energy allocation, and determines collaborative control parameters. The control feedback module provides control feedback to the controller in the photovoltaic panel area, updates the collaborative control parameters according to the feedback results, and executes intelligent weeding operations in the photovoltaic panel area. This achieves accurate identification and efficient removal of weeds in the photovoltaic panel area, effectively improving weeding efficiency and energy utilization efficiency. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This application provides a schematic diagram of the structure of an intelligent weed control system for photovoltaic panel areas based on multimodal perception.

[0012] Figure 2 This application provides a schematic diagram of the feature extraction module in an intelligent weed control system for photovoltaic panel areas based on multimodal perception.

[0013] Figure labeling: Feature extraction module 10, collaborative control parameter determination module 20, control feedback module 30. Detailed Implementation

[0014] This application provides an intelligent weeding control system for photovoltaic panel areas based on multimodal perception, which addresses the technical problems in existing technologies such as inaccurate weed identification, low weeding efficiency, and unreasonable energy distribution in photovoltaic panel areas, which easily lead to poor weeding results and energy waste.

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0016] Examples, such as Figure 1 As shown, this application provides an intelligent weed control system for photovoltaic panel areas based on multimodal sensing, the system comprising:

[0017] The feature extraction module 10 is used to collect multispectral image data of the photovoltaic panel area, generate a multimodal sensing dataset, extract features from the multimodal sensing dataset, and obtain vegetation analysis results based on the sensing feature set.

[0018] Specifically, firstly, based on the environmental characteristics of the photovoltaic panel area and the needs of weeding, the acquisition parameters of the multispectral camera are precisely set, including resolution parameters and near-infrared band parameters. Reasonable setting of these parameters is a prerequisite for obtaining high-quality image data. Next, the image acquisition unit activates the multispectral camera according to the set parameters, simultaneously acquiring RGB image parameter sets and near-infrared image sets to obtain rich spectral information. After acquisition, the RGB image parameter sets and near-infrared image sets are spatiotemporally registered to eliminate deviations caused by differences in acquisition time and angle, generating a spatiotemporally aligned image set to ensure accurate correspondence of information in different spectral images. Subsequently, the distortion correction unit performs distortion correction on the spatiotemporally aligned image set to further improve image quality, ultimately obtaining a multimodal perception dataset. Based on an improved YOLOv7 network, the multimodal perception dataset is traversed. The backbone network CSPDarknet53, combined with the SPPF module, enables rapid inference. Its multi-task branch design plays a crucial role. The weed detection branch (confidence > 0.9) extracts multi-scale weed features, performs matching calculations on the dataset based on these features, obtains the weed confidence of multiple pixels, performs regional probability statistics based on this, generates a regional weed probability distribution matrix, and then draws a weed probability heatmap to visually show the probability of weed distribution in the photovoltaic panel area. Simultaneously, the NDVI calculation branch assesses vegetation health and derives the NDVI index, reflecting the vegetation's growth status. Finally, based on the weed probability heatmap, connected component analysis is used to mark and determine the weed distribution area; weed density is calculated by combining the multimodal sensing dataset to generate a weed density distribution map; plant height data is obtained by combining the multimodal sensing dataset and the weed density distribution map; the weed density distribution map and plant height data are spatially overlaid according to weed confidence, and the comprehensive weed probability heatmap and NDVI index are combined to generate comprehensive and detailed vegetation analysis results, providing an important basis for the subsequent collaborative control parameter determination module to perform path planning and energy allocation.

[0019] The collaborative control parameter determination module 20 is used to perform path planning based on the vegetation analysis results, generate a weeding path, perform dynamic energy allocation according to the weeding path, and determine collaborative control parameters.

[0020] Specifically, the process begins with a 3D analysis of the weed density distribution map. Weed density datasets are extracted, gridded, and a density heatmap is created. 3D point cloud parameters are constructed using a multimodal sensing dataset, and plant height data is integrated to classify vegetation and determine treatment priorities. The density heatmap and plant height data are then linked and integrated according to priority to construct a comprehensive 3D environmental map reflecting the photovoltaic panel area. Next, the 3D environmental map is traversed to analyze the weeding needs and obstacle distribution in the photovoltaic panel area, constructing a reasonable path topology. Based on this structure, the coordinates of photovoltaic pile foundations and cable distribution information are identified, and cable safety distance constraints are set. A 3D model of the cables is constructed, the central axis is extracted, and a cylindrical safety space is built along the axis. Local paths are planned outside this space, weeding distance data is generated, path conflict judgment and handling are performed, and path curvature is calculated to determine the radius of the detour buffer zone, thus setting the buffer zone boundary. Then, candidate path nodes are constructed at equidistant markers at the buffer zone boundary, and a continuous zigzag detour path, i.e., the initial global path, is used as the weeding path, and dynamic energy allocation is performed based on the weed situation and environmental characteristics of different road sections. For example, more energy is allocated to densely weeded areas and less energy is allocated to sparsely weeded areas to determine multiple path segments. Finally, these path segments are mapped to multiple controllers for control analysis. For each path segment, considering factors such as weed density, plant height, and NDVI index, a fuzzy logic controller based on a dynamic energy allocation model is used to optimize the coordinated control parameters of laser power and blade rotation speed. When the weed density is less than 5 plants / m²... 2 When the plant height is less than 8cm and the NDVI is less than 0.3, the laser mode is used, with the laser power set to 15W and the blade speed set to 0. Otherwise, the hybrid mode is used, with the laser power set to 20W and the blade speed set to 1800rpm. Through these steps, the coordinated control parameters covering key parameters such as laser power and blade speed are finally determined, providing an accurate basis for the subsequent control feedback module to accurately control the weeding equipment, ensuring that the weeding operation is carried out efficiently, safely, and energy-savingly.

[0021] The control feedback module 30 is used to provide control feedback to the controller of the photovoltaic panel area according to the collaborative control parameters, and to update the collaborative control parameters according to the feedback results to perform intelligent weeding operations in the photovoltaic panel area.

[0022] Specifically, firstly, the collaborative control parameters are analyzed to obtain key information such as the target blade rotation speed, laser power setpoint, and energy distribution mode. These parameters are fundamental to controlling the weeding equipment and determine its operating status in different scenarios. Then, based on the target blade rotation speed parameter, it is input into a fuzzy PID controller. The fuzzy PID controller, combined with the actual collected blade rotation speed feedback signal, adaptively adjusts the control parameters through fuzzy logic. When the actual rotation speed is lower than the target value, the controller appropriately increases the control signal to raise the blade rotation speed; conversely, it lowers the speed, thus precisely adjusting the blade rotation speed to stabilize it near the target value and ensure consistent weeding results. Simultaneously, according to the laser power setpoint, the laser power is precisely controlled using a laser power PID controller. The laser power PID controller adjusts the laser power in real time based on the laser output power feedback signal, achieving energy feedback control. In areas with abundant and difficult-to-clear weeds, the laser power is appropriately increased; in areas with fewer weeds, the laser power is reduced, ensuring weeding effectiveness while avoiding energy waste. After completing the above control feedback operations, based on the energy distribution mode, a multi-parameter coupled control analysis is performed, integrating the first and second control feedback results. In laser mode, the focus is on stable laser power output and weeding effectiveness; in hybrid mode, laser power and blade speed are coordinated to achieve optimal synergy. This analysis allows for dynamic updates to the collaborative control parameters, generating control commands that better suit actual operating conditions. Throughout the weeding process, continuous feedback is acquired, and the collaborative control parameters are constantly updated, enabling the weeding equipment to adapt to the complex and ever-changing environment of the photovoltaic panel area, automatically adjusting its operating status to achieve efficient and intelligent weeding operations. This effectively improves weeding efficiency and quality, ensuring the normal operation of the photovoltaic panels.

[0023] In one possible implementation, such as Figure 2 As shown, the feature extraction module 10 further includes:

[0024] The acquisition parameter setting unit is used to set the acquisition parameters of the multispectral camera, including resolution parameters and near-infrared band parameters.

[0025] The image acquisition unit is used to activate the multispectral camera based on the acquisition parameters, acquire the RGB image parameter set according to the resolution parameters, and acquire the near-infrared image set according to the near-infrared band parameters.

[0026] The spatiotemporal registration unit is used to perform spatiotemporal registration of the RGB image parameter set with the near-infrared image set to generate a spatiotemporally aligned image set.

[0027] The distortion correction unit is used to perform distortion correction based on the spatiotemporally aligned image set to obtain the multimodal sensing dataset.

[0028] Specifically, setting the acquisition parameters of the multispectral camera is a crucial step in obtaining accurate image data. The resolution parameter setting needs to comprehensively consider the actual area of ​​the photovoltaic panel area, the distribution range of weeds, and the accuracy requirements for subsequent weed feature identification. If the photovoltaic panel area is large and the weeds are scattered, a higher resolution, such as 4096×3072 pixels, is required for comprehensive coverage and accurate weed identification. This will clearly present the morphological details of the weeds, including leaf texture and plant spacing, which will help in the accurate judgment of weed species and growth status. If the panel area is small and the key areas of focus are clearly defined, a lower resolution, such as 1920×1080 pixels, may be sufficient, while reducing data processing load and improving operational efficiency. Near-infrared band parameter settings focus on vegetation characteristics. Near-infrared light is sensitive to the reflectivity of plants, and vegetation with different health conditions shows significant differences in reflectivity in the near-infrared band. Typically, to effectively distinguish weeds from other vegetation and the background, the near-infrared band in the 760-1100 nanometer range is selected for acquisition. For photovoltaic panel areas dominated by grassy weeds, the focus should be on collecting data in the 780-900 nm band. This band can highlight the difference in reflectance between grassy weeds and the surrounding environment, enhancing the visibility of weeds in images. If the weeds in the panel area are diverse, the near-infrared band acquisition range can be appropriately broadened, such as 760-1000 nm, to obtain more comprehensive vegetation information and provide rich data support for subsequent vegetation health assessment (such as calculating the NDVI index) and accurate weed detection.

[0029] Once the acquisition parameters are set, the image acquisition phase begins. The multispectral camera is activated according to the pre-set parameters. Upon receiving the activation command, the camera first acquires RGB image parameters according to the resolution parameters. If the resolution is set to a high 4096×3072 pixels, the camera will capture high-definition color images of the photovoltaic panel area, comprehensively recording the color information of weeds, photovoltaic panels, and the surrounding environment, such as the green of weed leaves and the blue of photovoltaic panels. These RGB image parameters provide intuitive color characteristics for subsequent visual identification of weeds. Simultaneously, the camera acquires near-infrared image parameters. If the near-infrared parameters are set in the 760-1100 nanometer range, the camera will focus on the light reflection in this band, acquiring the near-infrared reflection image of the vegetation in the photovoltaic panel area. Weeds in different health states exhibit different reflectivities in the near-infrared band; healthy weeds have higher reflectivity, while weeds affected by pests or diseases or with poor growth have lower reflectivity. By collecting near-infrared image sets, key information such as the health status of vegetation can be obtained.

[0030] After acquiring the RGB image parameter set and the near-infrared image set, spatiotemporal registration is required to generate a spatiotemporally aligned image set to ensure the accuracy and effectiveness of subsequent analysis. Because even with extremely short time intervals, the acquisition of RGB and near-infrared images by multispectral cameras can lead to temporal and spatial discrepancies due to factors such as minor equipment vibrations and real-time changes in lighting, spatiotemporal registration first analyzes the timestamps of the RGB image parameter set and the near-infrared image set to accurately calculate the difference in acquisition time. Then, a time interpolation algorithm is used to synchronize the two sets of images in the temporal dimension, eliminating dynamic differences caused by asynchronous acquisition times, such as subtle changes in the growth state of the same weed at different times. In the spatial dimension, feature extraction algorithms are used to extract feature points from the RGB and near-infrared images, such as the edge contours of weeds and specific texture feature points. Then, a feature matching algorithm is used to find corresponding feature points in the two sets of images, and based on this, the spatial transformation relationship between the images is calculated, including transformation parameters such as translation, rotation, and scaling. Based on these parameters, the near-infrared image is geometrically transformed to ensure complete spatial alignment with the RGB image, guaranteeing that the same object in both sets of images corresponds precisely in spatial location. After dual temporal and spatial registration, the RGB image parameter set and the near-infrared image set are fused to finally generate a spatiotemporally aligned image set.

[0031] Camera calibration techniques are used to determine the intrinsic and extrinsic parameters of the camera, which reflect its optical structure and imaging geometry. Using these known calibration parameters, a mapping model between image pixel coordinates and actual physical coordinates is established. For image distortion, distortion correction algorithms, such as those based on polynomial models, are employed. By analyzing known feature points or checkerboard patterns in the image, distortion coefficients are calculated. These coefficients characterize the degree and type of image distortion, including radial and tangential distortion. Based on the calculated distortion coefficients, a correction transformation is performed on each pixel in the spatiotemporally aligned image set, remapping distorted pixels to their correct positions, thereby eliminating or reducing image distortion. After this series of correction operations, the resulting image more accurately reflects the actual situation of the photovoltaic panel area, containing rich spectral information (RGB and near-infrared) and accurate spatial information, ultimately forming a high-quality multimodal sensing dataset.

[0032] In one possible implementation, the feature extraction module 10 further includes:

[0033] The sensing feature set acquisition unit is used to traverse the multimodal sensing dataset to identify weed features in the photovoltaic panel area, obtain a sensing feature set, and draw a weed probability heat map based on the sensing feature set. The weed probability heat map includes weed confidence levels.

[0034] The weed distribution area determination unit is used to perform connected component analysis and labeling based on the weed probability heatmap to determine the weed distribution area.

[0035] The weed density distribution map acquisition unit is used to calculate the weed distribution area by combining the multimodal sensing dataset to obtain the weed density distribution map.

[0036] The plant height data acquisition unit is used to calculate the vegetation depth of the photovoltaic panel area based on the multimodal sensing dataset and the weed density distribution map, and obtain plant height data.

[0037] The spatial overlay unit is used to spatially overlay the weed density distribution map and the plant height data according to the weed confidence level to generate the vegetation analysis results.

[0038] Specifically, a comprehensive traversal of the multimodal perception dataset is first undertaken. This dataset integrates information from RGB and near-infrared images, providing a rich data foundation for weed feature recognition. Utilizing the improved YOLOv7 network, its powerful feature extraction capabilities are leveraged. Through the weed detection branch (confidence > 0.9) in the multi-task branching design, multi-scale weed features are extracted from the dataset, such as unique leaf textures, specific color combinations, and plant shape outlines. These features collectively constitute the perception feature set. Based on the perception feature set, a weed probability heatmap is further generated. The perception features are compared with pixels in the dataset to calculate the probability that each pixel belongs to a weed, i.e., the weed confidence score. The regional probability statistics subunit performs statistical analysis on the weed confidence scores of multiple pixels, generating a regional weed probability distribution matrix. This matrix reflects the probability of weed occurrence in different regions. Finally, bilinear interpolation is performed based on the regional weed probability distribution matrix to transform the discrete probability data into a continuous heatmap format. In the heat map, different colors represent different confidence levels of weeds. The darker the color, the higher the probability of weeds being present. This visually and clearly shows the possible distribution area and probability of weeds within the photovoltaic panel area, providing an important basis for accurately determining the distribution range and density of weeds in the future.

[0039] After obtaining the weed probability heatmap, connecting component analysis and labeling based on this heatmap is a crucial step in determining the weed distribution areas. The weed probability heatmap contains weed confidence information for each pixel. Connecting component analysis first sets a reasonable weed confidence threshold, focusing on analyzing pixels in the heatmap whose weed confidence is higher than this threshold. Starting from a point among these high-confidence pixels, following certain neighborhood rules (such as four-neighbor or eight-neighbor), it searches for adjacent pixels with the same weed confidence higher than the threshold. If adjacent matching pixels are found, they are grouped into the same connected region and assigned the same label. This neighborhood search continues until all matching pixels within the connected region have been labeled. The above operation is repeated for all unlabeled pixels in the heatmap with a weed confidence higher than the threshold, thus dividing the entire heatmap into multiple distinct connected regions. These marked connected regions are the weed distribution areas determined by the system. In this way, the location and range of concentrated weed growth in the photovoltaic panel area can be clearly and accurately identified, providing precise location information for subsequent operations such as calculating weed density and planning weeding paths.

[0040] Image data corresponding to weed distribution areas are extracted from a multimodal sensing dataset. Spectral information and texture features from these images are used to further distinguish individual weeds from the background. Image segmentation algorithms are used to accurately segment the weeds from the background, identifying the boundaries of each individual weed. Next, the number of weeds within the weed distribution area is counted. Simultaneously, the actual area of ​​the weed distribution area is determined based on geographical information or image scale from the multimodal sensing dataset. Dividing the number of weeds by the actual area yields the weed density for that area. This calculation process is repeated for each sub-region within the weed distribution area to obtain the weed density value for each sub-region. Finally, these weed density values ​​are visualized to form a weed density distribution map. In the map, different colors or gray levels represent different weed densities; darker colors or higher gray levels indicate greater weed density, thus visually displaying the density distribution of weeds within the photovoltaic panel area and providing a strong basis for subsequent weeding decisions.

[0041] A deep learning-based monocular depth estimation algorithm is employed to obtain plant height data based on a multimodal sensing dataset combined with weed density distribution maps. RGB images from the multimodal sensing dataset are used as the primary input due to their rich color and texture information. Leveraging the powerful feature extraction capabilities of Convolutional Neural Networks (CNNs), a backbone network structure such as ResNet or VGG is constructed to perform multi-layer convolution operations on the images, extracting image features at different scales, such as weed edges, textures, and differences from the surrounding environment. Simultaneously, the weed density distribution map is encoded as a feature vector and integrated into the intermediate layer features of the CNN through fully connected layers, allowing the network to learn the potential relationship between weed density and plant depth. During network training, supervised learning is performed using a photovoltaic panel area image dataset containing a large number of known plant height annotations. The mean squared error (MSE) loss function is used to measure the difference between the predicted depth value and the true height. Backpropagation is used to continuously adjust network parameters and optimize model performance. During the inference phase, the RGB images from the multimodal sensing dataset are input into the trained network, and the network outputs a preliminary vegetation depth prediction map. Then, based on the image capture parameters, such as focal length and camera height, the predicted depth values ​​are converted into actual plant height data. This deep learning-based monocular depth estimation algorithm fully utilizes multimodal sensing data and weed density information to effectively acquire plant height data in photovoltaic panel areas.

[0042] First, the weed density distribution map and plant height data are standardized to ensure consistency in spatial coordinates and compatibility of data scales. Using weed confidence level as the weighting criterion, areas with high weed confidence levels are given greater weight in the overlay process for weed density and plant height data, as the information in these areas is more reliable and has a greater impact on the analysis results. Specifically, for each spatial location, the weed density and plant height values ​​are weighted and fused according to the weed confidence level at that location. For example, in an area with a weed confidence level of 0.8, if the weed density is 10 plants / m² and the plant height is 20 cm, after weighted calculation, the comprehensive information in the vegetation analysis results for this area will more prominently reflect the characteristics of these two data points; while in areas with low weed confidence levels, the influence of the corresponding data will be weakened. By performing this weighted overlay operation on various spatial locations within the photovoltaic panel area, a vegetation analysis result containing comprehensive information such as weed distribution range, density, and plant height is finally generated. The results are presented in the form of visual maps or detailed data reports, which greatly improves the scientific decision-making of the intelligent weeding control system in the photovoltaic panel area and the efficiency of weeding operations.

[0043] In one possible implementation, the perceptual feature set acquisition unit further includes:

[0044] The weed feature extraction subunit is used to extract multi-scale weed features by traversing the multimodal sensing dataset based on the improved YOLOv7 network.

[0045] The matching calculation subunit is used to perform matching calculations on the multimodal sensing dataset based on the multi-scale weed features to obtain the weed confidence scores of multiple pixels in the multimodal sensing dataset.

[0046] The regional probability statistics subunit is used to perform regional probability statistics based on the weed confidence of the multiple pixels and generate a regional weed probability distribution matrix.

[0047] The bilinear interpolation subunit is used to perform bilinear interpolation based on the probability distribution matrix of weeds in the region to draw a probability heat map of weeds in the photovoltaic panel area.

[0048] Specifically, the improved YOLOv7 network processes multimodal sensing datasets to extract multi-scale weed features. The backbone of the improved YOLOv7 network consists of CSPDarknet53 and SPPF modules. CSPDarknet53 enhances the network's feature extraction capabilities through a cross-stage local network structure, while reducing computational cost and improving efficiency; the SPPF module further accelerates inference, allowing the network to process multimodal sensing data more quickly. The network employs a multi-task branch design, with the weed detection branch (confidence > 0.9) used to extract multi-scale weed features. When traversing the multimodal sensing dataset, the network's convolutional layers perform layer-by-layer convolution operations on the RGB and near-infrared images in the dataset. Shallow convolutional layers have smaller receptive fields, focusing on capturing detailed features such as weed leaf texture and edge contours; deep convolutional layers have larger receptive fields, extracting global features such as the overall shape and distribution patterns of weed plants, thus achieving multi-scale feature extraction. During training, various optimization techniques were employed to improve network performance. For data augmentation, random rotation (±30°), illumination perturbation (±20%), and scaling (0.5-2x) were applied to the training data to increase data diversity, enabling the network to learn weed features under different angles, lighting, and scales, thus enhancing the model's generalization ability. Regarding the loss function, FocalLoss and CIoU Loss were combined. FocalLoss addresses the class imbalance problem by reducing the weight of many simple negative samples (such as background areas), allowing the network to focus more on difficult-to-classify weed samples. CIoU Loss, on the other hand, focuses on improving localization accuracy, considering factors such as the overlap area between the predicted and ground truth boxes, the distance between the center points, and the aspect ratio, enabling the network to more accurately locate weeds during training. Through these training optimizations, the improved YOLOv7 network can more accurately extract multi-scale weed features from the multimodal perception dataset, providing crucial support for subsequent weed detection and analysis.

[0049] After acquiring multi-scale weed features, they are matched with a multimodal sensing dataset to determine the weed confidence level of multiple pixels in the dataset. First, feature vectors are constructed for the multi-scale weed features, containing key information such as color, texture, and shape of the weeds at different scales. Then, for each pixel in the multimodal sensing dataset, corresponding pixel features are extracted from its surrounding neighborhood and converted into feature vectors. Next, a matching algorithm is used to calculate the similarity between the pixel feature vector and the multi-scale weed feature vector, such as the cosine similarity algorithm. This algorithm measures the similarity by calculating the cosine of the angle between two vectors; the closer the cosine value is to 1, the higher the similarity between the two vectors. When calculating the similarity, the weights of features at different scales in the multi-scale feature vector are considered comprehensively, with higher weights assigned to key scale features that more accurately reflect the characteristics of the weeds. Based on the calculated similarity results and a pre-set threshold, the weed confidence level of each pixel is determined. If the similarity is higher than the threshold, the pixel is considered to be more likely to belong to weeds and is assigned a higher confidence score; conversely, if the similarity is lower than the threshold, a lower confidence score is assigned. By performing this matching calculation and confidence assignment on all pixels in the multimodal sensing dataset, we can obtain the weed confidence scores for multiple pixels in the dataset. This provides an important data foundation for subsequent operations such as drawing weed probability heatmaps and determining weed distribution areas, and helps to more accurately identify and analyze the weed distribution in photovoltaic panel areas.

[0050] After obtaining the weed confidence scores of multiple pixels in the multimodal sensing dataset, to more intuitively present the distribution probability of weeds in different areas of the photovoltaic panel area, regional probability statistics are performed and a regional weed probability distribution matrix is ​​generated. First, the image corresponding to the photovoltaic panel area is divided into multiple sub-regions of equal size. The division of these sub-regions must comprehensively consider computational efficiency and accuracy requirements, ensuring that it accurately reflects the weed distribution characteristics without excessive computation. For each sub-region, all pixels contained within it are traversed, and the weed confidence scores of these pixels are calculated. For example, in a certain sub-region, there are 100 pixels, each with its corresponding weed confidence score value. These values ​​are summed to obtain the total weed confidence scores of all pixels in that sub-region. Then, this sum is divided by the total number of pixels in the sub-region to obtain the average weed confidence score of that sub-region. This average weed confidence score represents the probability level of weed occurrence in that sub-region. Following the above method, the calculation is performed one by one for all sub-regions of the photovoltaic panel area image to obtain the weed probability value for each sub-region. Finally, the weed probability values ​​of these sub-regions are arranged according to their positions in the image, forming a two-dimensional matrix, namely the regional weed probability distribution matrix. In this matrix, each element corresponds to the weed probability of a sub-region, and the rows and columns of the matrix correspond to the sub-region division of the photovoltaic panel area image in different directions. Through the regional weed probability distribution matrix, the differences in weed distribution probabilities in different areas within the photovoltaic panel area can be clearly seen, providing intuitive and important data support for subsequent determination of weed distribution areas and planning of weeding paths.

[0051] After generating the regional weed probability distribution matrix, a weed probability heatmap is drawn using bilinear interpolation, combined with geographic coordinate information. This allows for a visual representation of the continuous changes in weed distribution probability within the photovoltaic panel area. First, the geographic coordinates corresponding to each element in the regional weed probability distribution matrix are defined. These coordinates precisely define the correspondence between matrix elements and their actual locations within the photovoltaic panel area. Since the matrix is ​​discrete, bilinear interpolation is required to obtain a continuous heatmap. For each target pixel in the heatmap, its relative position within the regional weed probability distribution matrix coordinate system is determined. The four nearest matrix elements to this point are found, forming a small rectangle enclosing the target pixel. Based on the relative distance from the target pixel to these four elements, interpolation weights are calculated in both the horizontal and vertical directions using the bilinear interpolation algorithm. These weights are then used to calculate a weighted average of the weed probability values ​​corresponding to the four elements, yielding an estimated weed probability value for the target pixel. This process is repeated for each pixel in the heatmap to obtain the weed probability value for each point. Based on pre-defined color mapping rules, different weed probability values ​​are mapped to different colors. For example, low probability values ​​are mapped to blue, high probability values ​​to red, and intermediate probability values ​​to colors transitioning from blue to red. In this way, after all pixels are assigned the corresponding color according to their respective weed probability values, a weed probability heatmap of the photovoltaic panel area is created. In the heatmap, the intensity of the color directly reflects the probability of weed distribution, allowing for the rapid identification of dense and sparse weed areas, providing crucial information for developing precise weed control strategies.

[0052] In one possible implementation, the cooperative control parameter determination module 20 further includes:

[0053] The three-dimensional environment map construction unit is used to perform three-dimensional analysis based on the weed density distribution map and construct a three-dimensional environment map.

[0054] The path topology construction unit is used to traverse the three-dimensional environment map to perform weeding analysis of the photovoltaic panel area and construct the path topology.

[0055] The initial global path generation unit is used to set cable safety distance constraints, and to plan the weeding path according to the path topology and the cable safety distance constraints to generate the initial global path.

[0056] The path segmentation determination unit is used to dynamically allocate energy to the initial global path as a weeding path and determine multiple path segments.

[0057] The control analysis unit is used to map the multiple paths into multiple controllers for control analysis and to determine the cooperative control parameters.

[0058] Specifically, a 3D environmental map is constructed based on the acquired weed density distribution map using 3D analysis. Weed density datasets are extracted from the map, gridded, and a density heatmap is created, visualizing the weed density distribution. Simultaneously, a 3D point cloud parameter model of the photovoltaic panel area is constructed using a multimodal sensing dataset, fused with plant height data, and categorized according to the fusion results to determine multiple vegetation treatment priorities. Finally, the density heatmap and plant height data are correlated and integrated according to these priorities to construct a 3D environmental map containing information on weed distribution, height, and treatment priorities, providing a comprehensive environmental information foundation for subsequent weed control analysis.

[0059] The process involves traversing the constructed 3D environment map to perform weeding analysis in the photovoltaic panel area and building a path topology. Based on the distribution of weeds, the location of photovoltaic panels, and other obstacles in the 3D environment map, the process analyzes passable areas and areas to be avoided, identifies possible path nodes for the weeding equipment to move within the photovoltaic panel area, and connects these nodes into a network to form a path topology. This structure clearly defines the potential routes and connections for the weeding equipment within the area.

[0060] The coordinates of photovoltaic pile foundations and cable distribution information are accurately identified from the path topology. Based on this information, cable safety distance constraints are set according to actual operational needs and safety standards, such as requiring a minimum 20cm safety distance between weeding equipment and cables. After determining the constraints, a 3D model of the cable is constructed with the cable as the center, and the central axis is extracted. Along the central axis, a cylindrical safety space is constructed according to the safety distance constraints to ensure that weeding equipment does not approach the cable within this space. Local paths are planned outside the cylindrical safety space, and corresponding weeding distance data is constructed. During the planning process, path conflict judgment is continuously performed. If a path conflict occurs, a detour sub-path is inserted to avoid obstacles or dangerous areas. After completing the local path planning, the path curvature of the detour sub-path is calculated to determine the radius of the detour buffer zone, and the buffer zone boundary is set according to the radius. Equidistant markers are made on the buffer zone boundary to construct multiple candidate path nodes. These nodes constitute a set of possible path choices. By filtering these paths and taking into account factors such as path length and obstacle density, a continuous zigzag bypass path is finally generated. This path can meet the cable safety distance constraints and cover the area that needs to be weeded to the greatest extent. Thus, it serves as the initial global path to provide a basic route framework for subsequent weeding operations, ensuring that the weeding equipment can operate efficiently under safe conditions.

[0061] Utilizing multimodal sensing data, based on weed confidence scores, regional weed probability distribution matrices, and weed density distribution maps obtained through multi-scale weed feature matching, detailed feature extraction is performed on the areas traversed by the initial global path. Sensors collect real-time information on weed density, plant height, and growth status at different locations along the path, quantifying this information into parameters usable for energy allocation decisions. Next, a dynamic energy allocation model is constructed. This model comprehensively considers factors such as weed characteristic parameters, the energy consumption characteristics of the weeding equipment, and operational efficiency. For example, for areas with high weed density and tall plants, more energy is allocated for weeding operations according to preset rules, manifested as increased blade speed and increased laser weeding power; while for areas with sparse and short weeds, energy output is correspondingly reduced. During energy allocation, the initial global path is divided based on energy allocation change nodes. When the energy allocation scheme changes, the path is divided into new segments. Simultaneously, the segmentation is further optimized and adjusted based on the path's geometric features, such as turns and slope changes. Because the energy consumption and operational difficulty of weeding equipment vary in areas with curves or slopes, energy allocation and operating modes need to be considered separately. Furthermore, the equipment's battery life and operating time limitations must also be taken into account. When dividing the path into segments, ensure that the energy consumption of each segment does not exceed the equipment's remaining battery power, and that the operating time of each segment is within a reasonable range to guarantee the successful completion of the entire weeding task. Through these steps, the initial global path is divided into multiple path segments, each with a corresponding dynamic energy allocation scheme, providing precise guidance for subsequent weeding operation control.

[0062] Multiple path segments are mapped to multiple controllers for control analysis to determine collaborative control parameters. Each path segment corresponds to a different operating condition, and these path segments are mapped to corresponding controllers, such as a fuzzy PID controller for blade speed adjustment and a laser power PID controller for laser power adjustment. By analyzing the required operating power, speed, and other parameters for each path segment, and comprehensively considering the collaborative work between the controllers, collaborative control parameters are determined. These parameters will be used to precisely control the operation of the weeding equipment, achieving efficient, safe, and energy-saving intelligent weeding operations in photovoltaic panel areas.

[0063] In one possible implementation, the 3D environment map building unit further includes:

[0064] The density heatmap drawing sub-unit is used to extract the weed density dataset based on the weed density distribution map, perform gridding on the weed density dataset, and draw the density heatmap.

[0065] The grading subunit is used to construct three-dimensional point cloud parameters of the photovoltaic panel area based on the multimodal perception dataset, fuse the three-dimensional point cloud parameters with the plant height data, and perform grading based on the fusion result to determine the processing priority of multiple vegetation types.

[0066] The association and integration subunit is used to associate and integrate the density heatmap with the plant height data according to the multiple vegetation processing priorities to construct the three-dimensional environment map.

[0067] Specifically, a weed density dataset was extracted from existing weed density distribution maps. This dataset contains weed density information at different locations within the photovoltaic panel area, accurately recording the density of weed growth in each region. To more intuitively present the weed density distribution characteristics and facilitate subsequent analysis, the extracted weed density dataset was gridded. Based on the actual extent and shape of the photovoltaic panel area, the entire region was divided into regular grids, each with corresponding coordinates. Then, the weed density data was filled in according to the grid positions, ensuring each grid had a clearly defined weed density value. After gridding, a density heatmap was drawn according to specific color mapping rules. A density value range was defined, and different density values ​​within this range were mapped to different colors; for example, lower weed density values ​​were mapped to light blue, higher density values ​​to dark red, and intermediate density values ​​were represented by a gradient from light blue to dark red. Each grid was filled with the corresponding color based on its weed density value, ultimately forming a complete density heatmap. In this image, the distribution of different colors clearly shows the variations in weed density within the photovoltaic panel area.

[0068] Utilizing LiDAR data from a multimodal sensing dataset, a voxel-based point cloud construction algorithm is employed. This algorithm divides the photovoltaic panel area into uniform voxels, filling the corresponding voxels with discrete point data obtained from LiDAR measurements according to their spatial locations. By calculating the statistical characteristics of points within the voxels, such as centroid and density, preliminary 3D point cloud parameters are generated, effectively reducing data redundancy and improving processing efficiency. In the data fusion stage, a weighted average-based fusion algorithm is used. Considering that the 3D point cloud parameters reflect spatial location information, while the plant height data reflects the vertical characteristics of the vegetation, weights are assigned based on their importance to the vegetation analysis. Assuming the weight of the 3D point cloud parameters is w1 and the weight of the plant height data is w2, and w1 + w2 = 1, for each point at a spatial location, its 3D point cloud coordinates are multiplied by w1, and its plant height value is multiplied by w2, then summed to obtain the fused coordinates, thus achieving the fusion of the two. In classifying and prioritizing vegetation treatment, the K-Means clustering algorithm is used. The merged data is used as input, and the number of clusters K is pre-defined, corresponding to different vegetation treatment priority levels. K initial cluster centers are randomly selected, and the distance from each data point to each cluster center is calculated, assigning the data point to the nearest cluster. Then, the center of each cluster is recalculated, iterating continuously until the cluster centers no longer change or change very little. After clustering, the clusters are sorted according to characteristics such as the average height and density of vegetation within each cluster. Areas corresponding to clusters with high height and density are determined as high-priority treatment areas, and vice versa, thus determining multiple vegetation treatment priorities.

[0069] The vegetation treatment priorities for each region were clearly defined. These priorities, derived from previous analysis of multimodal sensing data, represent the urgency and importance of weed control in different areas. Then, density heatmaps and plant height data were integrated based on these priorities. High-priority areas received more attention and detailed presentation. In the density heatmap, the distribution of weeds in the area was highlighted, clearly distinguishing its density information from the surrounding area, possibly through methods such as darkening colors or enhancing contrast. For plant height data, the height details of weeds within the area were accurately displayed, for example, using bar charts or contour lines representing different heights. For medium- and low-priority areas, their prominence on the map was appropriately reduced during the integration process, while retaining necessary information. Density heatmaps and plant height data were precisely matched spatially, ensuring that each location reflects both weed density and plant height information. Through this associative integration method, two-dimensional density heatmaps and one-dimensional plant height data were merged into a three-dimensional environmental map.

[0070] In one possible implementation, the initial global path generation unit further includes:

[0071] The safety distance constraint setting subunit is used to identify the coordinates of the photovoltaic pile foundation and the cable distribution information based on the path topology and set the cable safety distance constraint.

[0072] The buffer radius calculation subunit is used to calculate the radius of the bypass buffer according to the cable safety distance constraint, and to set the buffer boundary according to the buffer radius.

[0073] The candidate path node construction subunit is used to construct multiple candidate path nodes based on the buffer boundary using equidistant marking, filter the multiple candidate path nodes to generate a continuous Z-shaped detour path, and use the continuous Z-shaped detour path as the initial global path.

[0074] Specifically, a deep analysis of the established path topology is performed. This topology includes the positional relationships and traversable paths of various objects within the photovoltaic panel area, identifying the coordinates of the photovoltaic pile foundations. These coordinates precisely determine the location of the photovoltaic pile foundations within the panel area. Simultaneously, cable distribution information is obtained from the path topology, involving further processing of previously collected multimodal sensing data (such as lidar scanning data and electromagnetic induction data) to clarify the cable routes, burial depths, and specific location ranges. After identifying the coordinates of the photovoltaic pile foundations and the cable distribution information, cable safety distance constraints are set based on industry standards, equipment safety requirements, and practical operational experience. For example, considering factors such as vibration and steering deviation during the operation of weeding equipment, a safety distance of at least 20 centimeters must be maintained between the weeding equipment and the cables to prevent damage to the cables during weeding operations.

[0075] A 3D model of the cable is constructed using multimodal sensing data, and the cable's central axis is extracted through model processing. Based on pre-defined cable safety distance constraints, a cylindrical safety space is constructed along the central axis to ensure that weeding equipment does not intrude within a certain range around the cable. Next, local path planning is performed outside the cylindrical safety space, constructing weeding distance data containing information on the distance between each path and the cable. During this process, path conflict is continuously determined based on the weeding distance data. Once a conflict is detected between a path and other obstacles or safety areas, a detour sub-path is inserted. Then, the curvature of these detour sub-paths is calculated, and the detour buffer radius is determined by considering factors such as the turning performance and operational stability of the weeding equipment. Finally, based on the calculated buffer radius, a cylindrical surface is defined in 3D space using the cable's central axis as a reference, and equidistant closed curves are drawn on a 2D map to define the buffer boundary. This provides a clear safety limit for subsequent weeding path planning, preventing weeding equipment from approaching the cable during operation and ensuring the safety of the cable in the photovoltaic panel area.

[0076] Operating based on buffer boundaries, the algorithm evenly constructs multiple candidate path nodes by marking them at equal intervals along the buffer boundaries. These nodes constitute a potential path selection set, providing a foundation for generating suitable detour paths. Subsequently, a screening algorithm is used to filter the numerous candidate path nodes, comprehensively considering factors such as path length, path smoothness, obstacle density, and whether they meet the requirements of weeding operations. After screening, suitable nodes are connected to generate a continuous zigzag detour path. This zigzag path satisfies cable safety distance constraints while reducing path length to some extent, thus improving weeding efficiency. Ultimately, this continuous zigzag detour path is determined as the initial global path, providing an important basic framework for subsequent weeding operation planning and execution.

[0077] In one possible implementation, the buffer radius calculation subunit includes:

[0078] The central axis is extracted into micro-units to construct a three-dimensional model of the cable, and the central axis is extracted based on the three-dimensional model of the cable.

[0079] The safety space is constructed using micro-units, which are used to construct cylindrical safety spaces along the central axis based on the cable safety distance constraints.

[0080] Weeding distance data is used to construct micro-units for planning local paths outside the cylindrical safety space, thereby constructing weeding distance data.

[0081] The path conflict determination micro-unit is used to determine path conflicts based on the weeding distance data. If a path conflict exists, a detour sub-path is inserted.

[0082] A path curvature calculation micro-unit is used to calculate the path curvature based on the bypass sub-path and determine the radius of the bypass buffer zone.

[0083] Specifically, a lidar system is used to perform a comprehensive scan of the cable's location, acquiring high-density point cloud data. This data precisely records the spatial location information of the cable surface and its surrounding environment. Simultaneously, electromagnetic induction detection technology is used to obtain data on the magnetic field distribution generated by the current inside the cable, thus more accurately locating the cable's direction and position. The acquired point cloud data and electromagnetic induction data are fused to remove noise and outliers, improving data quality. Next, deep learning-based 3D reconstruction algorithms, such as PointNet++, are used to train and process the fused data, constructing an accurate 3D model of the cable. This model can meticulously represent the cable's shape, curvature, and spatial location. When extracting the central axis, the 3D cable model is first skeletonized. Through distance transformation and thinning algorithms, the cable model is simplified into a one-dimensional skeleton structure, which approximates the cable's central axis. Then, curve fitting algorithms, such as least squares B-spline curve fitting, are used to optimize the skeleton, removing redundant points and local fluctuations, making the central axis smoother and more accurate, ultimately yielding a central axis that represents the core direction of the cable.

[0084] After clearly defining the cable's centerline, constructing a cylindrical safety space based on cable safety distance constraints is crucial for ensuring the safety of weeding operations. First, the actual radius of the cable is obtained through prior cable laying records or measurements using high-precision detection equipment. Then, according to the established safety distance constraints, the cable radius is added to a fixed safety distance of 20cm to calculate the radius of the cylindrical safety space. For example, if the cable radius is 5cm, the radius of the cylindrical safety space is 25cm. Using the extracted cable centerline as a reference, a complete cylindrical safety space is generated in three-dimensional space, from the beginning to the end of the axis, using the calculated radius as a standard, through spatial geometry construction methods. This space strictly adheres to the cable safety distance constraints, effectively preventing contact between the weeding equipment and the cable during operation, providing a clear safety boundary for subsequent weeding path planning, and ensuring the safety of cables in the photovoltaic panel area during weeding operations.

[0085] Based on environmental information such as the topography and distribution of photovoltaic piles in the photovoltaic panel area, path planning algorithms (such as A* algorithm and Dijkstra's algorithm) are used to search for potential paths within the passable area outside the safe space. During the planning process, cable safety distance constraints are used as a key limiting condition to ensure that the minimum distance between any point on each generated local path and the cable is greater than or equal to 20cm. Along the planned local path, the location information of path points is collected at fixed intervals (e.g., every 10cm), and the vertical distance from these points to the cable's central axis is calculated. This distance is then subtracted from the cable radius to obtain the actual distance between the point and the cable. These distance data are recorded sequentially according to the path order to form weeding distance data. For example, a local path may contain 100 collection points, and the distance data corresponding to each point collectively constitutes the weeding distance data for that path. The weeding distance data constructed in this way not only ensures the safe distance between the weeding equipment and the cable but also provides accurate quantitative basis for subsequent path conflict determination and buffer radius calculation, ensuring the safety and feasibility of the weeding path planning.

[0086] A conflict detection threshold is set (e.g., less than 20cm of the safety distance constraint), and each distance value in the weeding distance data is compared with this threshold. If a distance value less than the threshold is found on a path segment, it is determined that the path segment conflicts with the cable safety space. Simultaneously, if the weeding distance data shows that the path is too close to photovoltaic piles, other obstacles, etc., it is also considered a path conflict. Once a conflict is detected, a path repair mechanism is immediately activated. Centered on the conflict point, within the surrounding passable area, a new detour sub-path is replanned using a preset path search strategy (e.g., bidirectional search, heuristic search) and the A* algorithm or RRT (Rapid Expanding Random Tree) algorithm to avoid the conflict area. When generating the detour sub-path, the cable safety distance constraint is also followed to ensure that the distance between every point on the new path and the cable meets safety requirements. Finally, the detour sub-path is inserted into the conflict location of the original path, replacing the conflicting path segment, thus forming a complete and safe new path, providing reliable assurance for subsequent path planning and weeding operations.

[0087] The discrete curvature calculation method is used to discretize the bypass sub-path into a series of ordered path points. The curvature value at each path point is calculated by the rate of change of the angle between the vectors formed by three adjacent points. For example, for three consecutive points P on the path... i-1 P i P i+1 The curvature ki of the point is calculated using vector cross product and dot product operations, combined with trigonometric function relationships, and then... The radius of curvature is obtained. The radius of curvature of all path points is compared to 1m. If any is less than 1m, the path is locally adjusted and optimized by inserting new path points or using a smoothing algorithm to ensure that the radius of curvature of the entire detour sub-path is greater than 1m, guaranteeing that the weeding equipment can turn safely and smoothly. When determining the detour buffer radius, the path curvature, the size of the weeding equipment, and safety margins are comprehensively considered. For sections with greater curvature (more curved paths), the buffer radius is appropriately increased to prevent the equipment from exceeding the safety range when turning; while for sections with less curvature (straighter paths), the buffer radius is correspondingly reduced to improve operational efficiency. For example, when the path curvature radius is 1.2m, the buffer radius is set to 0.5m, considering the equipment width and turning performance; if the path curvature radius reaches 2m, the buffer radius can be adjusted to 0.3m. Finally, through precise calculation of the path curvature and comprehensive consideration of multiple factors, the detour buffer radius that meets both safety and operational requirements is determined.

[0088] In one possible implementation, the control feedback module 30 further includes:

[0089] The parameter parsing unit is used to parse the cooperative control parameters to obtain the target blade rotation speed parameters, laser power setting value, and energy distribution mode.

[0090] The first control feedback result acquisition unit is used to perform control feedback on the blade speed PID controller according to the target blade speed parameters to obtain the first control feedback result.

[0091] The second control feedback result acquisition unit is used to perform control feedback on the laser power PID controller according to the laser power set value to obtain the second control feedback result.

[0092] The coupled control analysis unit is used to perform multi-parameter coupled control analysis on the first control feedback result and the second control feedback result based on the energy distribution mode, update the cooperative control parameters to generate control commands, and execute adaptive weeding operations in the photovoltaic panel area.

[0093] Specifically, data analysis algorithms are used to meticulously break down the determined collaborative control parameters, separating the target blade rotation speed parameter, laser power setpoint, and energy distribution mode from the parameter set. The target blade rotation speed parameter clarifies the rotational speed the weeding blade should achieve in different operating scenarios; the laser power setpoint specifies the energy output intensity of the laser weeding device; and the energy distribution mode comprehensively plans the energy supply ratio between blade rotation and laser emission. Through this analysis process, the originally comprehensive collaborative control parameters are transformed into specific, executable sub-parameters.

[0094] In the intelligent weeding system for photovoltaic panels, precise control of blade speed relies on the coordinated operation of a fuzzy PID controller and the target blade speed parameters. First, the target blade speed parameters (Nset) and laser power setpoint (Pset) are acquired to provide a reference for subsequent control. The fuzzy PID controller rapidly calculates the speed deviation (eN = Nset - Nactual) and the deviation change rate (ΔeN) by real-time acquisition of the blade speed encoder feedback value (Nactual), and uses these as input parameters. Based on pre-set fuzzy control rules, the controller fuzzifies the speed deviation and deviation change rate, determines the adjustment amounts of the proportional (P), integral (I), and derivative (D) parameters through fuzzy inference, and generates the first control feedback result. The corresponding PWM duty cycle signal is then output to the drive motor to achieve dynamic adjustment of the blade speed.

[0095] Meanwhile, the laser power PID controller collects the energy feedback value (Pactual) from the photoelectric sensor in real time, calculates the power deviation (eP = Pset - Pactual), and uses an incremental PID algorithm to calculate the increment of the control quantity based on the power deviation and the control output at the previous moment, generating a second control feedback result. This result is used to adjust the drive current and precisely control the laser power. The two work together to ensure that both the blade speed and laser power of the weeding equipment consistently meet the set requirements, providing a reliable guarantee for efficient and precise weeding operations.

[0096] Centered on the energy allocation mode, this system deeply integrates the first and second control feedback results to achieve precise multi-parameter coupled control. First, based on the energy allocation mode, the first control feedback result (blade speed PWM duty cycle signal) is input to the blade speed PID controller. This is combined with weeding environment parameters (such as weed density and path curvature) and equipment operating status (such as motor temperature and energy consumption) for multi-parameter coupled analysis. When energy allocation prioritizes laser operation, if excessively high blade speed and low weed density are detected, the controller generates a blade mode switching command to reduce blade speed or switch to energy-saving mode. Similarly, based on the second control feedback result (laser power drive current adjustment signal), combined with parameters such as laser spot size and energy loss rate, a multi-parameter coupled analysis is performed on the laser power PID controller. When energy allocation prioritizes blade operation, a laser mode switching command to reduce laser power is generated. Subsequently, the blade mode switching command and the laser mode switching command are incorporated into a closed-loop feedback control system. The system monitors the equipment response status in real time, collects data such as motor speed and actual laser power, and generates collaborative control status parameters. Finally, based on the collaborative control state parameters, the collaborative control parameters are dynamically updated, and the optimized parameters are transformed into control commands to precisely drive the weeding equipment, ensuring that the blades and lasers work together efficiently under different working conditions, and realizing the precision and intelligence of adaptive weeding operations in the photovoltaic panel area.

[0097] In one possible implementation, the coupling control analysis unit further includes:

[0098] The blade mode switching instruction generation subunit is used to perform multi-parameter coupling analysis on the blade speed PID controller based on the energy distribution mode and the first control feedback result, and generate blade mode switching instructions.

[0099] The laser mode switching instruction generation subunit is used to perform multi-parameter coupling analysis on the laser power PID controller based on the energy distribution mode and the second control feedback result, and generate laser mode switching instructions.

[0100] The closed-loop feedback control subunit is used to perform closed-loop feedback control according to the blade mode switching command and the laser mode switching command, and generate cooperative control state parameters.

[0101] A control command generation subunit is used to generate the control command based on the updated cooperative control parameters according to the cooperative control state parameters.

[0102] Specifically, based on the established energy allocation mode, multi-parameter coupling analysis is performed using the first control feedback result generated by the blade speed PID controller to generate blade mode switching commands. First, information such as weed density, plant height, and Normalized Difference Vegetation Index (NDVI) of the current working area is acquired. Based on the energy allocation mode, the appropriate weeding method is determined. For example, when weed density is low, plants are short, and the NDVI value indicates weak weed activity, energy allocation tends towards laser mode; conversely, it tends towards blade mode or a hybrid mode. Simultaneously, the first control feedback result is obtained from the blade speed PID controller, including data such as the deviation between the current actual blade speed and the target speed, and the rate of change of this deviation. If the energy distribution favors the laser mode and the first control feedback indicates that the blade speed is too high, combined with environmental parameters such as low weed density, multi-parameter coupling analysis determines that the blade speed can be reduced to save energy. At this time, a blade mode switching command to reduce the speed is generated. If it is in mixed mode and the weed density is high, and the first control feedback indicates that the blade speed is lower than expected, an command to increase the speed will be generated after analysis, so that the blade speed matches the energy distribution mode and the actual weeding needs, ensuring that the weeding operation is efficient and energy-saving.

[0103] Based on the energy allocation mode, the system deeply integrates the second control feedback results generated by the laser power PID controller, performs multi-parameter coupling analysis, and then generates precise laser mode switching commands. Real-time data such as weed density, plant height, and NDVI are acquired to determine the laser's operating mode preference based on the energy allocation mode. If weed density is low, plant height is short, and NDVI values ​​indicate low weed activity, the energy allocation mode typically favors low-power laser operation; conversely, in areas with vigorous weed growth, the energy allocation favors high-power laser or a hybrid mode. At this point, the second control feedback result, i.e., the deviation between the actual laser power and the set power, is obtained from the laser power PID controller. When the energy distribution mode favors low-power laser operation and the second control feedback indicates that the actual laser power is too high, the laser power can be reduced by combining the current weed density and multi-parameter coupling analysis, thereby generating a laser mode switching command to reduce power. If it is in a mixed mode or high-power demand mode and the feedback indicates that the laser power is insufficient, the laser power can be increased by combining parameters such as the density of weeds and plant height. This allows the laser power to be precisely matched with the energy distribution mode and the actual weeding needs, achieving efficient and energy-saving weeding operations.

[0104] A dual-loop PID control algorithm combined with state-space analysis is employed to implement closed-loop feedback control for blade mode switching commands and laser mode switching commands, generating coordinated control state parameters. In the blade control loop, the speed or torque set by the blade mode switching command is used as the target value. The actual blade speed and current are collected as feedback signals, and the control quantity is calculated using an incremental PID algorithm to continuously adjust the motor drive voltage to reduce deviation. Similarly, in the laser control loop, the power and spot parameters of the laser mode switching command are used as the benchmark. A positional PID algorithm is used to adjust the drive current in real time based on data from the photodetector and spot monitor. To achieve coordination between the two, state-space analysis is introduced. Variables such as blade speed, motor current, laser power, and equipment temperature are constructed into a state vector, and the dynamic relationship between variables is described by a state transition matrix. The changes in the state vector are monitored in real time. A Kalman filter algorithm is used to optimize the feedback data under noise interference. Then, a weighted summation algorithm is used to fuse the control parameters of the blade and laser. Finally, coordinated control state parameters containing key indicators such as equipment operating efficiency, energy consumption ratio, and coordination matching degree are output to ensure the accuracy and stability of weeding operations.

[0105] A mapping relationship is established between the collaborative control state parameters and the original collaborative control parameters, corresponding one-to-one with indicators such as blade operating status, laser output efficiency, equipment coordination level, and energy consumption level to original parameters such as target blade rotation speed parameters, laser power setpoint, and energy distribution mode. An adaptive genetic algorithm is used to iteratively optimize the original collaborative control parameters with the optimization objectives of maximizing weeding efficiency and minimizing energy consumption. For example, if the collaborative control state parameters indicate excessive laser energy consumption and weeding efficiency not meeting expectations, the algorithm will adjust the energy distribution mode, appropriately reduce the laser power setpoint, and correspondingly increase the blade rotation speed parameter. After multiple rounds of optimization, new collaborative control parameters are generated and transformed into control commands containing precise blade rotation speed control signals, laser power adjustment signals, and energy distribution strategies. These commands drive the precise execution of each component of the weeding equipment, achieving adaptive optimization and efficient operation of weeding in the photovoltaic panel area.

[0106] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0107] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0108] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A smart weed control system for photovoltaic panel areas based on multimodal sensing, characterized in that, The system includes: The feature extraction module is used to collect multispectral image data of the photovoltaic panel area, generate a multimodal sensing dataset, extract features from the multimodal sensing dataset, and obtain vegetation analysis results based on the sensing feature set. The collaborative control parameter determination module is used to perform path planning based on the vegetation analysis results, generate a weeding path, perform dynamic energy allocation according to the weeding path, and determine the collaborative control parameters. The control feedback module is used to provide control feedback to the controller of the photovoltaic panel area according to the collaborative control parameters, and to update the collaborative control parameters according to the feedback results to execute intelligent weeding operations in the photovoltaic panel area. The control feedback module also includes: The parameter parsing unit is used to parse the cooperative control parameters to obtain the target blade rotation speed parameters, laser power setpoint, and energy distribution mode. The first control feedback result acquisition unit is used to perform control feedback on the blade speed PID controller according to the target blade speed parameter to obtain the first control feedback result. The second control feedback result acquisition unit is used to perform control feedback on the laser power PID controller according to the laser power set value, and obtain the second control feedback result. The coupled control analysis unit is used to perform multi-parameter coupled control analysis on the first control feedback result and the second control feedback result based on the energy distribution mode, update the cooperative control parameters to generate control commands, and execute adaptive weeding operations in the photovoltaic panel area.

2. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 1, characterized in that, The feature extraction module further includes: The acquisition parameter setting unit is used to set the acquisition parameters of the multispectral camera, including resolution parameters and near-infrared band parameters. An image acquisition unit is used to activate a multispectral camera based on the acquisition parameters, acquire an RGB image parameter set according to the resolution parameters, and acquire a near-infrared image set according to the near-infrared band parameters. The spatiotemporal registration unit is used to perform spatiotemporal registration of the RGB image parameter set with the near-infrared image set to generate a spatiotemporally aligned image set. The distortion correction unit is used to perform distortion correction based on the spatiotemporally aligned image set to obtain the multimodal sensing dataset.

3. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 1, characterized in that, The feature extraction module further includes: The sensing feature set acquisition unit is used to traverse the multimodal sensing dataset to identify weed features in the photovoltaic panel area, obtain a sensing feature set, and draw a weed probability heat map based on the sensing feature set. The weed probability heat map includes weed confidence. The weed distribution area determination unit is used to perform connected component analysis and labeling based on the weed probability heatmap to determine the weed distribution area. The weed density distribution map acquisition unit is used to calculate the weed distribution area by combining the multimodal sensing dataset to obtain the weed density distribution map; The plant height data acquisition unit is used to calculate the vegetation depth of the photovoltaic panel area based on the multimodal sensing dataset and the weed density distribution map, and obtain plant height data. The spatial overlay unit is used to spatially overlay the weed density distribution map and the plant height data according to the weed confidence level to generate the vegetation analysis results.

4. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 3, characterized in that, The perceptual feature set acquisition unit further includes: The weed feature extraction subunit is used to traverse the multimodal sensing dataset based on the improved YOLOv7 network and extract multi-scale weed features. The backbone network of the improved YOLOv7 network consists of CSPDarknet53 and SPPF modules. The matching calculation subunit is used to perform matching calculations on the multimodal sensing dataset based on the multi-scale weed features to obtain the weed confidence scores of multiple pixels in the multimodal sensing dataset. The regional probability statistics subunit is used to perform regional probability statistics based on the weed confidence of the multiple pixels and generate a regional weed probability distribution matrix. The bilinear interpolation subunit is used to perform bilinear interpolation based on the probability distribution matrix of weeds in the region to draw a probability heat map of weeds in the photovoltaic panel area.

5. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 4, characterized in that, The collaborative control parameter determination module further includes: A 3D environment map construction unit is used to perform 3D analysis based on the weed density distribution map and construct a 3D environment map. The path topology construction unit is used to traverse the three-dimensional environment map to perform weeding analysis of the photovoltaic panel area and construct the path topology. The initial global path generation unit is used to set cable safety distance constraints, and to plan a weeding path according to the path topology and the cable safety distance constraints to generate an initial global path. The path segmentation determination unit is used to dynamically allocate energy by using the initial global path as a weeding path and determine multiple path segments. The control analysis unit is used to map the multiple paths into multiple controllers for control analysis and to determine the cooperative control parameters.

6. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 5, characterized in that, The three-dimensional environment map construction unit also includes: The density heatmap drawing subunit is used to extract the weed density dataset based on the weed density distribution map, perform gridding on the weed density dataset, and draw the density heatmap. The grading subunit is used to construct three-dimensional point cloud parameters of the photovoltaic panel area based on the multimodal perception dataset, fuse the three-dimensional point cloud parameters with the plant height data, and perform grading based on the fusion result to determine the processing priority of multiple vegetation types. The association and integration subunit is used to associate and integrate the density heatmap with the plant height data according to the multiple vegetation processing priorities to construct the three-dimensional environment map.

7. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 5, characterized in that, The initial global path generation unit further includes: The safety distance constraint setting subunit is used to identify the coordinates of the photovoltaic pile foundation and the cable distribution information based on the path topology, and to set the cable safety distance constraint. The buffer radius calculation subunit is used to calculate the radius of the bypass buffer according to the cable safety distance constraint, and to set the buffer boundary according to the buffer radius; The candidate path node construction subunit is used to construct multiple candidate path nodes based on the buffer boundary using equidistant marking, filter the multiple candidate path nodes to generate a continuous Z-shaped detour path, and use the continuous Z-shaped detour path as the initial global path.

8. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 7, characterized in that, The buffer radius calculation subunit includes: The central axis is extracted into micro-units to construct a three-dimensional model of the cable, and the central axis is extracted based on the three-dimensional model of the cable. A safety space construction micro-unit is used to construct a cylindrical safety space along the central axis based on the cable safety distance constraint. Weeding distance data is used to construct micro-units for planning local paths outside the cylindrical safety space and constructing weeding distance data; The path conflict determination micro-unit is used to determine path conflicts based on the weeding distance data. If a path conflict exists, a detour sub-path is inserted. A path curvature calculation micro-unit is used to calculate the path curvature based on the bypass sub-path and determine the radius of the bypass buffer zone.

9. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 1, characterized in that, The coupling control analysis unit further includes: The blade mode switching instruction generation subunit is used to perform multi-parameter coupling analysis on the blade speed PID controller based on the energy distribution mode and the first control feedback result, and generate blade mode switching instructions. The laser mode switching instruction generation subunit is used to perform multi-parameter coupling analysis on the laser power PID controller based on the energy distribution mode and the second control feedback result, and generate a laser mode switching instruction. The closed-loop feedback control subunit is used to perform closed-loop feedback control according to the blade mode switching command and the laser mode switching command, and generate cooperative control state parameters. A control command generation subunit is used to generate the control command based on the updated cooperative control parameters according to the cooperative control state parameters.

Images