Photovoltaic panel area intelligent weeding control system based on multi-mode sensing

Abstract

The invention discloses a photovoltaic panel area intelligent weeding control system based on multi-mode perception, and relates to the technical field of weeding control, and the system comprises a feature extraction module which is used for collecting multispectral image data of a photovoltaic panel area and obtaining a vegetation analysis result; the cooperative control parameter determination module is used for performing path planning, generating a weeding path, performing dynamic energy distribution and determining cooperative control parameters; and the control feedback module is used for performing control feedback on the controller of the photovoltaic panel area and executing intelligent weeding operation of the photovoltaic panel area. The technical problems that in the prior art, photovoltaic panel area weed identification is not accurate, the weeding efficiency is low, energy distribution is not reasonable, the weeding effect is poor and energy is wasted are solved, and the technical effects that photovoltaic panel area weeds are accurately identified and efficiently removed, and the weeding efficiency and the energy utilization efficiency are effectively improved are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of weed control, and in particular to a photovoltaic panel area intelligent weed control system based on multi-modal perception. BACKGROUND

[0002] With the continuous expansion of photovoltaic power generation scale, weed cleaning in photovoltaic panel areas has become a key work to ensure power generation efficiency and equipment safety. At present, manual weeding is inefficient, high-cost, and difficult to implement large-scale operation; mechanical weeding lacks automation, is easy to damage photovoltaic panels, and lacks comprehensive perception and intelligent analysis of environmental information, and cannot carry out targeted operation according to weed growth conditions, distribution density, etc., resulting in problems such as energy waste and poor weeding effect. At the same time, the existing technology lacks dynamic optimization mechanism in the aspects of weeding path planning and energy distribution, and is difficult to meet the complex and changeable environmental requirements of photovoltaic panel areas.

[0003] The existing technology has the technical problems of inaccurate weed identification in photovoltaic panel areas, low weeding efficiency, unreasonable energy distribution, which easily leads to poor weeding effect and energy waste. SUMMARY

[0004] The present application provides a photovoltaic panel area intelligent weed control system based on multi-modal perception, which is used to solve the technical problems of inaccurate weed identification in photovoltaic panel areas, low weeding efficiency, unreasonable energy distribution, which easily leads to poor weeding effect and energy waste in the prior art.

[0005] In view of the above problems, the present application provides a photovoltaic panel area intelligent weed control system based on multi-modal perception.

[0006] The present application provides a photovoltaic panel area intelligent weed control system based on multi-modal perception, which comprises:

[0007] A feature extraction module is configured to collect multispectral image data of the photovoltaic panel area, generate a multi-modal perception data set, perform feature extraction on the multi-modal perception data set, and obtain a vegetation analysis result according to the perception feature set; a cooperative control parameter determination module is configured to perform path planning according to the vegetation analysis result, generate a weeding path, perform dynamic energy distribution according to the weeding path, and determine cooperative control parameters; and a control feedback module is configured to perform control feedback on the controller of the photovoltaic panel area according to the cooperative control parameters, update the cooperative control parameters according to the feedback result, and perform intelligent weeding operation of the photovoltaic panel area.

[0008] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0009] The system includes a feature extraction module for collecting multispectral image data of the photovoltaic panel area, generating a multimodal sensing dataset, extracting features, and obtaining vegetation analysis results. A collaborative control parameter determination module is used to plan paths based on the vegetation analysis results, generate weeding paths, perform dynamic energy allocation, and determine collaborative control parameters. A control feedback module provides control feedback to the controller in the photovoltaic panel area, updates the collaborative control parameters according to the feedback results, and executes intelligent weeding operations in the photovoltaic panel area. This achieves accurate identification and efficient removal of weeds in the photovoltaic panel area, effectively improving weeding efficiency and energy utilization efficiency. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This application provides a schematic diagram of the structure of an intelligent weed control system for photovoltaic panel areas based on multimodal perception.

[0012] Figure 2 This application provides a schematic diagram of the feature extraction module in an intelligent weeding control system for photovoltaic panel areas based on multimodal perception.

[0013] Figure labeling: Feature extraction module 10, collaborative control parameter determination module 20, control feedback module 30. Detailed Implementation

[0014] This application provides an intelligent weeding control system for photovoltaic panel areas based on multimodal perception, which addresses the technical problems in existing technologies such as inaccurate weed identification, low weeding efficiency, and unreasonable energy distribution in photovoltaic panel areas, which easily lead to poor weeding results and energy waste.

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0016] Examples, such as Figure 1 As shown, this application provides an intelligent weed control system for photovoltaic panel areas based on multimodal sensing, the system comprising:

[0017] The feature extraction module 10 is configured to collect multispectral image data of the photovoltaic panel area, generate a multi-modal perception data set, perform feature extraction on the multi-modal perception data set, and obtain vegetation analysis results based on the perception feature set.

[0018] Specifically, first, the collection parameters of the multispectral camera are accurately set according to the environmental characteristics of the photovoltaic panel area and the weeding task requirements, including resolution parameters, near-infrared band parameters, and the like. Reasonable setting of these parameters is a prerequisite for obtaining high-quality image data. Next, the image set collection unit activates the multispectral camera according to the set parameters, synchronously collects the RGB image parameter set and the near-infrared image set, and thus obtains rich spectral information. After the collection is completed, the RGB image parameter set and the near-infrared image set are time and space registered to eliminate the deviation caused by the difference in collection time and angle, generate a time and space aligned image set, and ensure that the information in different spectral images can be accurately corresponded. Subsequently, the distortion correction unit performs distortion correction on the time and space aligned image set to further improve the image quality, and finally obtains a multi-modal perception data set. Based on the improved YOLOv7 network, the multi-modal perception data set is traversed. The backbone network CSPDarknet53 of the network combines the SPPF module to enable fast inference. The multi-task branch design plays a key role, the weed detection branch (confidence > 0.9) extracts multi-scale weed features, matches the data set based on these features, obtains the weed confidence of multiple pixels, performs regional probability statistics based on this, generates a regional weed probability distribution matrix, and then draws a weed probability heat map to intuitively display the distribution possibility of weeds in the photovoltaic panel area. At the same time, the NDVI calculation branch evaluates the vegetation health degree to obtain the NDVI index, which reflects the growth state of the vegetation. Finally, based on the weed probability heat map, the weed distribution area is determined through connected domain analysis marking; the weed density distribution map is generated by calculating the weed density based on the multi-modal perception data set; the plant height data are obtained based on the multi-modal perception data set and the weed density distribution map; and the weed density distribution map and the plant height data are spatially superimposed according to the weed confidence, the weed probability heat map and the NDVI index are comprehensively generated, and a comprehensive and detailed vegetation analysis result is generated, which provides an important basis for the subsequent cooperative control parameter determination module to perform path planning and energy allocation.

[0019] The cooperative control parameter determination module 20 is configured to perform path planning based on the vegetation analysis result, generate a weeding path, perform dynamic energy allocation according to the weeding path, and determine cooperative control parameters.

[0020] Specifically, first, a three-dimensional analysis is performed on the weed density distribution map, the weed density dataset is extracted and grid processed, and a density heat map is drawn; combined with the multi-modal perception dataset, a three-dimensional point cloud parameter is constructed, the plant height data is fused for grade division, and the vegetation processing priority is determined; according to the priority, the density heat map and the plant height data are associated and integrated to construct a three-dimensional environment map that comprehensively reflects the environment of the photovoltaic panel area. Then, traverse the three-dimensional environment map, analyze the weeding demand and obstacle distribution of the photovoltaic panel area, and construct a reasonable path topology structure. Based on this structure, the photovoltaic pile foundation coordinates and cable distribution information are identified, and the cable safety distance constraint is set. By constructing a three-dimensional cable model, the central axis is extracted, a cylindrical safety space is constructed along the axis, a local path is planned outside the space, a weeding distance data is constructed, a path conflict is determined and processed, a path curvature is calculated to determine the radius of the bypass buffer zone, and the buffer zone boundary is set accordingly. Then, candidate path nodes are constructed at equal intervals on the buffer zone boundary, and a continuous zigzag bypass path, i.e. an initial global path, is generated. Then, the initial global path is used as the weeding path, and dynamic energy allocation is performed according to the weed conditions and environmental characteristics of different sections. For example, more energy is allocated in areas with dense weeds, and less energy is allocated in areas with sparse weeds, to determine multiple path segments. Finally, these path segments are mapped to multiple controllers for control analysis. For each path segment, factors such as weed density, plant height, NDVI index, etc. are considered, and a fuzzy logic controller of the dynamic energy allocation model is used to optimize the collaborative control parameters of laser power and blade speed. When the weed density is less than 5 plants / m 2 , the plant height is less than 8 cm, and the NDVI is less than 0.3, the laser mode is used, the laser power is set to 15W, and the blade speed is 0; otherwise, the hybrid mode is used, the laser power is 20W, and the blade speed is 1800rpm. Through these steps, the collaborative control parameters including laser power, blade speed, etc. are finally determined, which provides accurate basis for the subsequent control feedback module to accurately control the weeding equipment, and ensures that the weeding operation is efficient, safe and energy-saving.

[0021] The control feedback module 30 is configured to control the controllers of the photovoltaic panel area according to the collaborative control parameters, update the collaborative control parameters according to the feedback results, and perform intelligent weeding operation of the photovoltaic panel area.

[0022] Specifically, first, the cooperative control parameters are analyzed to obtain key information such as target blade speed parameters, laser power set values, and energy distribution modes. These parameters are the basis for controlling the operation of the weeding device and determine the running state of the device in different scenarios. Subsequently, the target blade speed parameters are input into the fuzzy PID controller. The fuzzy PID controller combines the actual collected blade speed feedback signal and adjusts the control parameters adaptively through fuzzy logic. When the actual speed is lower than the target value, the controller appropriately increases the control signal to increase the blade speed; otherwise, it reduces the speed to accurately adjust the blade speed and keep it stable around the target value, ensuring the consistency of the weeding effect. At the same time, according to the laser power set value, the laser power PID controller is used to accurately control the laser power. The laser power PID controller adjusts the laser power in real time according to the feedback signal of the laser output power to realize energy feedback control. In areas with more weeds and more difficult to remove, the laser power is appropriately increased; in areas with fewer weeds, the laser power is reduced, ensuring the weeding effect and avoiding energy waste. After completing the above control feedback operations, based on the energy distribution mode, the first control feedback result and the second control feedback result are comprehensively analyzed for multi-parameter coupling control. If it is in laser mode, the stable output of laser power and the weeding effect will be focused on; if it is in mixed mode, the laser power and the blade speed will be coordinated to achieve the best cooperation of the two. Through this analysis, the cooperative control parameters are dynamically updated to generate control instructions that are more in line with the actual operation situation. During the entire weeding operation process, feedback information is continuously obtained, and the cooperative control parameters are continuously updated, so that the weeding device can adapt to the complex and variable environment of the photovoltaic panel area, automatically adjust the working state, realize efficient and intelligent weeding operation, effectively improve the weeding efficiency and quality, and ensure the normal operation of the photovoltaic panel.

[0023] In one possible implementation manner, as shown in Figure 2 The feature extraction module 10 further includes:

[0024] A collection parameter setting unit is configured to set collection parameters of the multi-spectral camera, wherein the collection parameters include a resolution parameter and a near-infrared wave band parameter.

[0025] An image set collection unit is configured to activate the multi-spectral camera based on the collection parameters, collect an RGB image parameter set according to the resolution parameter, and collect a near-infrared image set according to the near-infrared wave band parameter.

[0026] A space-time registration unit is configured to perform space-time registration on the RGB image parameter set and the near-infrared image set to generate a space-time aligned image set.

[0027] A distortion correction unit is configured to perform distortion correction based on the space-time aligned image set to obtain the multi-modal perception data set.

[0028] Specifically, setting the acquisition parameters of the multispectral camera is a key step to obtain accurate image data. The setting of the resolution parameter needs to consider the actual area of the photovoltaic panel area, the distribution range of the weeds, and the accuracy requirement of subsequent weed feature recognition. If the photovoltaic panel area is large and the weeds are distributed relatively dispersedly, in order to fully cover and accurately identify the weeds, a higher resolution, such as 4096x3072 pixels, needs to be set to clearly present the morphological details of the weeds, including leaf texture, plant spacing, etc., which is helpful for the accurate judgment of the weed species and growth conditions in the subsequent stage. If the panel area is small and the key area is clear, a lower resolution, such as 1920x1080 pixels, may be sufficient to meet the demand, while reducing the data processing amount and improving the running efficiency. The setting of the near-infrared waveband parameter focuses on the characteristics of the vegetation. The near-infrared light is sensitive to the reflection characteristics of plants, and the reflectivity of vegetation with different health conditions in the near-infrared waveband is significantly different. Generally, in order to effectively distinguish weeds from other vegetation and background, the near-infrared waveband in the range of 760-1100 nanometers will be selected for acquisition. For photovoltaic panel areas dominated by gramineous weeds, the 780-900 nanometer waveband data can be collected, which can highlight the reflection difference between gramineous weeds and the surrounding environment, and enhance the recognition degree of weeds in the image. If the panel area has diverse weed species, the near-infrared waveband acquisition range can be appropriately widened, such as 760-1000 nanometers, to obtain more comprehensive vegetation information, and provide rich data support for subsequent vegetation health assessment (such as calculating the NDVI index) and accurate weed detection.

[0029] When the acquisition parameter setting is completed, the image acquisition stage is entered. The multispectral camera is activated according to the set acquisition parameters. After receiving the activation instruction, the camera will first carry out the acquisition of the RGB image parameter set according to the resolution parameter. If the resolution parameter is set to a higher 4096x3072 pixels, the camera will capture the color image of the photovoltaic panel area with high definition, and record the color information of the weeds, photovoltaic panels and surrounding environment in the area, such as the green of the weed leaves and the blue of the photovoltaic panels, etc. These RGB image parameter sets can provide intuitive color features for the subsequent visual recognition of weeds. At the same time, the camera will collect the near-infrared image set according to the near-infrared waveband parameter. If the near-infrared waveband parameter is set in the range of 760-1100 nanometers, the camera will focus on the light reflection in this waveband to obtain the reflection image of the vegetation in the photovoltaic panel area in the near-infrared waveband. Different health status of weeds will show different reflectivity in the near-infrared waveband. Healthy weeds have higher reflectivity, while weeds affected by diseases and pests or growing poorly have lower reflectivity. By collecting the near-infrared image set, key information such as the health status of the vegetation can be obtained.

[0030] After obtaining the RGB image parameter set and the near-infrared image set, in order to ensure the accuracy and effectiveness of subsequent analysis, it is necessary to perform time-space registration on the two groups of images to generate a time-space aligned image set. Due to the slight vibration of the device, the real-time change of light, and other factors, even if the time interval is extremely short, there may be a certain deviation in time and space between the two groups of images when the multispectral camera collects RGB images and near-infrared images. Time-space registration first analyzes the time stamps of the RGB image parameter set and the near-infrared image set, accurately calculates the difference in collection time, and then uses a time interpolation algorithm to synchronize the two groups of images in the time dimension, eliminating the dynamic change difference caused by the asynchronous collection time, such as the subtle changes in the growth state of the same weed at different times. In the spatial dimension, feature extraction algorithms are used to extract feature points from RGB images and near-infrared images, such as the edge profile of the weed and specific texture feature points. Then, using a feature matching algorithm, the corresponding feature points in the two groups of images are found, and based on this, the spatial transformation relationship between the images is calculated, including translation, rotation, and scaling transformation parameters. According to these parameters, the near-infrared image is geometrically transformed to completely align with the RGB image in space, ensuring that the same object in the two groups of images corresponds accurately in space. After time and space registration, the RGB image parameter set and the near-infrared image set are fused to generate a time-space aligned image set.

[0031] The intrinsic and extrinsic parameters of the camera are determined through camera calibration technology, which reflects the optical structure and imaging geometry of the camera. Using known calibration parameters, a mapping model between image pixel coordinates and actual physical coordinates is established. For image distortion, a distortion correction algorithm is used, such as a correction method based on a polynomial model. By analyzing known feature points or checkerboard patterns in the image, distortion coefficients are calculated, which represent the degree and type of image distortion, including radial and tangential distortion. According to the calculated distortion coefficients, each pixel in the time-space aligned image set is corrected and transformed, and the distorted pixels are remapped to the correct position, thereby eliminating or reducing image distortion. After a series of correction operations, the resulting image can more accurately reflect the actual situation of the photovoltaic panel area, containing rich spectral information (RGB and near-infrared) and accurate spatial information, ultimately forming a high-quality multi-modal perception dataset.

[0032] In one possible implementation, the feature extraction module 10 further includes:

[0033] The perception feature set acquisition unit is configured to traverse the multi-modal perception dataset to identify the weed features of the photovoltaic panel area, obtain a perception feature set, and draw a weed probability heat map based on the perception feature set, wherein the weed probability heat map contains a weed confidence.

[0034] The weed distribution area determination unit is configured to determine a weed distribution area based on the connected component analysis marking of the weed probability heat map.

[0035] The weed density distribution map acquisition unit is configured to calculate the weed distribution area based on the multi-modal perception dataset to obtain a weed density distribution map.

[0036] The plant height data acquisition unit is configured to perform vegetation depth calculation on the photovoltaic panel area based on the multi-modal perception dataset in combination with the weed density distribution map to obtain plant height data.

[0037] The spatial superposition unit is configured to perform spatial superposition of the weed density distribution map and the plant height data according to the weed confidence to generate the vegetation analysis result.

[0038] Specifically, first, a comprehensive traversal is performed on a multi-modal perception dataset, which integrates the information of RGB images and near-infrared images, providing a rich data basis for weed feature recognition. The improved YOLOv7 network is used, which has strong feature extraction capability. Through the weed detection branch (confidence > 0.9) in the multi-task branch design, multi-scale weed features such as unique leaf texture, specific color combination, and plant shape contour are extracted from the dataset, which together constitute the perception feature set. Based on the perception feature set, a weed probability heat map is further drawn. The perception features are compared with the pixel points in the dataset, and the possibility of each pixel point belonging to weeds, i.e., the weed confidence, is calculated. The regional probability statistical sub-unit statistically analyzes the weed confidence of multiple pixel points to generate a regional weed probability distribution matrix, which reflects the probability of weeds appearing in different regions. Finally, according to the regional weed probability distribution matrix, bilinear interpolation processing is performed to convert discrete probability data into a continuous heat map form. In the heat map, different colors represent different weed confidences, and the deeper the color, the higher the probability of weeds existing in the area, thereby intuitively and clearly showing the possible distribution area and probability of weeds in the photovoltaic panel area, providing an important basis for subsequent accurate determination of the distribution range and density of weeds.

[0039] After obtaining the weed probability heat map, the key step to determine the weed distribution area is to mark the connected component based on the heat map. Each pixel in the weed probability heat map contains weed confidence information. The connected component analysis first sets a reasonable weed confidence threshold. For the pixel points in the heat map whose weed confidence is higher than the threshold, they are analyzed in detail. Starting from a certain point in these high-confidence pixel points, search for adjacent pixel points with the same high weed confidence according to certain neighborhood rules (such as four-neighborhood or eight-neighborhood). If adjacent pixel points that meet the conditions are found, they are classified into the same connected region and assigned the same label. Continue this neighborhood search until all pixel points in the connected region that meet the conditions are labeled. Repeat the above operation for all unmarked pixel points in the heat map whose weed confidence is higher than the threshold, so as to divide the entire heat map into multiple different connected regions. These labeled connected regions are the weed distribution areas determined by the system, which can clearly and accurately identify the location and range of weed concentration in the photovoltaic panel area, and provide accurate location information for subsequent operations such as calculating weed density and planning weed removal path.

[0040] Extract the image data corresponding to the weed distribution area from the multi-modal perception dataset. Use the spectral information and texture features in these images to further distinguish weed individuals from the background. Through image segmentation algorithm, accurately segment the weeds from the background and identify the boundary of each weed individual. Then, within the weed distribution area, count the number of weed individuals. At the same time, according to the geographic information or image scale in the multi-modal perception dataset, determine the actual area of the weed distribution area. Divide the number of weed individuals by the actual area of the region to obtain the weed density of the region. Repeat the above calculation process for each sub-region within the weed distribution area to obtain the weed density value of each sub-region. Finally, present these weed density values in a visual way to form a weed density distribution map. In the map, different colors or gray scales represent different weed densities, and the deeper the color or the higher the gray scale, the greater the weed density, thus intuitively showing the distribution of weeds in the photovoltaic panel area, providing a strong basis for subsequent weed removal decisions.

[0041] A monocular depth estimation algorithm based on deep learning is used to obtain plant height data based on a multi-modal perception dataset combined with weed density distribution maps. The RGB images in the multi-modal perception dataset are used as the main input because they contain rich color and texture information. The powerful feature extraction capability of the convolutional neural network (CNN) is used to construct a backbone network structure such as ResNet or VGG, and multi-layer convolution operations are performed on the images to extract image features from different scales, such as the edges, textures, and difference features of weeds and the surrounding environment. At the same time, the weed density distribution map is encoded into a feature vector, which is fused into the intermediate layer features of the CNN through a fully connected layer, allowing the network to learn the potential relationship between weed density and plant depth. During the training process of the network, a photovoltaic panel area image dataset containing a large number of known plant height annotations is used for supervised learning, and the mean square error (MSE) is used as the loss function to measure the difference between the predicted depth value and the true height. The network parameters are continuously adjusted through the backpropagation algorithm to optimize the model performance. In the inference stage, the RGB images in the multi-modal perception dataset to be processed are input into the trained network, and the network outputs a preliminary vegetation depth prediction map. According to the shooting parameters of the image, such as focal length and camera height, the predicted depth value is converted into actual plant height data. Through this monocular depth estimation algorithm based on deep learning, the multi-modal perception data and weed density information are fully utilized to effectively obtain the plant height data in the photovoltaic panel area.

[0042] First, the weed density distribution map and the plant height data are standardized to ensure consistency in spatial coordinates and compatibility in data scales. The weed confidence is used as the weight basis. For areas with high weed confidence, the weed density and plant height data are given greater weight during the superimposition process because the information in these areas is more reliable and has a greater impact on the analysis results. During the specific superimposition, for each spatial location, the weed density value and the plant height value are weighted and fused according to the weed confidence of that location. For example, in an area with a weed confidence of 0.8, if the weed density is 10 plants per square meter and the plant height is 20 centimeters, the weighted calculation will make the comprehensive information in the vegetation analysis results more prominent, highlighting the features of these two data. In areas with low weed confidence, the influence of the corresponding data will be weakened. Through such weighted superimposition operations on each spatial location in the photovoltaic panel area, the vegetation analysis results containing comprehensive information such as weed distribution range, density, and plant height are finally generated. The results are presented in the form of visual maps or detailed data reports, greatly improving the decision-making scientificity of the intelligent weed control system in the photovoltaic panel area and the efficiency of the weed control operation.

[0043] In one possible implementation manner, the perception feature set acquisition unit further includes:

[0044] a weed feature extraction subunit configured to traverse the multi-modal perception dataset based on the improved YOLOv7 network and extract multi-scale weed features.

[0045] a matching calculation subunit configured to perform matching calculation on the multi-modal perception dataset according to the multi-scale weed features and obtain weed confidence of a plurality of pixel points in the multi-modal perception dataset.

[0046] a region probability statistics subunit configured to perform region probability statistics based on the weed confidence of the plurality of pixel points and generate a region weed probability distribution matrix.

[0047] a bilinear interpolation subunit configured to perform bilinear interpolation according to the region weed probability distribution matrix and draw the weed probability heat map of the photovoltaic panel region.

[0048] Specifically, the improved YOLOv7 network processes the multi-modal perception dataset to extract multi-scale weed features. The backbone network of the improved YOLOv7 network is composed of CSPDarknet53 and SPPF module. CSPDarknet53 enhances the feature extraction capability of the network through the cross-stage local network structure, while reducing the computational complexity and improving the computational efficiency; the SPPF module further speeds up the inference, so that the network can process the multi-modal perception data more quickly. The network adopts a multi-task branch design, in which the weed detection branch (confidence > 0.9) is used to extract multi-scale weed features. When traversing the multi-modal perception dataset, the convolutional layer of the network performs layer-by-layer convolution operation on the RGB image and near-infrared image in the dataset. The shallow convolutional layer has a small receptive field and focuses on capturing detailed features such as weed leaf texture, edge contour, etc.; the deep convolutional layer has a large receptive field and can extract global features such as the overall shape of the weed plant and the distribution pattern, thereby realizing multi-scale feature extraction. In the training process, in order to improve the network performance, a variety of optimization methods are adopted. In terms of data augmentation, random rotation (±30°), illumination disturbance (±20%), and scale scaling (0.5-2 times) operations are performed on the training data to increase the diversity of the data, so that the network can learn the weed features under different angles, illumination and scales, and enhance the generalization ability of the model. In terms of loss function, FocalLoss and CIoU Loss are combined. FocalLoss solves the class imbalance problem, reduces the weight of a large number of simple negative samples (such as background regions), and makes the network pay more attention to difficult-to-classify weed samples; CIoU Loss focuses on improving the positioning accuracy, considering the overlapping area, center point distance and aspect ratio of the predicted box and the real box, so that the network can more accurately locate the weeds during training. Through these training optimizations, the improved YOLOv7 network can more accurately extract multi-scale weed features from the multi-modal perception dataset, providing key support for subsequent weed detection and analysis.

[0049] After obtaining the multi-scale weed features, they are matched with the multi-modal perception dataset to determine the weed confidence of multiple pixels in the dataset. First, feature vectors are constructed for the multi-scale weed features, which contain key information such as color, texture, shape, etc. of weeds at different scales. Then, for each pixel point in the multi-modal perception dataset, the corresponding pixel features are extracted from its surrounding neighborhood and also converted into feature vectors. Next, a matching algorithm is used to calculate the similarity between the pixel point feature vectors and the multi-scale weed feature vectors, such as the cosine similarity algorithm, which measures the similarity of two vectors by calculating the cosine value of the angle between them. The closer the cosine value is to 1, the higher the similarity of the two vectors. In calculating the similarity, the weights of different scale features in the multi-scale feature vectors are considered. For key scale features that can more accurately reflect the characteristics of weeds, higher weights are given. According to the calculated similarity results, combined with the pre-set threshold, the weed confidence of the pixel point is determined. If the similarity is higher than the threshold, it is considered that the pixel point belongs to the weed with a higher possibility, and a higher weed confidence is given; otherwise, if the similarity is lower than the threshold, a lower confidence is given. By performing such matching calculation and confidence assignment on all pixel points in the multi-modal perception dataset, the weed confidence of multiple pixels in the dataset is obtained, providing an important data basis for subsequent operations such as drawing weed probability heat map and determining weed distribution area, which helps to more accurately identify and analyze the weed distribution situation in the photovoltaic panel area.

[0050] After obtaining the weed confidence of multiple pixels in the multi-modal perception dataset, in order to more intuitively present the distribution probability of weeds in different areas of the photovoltaic panel area, area probability statistics are performed and a regional weed probability distribution matrix is generated. First, the image corresponding to the photovoltaic panel area is divided into multiple sub-regions of equal size. The division of these sub-regions takes into account the computational efficiency and accuracy requirements to ensure that the distribution characteristics of weeds can be accurately reflected without excessive computational load. For each sub-region, all the pixels contained therein are traversed, and the weed confidence of these pixels is counted. For example, in a certain sub-region, there are 100 pixels, each pixel has its corresponding weed confidence value. Sum these values to get the sum of the weed confidence of all pixels in the sub-region. Then, divide this sum by the total number of pixels in the sub-region to get the average weed confidence of the sub-region. This average weed confidence represents the probability level of weeds appearing in the sub-region. According to the above method, the weed probability value of each sub-region is calculated one by one. Finally, the weed probability values of these sub-regions are arranged according to their positions in the image to form a two-dimensional matrix, i.e. the regional weed probability distribution matrix. In this matrix, each element corresponds to the weed probability of a sub-region, and the rows and columns of the matrix correspond to the division of the sub-regions in different directions of the photovoltaic panel area image. Through the regional weed probability distribution matrix, the differences in the distribution probability of weeds in different areas of the photovoltaic panel area can be clearly seen, providing intuitive and important data support for subsequent determination of weed distribution areas and planning of weed removal paths.

[0051] After generating the regional weed probability distribution matrix, combined with geographic coordinate information, the weed probability heat map is drawn by using bilinear interpolation, which can intuitively show the continuous change of the weed distribution probability in the photovoltaic panel area. First, the geographic coordinates corresponding to each element in the regional weed probability distribution matrix are determined, which accurately defines the corresponding relationship between the matrix elements and the actual location in the photovoltaic panel area. Since the matrix is discrete, bilinear interpolation is needed to obtain a continuous heat map. For each target pixel in the heat map to be drawn, determine its relative position in the coordinate system of the regional weed probability distribution matrix, find the nearest four matrix elements around the target pixel, which form a small rectangle enclosing the target pixel. According to the relative distance of the target pixel to the four elements, the bilinear interpolation algorithm is used to calculate the interpolation weights in the horizontal and vertical directions respectively. Using these weights, the weighted average of the weed probability values corresponding to the four elements is calculated to obtain the estimated value of the weed probability of the target pixel. Repeat the above steps to traverse each pixel in the heat map to obtain the weed probability value of each point. According to the pre-set color mapping rule, different weed probability values are mapped to different colors, such as mapping low probability values to blue, high probability values to red, and intermediate probability values to colors transitioning from blue to red. In this way, all pixel points are assigned corresponding colors according to their respective weed probability values, and the weed probability heat map of the photovoltaic panel area is drawn. In the heat map, the darkness of the color directly reflects the high and low of the weed distribution probability, which can be used to quickly identify the dense and sparse areas of weeds and provide key basis for formulating accurate weed control strategies.

[0052] In one possible implementation, the cooperative control parameter determination module 20 further includes:

[0053] A three-dimensional environment map construction unit is configured to perform three-dimensional analysis based on the weed density distribution map and construct a three-dimensional environment map.

[0054] A path topology construction unit is configured to traverse the three-dimensional environment map to perform weed control analysis of the photovoltaic panel area and construct a path topology.

[0055] An initial global path generation unit is configured to set a cable safety distance constraint, perform weed control path planning according to the cable safety distance constraint based on the path topology, and generate an initial global path.

[0056] A path segmentation determination unit is configured to perform dynamic energy allocation for the initial global path as a weed control path and determine a plurality of path segments.

[0057] A control analysis unit is configured to map the plurality of path segments to a plurality of controllers for control analysis and determine the cooperative control parameters.

[0058] Specifically, a three-dimensional environment map is constructed based on a three-dimensional analysis of the acquired weed density distribution map. Weed density data sets are extracted from the weed density distribution map and are processed in a grid to draw a density heat map, which presents the distribution of weed density in a visual form. At the same time, a three-dimensional point cloud parameter of the photovoltaic panel area is constructed using a multi-modal perception data set, and plant height data is fused. According to the fusion result, a hierarchical division is performed to determine multiple vegetation treatment priorities. Finally, the density heat map and the plant height data are associated and integrated according to these priorities to construct a three-dimensional environment map containing information such as weed distribution, height, and treatment priority, providing a comprehensive environmental information basis for subsequent weed removal analysis.

[0059] The constructed three-dimensional environment map is traversed to perform weed removal analysis in the photovoltaic panel area, and a path topology structure is constructed. In the three-dimensional environment map, according to the position information of weed distribution, photovoltaic panels and other obstacles, the passable area and the area to be avoided are analyzed, the possible path nodes of the weed removal equipment moving in the photovoltaic panel area are determined, and these nodes are connected into a network to form a path topology structure, which clearly defines the potential routes and connection relationships of the weed removal equipment moving in the area.

[0060] The photovoltaic pile foundation coordinates and cable distribution information are accurately identified from the path topology structure. Based on these information, combined with actual operation requirements and safety standards, a cable safety distance constraint is set, for example, the weed removal equipment and the cable need to maintain a safety distance of at least 20 cm. After determining the constraint, a cable three-dimensional model is constructed around the cable, and the center axis is extracted from it. Along the center axis, a cylindrical safety space is constructed according to the safety distance constraint to ensure that the weed removal equipment will not approach the cable in this space. Local paths are planned outside the cylindrical safety space, and the corresponding weed removal distance data is constructed. During the planning process, path conflict determination is continuously performed, and if a path conflict occurs, a detour sub-path is inserted to avoid obstacles or dangerous areas. After completing the local path planning, the path curvature of the detour sub-path is calculated to determine the detour buffer zone radius, and the buffer zone boundary is set according to the radius. Equidistant markers are placed on the buffer zone boundary to construct multiple candidate path nodes. These nodes constitute a possible path selection set, which is screened by considering factors such as path length and obstacle density, and finally a continuous zigzag detour path is generated, which can not only meet the cable safety distance constraint, but also maximize the coverage of the area that needs to be weeded, thereby providing a basic route framework for subsequent weed removal operations as an initial global path, ensuring that the weed removal equipment can efficiently carry out operations under the premise of safety.

[0061] With multi-modal perception data, the initial global path is divided into multiple path segments, each with a corresponding dynamic energy allocation scheme. The energy allocation scheme is determined based on the characteristics of the weeds, the energy consumption characteristics of the weeding equipment, and the operation efficiency. For example, for areas with high weed density and tall plants, more energy is allocated for weeding operations, such as increasing the speed of the blades or increasing the power of the laser weeding. For areas with sparse weeds and small plants, the energy output is correspondingly reduced. In the process of energy allocation, the initial global path is divided into new segments when the energy allocation scheme changes. At the same time, the path segments are further optimized and adjusted based on the geometric characteristics of the path, such as turns and slope changes. Because the energy consumption and operation difficulty of the weeding equipment are different at turns or slopes, the energy allocation and operation mode need to be considered separately. In addition, the endurance and operation time limit of the equipment also need to be considered. When dividing the path segments, it is ensured that the energy consumption of each segment does not exceed the remaining power of the equipment, and the operation time of each segment is within a reasonable range, to ensure that the entire weeding task can be completed smoothly. Through the above steps, the initial global path is divided into multiple path segments, each with a corresponding dynamic energy allocation scheme, providing accurate guidance for subsequent weeding operation control.

[0062] The multiple path segments are mapped to multiple controllers for control analysis to determine the cooperative control parameters. Each path segment corresponds to different operation conditions, and these path segments are mapped to corresponding controllers, such as a fuzzy PID controller for blade speed adjustment and a laser power PID controller for laser power adjustment. Through control analysis of the operation power, speed, and other parameters required by each path segment, and considering the cooperative work between controllers, cooperative control parameters are determined, which will be used to accurately control the operation of the weeding equipment, achieving efficient, safe, and energy-saving intelligent weeding operation in the photovoltaic panel area.

[0063] In one possible implementation, the three-dimensional environment map construction unit further includes:

[0064] The density heat map drawing sub-unit is configured to extract a weed density data set based on the weed density distribution map, perform grid processing on the weed density data set, and draw a density heat map.

[0065] The hierarchical division subunit is configured to construct a three-dimensional point cloud parameter of the photovoltaic panel area based on the multi-modal perception data set, fuse the three-dimensional point cloud parameter with the plant height data, perform hierarchical division according to a fusion result, and determine a plurality of vegetation treatment priorities.

[0066] The correlation integration subunit is configured to correlate and integrate the density heat map and the plant height data according to the plurality of vegetation treatment priorities, and construct the three-dimensional environment map.

[0067] Specifically, a weed density data set is extracted from an existing weed density distribution map. This data set contains weed density information at different locations within the photovoltaic panel area, which accurately records the density of weed growth in each region. In order to more intuitively present the characteristics of weed density distribution and facilitate subsequent analysis, the extracted weed density data set is subjected to gridding processing. It divides the entire area into regular grids according to the actual range and shape of the photovoltaic panel area, and each grid has a corresponding coordinate position. Then, the weed density data is filled one by one according to the grid position, so that each grid has a clear weed density value. After completing the gridding processing, a density heat map is drawn according to a specific color mapping rule. A density value range is set, and different density values within the range are mapped to different colors, such as lower weed density values mapped to light blue and higher density values mapped to dark red, and intermediate density values represented by colors gradually changing from light blue to dark red. According to the weed density value of each grid, the corresponding color is filled, and finally a complete density heat map is formed. In this map, the change of weed density in the photovoltaic panel area can be clearly observed through the distribution of different colors.

[0068] The laser radar data in the multi-modal perception dataset is used to adopt a voxel-based point cloud construction algorithm. The algorithm divides the photovoltaic panel area space into uniform voxels, fills the discrete point data measured by the laser radar into the corresponding voxels according to their spatial positions, generates preliminary three-dimensional point cloud parameters by calculating the statistical characteristics of the points in the voxels, such as the centroid and density, effectively reduces data redundancy, and improves processing efficiency. In the data fusion stage, a weighted average-based fusion algorithm is used. Considering that the three-dimensional point cloud parameters reflect spatial position information, while the plant height data reflect the characteristics of the vegetation in the vertical direction, weights are assigned to them according to their importance in vegetation analysis. Assuming that the weight of the three-dimensional point cloud parameters is w1 and the weight of the plant height data is w2, and w1+w2=1, for each spatial position point, multiply its three-dimensional point cloud coordinate value by w1, multiply its plant height value by w2, and then add them to obtain the fused coordinate value, thereby realizing the fusion of the two. In the grade division and determination of the vegetation treatment priority, the K-Means clustering algorithm is used, the fused data is taken as the input, the number of clusters K is pre-set, which corresponds to the number of different vegetation treatment priority levels, K initial cluster centers are randomly selected, the distance of each data point to each cluster center is calculated, and the data points are divided into the nearest cluster. Then, the center of each cluster is recalculated, and the iteration is continued until the cluster center no longer changes or changes very little. After clustering, the clusters are sorted according to the average height, density and other characteristics of the vegetation in each cluster. The area corresponding to the cluster with high height and density is determined as the high-priority processing area, and vice versa, to determine multiple vegetation treatment priorities.

[0069] The vegetation treatment priorities of each area are determined based on the analysis of the multi-modal perception data, which represent the urgency and importance of weed treatment in different areas. Then, the density heat map and plant height data are integrated based on the vegetation treatment priority. For high-priority areas, more attention and detailed presentation are given. In terms of density heat map, the distribution of weeds in this area is highlighted, and its weed density information is clearly distinguished from the surrounding areas, which may be achieved by deepening the color or enhancing the contrast. For plant height data, the height details of weeds in this area are accurately displayed, such as using different height histograms or contour lines to represent. For low-priority areas, their prominence in the map is appropriately reduced during the integration process according to their priority order, but necessary information is still retained. The density heat map and plant height data are accurately matched in spatial position, so that each position can reflect the density of weeds and the height of plants. Through this associated integration method, the two-dimensional density heat map and one-dimensional plant height data are fused into a three-dimensional environmental map.

[0070] In a possible implementation manner, the initial global path generation unit further includes:

[0071] A safety distance constraint setting subunit is configured to identify photovoltaic pile foundation coordinates and cable distribution information based on the path topology structure, and set a cable safety distance constraint.

[0072] A buffer zone radius calculation subunit is configured to calculate a detour buffer zone radius according to the cable safety distance constraint, and set a buffer zone boundary according to the buffer zone radius.

[0073] A candidate path node construction subunit is configured to identify equidistance based on the buffer zone boundary, construct a plurality of candidate path nodes, screen the plurality of candidate path nodes, generate a continuous zigzag detour path, and take the continuous zigzag detour path as the initial global path.

[0074] Specifically, the constructed path topology structure is deeply analyzed. The path topology structure contains the positional relationship of various objects in the photovoltaic panel area and the passable path information, and the coordinates of the photovoltaic pile foundation are identified. These coordinates accurately determine the position of the photovoltaic pile foundation in the panel area. At the same time, the cable distribution information is obtained from the path topology structure, which involves further processing of the multi-modal perception data (such as laser radar scanning data, electromagnetic induction data, etc.) collected in the early stage, to clarify the direction, depth and specific position range of the cable. After identifying the coordinates of the photovoltaic pile foundation and the cable distribution information, the cable safety distance constraint is set according to industry standards, equipment safety requirements and actual operation experience. For example, considering the vibration, turning deviation and other factors of the weeding equipment during operation, it is stipulated that the weeding equipment and the cable must maintain a safety distance of at least 20 centimeters to prevent the equipment from damaging the cable during weeding operations.

[0075] A three-dimensional cable model is constructed using multi-modal perception data, and the cable center axis is extracted through model processing. Based on the set cable safety distance constraint, a cylindrical safety space is constructed along the center axis to ensure that the weeding equipment does not intrude within a certain range around the cable. Then, local path planning is performed outside the cylindrical safety space, and weeding distance data containing the distance information between each path and the cable are constructed. In this process, path conflict judgment is continuously performed according to the weeding distance data, and once a conflict between the path and other obstacles or safety areas is found, a detour sub-path is inserted. Then, the path curvature of these detour sub-paths is calculated, and the turning performance, running stability and other factors of the weeding equipment are considered to determine the detour buffer zone radius. Finally, according to the calculated buffer zone radius, the cylindrical surface is determined in the three-dimensional space with the cable center axis as the reference, and the equidistant closed curve is drawn on the two-dimensional map, so as to set the buffer zone boundary, provide clear safety limits for subsequent weeding path planning, prevent the weeding equipment from approaching the cable during operation, and protect the safety of the cable in the photovoltaic panel area.

[0076] Based on the buffer boundary, equidistant identification is performed on the buffer boundary, and multiple candidate path nodes are uniformly constructed. These nodes constitute a potential path selection set, providing a basis for generating a suitable bypass path. Then, a screening algorithm is used to consider multiple factors such as path length, path smoothness, obstacle density, and whether it meets the weeding operation requirements, and screen the numerous candidate path nodes. After screening, suitable nodes are connected to generate a continuous zigzag bypass path. This zigzag path can not only meet the cable safety distance constraint, but also reduce the path length to some extent and improve the weeding efficiency. Finally, the continuous zigzag bypass path is determined as the initial global path, providing an important framework for subsequent weeding operation planning and execution.

[0077] In a possible implementation manner, the buffer radius calculation subunit comprises:

[0078] A center axis extraction micro-unit is configured to construct a cable three-dimensional model and extract a center axis based on the cable three-dimensional model.

[0079] A safety space construction micro-unit is configured to construct a cylindrical safety space along the center axis based on the cable safety distance constraint.

[0080] A weeding distance data construction micro-unit is configured to plan a local path outside the cylindrical safety space and construct weeding distance data.

[0081] A path conflict judgment micro-unit is configured to perform path conflict judgment according to the weeding distance data, and insert a bypass sub-path when there is path conflict.

[0082] A path curvature calculation micro-unit is configured to calculate path curvature based on the bypass sub-path and determine the bypass buffer radius.

[0083] Specifically, the laser radar is used to conduct a comprehensive scan of the area where the cable is located, obtaining high-density point cloud data that accurately records the spatial position information of the cable surface and the surrounding environment. At the same time, combined with electromagnetic induction detection technology, the magnetic field distribution data generated by the current inside the cable is obtained, so as to more accurately locate the direction and position of the cable. The obtained point cloud data and electromagnetic induction data are fused and processed to remove noise and abnormal points, improving data quality. Then, a three-dimensional reconstruction algorithm based on deep learning, such as the PointNet++ network model, is used to train and process the fused data, constructing an accurate three-dimensional cable model that can present the shape, bending degree and spatial position of the cable in detail. When extracting the central axis, the three-dimensional cable model is first skeletonized, and through distance transformation and thinning algorithm, the cable model is simplified to a one-dimensional skeleton structure, which is approximately the central axis of the cable. Then, a curve fitting algorithm such as the least squares method is used to fit a B-spline curve to optimize the skeleton, remove redundant points and local fluctuations in the skeleton, and make the central axis smoother and more accurate, finally obtaining the central axis that can represent the core direction of the cable.

[0084] After the cable central axis is determined, constructing a cylindrical safety space based on the cable safety distance constraint is the key to ensuring the safety of weeding operation. First, the actual radius parameter of the cable is obtained through the pre-cable laying record or high-precision detection equipment measurement. Then, according to the established safety distance constraint rule, the cable radius is added to the fixed safety distance of 20 cm to calculate the radius of the cylindrical safety space. For example, if the cable radius is 5 cm, the cylindrical safety space radius is 25 cm. With the extracted cable central axis as the reference, a complete cylindrical safety space is generated in three-dimensional space from the starting end to the end of the axis with the calculated radius as the standard through spatial geometric construction method. This space strictly follows the cable safety distance constraint and can effectively avoid contact between the weeding equipment and the cable during operation, providing a clear safety boundary for subsequent weeding path planning and ensuring the safety of the cable in the photovoltaic panel area during weeding operation.

[0085] Based on the environmental information such as topography and photovoltaic pile distribution of the photovoltaic panel area, a path planning algorithm (such as A* algorithm, Dijkstra algorithm) is used to search for potential paths in the passable area outside the safe space. In the planning process, the cable safety distance constraint is taken as a key limiting condition to ensure that the minimum distance between any point on each local path generated and the cable is greater than or equal to 20 cm. Along the planned local path, the position information of the path points is collected at fixed intervals (such as every 10 cm), and the vertical distance of these points to the center axis of the cable is calculated, and then the cable radius is subtracted to obtain the actual distance of the point to the cable. These distance data are recorded in sequence according to the path order to form the weeding distance data. For example, the distance data corresponding to each of the 100 collection points on a local path collectively constitute the weeding distance data of the path. The weeding distance data constructed in this way not only ensures the safety distance between the weeding equipment and the cable, but also provides accurate quantitative basis for subsequent path conflict judgment, buffer radius calculation, etc., ensuring the safety and feasibility of the weeding path planning.

[0086] A conflict judgment threshold (such as 20 cm less than the safety distance constraint) is set, and each distance value in the weeding distance data is compared with the threshold. When it is found that there is a distance value less than the threshold on a certain path, it is judged that there is a conflict between the path and the safe space of the cable; at the same time, if the weeding distance data shows that the path is too close to the photovoltaic pile, other obstacles, etc., it will also be identified as a path conflict. Once a conflict is detected, the path repair mechanism is immediately started, and based on the preset path search strategy (such as bidirectional search, heuristic search), a detour sub-path that can avoid the conflict area is re-planned in the passable area around the conflict point using the A* algorithm or the RRT (rapidly-exploring random tree) algorithm. When generating the detour sub-path, the cable safety distance constraint is also followed to ensure that the distance between each point on the new path and the cable meets the safety requirement. Finally, the detour sub-path is inserted into the conflict position of the original path to replace the path segment with conflict, thereby forming a complete and safe new path to provide reliable guarantee for subsequent path planning and weeding operation.

[0087] The detour sub-path is discretized into a series of ordered path points by using the discrete curvature calculation method, and the curvature value at each path point is calculated by the vector angle change rate formed by adjacent three points. For example, for the continuous three points P i-1 , P i , P i+1 , the curvature ki of the point is calculated by using vector cross product and dot product operations combined with trigonometric relationships, and the curvature ki of the point is calculated by The curvature radius is obtained. The curvature radius of all path points is compared with 1 m. If there is a case less than 1 m, the path is locally adjusted and optimized by inserting a new path point or a smoothing algorithm, so that the curvature radius of the whole bypass sub-path is greater than 1 m, to ensure that the weeding equipment can safely and smoothly turn. When determining the bypass buffer radius, the path curvature, the size of the weeding equipment and the safety margin are considered comprehensively. For the road section with large curvature (the path is relatively curved), the buffer radius is appropriately increased to avoid the equipment exceeding the safety range when turning; for the road section with small curvature (the path is relatively straight), the buffer radius is correspondingly reduced to improve the working efficiency. For example, when the path curvature radius is 1.2 m, the buffer radius is set to 0.5 m in combination with the equipment width and the turning performance; if the path curvature radius reaches 2 m, the buffer radius can be adjusted to 0.3 m. Finally, through accurate calculation of the path curvature and comprehensive consideration of multiple factors, the bypass buffer radius meeting the safety and working requirements is determined.

[0088] In one possible implementation manner, the control feedback module 30 further includes:

[0089] A parameter analysis unit is configured to analyze the cooperative control parameter to obtain a target blade rotating speed parameter, a laser power setting value and an energy distribution mode.

[0090] A first control feedback result acquisition unit is configured to perform control feedback on a blade rotating speed PID controller according to the target blade rotating speed parameter to obtain a first control feedback result.

[0091] A second control feedback result acquisition unit is configured to perform control feedback on a laser power PID controller according to the laser power setting value to obtain a second control feedback result.

[0092] A coupling control analysis unit is configured to perform multi-parameter coupling control analysis on the first control feedback result and the second control feedback result based on the energy distribution mode, update the cooperative control parameter to generate a control instruction, and perform adaptive weeding operation on the photovoltaic panel area.

[0093] Specifically, a data analysis algorithm is used to carefully disassemble the determined cooperative control parameter, and separate the target blade rotating speed parameter, the laser power setting value and the energy distribution mode from the parameter set. The target blade rotating speed parameter determines the rotating speed of the weeding blade in different working scenarios, the laser power setting value specifies the energy output intensity of the laser weeding device, and the energy distribution mode plans the energy supply proportion relationship between the blade rotation and the laser emission. Through this analysis process, the originally comprehensive cooperative control parameter is converted into specific and executable sub-parameters.

[0094] In the intelligent weeding system of photovoltaic panel area, the precise control of blade speed relies on the coordinated operation of fuzzy PID controller and target blade speed parameter. First, the target blade speed parameter (Nset) and laser power setting value (Pset) are obtained to provide a reference for subsequent control. The fuzzy PID controller rapidly calculates the speed deviation (eN = Nset - Nactual) and deviation rate (ΔeN) by real-time acquisition of the blade speed encoder feedback value (Nactual), and takes them as input parameters. According to the pre-set fuzzy control rules, the controller performs fuzzy processing on the speed deviation and deviation rate, determines the adjustment amount of proportional (P), integral (I), and differential (D) parameters through fuzzy reasoning, and then generates the first control feedback result to output the corresponding PWM duty cycle signal to the driving motor, realizing dynamic adjustment of the blade speed.

[0095] At the same time, the laser power PID controller acquires the energy feedback value (Pactual) of the photoelectric sensor in real time, calculates the power deviation (eP = Pset - Pactual), and uses the incremental PID algorithm to calculate the increment of the control amount according to the power deviation and the control output at the last moment, generates the second control feedback result, and adjusts the driving current to precisely control the laser power. The two work together to ensure that the blade speed and laser power of the weeding equipment can both meet the set requirements, providing reliable protection for efficient and precise weeding operations.

[0096] Taking the energy distribution mode as the core, the first and second control feedback results are deeply integrated to realize precise multi-parameter coupling control. First, according to the energy distribution mode, the first control feedback result (blade speed PWM duty cycle signal) is input into the blade speed PID controller, and multi-parameter coupling analysis is performed in combination with environmental parameters (such as weed density and path curvature) and equipment operating status (such as motor temperature and energy consumption). When the energy distribution tends to prioritize laser, if the blade speed is detected to be too high and the weed density is low, the controller generates a blade mode switching instruction to reduce the blade speed or switch to energy-saving mode. Similarly, for the second control feedback result (laser power driving current adjustment signal), multi-parameter coupling analysis is performed on the laser power PID controller in combination with parameters such as laser spot size and energy loss rate, and a laser mode switching instruction to reduce the laser power is generated when the energy distribution focuses on blade operation. Subsequently, the blade mode switching instruction and the laser mode switching instruction are included in the closed-loop feedback control system to monitor the equipment response state in real time, collect data such as motor speed and actual laser power, and generate cooperative control state parameters. Finally, based on the cooperative control state parameters, the cooperative control parameters are dynamically updated, and the optimized parameters are converted into control instructions to precisely drive the weeding equipment, ensuring efficient cooperation of the blade and laser under different working conditions and realizing the precision and intelligence of adaptive weeding operation in photovoltaic panel area.

[0097] In a possible implementation manner, the coupling control analysis unit further includes:

[0098] The blade mode switching instruction generation subunit is configured to perform multi-parameter coupling analysis on the blade rotating speed PID controller based on the energy distribution mode and the first control feedback result, and generate a blade mode switching instruction.

[0099] The laser mode switching instruction generation subunit is configured to perform multi-parameter coupling analysis on the laser power PID controller based on the energy distribution mode and the second control feedback result, and generate a laser mode switching instruction.

[0100] The closed-loop feedback control subunit is configured to perform closed-loop feedback control according to the blade mode switching instruction and the laser mode switching instruction, and generate a cooperative control state parameter.

[0101] The control instruction generation subunit is configured to update the cooperative control parameter based on the cooperative control state parameter, and generate the control instruction.

[0102] Specifically, multi-parameter coupling analysis is performed in combination with the first control feedback result generated by the blade rotating speed PID controller based on the energy distribution mode, and then the blade mode switching instruction is generated. First, the information such as the grass density, plant height, and normalized difference vegetation index (NDVI) of the current working area is acquired, and the energy distribution mode is determined according to the current working mode, for example, when the grass density is low, the plant is short, and the NDVI value shows that the grass vitality is weak, the energy distribution tends to be the laser mode, and vice versa, the energy distribution tends to be the blade mode or the mixed mode. Meanwhile, the first control feedback result is acquired from the blade rotating speed PID controller, that is, the deviation and the change rate of the deviation between the current blade actual rotating speed and the target rotating speed. If the energy distribution tends to be the laser mode, and the first control feedback result shows that the blade rotating speed is too high, in combination with the environmental parameters such as the low grass density, multi-parameter coupling analysis is performed, and it is determined that the blade rotating speed can be reduced to save energy. At this time, the blade mode switching instruction of reducing the rotating speed is generated. If the working mode is the mixed mode, and the grass density is high, and the first control feedback shows that the blade rotating speed is lower than expected, the instruction of increasing the rotating speed is generated after analysis, so that the blade rotating speed is matched with the energy distribution mode and the actual weeding demand, and the weeding work is ensured to be efficient and energy-saving.

[0103] According to the energy distribution mode, the second control feedback result generated by the laser power PID controller is deeply fused, multi-parameter coupling analysis is carried out, and then accurate laser mode switching instructions are generated. Real-time acquisition of weed density, plant height and NDVI information, according to the energy distribution mode to determine the working mode tendency of the laser. If the weed density is low, the plant height is not high, and the NDVI value shows that the weed vigor is low, the energy distribution mode usually tends to work mainly with low-power laser; on the contrary, in the area where the weed growth is vigorous, the energy distribution is biased towards high-power laser or mixed mode. At this time, the second control feedback result, that is, the deviation value of the actual power of the laser from the set power, is obtained from the laser power PID controller. When the energy distribution mode tends to low-power laser operation, and the second control feedback result shows that the actual power of the laser is too high, combined with the current sparse situation of weeds, after multi-parameter coupling analysis, it is judged that the laser power can be reduced, and the laser mode switching instruction of reducing power is generated; if it is in mixed mode or high-power demand mode, and the feedback result shows that the laser power is insufficient, combined with the parameters such as the density and height of the weeds, the instruction of increasing the laser power is generated, so that the laser power is accurately matched with the energy distribution mode and the actual weed control demand, and efficient and energy-saving weed control operation is realized.

[0104] The double closed-loop PID combined with the state space analysis algorithm is adopted to implement closed-loop feedback control on the blade mode switching instruction and the laser mode switching instruction and generate cooperative control state parameters. In the blade control loop, the rotational speed or torque set by the blade mode switching instruction is taken as the target value, the actual rotational speed and current of the blade are taken as the feedback signals, the control amount is calculated through the incremental PID algorithm, and the driving voltage of the motor is adjusted to reduce the deviation; the laser control loop is the same, the power and spot parameters of the laser mode switching instruction are taken as the reference, the position PID algorithm is used, and the driving current is adjusted in real time according to the feedback data of the photoelectric detector and the spot monitor. In order to realize cooperation, the state space analysis method is introduced, the blade speed, motor current, laser power and equipment temperature are constructed into a state vector, and the dynamic relationship between the variables is described through a state transition matrix. Real-time monitoring of the state vector change, using Kalman filtering algorithm to optimize the feedback data under noise interference, and then using weighted summation algorithm to fuse the control parameters of the blade and the laser, finally outputting the cooperative control state parameters containing the key indicators such as equipment operation efficiency, energy consumption ratio and cooperative matching degree, to ensure the accuracy and stability of the weed control operation.

[0105] The mapping relationship between the cooperative control state parameters and the original cooperative control parameters is established, and the blade running state, laser output efficiency, equipment cooperation degree, energy consumption level and other indicators are one-to-one corresponding to the target blade speed parameter, laser power setting value, energy distribution mode and other original parameters. Using the adaptive genetic algorithm, the original cooperative control parameters are iteratively optimized with the optimization objectives of maximizing the weeding efficiency and minimizing the energy consumption. For example, if the cooperative control state parameters show that the laser energy consumption is too high and the weeding efficiency is not expected, the algorithm will adjust the energy distribution mode, appropriately reduce the laser power setting value, and correspondingly increase the blade speed parameter. After multiple rounds of optimization, new cooperative control parameters are generated, which are converted into control instructions containing precise blade speed control signals, laser power adjustment signals and energy distribution strategies to drive the components of the weeding equipment to execute accurately, and realize the adaptive optimization and efficient operation of the photovoltaic panel area weeding operation.

[0106] It should be noted that the above sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the above describes the specific embodiments of the present application. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0107] The above only describes the preferred embodiments of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0108] The present application and the drawings are only exemplary descriptions of the present application, and are considered to cover any and all modifications, changes, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art can make various modifications and changes to the present application without departing from the scope of the present application. Thus, if these modifications and changes of the present application belong to the scope of the present application and its equivalent technology, the present application intends to include these modifications and changes.

Claims

1. A smart weed control system for photovoltaic panel areas based on multimodal sensing, characterized in that, The system includes: The feature extraction module is used to collect multispectral image data of the photovoltaic panel area, generate a multimodal sensing dataset, extract features from the multimodal sensing dataset, and obtain vegetation analysis results based on the sensing feature set. The collaborative control parameter determination module is used to perform path planning based on the vegetation analysis results, generate a weeding path, perform dynamic energy allocation according to the weeding path, and determine the collaborative control parameters. The control feedback module is used to provide control feedback to the controller of the photovoltaic panel area according to the collaborative control parameters, and to update the collaborative control parameters according to the feedback results to execute intelligent weeding operations in the photovoltaic panel area.

2. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 1, characterized in that, The feature extraction module further includes: The acquisition parameter setting unit is used to set the acquisition parameters of the multispectral camera, including resolution parameters and near-infrared band parameters. An image acquisition unit is used to activate a multispectral camera based on the acquisition parameters, acquire an RGB image parameter set according to the resolution parameters, and acquire a near-infrared image set according to the near-infrared band parameters. The spatiotemporal registration unit is used to perform spatiotemporal registration of the RGB image parameter set with the near-infrared image set to generate a spatiotemporally aligned image set. The distortion correction unit is used to perform distortion correction based on the spatiotemporally aligned image set to obtain the multimodal sensing dataset.

3. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 1, characterized in that, The feature extraction module further includes: The sensing feature set acquisition unit is used to traverse the multimodal sensing dataset to identify weed features in the photovoltaic panel area, obtain a sensing feature set, and draw a weed probability heat map based on the sensing feature set. The weed probability heat map includes weed confidence. The weed distribution area determination unit is used to perform connected component analysis and labeling based on the weed probability heatmap to determine the weed distribution area. The weed density distribution map acquisition unit is used to calculate the weed distribution area by combining the multimodal sensing dataset to obtain the weed density distribution map; The plant height data acquisition unit is used to calculate the vegetation depth of the photovoltaic panel area based on the multimodal sensing dataset and the weed density distribution map, and obtain plant height data. The spatial overlay unit is used to spatially overlay the weed density distribution map and the plant height data according to the weed confidence level to generate the vegetation analysis results.

4. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 3, characterized in that, The perceptual feature set acquisition unit further includes: The weed feature extraction subunit is used to traverse the multimodal sensing dataset based on the improved YOLOv7 network to extract multi-scale weed features; The matching calculation subunit is used to perform matching calculations on the multimodal sensing dataset based on the multi-scale weed features to obtain the weed confidence scores of multiple pixels in the multimodal sensing dataset. The regional probability statistics subunit is used to perform regional probability statistics based on the weed confidence of the multiple pixels and generate a regional weed probability distribution matrix. The bilinear interpolation subunit is used to perform bilinear interpolation based on the probability distribution matrix of weeds in the region to draw a probability heat map of weeds in the photovoltaic panel area.

5. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 4, characterized in that, The collaborative control parameter determination module further includes: A 3D environment map construction unit is used to perform 3D analysis based on the weed density distribution map and construct a 3D environment map. The path topology construction unit is used to traverse the three-dimensional environment map to perform weeding analysis of the photovoltaic panel area and construct the path topology. The initial global path generation unit is used to set cable safety distance constraints, and to plan a weeding path according to the path topology and the cable safety distance constraints to generate an initial global path. The path segmentation determination unit is used to dynamically allocate energy by using the initial global path as a weeding path and determine multiple path segments. The control analysis unit is used to map the multiple paths into multiple controllers for control analysis and to determine the cooperative control parameters.

6. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 5, characterized in that, The three-dimensional environment map construction unit also includes: The density heatmap drawing subunit is used to extract the weed density dataset based on the weed density distribution map, perform gridding on the weed density dataset, and draw the density heatmap. The grading subunit is used to construct three-dimensional point cloud parameters of the photovoltaic panel area based on the multimodal perception dataset, fuse the three-dimensional point cloud parameters with the plant height data, and perform grading based on the fusion result to determine the processing priority of multiple vegetation types. The association and integration subunit is used to associate and integrate the density heatmap with the plant height data according to the multiple vegetation processing priorities to construct the three-dimensional environment map.

7. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 5, characterized in that, The initial global path generation unit further includes: The safety distance constraint setting subunit is used to identify the coordinates of the photovoltaic pile foundation and the cable distribution information based on the path topology, and to set the cable safety distance constraint. The buffer radius calculation subunit is used to calculate the radius of the bypass buffer according to the cable safety distance constraint, and to set the buffer boundary according to the buffer radius; The candidate path node construction subunit is used to construct multiple candidate path nodes based on the buffer boundary using equidistant marking, filter the multiple candidate path nodes to generate a continuous Z-shaped detour path, and use the continuous Z-shaped detour path as the initial global path.

8. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 7, characterized in that, The buffer radius calculation subunit includes: The central axis is extracted into micro-units to construct a three-dimensional model of the cable, and the central axis is extracted based on the three-dimensional model of the cable. A safety space construction micro-unit is used to construct a cylindrical safety space along the central axis based on the cable safety distance constraint. Weeding distance data is used to construct micro-units for planning local paths outside the cylindrical safety space and constructing weeding distance data; The path conflict determination micro-unit is used to determine path conflicts based on the weeding distance data. If a path conflict exists, a detour sub-path is inserted. A path curvature calculation micro-unit is used to calculate the path curvature based on the bypass sub-path and determine the radius of the bypass buffer zone.

9. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 1, characterized in that, The control feedback module also includes: The parameter parsing unit is used to parse the cooperative control parameters to obtain the target blade rotation speed parameters, laser power setpoint, and energy distribution mode. The first control feedback result acquisition unit is used to perform control feedback on the blade speed PID controller according to the target blade speed parameter to obtain the first control feedback result. The second control feedback result acquisition unit is used to perform control feedback on the laser power PID controller according to the laser power set value, and obtain the second control feedback result. The coupled control analysis unit is used to perform multi-parameter coupled control analysis on the first control feedback result and the second control feedback result based on the energy distribution mode, update the cooperative control parameters to generate control commands, and execute adaptive weeding operations in the photovoltaic panel area.

10. The intelligent weeding control system for photovoltaic panel areas based on multimodal perception as described in claim 9, characterized in that, The coupling control analysis unit further includes: The blade mode switching instruction generation subunit is used to perform multi-parameter coupling analysis on the blade speed PID controller based on the energy distribution mode and the first control feedback result, and generate blade mode switching instructions. The laser mode switching instruction generation subunit is used to perform multi-parameter coupling analysis on the laser power PID controller based on the energy distribution mode and the second control feedback result, and generate a laser mode switching instruction. The closed-loop feedback control subunit is used to perform closed-loop feedback control according to the blade mode switching command and the laser mode switching command, and generate cooperative control state parameters. A control command generation subunit is used to generate the control command based on the updated cooperative control parameters according to the cooperative control state parameters.

Images