A Deep Learning-Based Method for Decision-Making Operation Paths of Cleaning Robots in Photovoltaic Power Plants
By developing a deep learning-based path decision-making method for photovoltaic power plant cleaning robots, action preference bias information is generated using the stain distribution feature map to correct the action probability distribution of the policy network. This solves the information gap problem between perception and decision-making in photovoltaic power plant cleaning robots and achieves efficient cleaning path planning for stain areas.
Patent Information
- Application Number
- CN202611371651.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-05
- Publication Date
- 2026-11-17
AI Technical Summary
In the path planning of existing photovoltaic power station cleaning robots, there is an information gap between perception and decision-making. The characteristics of stains and the probability distribution of actions lack a direct correlation, resulting in the adoption of indiscriminate action strategies for areas with different stain densities, and the inability to effectively utilize real-time perception information for differentiated cleaning.
The deep learning-based method for decision-making on the operation path of a photovoltaic power station cleaning robot generates action preference bias information by acquiring the feature map of dirt distribution on the surface of photovoltaic modules, corrects the action probability distribution of the policy network, and updates the policy network through reward feedback and experience replay buffer, thereby achieving the coupling of perception and cleaning effect.
This design ensures that each movement of the cleaning robot tends to target areas with higher dirt density, avoiding resource waste. The action search space gradually converges as cleaning progresses, improving cleaning efficiency and strategy convergence speed.
Smart Images

Figure CN123113869A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent operation and maintenance technology for photovoltaic power plants, specifically to a method for decision-making on the operation path of a cleaning robot for photovoltaic power plants based on deep learning. Background Technology
[0002] Current path planning for cleaning robots in photovoltaic power plants primarily employs traditional algorithms (such as A*, DWA, or genetic algorithms). Stain visual detection (based on models like CNN or YOLO) and path planning are processed independently in two stages: after the vision module outputs the detection results, the path planning module generates a cleaning path based on preset rules. The detection results are then input into the path planning module after intermediate transformation, rather than directly participating in action decision-making. This architecture leads to an information gap between perception and decision-making: the cleaning robot cannot differentiate candidate actions based on the current stain distribution characteristics at each step, resulting in indiscriminate action strategies for areas with different stain densities. While existing deep learning methods have achieved end-to-end path decision-making in indoor cleaning scenarios, their policy networks only encode stain information as part of a state vector, outputting action probability distributions and directly sampling and executing them, lacking a direct corrective correlation between stain features and action probability distributions. Meanwhile, the training loop of existing methods only implements the standard process of state-action-reward-update, without feeding back the cleaning progress to the sensory input, and without coupling the reward signal with the physical effect of photovoltaic cleaning. This causes the policy network to explore without a target in a fixed state space and cannot adaptively converge the search space as cleaning progresses.
[0003] In existing deep learning methods, stain distribution features are only used as state inputs to the policy network. After outputting the action probability distribution, the network directly samples and executes the action. There is no direct correction correlation between stain features and action probability distribution, which causes the policy network to adopt indiscriminate action policies for areas with different stain densities. It cannot guide the action search to the area to be cleaned based on the real-time perceived stain distribution status. Summary of the Invention
[0004] The purpose of this invention is to provide a deep learning-based method for decision-making on the operation path of a cleaning robot in a photovoltaic power plant, in order to address the shortcomings in the prior art.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a deep learning-based method for determining the work path of a photovoltaic power plant cleaning robot, wherein the cleaning robot moves on the surface of a photovoltaic array to perform cleaning operations, including: S1. Obtain an image of the photovoltaic module surface and extract a stain distribution feature map, wherein the stain distribution feature map represents the spatial distribution of stain density in each region; S2. Generate action preference bias information based on the stain distribution feature map, construct the current state with the stain distribution feature map and state information and input it into the policy network to obtain the action probability distribution, and sample cleaning actions after correcting the action probability distribution with the action preference bias information. The probability value of each candidate cleaning action is positively correlated with the stain density of the area it points to. S3. Perform the cleaning action, calculate the reward value based on the reward function and store it in the experience replay buffer; S4. Sample experience data from the experience replay buffer and update the policy network based on the reward value; S5. The cleaning progress of the previous iteration updates the stain distribution feature map and feeds it back to the subsequent iterations to make the action search space converge. The accumulated reward feedback guides the parameter update towards the recovery of light transmittance until the preset termination condition is met, and the cleaning path is obtained.
[0006] In a preferred embodiment, the photovoltaic power station includes multiple photovoltaic arrays arranged at a preset row and column spacing, each photovoltaic array having a preset tilt angle.
[0007] In a preferred embodiment, in step S1, the stain distribution feature map is extracted by a pre-trained visual encoder. The pre-trained visual encoder is a feature extraction network finely tuned on a photovoltaic module stain dataset. The feature extraction network outputs a multi-dimensional feature vector, which is mapped to the stain distribution feature map of a fixed size through spatial pyramid pooling.
[0008] In a preferred embodiment, the status information of the cleaning robot in step S2 includes: The robot's current location coordinates, remaining battery power, cleaned area records, and current weather conditions.
[0009] In a preferred embodiment, the candidate cleaning action includes: Forward, backward, turn, cross arrays, and switch cleaning modes; The action preference bias information includes the preference weights corresponding to each candidate cleaning action, and the preference weights are generated as follows: Centered on the current position of the cleaning robot, the stain density statistics of the corresponding area in the stain distribution feature map are extracted along the direction corresponding to each candidate cleaning action. The stain density statistics are normalized and mapped to continuous real values in the interval between zero and one as the preference weight of the corresponding candidate cleaning action. The preference weight is positively correlated with the stain density of the area pointed to by the candidate cleaning action. The stain distribution feature map serves as both a component of the current state and the basis for generating the preference weights. The correction continuously scales the probability values of each candidate cleaning action within the range of zero to one, and the probability values of each candidate cleaning action are positively correlated with the stain density of the area they point to.
[0010] In a preferred embodiment, before step S2, the method further includes: encoding the geometric topology information of the photovoltaic array into graph structure data, and embedding the graph structure data into the policy network through a graph attention network; The policy network is constructed using an adaptive temperature soft actor-critic algorithm, including an Actor network and a Critic network. The Actor network outputs the action probability distribution, and the Critic network estimates the action value of the current state. The temperature coefficient of the adaptive temperature is dynamically determined based on the graph distance between the current position node of the cleaning robot and each uncleaned area node in the graph structure data. The larger the graph distance, the higher the temperature coefficient to enhance exploration, and the smaller the graph distance, the lower the temperature coefficient to enhance execution determinism. The graph distance is the number of edges or the weighted path length on the shortest path from the current position node to the uncleaned area node.
[0011] In a preferred embodiment, when updating the parameters of the policy network in step S4, a counterfactual path evaluation step is further included: For each piece of experience data sampled from the experience playback buffer, the counterfactual reward value of the candidate cleaning action that was not performed in the experience data is calculated based on the mapping relationship between stain density and the change in light transmittance after cleaning fitted from historical cleaning data. The difference between the counterfactual reward value and the reward value is used as the regret value, and the regret value is used as an auxiliary gradient signal to guide the parameter update of the policy network. The auxiliary gradient signal and the gradient signal based on the reward value together constitute a multi-signal driven parameter update.
[0012] In a preferred embodiment, the reward function integrates cleaning revenue, cleaning energy consumption, cleaning coverage, and operational safety constraints in a weighted manner. The cleaning revenue is calculated based on the change in light transmittance of the photovoltaic module surface before and after cleaning. The cleaning energy consumption is calculated based on the measured value of the motor power consumption of the cleaning robot. The cleaning coverage is calculated based on the ratio of the area of the cleaned module to the total area of the soiled module. The operational safety constraints are calculated based on the distance between the cleaning robot and the boundary of the photovoltaic array. The weighting coefficients for each reward item are dynamically determined based on environmental condition parameters and robot operating status parameters; When sandstorm weather conditions are detected, increase the weight values of cleaning benefits and cleaning coverage to prioritize large-area cleaning; When the remaining battery power of the cleaning robot is lower than the power threshold determined based on the return path distance, the weight value of cleaning energy consumption is increased to reduce unnecessary energy-consuming actions.
[0013] In a preferred embodiment, the training process of the policy network employs a course learning strategy, including: The first stage involves pre-training the policy network in a simulation environment using a simple scenario with dense stains. The second phase gradually increases the complexity of stain distribution, changes in weather conditions, and the scale of photovoltaic arrays; The third stage involves deploying the pre-trained strategy network to a real photovoltaic power plant for online fine-tuning.
[0014] In a preferred embodiment, the preset termination condition includes: The stain coverage rate on the surface of all photovoltaic modules is lower than the preset cleaning standard threshold, the battery power of the cleaning robot is lower than the return power threshold determined based on the return path distance, or the cumulative cleaning time reaches the preset maximum working time.
[0015] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This invention generates action preference bias information based on a stain distribution feature map and directly corrects the action probability distribution output by the strategy network. This makes the probability value of each candidate cleaning action positively correlated with the stain density of the area it points to. At the decision level, a direct correction correlation is established between stain perception features and action probability distribution. This ensures that each step of the cleaning robot's action sampling tends to point to the area to be cleaned with higher stain density, avoiding the waste of cleaning resources caused by applying indiscriminate action strategies to areas with different stain densities. At the same time, the cleaning progress feedback of the previous iteration updates the stain distribution feature map, so that the action search space gradually converges to the remaining area to be cleaned as cleaning progresses. The cumulative reward feedback guides the strategy network parameters to be updated in the direction of increasing light transmittance recovery. This achieves the coupling of perception feedback and cleaning physical effects, reduces invalid exploration and repeated cleaning, and improves the strategy convergence speed and cleaning efficiency. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.
[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] Example 1, please refer to Figure 1 As shown in this embodiment, the deep learning-based photovoltaic power plant cleaning robot operation path decision method, in which the cleaning robot moves on the photovoltaic array surface to perform cleaning operations, includes: S1. Obtain an image of the photovoltaic module surface and extract a stain distribution feature map, wherein the stain distribution feature map represents the spatial distribution of stain density in each region; S2. Generate action preference bias information based on the stain distribution feature map, construct the current state with the stain distribution feature map and state information and input it into the policy network to obtain the action probability distribution, and sample cleaning actions after correcting the action probability distribution with the action preference bias information. The probability value of each candidate cleaning action is positively correlated with the stain density of the area it points to. S3. Perform the cleaning action, calculate the reward value based on the reward function and store it in the experience replay buffer; S4. Sample experience data from the experience replay buffer and update the policy network based on the reward value; S5. The cleaning progress of the previous iteration updates the stain distribution feature map and feeds it back to the subsequent iterations to make the action search space converge. The accumulated reward feedback guides the parameter update towards the recovery of light transmittance until the preset termination condition is met, and the cleaning path is obtained.
[0020] In step S1, an image of the photovoltaic module surface is acquired, and a stain distribution feature map is extracted. This feature map characterizes the spatial distribution of stain density in each region. Specifically, this step provides accurate environmental perception input for path decision-making. The stain distribution feature map can be represented as a two-dimensional matrix or tensor, where each element corresponds to a local area on the photovoltaic module surface, and its value reflects the stain density in that area. For example, the photovoltaic module surface can be divided into several grids, and a visual processing algorithm can be used to estimate the coverage ratio or optical shading rate of pollutants such as dust and bird droppings in each grid, thereby forming a feature map that can quantitatively describe the spatial heterogeneity of stains. This representation method enables the subsequent decision-making module to obtain the location and density information of stains in each region, providing data support for path planning.
[0021] In step S2, action preference bias information is generated based on the stain distribution feature map. The current state is constructed using the stain distribution feature map and state information and input into the policy network to obtain the action probability distribution. After correcting the action probability distribution with the action preference bias information, cleaning actions are sampled. The probability value of each candidate cleaning action is positively correlated with the stain density of the area it points to. This step is the core link in realizing perception-driven decision-making in this application. Its key lies in the dual role of the stain distribution feature map. On the one hand, the stain distribution feature map, as a component of the current state, is input into the policy network along with the state information, enabling the neural network to understand the current stain distribution pattern through internal feature extraction and output an action probability distribution representing the likelihood of each candidate cleaning action being selected. On the other hand, the stain distribution feature map is also used independently to calculate the action preference bias information, directly externally correcting the action probability distribution output by the policy network during the probability sampling stage. Specifically, the system calculates the corresponding preference weights based on the robot's current position and the stain density statistics in the direction pointed to by each candidate action (such as forward, left turn, right turn, etc.), and then corrects the probability values of the corresponding actions in the action probability distribution with the preference weights. Actions targeting areas with high dirt density are assigned higher preference weights, thus gaining a higher probability of being selected in the corrected action probability distribution; conversely, actions targeting clean areas are suppressed. This mechanism adds domain-knowledge-based heuristic guidance to the autonomous exploration of reinforcement learning, enabling the robot to tend towards high-value cleaning targets in the early stages of training and accelerating the policy convergence process. Furthermore, state information can also include geometric environmental parameters of the photovoltaic power station as background constraints. For example, a photovoltaic power station consists of multiple photovoltaic arrays arranged at preset row and column spacing, each with a preset tilt angle. The row and column spacing determines the robot's movement cost and feasibility when traversing the array, while the tilt angle affects the robot's adhesion and energy consumption characteristics on the component surfaces. Integrating this structured prior information into the current state helps the policy network learn safe and efficient paths that conform to physical laws.
[0022] In step S3, a cleaning action is performed, and a reward value is calculated based on a reward function and stored in the experience replay buffer. Specifically, the cleaning robot drives the actuators (such as tracked chassis, roller brush motors, etc.) to perform actual work according to the corrected and sampled action instructions. The reward value is obtained by real-time quantitative evaluation of the current action effect by the reward function. The reward function usually considers multiple dimensions such as cleaning effect, energy consumption, and safety. Its specific structure will be explained in subsequent embodiments. The experience replay buffer is used to store interaction tuples containing state, action, reward, and next state. By randomly sampling, the temporal correlation between continuous data is broken, improving the stability of subsequent model training and sample utilization efficiency.
[0023] In step S4, experience data is sampled from the experience replay buffer, and the policy network is updated based on the reward value. This step uses gradient descent optimization to adjust the parameters of the policy network by batch sampling historical experience, enabling it to output a probability distribution of actions with higher expected cumulative rewards when encountering similar states in the future. As training progresses, the policy network gradually learns the mapping relationship between the stain distribution and the optimal action, reducing its dependence on external bias information and enhancing its generalization and adaptive capabilities.
[0024] In step S5, the cleaning progress of the previous iteration updates the stain distribution feature map and feeds it back to subsequent iterations, causing the action search space to converge. The cumulative reward feedback guides parameter updates towards transmittance recovery until a preset termination condition is met, resulting in a cleaning path. This step describes the dynamic closed-loop convergence mechanism in multi-round iterations. As the cleaning operation progresses, the stain density in the cleaned areas is updated to low or zero values in the feature map in real time. Since the action preference bias information is positively correlated with stain density, the preference weight in the direction corresponding to the cleaned area decreases, leading to a decrease in the sampling probability of the policy network in that direction. Simultaneously, the relative preference weight of the remaining uncleaned areas increases, causing the sampling probability of the policy network to concentrate on the areas to be cleaned. This phenomenon manifests as the action search space dynamically converging with the cleaning progress, effectively avoiding repeated cleaning of ineffective areas and improving operational efficiency. Meanwhile, the transmittance recovery index included in the reward function continuously guides the policy network parameters towards maximizing power generation revenue through a cumulative feedback mechanism, forming a positive feedback loop between perception update, bias adjustment, action selection, and reward enhancement. When the preset termination condition is met, the iteration ends and a complete cleaning path is output. The preset termination conditions include: the stain coverage rate on the surface of all photovoltaic modules is lower than the preset cleaning compliance threshold, the battery power of the cleaning robot is lower than the return power threshold determined based on the return path distance, or the cumulative cleaning time reaches the preset maximum working time. These three termination conditions correspond to three scenarios: successful task completion, energy security assurance, and work efficiency control, respectively, ensuring that the cleaning robot meets the constraints of energy security and work efficiency while pursuing cleaning effectiveness.
[0025] Example 2: In this example, the extraction process of the stain distribution feature map in step S1 and the construction method of the state information in step S2 are further specified. As an optional implementation, in S1, the stain distribution feature map is extracted by a pre-trained visual encoder. The pre-trained visual encoder is a feature extraction network finely tuned on the photovoltaic module stain dataset. The feature extraction network outputs a multi-dimensional feature vector, which is mapped to a fixed-size stain distribution feature map through spatial pyramid pooling.
[0026] Specifically, to overcome the problem of insufficient semantic understanding of specific stains on photovoltaic surfaces by general image recognition models, this embodiment uses a ResNet-50 network fine-tuned on a dedicated dataset as the feature extraction backbone. The photovoltaic module stain dataset contains no fewer than 10,000 high-resolution labeled images, covering pixel-level mask annotations for four typical stain types: dust, bird droppings, industrial dust, and mud. During fine-tuning, a weighted sum of the cross-entropy loss function and the Dice loss function is used as the total loss function. This combined design effectively alleviates the sample imbalance problem between the stained area and the clean background, improving the segmentation accuracy for small-area, high-density stains (such as bird droppings). The Adam optimizer is used during training, with an initial learning rate set to 1× The batch size is 16, and the training is performed for 50 epochs to ensure that the feature extractor converges fully.
[0027] During the feature mapping stage, the 2048-dimensional high-dimensional feature vector output by ResNet-50 is fed into the spatial pyramid pooling layer for processing. Spatial pyramid pooling employs a four-level grid partitioning strategy (1×1, 2×2, 3×3, 6×6) to divide the feature map at any scale into sub-regions of different granularities and perform max pooling on each. The pooling results are then concatenated and mapped through a fully connected layer to a fixed 8×8 stain distribution feature map. The core technical effect of this design lies in decoupling the strong binding relationship between the input image resolution and the input dimension of the policy network: regardless of changes in the original image size captured by the front-end camera, or variations in the scale of the stains within the image, the final feature tensor input to the policy network always maintains a constant 64-dimensional (8×8) structure. This not only ensures the stability of the subsequent reinforcement learning policy network input layer, avoiding training oscillations caused by fluctuations in input dimension, but also enhances the model's robustness in representing stains of different morphologies by fusing local texture details and global distribution patterns through multi-scale pooling. It should be understood that, in addition to ResNet-50, the feature extraction network can also use EfficientNet-B3 or YOLOv8 backbone network. The number of grid levels in spatial pyramid pooling can also be adjusted according to actual computing power and accuracy requirements. For example, a 4×4 level grid can be added to the higher resolution input to improve fine-grained perception capability.
[0028] As an optional implementation, the status information of the cleaning robot in S2 includes: the robot's current position coordinates, remaining battery power, records of cleaned areas, and current weather condition parameters.
[0029] In the specific implementation, the aforementioned multi-dimensional state information is constructed as a compact 134-dimensional state vector. Its construction method and the physical meaning of each dimension are as follows: First, the 8×8 stain distribution feature map is flattened into a 64-dimensional floating-point vector in row-major order, representing the spatial distribution of stain density within the current field of view. Second, the 8×8 cleaned area record matrix is also flattened into a 64-dimensional Boolean vector in row-major order, where cleaned grids are marked as 1 and uncleaned grids as 0. These two sets of 64-dimensional vectors are strictly one-to-one corresponding in spatial index; that is, the i-th element of the feature map and the i-th element of the Boolean matrix represent the same physical area on the photovoltaic module surface. This data alignment logic ensures that the policy network can accurately distinguish between three semantic states: stained and uncleaned, stained but cleaned, and no stains, avoiding invalid cleaning decisions due to spatial misalignment. Based on this, the robot's current position coordinates (2 dimensions, x and y, provided by the RTK-GPS module, positioning accuracy ±2cm), remaining battery power (1 dimension, read in real time by the battery management system), and current weather condition parameters (3 dimensions, light intensity, wind speed, and precipitation probability, obtained by onboard sensors or the power station's weather station) are sequentially assembled. The final 134-dimensional state vector is then formed. It is composed of the above five sets of data concatenated in sequence, specifically as follows: =[Stain distribution feature map flattened vector (64-dimensional floating-point value), cleaned area record matrix flattened vector (64-dimensional Boolean value), robot current position coordinates (2-dimensional), battery remaining power (1-dimensional), current weather condition parameters (3-dimensional)], which fully encodes four types of decision-making elements: environmental perception, task memory, ontological constraints, and external disturbances.
[0030] The above 134-dimensional state vector This is a specific implementation of the current state described in step S2: the flattened vector of the stain distribution feature map and the flattened vector of the cleaned area record matrix together constitute the part of the current state borne by the stain distribution feature map, while the location coordinates, remaining battery power, and weather condition parameters constitute the part of the current state borne by the state information. The two are concatenated together to serve as the complete input of the policy network.
[0031] This multidimensional state vector provides the state space required for the Markov decision-making process of the policy network: position coordinates are used for global localization to prevent the robot from idling in repetitive areas; battery power is used for energy constraints, causing the policy to favor returning home or low-power actions when the battery is low; cleaned records are used to record task progress, enabling the policy network to distinguish between cleaned and uncleaned areas; weather parameters are used to reflect the impact of the external environment on cleaning effectiveness and safety risks. The fusion of multi-source information allows the policy network to simultaneously handle cleaning benefits, energy security, and environmental adaptability in a single forward propagation, without relying on the serial judgment of multiple independent modules, thereby improving the real-time performance and global optimality of the decision.
[0032] Example 3, as an optional implementation, includes candidate cleaning actions such as forward, backward, turning, crossing the array, and switching cleaning modes.
[0033] Specifically, these five types of actions constitute a complete motion space covering the complex operational scenarios of photovoltaic power plants. Forward, backward, and turning are used for precise movement control of the robot on the surface of a single photovoltaic module or within array channels, enabling thorough cleaning of localized contaminated areas; traversing arrays allows the robot to move from the current photovoltaic array to an adjacent array, solving the connectivity problem of discontinuous work surfaces; switching cleaning modes allows the robot to dynamically switch between active cleaning operations and pure mobile cruising states, avoiding ineffective energy consumption and component wear caused by the brush spinning idly in clean areas or during transfers. This discretized motion space design ensures the feasibility of robot movement in structured environments and provides clear decision boundaries for reinforcement learning strategies.
[0034] As an optional implementation, the action preference bias information includes the preference weights corresponding to each candidate cleaning action. The preference weights are generated as follows: taking the current position of the cleaning robot as the center, extracting the stain density statistics of the corresponding area in the stain distribution feature map along the direction corresponding to each candidate cleaning action, and normalizing the stain density statistics to map them into continuous real values in the interval between zero and one as the preference weights of the corresponding candidate cleaning actions. The preference weights are positively correlated with the stain density of the area pointed to by the candidate cleaning action.
[0035] Specifically, this generation process achieves a quantitative mapping from environmental perception data to prior decision-making knowledge. In actual calculations, the system first uses the robot's current grid as the center and extracts a local region from the stain distribution feature map along the direction indicated by each candidate action. For example, the average stain density of the first three grid cells in that direction is taken as the stain density statistical value corresponding to that action. ,in This represents the candidate action index. Subsequently, a min-max normalization method is used to map the stain density statistics of all candidate actions to... The interval is calculated using the following formula: ; in, Indicates the first The preference weights of each candidate cleaning action. This represents the statistical value of stain density in the direction corresponding to the action. and These represent the minimum and maximum values of the stain density statistics for all current candidate actions, respectively. To prevent division by zero of small constants, for example, take Introduction The design has significant engineering implications: when the robot is in a completely clean area or in a special state where the density of dirt is completely uniform around it... and Equal, if not This term will cause the denominator to be zero, leading to numerical calculation anomalies; adding After this, the preference weights for each action will approach the same value (e.g., 0.5), causing the strategy to degenerate into uniform random exploration. This ensures the numerical stability of the algorithm while avoiding directional misdirection in situations of missing information. Through the above normalization process, the preference weights... It becomes a continuously differentiable variable that characterizes the level of cleanliness in each direction, providing a standardized input benchmark for subsequent probability adjustments.
[0036] As an optional implementation, the stain distribution feature map serves simultaneously as a component of the current state and the basis for generating preference weights. The correction continuously scales the probability values of each candidate cleaning action within the range of zero to one, with the probability value of each candidate cleaning action being positively correlated with the stain density of the area it points to. This feature embodies the design concept of the synergistic effect of perception and decision-making in this application. Stain distribution feature Figure 1 On one hand, the sampling probability is fed into the neural network as part of the policy network's state vector, allowing it to implicitly learn environmental features through nonlinear transformations. On the other hand, it is also independently extracted for calculating preference weights, explicitly correcting the network output during the sampling phase. The technical significance of this dual-action mechanism lies in its construction of an external heuristic guidance mechanism. Without altering the internal parameter structure of the policy network or adding extra training burden, it directly applies domain knowledge—specifically prioritizing the cleaning of highly soiled areas—to the decision-making process. In the early stages of training or in unfamiliar environments, even if the policy network has not yet converged and its output is close to a random distribution, the sampling probability after preference weight correction can still automatically favor high-value areas, significantly alleviating the problem of low sample efficiency in the cold start phase of reinforcement learning. Simultaneously, since the correction operation only continuously scales the original probabilities rather than hard-truncating or forcibly replacing them, the policy network always retains a non-zero sampling probability for low-weight actions. This means that the autonomous exploration capability of reinforcement learning is preserved, and the robot still has the opportunity to discover potential optimal paths not covered by preference rules, achieving an effective combination of expert guidance and autonomous exploration.
[0037] In practical implementation, a preferred method for probability correction is to multiply element-wise and then renormalize, that is: ; in, Indicates the corrected number The probability of each candidate cleaning action being selected. This represents the raw probability output by the policy network. This represents the corresponding preference weight. This represents the total number of candidate actions. This method intuitively uses preference weights as scaling factors for probability, with clear physical meaning, facilitating engineering implementation and debugging. It should be understood that the scope of protection of this application is not limited to the multiplicative modification form.
[0038] As an equivalent alternative, probability correction can also be implemented in the logarithmic field using addition, i.e.: ; in, The bias strength coefficient, for example, is 2.0. After calculation, the probability distribution is normalized using the softmax function to obtain the corrected probability distribution. Mathematically, addition in the logarithmic field is equivalent to multiplication in the probability field; they are functionally completely equivalent. However, at the numerical computation level, addition in the logarithmic field effectively avoids floating-point underflow caused by multiplying multiple low-probability values, exhibiting superior numerical stability in scenarios with a large number of candidate actions or extremely sharp original probability distributions. Both of these implementation methods fall within the technical concept of correcting action probabilities with preference weights in this application, forming a multi-layered protection for this technical feature.
[0039] Example 4, as an optional implementation, further includes, before step S2: encoding the geometric topology information of the photovoltaic array into graph structure data, and embedding the graph structure data into a policy network through a graph attention network.
[0040] Specifically, the physical layout of a photovoltaic (PV) power plant is not a regular Euclidean grid, but a complex topological system composed of discrete component units connected non-uniformly through maintenance channels, cable trays, and other means. To accurately represent this non-Euclidean spatial relationship, this embodiment models the PV array as graph-structured data. Nodes in the graph correspond to the smallest cleaning unit in the PV array, such as a single PV module or an electrical string; the node feature vector contains multi-dimensional attributes such as the unit's real-time average dirt density, number of modules, installation tilt angle, and number of days since the last cleaning. Edges in the graph correspond to the physically accessible channels between cleaning units, and edge weights can be set as the actual physical distance of the channel or the difficulty coefficient of passage. Based on this, a graph attention network is used to extract and embed features from the graph-structured data. The graph attention network learns the importance weights between nodes through a self-attention mechanism; its core lies in calculating the attention coefficient between node i and its neighbor node j. The specific calculation formula is as follows: ; in, and Let be the feature vectors of node i and node j, respectively. It is a learnable linear transformation matrix. For learnable attention vectors, This represents the vector concatenation operation. The attention coefficients are then normalized using the softmax function to obtain the importance weights of node j relative to node i. This process enables the policy network to explicitly perceive the connectivity and spatial adjacency of the photovoltaic array, overcoming the limitation of relying solely on visual feature maps to obtain physical access constraints, and providing topological information for subsequent path decision-making.
[0041] As an optional implementation, the policy network is constructed using an adaptive temperature soft actor-critic algorithm, comprising an Actor network and a Critic network. The Actor network outputs the action probability distribution, while the Critic network estimates the action value of the current state. The soft actor-critic algorithm encourages exploration while ensuring task performance by maximizing the weighted sum of cumulative reward and policy entropy, avoiding premature convergence of the policy to a suboptimal solution. However, traditional soft actor-critic algorithms typically apply a single global scalar temperature coefficient to impose a uniform exploration intensity on all states and actions, which has significant limitations in the scenario of large-scale sparse contamination in photovoltaic power plants: when the robot moves away from uncleaned areas, excessively low temperatures lead to insufficient exploration, making it difficult to discover potential optimal paths; while when the robot approaches the target area, excessively high temperatures introduce unnecessary randomness, reducing cleaning efficiency. To address this issue, this embodiment proposes a spatially heterogeneous adaptive temperature regulation mechanism.
[0042] As an optional implementation, the temperature coefficient of the adaptive temperature is dynamically determined based on the graph distances between the cleaning robot's current position node and each uncleaned area node in the graph structure data. A larger graph distance results in a higher temperature coefficient to enhance exploration, while a smaller graph distance results in a lower temperature coefficient to enhance execution determinism. The graph distance is the number of edges or the weighted path length on the shortest path from the current position node to the uncleaned area node. Specifically, this mechanism no longer uses a single scalar but generates a temperature vector corresponding to each uncleaned area. Assume there are M uncleaned area nodes in the current graph structure data. The system first calculates the distance from the current location node to each uncleaned area node. The shortest path distance in the graph This can be achieved using Dijkstra's algorithm or breadth-first search. The distance is then normalized to obtain... ,in The maximum and shortest path edge count is given for the entire graph. The temperature coefficient for each uncleaned area is calculated based on the normalized distance. ; in, Use the base temperature coefficient, for example, 0.2; This is a distance adjustment factor, for example, 0.1. The resulting temperature vector This mechanism achieves spatially differentiated allocation of exploration intensity: for distant uncleaned areas with larger graph distances, the temperature coefficient is higher, and the policy distribution maintains a large entropy value, prompting the robot to try diverse paths to explore long-distance areas; for near uncleaned areas with smaller graph distances, the temperature coefficient is lower, and the policy distribution tends to be sharper, ensuring that the robot performs efficient cleaning actions with high determinism. This mechanism solves the problem of balancing exploration and utilization under non-uniform task distribution with fixed temperature parameters.
[0043] Furthermore, the aforementioned spatial heterogeneous temperature mechanism achieves precise gradient control by modifying the loss functions of the Actor and Critic networks. During the parameter update process of the policy network, when candidate cleaning actions... The area it points to corresponds to the uncleaned area node. At that time, the entropy regularization coefficient for this action no longer uses the global value. Instead, replace with the corresponding .
[0044] The modified Actor network loss function is as follows: ; in, The number of candidate cleaning actions. For the first The estimated value of the Critic network is calculated. Correspondingly, the target value of the Critic network is also revised accordingly: ; In the above loss function, The entropy regularization coefficient, acting as an action, directly participates in gradient backpropagation. This means that under the same current state, actions pointing in different directions will experience differentiated optimization pressures: actions pointing to distant areas maintain higher randomness to preserve exploration capabilities, while actions pointing to nearby areas converge to high-value patterns to improve utilization efficiency. It should be understood that when the region targeted by a candidate cleaning action does not belong to any uncleaned area node, such as when the robot is moving within a cleaned area, the temperature coefficient of this action can regress to the base temperature. When all uncleaned areas have been cleaned, the temperature vector degenerates into an empty set, and the policy network automatically switches to pure utilization mode to accelerate the final operation. This design ensures the integrity and numerical stability of the algorithm throughout the entire operation cycle.
[0045] Example 5, as an optional implementation, includes a counterfactual path evaluation step when updating the parameters of the policy network in step S4. This step is a significant improvement on the traditional reinforcement learning update mechanism, aiming to address the problems of high cost of acquiring real interaction samples and low training efficiency caused by the scarcity of effective samples in the clean photovoltaic power plant scenario. By introducing counterfactual reasoning, the system can extract supervision information of actions that have not been performed in each actual interaction, thereby improving sample utilization.
[0046] Specifically, counterfactual path evaluation relies on a physical model capable of predicting cleaning effectiveness. For each piece of empirical data sampled from the experience playback buffer, based on the mapping relationship between stain density and the change in light transmittance after cleaning fitted from historical cleaning data, the counterfactual reward value for candidate cleaning actions that were not performed in that empirical data is calculated. The construction of this mapping relationship is a prerequisite for the validity of counterfactual reasoning, namely, establishing a lightweight environmental dynamics model. In practical implementation, a linear regression method can be used to fit the historical cleaning data to obtain the formula. ,in This indicates the change in light transmittance after cleaning. This represents the estimated density of the stain before cleaning. and The fitting coefficients are used. For example, by statistically analyzing the actual cleaning records of a photovoltaic power station over six months, the fitting coefficients can be obtained. , The empirical parameters are used. This linear model has extremely low computational overhead and is suitable for real-time inference on embedded devices. It should be understood that when the stain type is complex or the cleaning effect exhibits strong nonlinear characteristics, a two-layer fully connected neural network can be used to fit this mapping relationship. The network input layer is a 1-dimensional stain density, the hidden layer is 32-dimensional and uses the ReLU activation function, and the output layer is a 1-dimensional transmittance change. A higher-precision prediction model is obtained through mean squared error loss and Adam optimizer training. Regardless of the fitting method used, the goal is to establish a differentiable or queryable mapping from stain state to cleaning benefits, providing a quantitative basis for evaluating the benefits of inactive actions.
[0047] After obtaining the mapping relationship, for sampling experience Any candidate cleaning action that was not executed The system first extracts the action. Stain density in the indicated area Substituting this into the above mapping relationship yields the estimated change in transmittance. Then, based on the reward function, the counterfactual reward value that could be obtained if the action were performed is calculated. Subsequently, the difference between the counterfactual reward and the actual reward is used as the regret value. This regret value is then used as an auxiliary gradient signal to guide the parameter updates of the policy network. The auxiliary gradient signal and the reward-based gradient signal together constitute a multi-signal driven parameter update. The formula for calculating the regret value is as follows: .when When, it indicates the current state. Below, actions not performed The expected result is greater than the actual action taken. Higher returns, i.e., the policy network does not select actions. If the expected return is lower than the projected return of the action, then the actual decision is better than or equal to the alternative. This value directly quantifies the gap between the current strategy and the potentially optimal strategy.
[0048] To utilize this difference for parameter optimization, this embodiment employs an auxiliary gradient stacking mechanism. The specific formula for the total gradient update is as follows: ; in, Based on actual rewards The calculated standard policy gradient, Actions not performed The logarithmic probability gradient, This is the auxiliary gradient weight coefficient, for example, set to 0.1. The physical meaning of this formula is: when the regret value of a failed action is positive, the auxiliary gradient term will adjust the network parameters in the direction that increases the probability of that action; when the regret value is negative, the probability of that action will be suppressed. (Coefficient) This plays a crucial balancing role, ensuring that the main gradient dominates the optimization direction to prevent environmental model errors from misleading the strategy, while allowing auxiliary gradients to provide additional correction signals to accelerate convergence. Besides the aforementioned method of directly stacking gradients, regret values can also be incorporated as an additional reward component into the target value calculation, or variations such as advantage-weighted regression can be used. As long as the core idea is to utilize the estimated value of unexecuted actions to assist parameter updates, these methods fall within the scope of this application.
[0049] This multi-signal-driven update mechanism has significant technical advantages over the standard soft actor-critic algorithm. In traditional reinforcement learning, policy updates rely solely on the reward signals from actual sampled trajectories. For a large number of unsampled actions in the same state, their value information is completely discarded. This can easily lead to sample waste and slow convergence in scenarios like photovoltaic power plants where the state space is huge and sampling is expensive. This embodiment constructs multiple additional positive or negative samples at each time step through counterfactual path evaluation, improving the efficiency of information utilization in a single interaction by several times. More importantly, since the regret value comes directly from the physical mapping of stain density and transmittance rather than the estimation of the neural network, it can provide accurate gradient guidance in the early stages of training. This effectively alleviates the gradient noise problem caused by overestimation or underestimation of the value network, enabling the policy network to correct sampling bias more quickly and converge towards the globally optimal cleaning path.
[0050] Example 6, as an optional implementation, uses a weighted approach to integrate cleaning benefits, cleaning energy consumption, cleaning coverage, and operational safety constraints. Cleaning benefits are calculated based on the change in light transmittance of the photovoltaic module surface before and after cleaning; cleaning energy consumption is calculated based on the measured power consumption of the cleaning robot's motor; cleaning coverage is calculated based on the ratio of the cleaned module area to the total area of the soiled modules; and operational safety constraints are calculated based on the distance between the cleaning robot and the photovoltaic array boundary. Specifically, this reward function constructs a multi-dimensional comprehensive evaluation system to quantify the immediate effectiveness of cleaning actions. Total reward value. The calculation formula is as follows: ; in, , , , These are the weighting coefficients for each reward. Cleaning Revenue Characterizing the degree of recovery of power generation performance after a single action, its calculation is based on the change in light transmittance of the photovoltaic module surface before and after cleaning. In practical engineering, the change in transmittance can be approximately estimated by the difference in the mean grayscale values of the image, i.e. ,in The average gray level of the image, divided by the normalization constant. To obtain standardization Clean energy consumption The energy cost representing an action is usually a negative value, and the calculation formula is as follows: Among them, the measured value of motor power consumption Voltage measured by current sensor With current Multiplying them together, we get For the duration of the action execution, Rated power of the motor. Cleaning coverage. Characterizes task progress and is calculated as the area of cleaned components. Total area of stained components The ratio. Operating safety constraints. Used to punish dangerous behavior, the calculation formula is: ,in The real-time distance between the robot and the boundary of the photovoltaic array. A preset safe distance threshold, such as 0.3 meters, is used. When the distance is greater than the threshold, this item is zero; when the distance is less than the threshold, a negative reward is generated, and the penalty increases as the distance gets closer. This multi-objective weighted design ensures that the policy network can balance economic benefits, energy efficiency, task integrity, and equipment safety during the optimization process.
[0051] As an optional implementation, the weight coefficients of each reward item are dynamically determined based on environmental condition parameters and robot operating state parameters. Specifically, to cope with the complex and ever-changing actual operating conditions of photovoltaic power plants, this embodiment does not adopt the fixed weight approach of traditional methods, but instead uses a context-aware dynamic weight adjustment mechanism. The initial values of the weight coefficients can be set to... , , , However, during operation, it will adaptively correct itself based on the real-time monitored environment and system status. The purpose of this dynamic adjustment is to automatically adjust the decision-making objectives according to changes in operating conditions, thereby achieving targeted optimization for specific scenarios while ensuring basic performance.
[0052] As an optional implementation, when dusty weather conditions are detected, the weighting of cleaning benefits and cleaning coverage is increased to prioritize large-area cleaning. Specifically, dusty weather is a typical high-value operational window for photovoltaic power plants, during which dirt accumulates rapidly on module surfaces, power generation losses are significant, and severe weather can cause operational interruptions at any time. Therefore, when meteorological sensors detect dusty weather conditions, such as PM10 concentrations exceeding 150 μg / m³, priority is given to cleaning. 3 At that time, the system will weight the cleaning revenue. And cleaning coverage weight The values were increased to 1.5 times their original values. This adjustment makes the reward function more positively incentivized for actions that quickly remove stains and expand the cleaning area, prompting the robot to maximize the recovery of power generation revenue within a limited time window, prioritizing the recovery of power generation revenue.
[0053] As an optional implementation, when the cleaning robot's remaining battery power falls below a threshold determined based on the return path distance, the weight of cleaning energy consumption is increased to reduce unnecessary energy-consuming actions. Specifically, energy security is a fundamental constraint for mobile robot operation. When the remaining battery power reported by the battery management system is below the return path threshold, this threshold is determined by the shortest path distance from the current location to the charging station. Multiply by energy consumption per unit distance Multiplying this by a safety factor of 1.2, the system will then calculate the clean energy consumption weight. The value has been increased to twice its original value. This adjustment significantly amplifies the negative reward penalty for high-energy-consuming actions, causing the policy network to tend to choose low-power movement modes or shorten unnecessary paths, and even terminate long-distance low-value cleaning tasks to ensure a reliable return to base, prioritizing safe return.
[0054] After any of the above weight adjustments are triggered, the system must renormalize all weight coefficients, that is, let This step is crucial for maintaining the stability of reinforcement learning training. Because the absolute scale of the reward value directly affects the value estimation range of the Critic network and the gradient update step size of the Actor network, simply amplifying certain weights without normalization can lead to drastic fluctuations in reward values, resulting in problems such as overestimation of the value network or gradient explosion. By re-normalizing, the dynamic changes in the relative importance of each objective are preserved, while maintaining the overall stability of the reward signal amplitude, ensuring the convergence of the policy optimization process. It should be understood that the aforementioned weight adjustment multiples (1.5x, 2x) and safety factor (1.2) are merely exemplary parameters. In actual deployment, they can be calibrated based on the power plant's historical operation and maintenance data and battery aging characteristics. Furthermore, in addition to the two typical operating conditions of sandstorms and low battery levels, this dynamic weight mechanism can also be extended to other environmental or ontological abnormal states such as strong winds, rain, snow, and high temperatures. As long as its core idea is to adaptively adjust the priority of optimization objectives according to the real-time situation, it falls within the scope of protection of this application.
[0055] Example 7, as an optional implementation, uses a course learning strategy in the training process of the policy network.
[0056] Specifically, when deep reinforcement learning algorithms are directly applied to real physical systems, they often face challenges such as low sample acquisition efficiency, high trial-and-error costs, and difficulties in cold start. This embodiment effectively avoids the above risks by constructing a progressive training system from simple to complex and from simulation to reality, providing a robust parameter convergence path for the complex mechanisms involved in the aforementioned embodiments, such as perception-driven decision-making, spatial heterogeneous temperature regulation, and counterfactual path evaluation. The course learning strategy specifically includes three progressive stages, aiming to achieve effective transfer from the simulation environment to the real power plant.
[0057] As an alternative implementation, the first stage involves pre-training the policy network in a simulation environment with a simple scenario of dense stains.
[0058] Specifically, the pre-training phase is built on the Isaac Gym high-performance parallel simulation platform, with the scenario set as a 10×10 photovoltaic array and a uniformly distributed dirt density ranging from 0.6 to 1.0. The core purpose of this high-density dirt design is to provide the policy network with sufficient and dense positive feedback signals, enabling it to frequently trigger cleaning rewards in the early stages of training. This quickly establishes a basic mapping relationship between dirt distribution feature maps, action preference bias information, and cleaning actions. During this phase, the policy network can rapidly learn to utilize the preference weight correction mechanism described in Example 3, exhibiting a basic behavioral pattern that tends towards high-dirt areas, avoiding policy degradation or oscillations caused by a long-term lack of effective feedback in a sparse reward environment. Pre-training typically lasts approximately 1 million steps until the basic cleaning success rate stabilizes above a preset threshold.
[0059] As an optional implementation, the second phase gradually increases the complexity of stain distribution, weather condition changes, and photovoltaic array size.
[0060] Specifically, as the basic behaviors are solidified, the training environment gradually approximates real-world operating conditions. The array size is expanded to 50×50 to simulate the long-term planning requirements of large-scale power plants, and the pollution density range is broadened to 0.1 to 1.0 to cover the full spectrum of scenarios from light dust accumulation to heavy pollution. Dynamic meteorological disturbances such as wind speeds of 0 to 15 m / s and precipitation probabilities of 0 to 30% are also introduced. More importantly, domain randomization technology is introduced in this stage to reduce the discrepancy between simulation and reality.
[0061] Specifically, at the simulation rendering and physics engine level, parameters such as light intensity, camera exposure parameters, image noise level, component surface friction coefficient, and motor response delay are sampled extensively and randomly. This approach makes it difficult for the policy network to overfit to the specific visual texture or ideal dynamic model of the simulation environment, and instead learns robust feature representations that are invariant to underlying physical changes. After approximately 3 million steps of progressively more complex training, the policy network not only possesses the ability to cope with variable environments and sparse stains, but also learns the graph distance-based adaptive temperature regulation mechanism in Example 4, enabling it to autonomously explore and utilize a wide space.
[0062] As an optional implementation, the third stage involves deploying the pre-trained policy network to a real photovoltaic power plant for online fine-tuning.
[0063] Specifically, this is the final step in eliminating residual domain gaps and adapting to the unique characteristics of specific power plants. To ensure safe operation of the actual equipment and prevent catastrophic amnesia, a conservative parameter update strategy was adopted for online fine-tuning. The learning rate was reduced to one-tenth of that in the pre-training phase, for example, set to 3× This ensures that parameter adjustments are made only within the neighborhood of the current optimal strategy, rather than deviating significantly from the learned effective behavior patterns. Simultaneously, a batch cumulative update mechanism is employed, meaning that a gradient update is performed only after accumulating 5000 real-world interaction experience data points, rather than a single-step real-time update. This approach smooths out gradient fluctuations caused by real sensor noise and reduces the real-time load on the embedded computing unit. Through online fine-tuning, the policy network can further calibrate the prediction bias of the counterfactual evaluation model in Example 5 and adapt to the unique component aging, installation errors, and microclimate characteristics of real power plants, ultimately achieving engineering-usable performance indicators.
[0064] To more clearly illustrate the synergistic effect of the technical solution in actual operation, the application of the above training results is demonstrated below in a specific photovoltaic power station scenario. In a cleaning operation during a sandstorm, a strategy network trained through a complete course was deployed on a cleaning robot. At the beginning of the operation, the onboard meteorological sensor detected excessive PM10 concentration. Based on the dynamic weighting mechanism described in Example 6, the strategy network automatically increased the cleaning benefit and coverage weights to 1.5 times their original values. The robot then exhibited a strong tendency towards large areas of accumulated dust, efficiently performing cross-array cleaning actions. As the operation progressed, the battery management system reported that the remaining battery power was lower than the return-to-home battery power threshold calculated based on the shortest path in the current graph. The strategy network immediately triggered the energy consumption weight doubling mechanism, abandoning the cleaning of distant, low-value stains and instead planning the shortest path to move towards the charging station. When the robot arrived near the charging station and the battery power reached the return-to-home safety threshold, the termination condition defined in Example 1 was precisely triggered, the operation safely ended, and a complete cleaning path record was output. The entire process requires no human intervention, fully verifying the decision-making rationality, environmental adaptability, and operational safety of the proposed method in complex dynamic environments. It also demonstrates the crucial role of the course learning strategy in ensuring the effective operation of the multi-module collaborative mechanism in real-world scenarios. It should be understood that the above demonstration is merely illustrative. In practical applications, the number of training steps, the range of domain randomization parameters, and the fine-tuning learning rate at each stage can be adaptively adjusted according to specific hardware computing power and power plant operation and maintenance requirements. As long as it follows a progressive training approach combining simplicity and reality, it falls within the scope of protection of this application.
[0065] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A deep learning-based method for determining the working path of a cleaning robot in a photovoltaic power station, wherein the cleaning robot moves on the surface of a photovoltaic array to perform cleaning operations, characterized in that... include: S1. Obtain an image of the photovoltaic module surface and extract a stain distribution feature map, wherein the stain distribution feature map represents the spatial distribution of stain density in each region; S2. Generate action preference bias information based on the stain distribution feature map, construct the current state with the stain distribution feature map and state information and input it into the policy network to obtain the action probability distribution, and sample cleaning actions after correcting the action probability distribution with the action preference bias information. The probability value of each candidate cleaning action is positively correlated with the stain density of the area it points to. S3. Perform the cleaning action, calculate the reward value based on the reward function and store it in the experience replay buffer; S4. Sample experience data from the experience replay buffer and update the policy network based on the reward value; S5. The cleaning progress of the previous iteration updates the stain distribution feature map and feeds it back to the subsequent iterations to make the action search space converge. The accumulated reward feedback guides the parameter update towards the recovery of light transmittance until the preset termination condition is met, and the cleaning path is obtained.
2. The method for determining the operation path of a photovoltaic power plant cleaning robot based on deep learning according to claim 1, characterized in that: The photovoltaic power station includes multiple photovoltaic arrays arranged at a preset row and column spacing, and each photovoltaic array has a preset tilt angle.
3. The method for determining the operation path of a photovoltaic power plant cleaning robot based on deep learning according to claim 1, characterized in that: In step S1, the stain distribution feature map is extracted by a pre-trained visual encoder. The pre-trained visual encoder is a feature extraction network finely tuned on a photovoltaic module stain dataset. The feature extraction network outputs a multi-dimensional feature vector, which is mapped to the stain distribution feature map of a fixed size through spatial pyramid pooling.
4. The method for determining the operation path of a photovoltaic power plant cleaning robot based on deep learning according to claim 1, characterized in that: The status information of the cleaning robot in S2 includes: The robot's current location coordinates, remaining battery power, cleaned area records, and current weather conditions.
5. The method for determining the operation path of a photovoltaic power plant cleaning robot based on deep learning according to claim 1, characterized in that: The candidate cleaning actions include: Forward, backward, turn, cross arrays, and switch cleaning modes; The action preference bias information includes the preference weights corresponding to each candidate cleaning action, and the preference weights are generated as follows: Centered on the current position of the cleaning robot, the stain density statistics of the corresponding area in the stain distribution feature map are extracted along the direction corresponding to each candidate cleaning action. The stain density statistics are normalized and mapped to continuous real values in the interval between zero and one as the preference weight of the corresponding candidate cleaning action. The preference weight is positively correlated with the stain density of the area pointed to by the candidate cleaning action. The stain distribution feature map serves as both a component of the current state and the basis for generating the preference weights. The correction continuously scales the probability values of each candidate cleaning action within the range of zero to one, and the probability values of each candidate cleaning action are positively correlated with the stain density of the area they point to.
6. The method for determining the operation path of a photovoltaic power plant cleaning robot based on deep learning according to claim 1, characterized in that: Before step S2, the method further includes: encoding the geometric topology information of the photovoltaic array into graph structure data, and embedding the graph structure data into the policy network through a graph attention network; The policy network is constructed using an adaptive temperature soft actor-critic algorithm, including an Actor network and a Critic network. The Actor network outputs the action probability distribution, and the Critic network estimates the action value of the current state. The temperature coefficient of the adaptive temperature is dynamically determined based on the graph distance between the current position node of the cleaning robot and each uncleaned area node in the graph structure data. The larger the graph distance, the higher the temperature coefficient to enhance exploration, and the smaller the graph distance, the lower the temperature coefficient to enhance execution determinism. The graph distance is the number of edges or the weighted path length on the shortest path from the current position node to the uncleaned area node.
7. The method for determining the operation path of a photovoltaic power plant cleaning robot based on deep learning according to claim 1, characterized in that: When updating the parameters of the policy network in S4, a counterfactual path evaluation step is also included: For each piece of experience data sampled from the experience playback buffer, the counterfactual reward value of the candidate cleaning action that was not performed in the experience data is calculated based on the mapping relationship between stain density and the change in light transmittance after cleaning fitted by historical cleaning data. The difference between the counterfactual reward value and the reward value is used as the regret value, and the regret value is used as an auxiliary gradient signal to guide the parameter update of the policy network. The auxiliary gradient signal and the gradient signal based on the reward value together constitute a multi-signal driven parameter update.
8. The method for determining the operation path of a photovoltaic power plant cleaning robot based on deep learning according to claim 1, characterized in that: The reward function integrates cleaning revenue, cleaning energy consumption, cleaning coverage, and operational safety constraints in a weighted manner. The cleaning revenue is calculated based on the change in light transmittance of the photovoltaic module surface before and after cleaning. The cleaning energy consumption is calculated based on the measured value of the motor power consumption of the cleaning robot. The cleaning coverage is calculated based on the ratio of the area of the cleaned module to the total area of the soiled module. The operational safety constraints are calculated based on the distance between the cleaning robot and the boundary of the photovoltaic array. The weighting coefficients for each reward item are dynamically determined based on environmental condition parameters and robot operating status parameters; When sandstorm weather conditions are detected, increase the weight values of cleaning benefits and cleaning coverage to prioritize large-area cleaning; When the remaining battery power of the cleaning robot is lower than the power threshold determined based on the return path distance, the weight value of cleaning energy consumption is increased to reduce unnecessary energy-consuming actions.
9. The method for determining the operation path of a photovoltaic power plant cleaning robot based on deep learning according to claim 1, characterized in that: The training process of the policy network adopts a curriculum learning strategy, including: The first stage involves pre-training the policy network in a simulation environment using a simple scenario with dense stains. The second phase gradually increases the complexity of stain distribution, changes in weather conditions, and the scale of photovoltaic arrays; The third stage involves deploying the pre-trained strategy network to a real photovoltaic power plant for online fine-tuning.
10. The method for determining the operation path of a photovoltaic power plant cleaning robot based on deep learning according to claim 1, characterized in that: The preset termination conditions include: The stain coverage rate on the surface of all photovoltaic modules is lower than the preset cleaning standard threshold, the battery power of the cleaning robot is lower than the return power threshold determined based on the return path distance, or the cumulative cleaning time reaches the preset maximum working time.
