Cleaning head path planning method for glass curtain wall cleaning robot

Abstract

The invention provides a cleaning head path planning method for a glass curtain wall cleaning robot, and belongs to the technical field of glass curtain wall cleaning robots. Curtain wall images are collected through a multispectral visual sensor, and stain distribution is identified and energy consumption is predicted by using a probability graph structured prediction model; cleaning sub-regions are divided according to the stain coordinates, a grid map is established, a double-layer game optimization model with minimization of chassis movement energy consumption as an upper layer target and minimization of cleaning head operation time as a lower layer target is constructed, and two layers of paths are alternately optimized by adopting a Nash equilibrium algorithm until convergence; and through a resource constraint penalty term iteration adjustment path scheme, it is ensured that resource consumption does not exceed carrying capacity, and finally a robot chassis moving path and a cleaning head operation path are output, so that the technical problem that global optimization is difficult due to mutual coupling of the robot chassis moving path and the cleaning head operation path is solved.

Description

Technical Field

[0001] This invention belongs to the technical field of glass curtain wall cleaning robots, and more specifically, relates to a path planning method for a cleaning head of a glass curtain wall cleaning robot. Background Technology

[0002] Glass curtain wall cleaning robots typically employ a robotic arm carrying a cleaning head. Traditional path planning methods treat chassis movement and cleaning head operation as independent optimization problems, planning the chassis movement path first and then the cleaning head trajectory, or using a unified optimization framework to treat both as a single objective. In actual operation scenarios, the energy consumption of robotic arm posture adjustment affects both chassis movement efficiency and cleaning operation time. Each chassis movement changes the robotic arm's workspace, thus affecting the cleaning head's reachability and optimal cleaning sequence. Conversely, changes in the cleaning head's position require frequent posture adjustments by the robotic arm, increasing the chassis's energy consumption burden. Traditional methods, when dealing with this two-way coupling, either oversimplify, leading to optimization results deviating from the actual optimal solution, or get trapped in a high-dimensional joint optimization space, causing a surge in computational complexity that fails to meet real-time requirements. In other words, existing technologies suffer from the technical problem of global optimization difficulties due to the coupling between the robot chassis movement path and the cleaning head operation path. Summary of the Invention

[0003] In view of this, the present invention provides a path planning method for a cleaning head of a glass curtain wall cleaning robot, which can solve the technical problem in the prior art where the robot chassis movement path and the cleaning head operation path are coupled, resulting in difficulties in global optimization.

[0004] This invention is implemented as follows: It provides a path planning method for a cleaning head of a glass curtain wall cleaning robot. This method acquires image information of the glass curtain wall surface using a multispectral vision sensor, inputs it into a stain recognition energy consumption prediction model to obtain a set of coordinates for the stain distribution area, predicted values ​​for pose adjustment power consumption, predicted values ​​for cleaning power consumption, predicted values ​​for cleaning water consumption, and predicted values ​​for cleaning agent consumption. Based on the stain distribution area coordinate set, the method divides the cleaning operation into sub-regions and establishes a cleaning coverage grid map for each sub-region. A two-layer game optimization model is established between the robot path and the cleaning head path. This model includes an upper-layer optimization model aiming to minimize robot movement energy consumption and a lower-layer optimization model aiming to minimize the cleaning head operation time. The energy consumption for robotic arm posture adjustment, as a coupling term, simultaneously affects both the upper-level and lower-level optimization models to achieve path collaborative optimization. A Nash equilibrium solution algorithm is used to iteratively calculate the two-level game optimization model, obtaining initial solutions for the robot path and the cleaning head path. The total resource consumption is calculated based on predicted values ​​for posture adjustment power consumption, cleaning power consumption, cleaning water consumption, and cleaning agent consumption. It is then determined whether the total resource consumption exceeds the resource carrying capacity threshold. If the total resource consumption does not exceed the resource carrying capacity threshold, the initial solutions for the robot path and the cleaning head path are output as the final planned paths. If the total resource consumption exceeds the resource carrying capacity threshold, a resource constraint penalty term is added to the two-level game optimization model, and the calculation is recalculated.

[0005] The stain recognition energy consumption prediction model has a three-branch architecture consisting of a multi-scale feature extraction backbone network, a probabilistic graphical structured prediction branch, and an energy consumption regression prediction branch.

[0006] Among them, the probabilistic graph structured prediction branch models the stain identification task as a conditional random field, defines a univariate potential function, a pairwise potential function, and a higher-order potential function, and uses a mean-field variational inference algorithm to iteratively propagate probabilistic information between graph nodes.

[0007] The number of iterations of the mean field variational inference algorithm is equal to the natural logarithm of the curtain wall surface area divided by the standard curtain wall area, multiplied by the product of the cleaning head working width divided by the standard cleaning width and the maximum extension length of the robotic arm divided by the standard extension length, and finally multiplied by the iteration adjustment coefficient, which takes values ​​in the range of a∈[1.2, 1.8].

[0008] The standard curtain wall area is set to 1000 by default. The standard cleaning width is 0.26m by default, and the standard extension length is 0.59m by default.

[0009] The training dataset for the stain recognition energy consumption prediction model contains 12,000 training sample pairs. Each training sample pair consists of a combination of multispectral images, stain category labels, stain boundary coordinate labels, stain severity level labels, actual values ​​of pose adjustment power consumption, actual values ​​of cleaning power consumption, actual values ​​of cleaning water consumption, and actual values ​​of cleaning agent consumption.

[0010] The stain recognition energy consumption prediction model training adopts a joint loss function, which is a weighted sum of stain recognition cross-entropy loss, structured prediction potential energy loss and energy consumption regression mean square error loss. The weight coefficient of stain recognition cross-entropy loss is 0.5, the weight coefficient of structured prediction potential energy loss is 0.3, and the weight coefficient of energy consumption regression mean square error loss is 0.2.

[0011] The multispectral vision sensor includes a visible light camera, a near-infrared camera, and a polarization camera. The visible light camera is used to collect color and texture information of the curtain wall surface, the near-infrared camera is used to collect thermal radiation information of the curtain wall surface, and the polarization camera is used to collect light polarization information of the curtain wall surface.

[0012] The method for dividing the cleaning operation sub-areas is to divide the curtain wall surface into multiple cleaning operation sub-areas according to the maximum extension length of the robotic arm and the movement range of the robot chassis, with the width of each cleaning operation sub-area not exceeding twice the maximum extension length of the robotic arm.

[0013] The grid size of the cleaning coverage grid map corresponds to the working width of the cleaning head. Each grid corresponds to a unit to be cleaned on the curtain wall surface. The grid status includes uncleaned status, cleaned status, and obstacle status.

[0014] The objective function of the upper-level optimization model is expressed as follows: robot movement energy consumption equals the product of the robot chassis movement distance divided by the reference movement distance and the ground friction coefficient divided by the reference friction coefficient, multiplied by the chassis mass divided by the reference mass and the product of gravitational acceleration. The objective function of the lower-level optimization model is expressed as follows: the cleaning head operation time equals the ratio of the total length of the cleaning head trajectory divided by the reference trajectory length and the average speed of the cleaning head divided by the reference speed, plus the product of the total area of ​​the soiled area divided by the reference area and the cleaning time per unit area. The coupling term in the two-level game optimization model is the robot arm posture adjustment energy consumption, which equals the square of the robot movement step length divided by the reference step length, multiplied by the square of the cleaning head position change divided by the reference position change, and multiplied by the product of the number of robot arm joints and the energy consumption of a single joint adjustment. The steps of the Nash equilibrium solution algorithm include initializing the robot path initial solution and the cleaning head path initial solution, fixing the cleaning head path initial solution, solving the upper-level optimization model to obtain the robot path update solution, fixing the robot path update solution, solving the lower-level optimization model to obtain the cleaning head path update solution, and determining whether the changes in the objective function values ​​of the upper-level optimization model and the lower-level optimization model are both less than the convergence threshold. If so, the Nash equilibrium solution is output; otherwise, the iteration continues.

[0015] The resource carrying capacity thresholds include battery capacity thresholds, cleaning water tank capacity thresholds, and cleaning agent storage tank capacity thresholds. The battery capacity threshold is determined based on the battery's rated capacity and a safety margin coefficient, with the default value for the safety margin coefficient being 0.8. The resource constraint penalty is equal to the electricity excess penalty coefficient multiplied by the difference between the total electricity consumption and the battery capacity threshold, plus the water excess penalty coefficient multiplied by the difference between the total cleaning water consumption and the cleaning water tank capacity threshold, and the cleaning agent excess penalty coefficient multiplied by the difference between the total cleaning agent consumption and the cleaning agent storage tank capacity threshold.

[0016] This invention establishes a two-layer game-theoretic optimization model for the robot path and the cleaning head path. Minimizing chassis movement energy consumption and minimizing cleaning head operation time are modeled as upper and lower layer optimization objectives, respectively. The energy consumption for robotic arm posture adjustment is explicitly introduced into the model as a coupling term. A Nash equilibrium algorithm is used to alternately optimize the two paths until convergence. This method decouples the originally high-dimensional coupled joint optimization problem into a game process of two low-dimensional subproblems. The upper layer optimizes the fixed cleaning head path to solve for the optimal chassis movement scheme, while the lower layer optimizes the fixed chassis path to solve for the optimal cleaning head trajectory. Through iterative game theory, the two paths reach a Nash equilibrium state under the constraints of the coupling term. At this point, either side cannot further optimize its own objective by changing its strategy alone. In summary, this invention solves the technical problem mentioned in the background art, where the coupling between the robot chassis movement path and the cleaning head operation path leads to difficulties in global optimization. Attached Figure Description

[0017] Figure 1 This is a three-dimensional diagram of the glass curtain wall cleaning robot involved in the present invention.

[0018] Figure 2 This is a schematic diagram of the stain distribution recognition results and grid map division in the embodiment.

[0019] Figure 3 This is a three-dimensional spatial trajectory diagram of the robot path and the cleaning head path planning results in the embodiment.

[0020] Figure 4 This is a graph showing the changes in resource consumption during the iterative optimization process in the example.

[0021] In the attached figures, the reference numerals are explained as follows: 1. Robot base, 2. Robotic arm, 3. Cleaning head, 4. Multispectral vision sensor. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0023] This invention provides a path planning method for a cleaning head of a glass curtain wall cleaning robot, involving cleaning robots such as... Figure 1 As shown, the system includes a robot base 1, a robotic arm 2, a cleaning head 3, and a multispectral vision sensor 4. The method includes the following steps:

[0024] S01. Collect image information of the glass curtain wall surface through a multispectral vision sensor, input it into the stain recognition energy consumption prediction model to obtain the stain distribution area coordinate set, pose adjustment power consumption prediction value, cleaning power consumption prediction value, cleaning water consumption prediction value and cleaning agent consumption prediction value.

[0025] S02. Divide the cleaning operation sub-regions according to the coordinate set of the stain distribution area, and establish a cleaning coverage grid map for each cleaning operation sub-region. The grid size of the cleaning coverage grid map corresponds to the working width of the cleaning head.

[0026] S03. Establish a two-layer game optimization model for the robot path and the cleaning head path. The two-layer game optimization model includes an upper-layer optimization model with the goal of minimizing the robot's movement energy consumption and a lower-layer optimization model with the goal of minimizing the cleaning head's operation time.

[0027] S04. The Nash equilibrium solution algorithm is used to iteratively calculate the two-layer game optimization model to obtain the initial solution of the robot path and the initial solution of the cleaning head path.

[0028] S05. Calculate the total resource consumption based on the predicted power consumption for posture adjustment, the predicted power consumption for cleaning, the predicted water consumption for cleaning, and the predicted consumption for cleaning agent, and determine whether the total resource consumption exceeds the resource carrying capacity threshold.

[0029] S06. If the total resource consumption does not exceed the resource carrying capacity threshold, the initial solution of the robot path and the initial solution of the cleaning head path are output as the final planned path; if the total resource consumption exceeds the resource carrying capacity threshold, a resource constraint penalty term is added to the two-layer game optimization model and the process returns to step S04 for recalculation.

[0030] The stain recognition energy consumption prediction model is structured as a three-branch architecture consisting of a multi-scale feature extraction backbone network, a probabilistic graphical structured prediction branch, and an energy consumption regression prediction branch. The multi-scale feature extraction backbone network uses convolutional layers with residual connections to extract image features at different resolutions, outputting feature maps at four scales. The probabilistic graphical structured prediction branch models the stain recognition task as a conditional random field. It models the stain category relationships, spatial adjacency relationships, and illumination consistency relationships in the output space as edge connections in a graph, constructing potential functions between nodes. A mean-field variational inference algorithm iteratively propagates probability information between graph nodes, updating the edge probability distribution of nodes in each iteration. After a preset number of iterations, the coordinate set of the stain distribution area is obtained. The energy consumption regression prediction branch contains three parallel fully connected layers that predict the pose adjustment power consumption, the cleaning power consumption, and the resource consumption, respectively. The resource consumption prediction includes the cleaning water consumption and the cleaning agent consumption. The number of iterations of the mean-field variational inference algorithm in the stain recognition energy consumption prediction model is determined based on the working width of the cleaning head, the maximum extension length of the robotic arm, and the surface area of ​​the curtain wall. The calculation of the number of iterations of the mean-field variational inference algorithm is expressed as follows: the number of iterations equals the natural logarithm of the curtain wall surface area divided by the standard curtain wall area, multiplied by the product of the working width of the cleaning head divided by the standard cleaning width and the maximum extension length of the robotic arm divided by the standard extension length, and finally multiplied by an iteration adjustment coefficient. The standard curtain wall area is obtained from experiments in typical operating scenarios of the glass curtain wall cleaning robot and is defaulted to 1000. The standard cleaning width is obtained experimentally based on the size of the cleaning head of the glass curtain wall cleaning robot and is set to a default value of 0.26m. The standard extension length is obtained experimentally based on the specifications of the robotic arm of the glass curtain wall cleaning robot and is set to a default value of 0.59m. The iterative adjustment coefficient is set to a closed interval of 1.2 to 1.8.

[0031] The steps for establishing the training dataset for the stain recognition energy consumption prediction model include: collecting multispectral images of the glass curtain wall surface as input samples under different time periods and different pedestrian traffic conditions; manually annotating each multispectral image, including stain category labeling, stain boundary coordinate labeling, and stain severity level labeling; performing actual cleaning operations on the curtain wall area corresponding to the annotated multispectral images, recording the sequence of joint angle changes of the robotic arm, the sequence of cleaning head rotation speed changes, water pump flow data, and cleaning agent spraying volume data; calculating the actual power consumption value for pose adjustment based on the sequence of joint angle changes of the robotic arm combined with the kinematic model of the robotic arm, and based on... The actual power consumption of cleaning is calculated by combining the sequence of changes in the rotational speed of the cleaning head with the power of the cleaning head motor. The actual power consumption of cleaning water is calculated by time integration based on the water pump flow data. The actual power consumption of cleaning agent is calculated by time integration based on the cleaning agent spraying data. The multispectral image, the stain category label, the stain boundary coordinate label, the stain severity level label, the actual power consumption of pose adjustment, the actual power consumption of cleaning, the actual water consumption of cleaning, and the actual power consumption of cleaning agent are combined to form training sample pairs. A total of 12,000 sets of training sample pairs are collected to form the training dataset of the stain recognition energy consumption prediction model.

[0032] The steps for training the stain recognition energy consumption prediction model include: dividing the training dataset of the stain recognition energy consumption prediction model into a training set, a validation set, and a test set in a ratio of 8:1:1; initializing the multi-scale feature extraction backbone network with weights pre-trained on a large-scale image dataset; randomly initializing the potential function parameters of the probabilistic graphical structured prediction branch; using the Xavier initialization method for the fully connected layer parameters of the energy consumption regression prediction branch; and defining the joint loss function as a weighted sum of the stain recognition cross-entropy loss, the structured prediction potential loss, and the energy consumption regression mean square error loss, wherein the stain recognition cross-entropy loss in the weighted sum... The weighting coefficients are 0.5, 0.3, and 0.2 for the structured prediction potential loss and the energy consumption regression mean square error loss, respectively. An adaptive moment estimation optimization algorithm is used to update the parameters. The initial learning rate is set to 0.0001, and it decays to 0.5 times its original value every 30 training cycles. During training, the model performance is evaluated on the validation set every 5 training cycles. Training stops when the validation set loss does not decrease for 10 consecutive training cycles. The model parameters with the smallest validation set loss are selected as the final training result, and performance is evaluated on the test set to obtain the stain recognition energy consumption prediction model.

[0033] The probabilistic graphical structured prediction branch explicitly models the output spatial constraints of stain recognition as a graph structure of a conditional random field. It utilizes variational inference techniques to propagate probabilistic information between graph nodes, achieving joint inference between stain category, spatial location, and complex shadow conditions. Traditional independent pixel classification methods ignore the correlation between stain types in adjacent regions, leading to numerous isolated misclassifications and blurred boundaries in the recognition results. The probabilistic graphical structured prediction branch defines a univariate potential function to describe the probability of a single pixel belonging to each stain category, a pairwise potential function to describe the category consistency constraints between adjacent pixels, and a higher-order potential function to describe the similarity constraints of stain features within the same illumination area. The constraint relationships corresponding to these univariate, pairwise, and higher-order potential functions are encoded as edge connections and potential functions in the graph. During the inference phase, the mean-field variational inference algorithm iteratively updates the edge probability distribution of each graph node, allowing the edge probability distributions of adjacent graph nodes to influence each other and gradually achieve global consistency, thereby obtaining stain distribution prediction results with better spatial continuity and clearer boundaries. Compared to post-processing smoothing methods, the probabilistic graphical structured prediction branch learns spatial constraints during model training, avoiding information loss caused by forcibly imposing constraints after independent prediction. The potential function parameters of the conditional random field are automatically learned through end-to-end training, eliminating the need for manual constraint rule design. This allows the stain recognition energy consumption prediction model to adapt to the stain distribution characteristics of different curtain wall surfaces. The mean-field variational inference algorithm performs probabilistic inference on the graph through a message passing mechanism, reducing computational complexity compared to precise inference methods and enabling real-time stain recognition of large-scale curtain wall surfaces. Gibbs sampling technology generates diverse candidate prediction results from the posterior distribution. By sampling multiple possible stain distribution schemes, the stain recognition energy consumption prediction model can estimate the uncertainty of the prediction, providing a reliability assessment basis for subsequent path planning. The introduction of the probabilistic graphical structured prediction branch improves the accuracy of stain boundary positioning, reduces category confusion between adjacent stain areas, and increases overall recognition accuracy, providing more accurate stain distribution information input for cleaning path planning. By modeling the structured relationships of the output space as a probabilistic graphical model, the stain recognition energy consumption prediction model can globally optimize the preliminary classification results during the recognition stage using spatial adjacency information and illumination consistency information, avoiding the local optimum problem caused by independent classification. The conditional random field framework allows for the flexible definition of different types of potential functions to encode domain knowledge. The univariate potential function captures pixel-level features, the pairwise potential function captures local smoothness, and the higher-order potential function captures region-level consistency. This multi-level constraint modeling enables the model to perform inference at different scales.The mean-field variational inference algorithm decomposes the global optimization problem into local update operations through iterative updates. Each graph node updates its own state based on the current state of its neighboring nodes. After multiple iterations, the state of the entire graph converges to a consistent state, which corresponds to the optimal stain distribution prediction that satisfies all constraints.

[0034] The multispectral vision sensor includes a visible light camera, a near-infrared camera, and a polarization camera. The visible light camera is used to collect color and texture information of the curtain wall surface. The near-infrared camera is used to collect thermal radiation information of the curtain wall surface to distinguish between organic and inorganic stains. The polarization camera is used to collect light polarization information of the curtain wall surface to identify water stains and oil films. The visible light camera, the near-infrared camera, and the polarization camera output visible light images, near-infrared images, and polarization images, respectively. The visible light images, near-infrared images, and polarization images are stitched together according to spectral dimensions to form the image information of the glass curtain wall surface.

[0035] The method for dividing the cleaning operation sub-areas is to divide the curtain wall surface into multiple cleaning operation sub-areas according to the maximum extension length of the robotic arm and the movement range of the robot chassis, with the width of each cleaning operation sub-area not exceeding twice the maximum extension length of the robotic arm.

[0036] Each grid in the cleaning coverage grid map corresponds to a unit on the curtain wall surface to be cleaned. The grid state includes three types: uncleaned state, cleaned state, and obstacle state. The obstacle state corresponds to the window frame, decorative strip, and non-glass area on the curtain wall surface.

[0037] The objective function of the upper-level optimization model minimizes the total energy consumption of the robot chassis during ground movement. The inputs to the objective function include the robot chassis movement distance, the ground friction coefficient, and the chassis mass. The output is the robot path optimization direction vector. The objective function is expressed as follows: robot movement energy consumption equals the product of the robot chassis movement distance divided by a reference movement distance, the ground friction coefficient divided by a reference friction coefficient, the chassis mass divided by a reference mass, and the acceleration due to gravity. The reference movement distance is obtained from experiments in typical operating scenarios of the glass curtain wall cleaning robot and is assumed to be 100m. The reference friction coefficient is obtained from experiments on the working ground material of the glass curtain wall cleaning robot and is assumed to be 0.3. The reference mass is obtained from experiments based on the chassis design parameters of the glass curtain wall cleaning robot and is assumed to be 300kg. The constraints of the upper-level optimization model include: the robot movement path must be parallel to the curtain wall; the robot chassis must not deviate from the designated safety guide yellow line; and the robot chassis must stop at resupply points to replenish resources. The robot chassis travel distance is obtained by accumulating the distances between adjacent path points in the initial solution of the robot path. The ground friction coefficient is obtained by measuring the surface of the set safety guide yellow line using a friction tester. The chassis mass is obtained by measuring the robot chassis using a weighing sensor. The robot path optimization direction vector is used to guide the Nash equilibrium solution algorithm to adjust the direction of the initial solution of the robot path.

[0038] The objective function of the lower-level optimization model minimizes the total operation time required for the cleaning head to complete cleaning of all stained areas. The inputs to the objective function include the total length of the cleaning head's trajectory, the average speed of the cleaning head, and the total area of ​​the stained areas. The output is the path optimization direction vector of the cleaning head. The objective function is expressed as follows: the cleaning head operation time equals the ratio of the total length of the cleaning head's trajectory divided by the reference trajectory length, to the ratio of the average speed of the cleaning head divided by the reference speed, plus the product of the total area of ​​the stained areas divided by the reference area and the cleaning time per unit area. The reference trajectory length is obtained from typical movement path experiments of the glass curtain wall cleaning robot's cleaning head and is defaulted to 50m. The reference speed is obtained from motion performance experiments of the glass curtain wall cleaning robot's cleaning head and is defaulted to 0.5m / s. The reference area is obtained from single-operation area experiments of the glass curtain wall cleaning robot and is defaulted to 100. The cleaning time per unit area is obtained from the cleaning efficiency experiment of the glass curtain wall cleaning robot and is defaulted to 10s / The constraints of the lower-level optimization model include: the cleaning head's movement range must not exceed the robotic arm's workspace; the cleaning head's path must cover all stain area grids in the coordinate set of the stain distribution areas; and the cleaning head's repeated cleaning of the same grid cell must not exceed once. The total length of the cleaning head's movement trajectory is obtained by summing the distances between adjacent path points in the initial solution of the cleaning head's path. The average movement speed of the cleaning head is obtained by dividing the total length of the cleaning head's movement trajectory by the total movement time corresponding to the initial solution of the cleaning head's path. The total area of ​​the stain area is obtained by summing the areas of all stain areas in the coordinate set of the stain distribution areas. The cleaning head path optimization direction vector is used to guide the Nash equilibrium solution algorithm to adjust the direction of the initial solution of the cleaning head's path.

[0039] The coupling term in the two-layer game optimization model is the energy consumption for robot arm posture adjustment. This energy consumption simultaneously affects the robot movement energy consumption in the upper-layer optimization model and the cleaning head operation time in the lower-layer optimization model. The robot arm posture adjustment energy consumption is related to the robot's movement step length and the change in the cleaning head position. The coupling term is calculated as follows: the robot arm posture adjustment energy consumption equals the square of the robot's movement step length divided by the reference step length, multiplied by the square of the change in the cleaning head position divided by the reference position change, and then multiplied by the product of the number of robot arm joints and the energy consumption of a single joint adjustment. The reference step length is obtained from experiments on the chassis movement step distance of the glass curtain wall cleaning robot and is assumed to be 2m. The reference position change is obtained from experiments on the adjustment range of the cleaning head position of the glass curtain wall cleaning robot and is assumed to be 1m. The energy consumption of a single joint adjustment is obtained from experiments on the energy consumption of the robot arm joint motors of the glass curtain wall cleaning robot and is assumed to be 50J. The robot's step size is the distance between two adjacent path points in the initial solution of the robot path, the change in the cleaning head position is the straight-line distance between two adjacent path points in the initial solution of the cleaning head path, and the number of robotic arm joints is obtained based on the robotic arm structure of the glass curtain wall cleaning robot.

[0040] The steps of the Nash equilibrium solution algorithm include: initializing the robot path initial solution and the cleaning head path initial solution as greedy search results; fixing the cleaning head path initial solution and solving the upper-level optimization model to obtain the robot path update solution; fixing the robot path update solution and solving the lower-level optimization model to obtain the cleaning head path update solution; calculating the change in the objective function value of the upper-level optimization model and the change in the objective function value of the lower-level optimization model; if the absolute values ​​of the changes in the objective function value of the upper-level optimization model and the lower-level optimization model are both less than the convergence threshold, then outputting the robot path update solution and the cleaning head path update solution. The updated head path solution is used as the Nash equilibrium solution and assigned to the initial solution of the robot path and the initial solution of the cleaning head path, respectively. If the absolute value of the change in the objective function value of the upper-level optimization model or the absolute value of the change in the objective function value of the lower-level optimization model is greater than or equal to the convergence threshold, the updated robot path solution is assigned to the initial robot path solution and the updated cleaning head path solution is assigned to the initial cleaning head path solution. Then, the process of solving the upper-level optimization model with the fixed initial cleaning head path solution is returned to continue iterating. The convergence threshold is obtained experimentally based on the accuracy requirements of the glass curtain wall cleaning robot path planning and is set to 0.01 by default.

[0041] The resource carrying capacity thresholds include battery capacity thresholds, cleaning water tank capacity thresholds, and cleaning agent storage tank capacity thresholds. The battery capacity threshold is determined based on the battery's rated capacity and safety margin coefficient. The cleaning water tank capacity threshold is determined based on the tank volume and water density. The cleaning agent storage tank capacity threshold is determined based on the tank volume and cleaning agent density. The battery rated capacity is obtained by reading the battery specifications of the glass curtain wall cleaning robot. The safety margin coefficient is obtained experimentally based on the battery safety usage specifications of the glass curtain wall cleaning robot and is defaulted to 0.8. The tank volume is obtained by measuring the tank dimensions of the glass curtain wall cleaning robot. The water density is taken as 1000 kg / m³ under standard temperature and pressure conditions. The volume of the storage tank is obtained by measuring the size of the cleaning agent storage tank of the glass curtain wall cleaning robot, and the density of the cleaning agent is obtained according to the product instructions of the cleaning agent.

[0042] The total resource consumption includes total power consumption, total cleaning water consumption, and total cleaning agent consumption. The total power consumption is the sum of the predicted power consumption for posture adjustment and the predicted power consumption for cleaning. The total cleaning water consumption is equal to the predicted cleaning water consumption, and the total cleaning agent consumption is equal to the predicted cleaning agent consumption.

[0043] The resource constraint penalty term is a weighted penalty for the portion of the total resource consumption exceeding the resource carrying capacity threshold. This penalty term is applied to the objective functions of both the upper-level and lower-level optimization models, causing the initial solutions of the robot path and the cleaning head path to adjust towards satisfying the resource constraints during the iteration process. The resource constraint penalty term is calculated as follows: it equals the power excess penalty coefficient multiplied by the difference between the total power consumption and the battery capacity threshold, plus the water excess penalty coefficient multiplied by the difference between the total cleaning water consumption and the cleaning water tank capacity threshold, plus the cleaning agent excess penalty coefficient multiplied by the difference between the total cleaning agent consumption and the cleaning agent storage tank capacity threshold. The power excess penalty coefficient is obtained from experiments using the glass curtain wall cleaning robot's energy management strategy and is set to a default value of 1.5; the water excess penalty coefficient is obtained from experiments using the glass curtain wall cleaning robot's water resource management strategy and is set to a default value of 1.2; and the cleaning agent excess penalty coefficient is obtained from experiments using the glass curtain wall cleaning robot's cleaning agent management strategy and is set to a default value of 1.3.

[0044] The final planned path includes the final robot path and the final cleaning head path. The final robot path is the initial solution of the robot path, and the final cleaning head path is the initial solution of the cleaning head path. The final robot path is used to control the movement trajectory of the robot chassis, and the final cleaning head path is used to control the working trajectory of the cleaning head.

[0045] The specific implementation methods of the above steps are described in detail below.

[0046] The specific implementation of step S01 involves acquiring multimodal image information of the glass curtain wall surface using a multispectral vision sensor and inputting it into a stain recognition energy consumption prediction model for intelligent analysis. First, a visible light camera captures the color and texture features of the curtain wall surface, obtaining visible light image data, which includes information about the appearance and morphology of stains. Then, a near-infrared camera collects the thermal radiation characteristics of the curtain wall surface, obtaining near-infrared image data, which effectively distinguishes the thermal radiation differences between organic and inorganic stains. Next, a polarization camera detects changes in the polarization state of light on the curtain wall surface, obtaining polarization image data, which has high sensitivity for recognizing transparent stains such as water stains and oil films. The visible light image, near-infrared image, and polarization image are stitched together according to spectral dimensions to form a multispectral fusion image, which serves as the input to the stain recognition energy consumption prediction model. The stain recognition energy consumption prediction model adopts a three-branch network architecture, including a multi-scale feature extraction backbone network, a probabilistic graphical structured prediction branch, and an energy consumption regression prediction branch. The multi-scale feature extraction backbone network extracts image features at different resolutions through multiple residual convolutional layers, outputting four feature maps at different scales to capture the detailed texture and overall distribution pattern of stains. The probabilistic graphical structured prediction branch models the stain recognition task as a conditional random field. It defines a univariate potential function to describe the stain category probability of a single pixel, a pairwise potential function to describe the category consistency constraint of adjacent pixels, and a higher-order potential function to describe the feature similarity constraint within the same illumination area. A mean-field variational inference algorithm iteratively propagates probability information between graph nodes, updating the edge probability distribution of each node in each iteration. After a preset number of iterations, it outputs the coordinate set of the stain distribution area. The energy consumption regression prediction branch contains three parallel fully connected layers that predict the power consumption for pose adjustment, cleaning, and resource consumption, including predictions for cleaning water and cleaning agent consumption. The purpose of this step is to obtain accurate stain distribution information and energy consumption prediction data, providing a reliable decision-making basis for subsequent path planning.

[0047] The specific implementation of step S02 involves dividing the cleaning operation into sub-regions based on the coordinate set of the stain distribution area and establishing a cleaning coverage grid map. First, the coverage area of ​​a single operation sub-region is determined based on the maximum extension length of the robotic arm and the movement range of the robot chassis. The curtain wall surface is divided into multiple cleaning operation sub-regions along longitudinal strips, with the width of each sub-region not exceeding twice the maximum extension length of the robotic arm, ensuring that the robot can adjust and cover the entire sub-region at a single docking position. Then, a cleaning coverage grid map is established for each cleaning operation sub-region. The grid size corresponds to the working width of the cleaning head, so that each grid represents the curtain wall surface unit that the cleaning head can cover in one sweep. The coordinate set of the stain distribution area is mapped onto the grid map. Grids containing stains are marked as uncleaned, grids corresponding to window frames, decorative strips, and non-glass areas of the curtain wall surface are marked as obstacles, and the remaining grids are marked as cleaned. The purpose of this step is to discretize the continuous curtain wall surface into regular grid units, simplifying the search space for path planning and improving computational efficiency.

[0048] The specific implementation of step S03 involves establishing a two-layer game optimization model for the robot path and the cleaning head path. The upper-layer optimization model aims to minimize the robot's movement energy consumption. The objective function inputs include the robot chassis movement distance, the ground friction coefficient, and the chassis mass. By calculating the energy consumption required for the robot chassis to overcome friction during ground movement, the model outputs a robot path optimization direction vector. Constraints of the upper-layer optimization model include that the robot's movement path must be parallel to the curtain wall, the robot chassis must not deviate from the designated safety guide yellow line, and the robot chassis must stop at replenishment points to replenish resources. The lower-layer optimization model aims to minimize the cleaning head's operation time. The objective function inputs include the total length of the cleaning head's movement trajectory, the average movement speed of the cleaning head, and the total area of ​​the stained area. By calculating the movement time and cleaning time required for the cleaning head to cover all stained areas, the model outputs a cleaning head path optimization direction vector. Constraints of the lower-layer optimization model include that the cleaning head's movement range must not exceed the robotic arm's workspace, the cleaning head path must cover all stained area grids, and the cleaning head must repeat cleaning the same grid no more than once. The coupling term in the two-layer game optimization model is the energy consumption for robot arm posture adjustment. This coupling term simultaneously affects the robot's movement energy consumption in the upper layer and the cleaning head's operation time in the lower layer. The robot arm's posture adjustment energy consumption is related to the robot's step size and the change in the cleaning head's position. When the robot's step size or the change in the cleaning head's position increases, the robot arm needs to perform a larger posture adjustment, leading to increased energy consumption. The purpose of this step is to establish a collaborative optimization framework for the robot chassis's movement path and the cleaning head's operation path, capturing the mutual influence between them through a two-layer game model.

[0049] The specific implementation of step S04 involves iteratively calculating the two-layer game optimization model using a Nash equilibrium solution algorithm. First, the initial solutions for the robot path and the cleaning head path are initialized as greedy search results. Greedy search quickly generates initial feasible paths through local optimal choices. Then, the initial solution for the cleaning head path is fixed, and the upper-level optimization model is solved to obtain the updated robot path solution. This step reduces movement energy consumption by optimizing the robot chassis's movement path. Next, the updated robot path solution is fixed, and the lower-level optimization model is solved to obtain the updated cleaning head path solution. This step shortens operation time by optimizing the cleaning head's operation path. The changes in the objective function values ​​of the upper-level and lower-level optimization models are calculated, and it is determined whether the absolute values ​​of both changes are less than a convergence threshold. The convergence threshold is experimentally obtained based on the path planning accuracy requirements and is defaulted to 0.01. If the absolute values ​​of both changes are less than the convergence threshold, it indicates that the iteration has converged to the Nash equilibrium point. The updated robot path solution and the updated cleaning head path solution are output as Nash equilibrium solutions and assigned to the initial solutions for the robot path and the cleaning head path, respectively. If the absolute value of any change is greater than or equal to the convergence threshold, it indicates that the iteration has not yet converged. The updated robot path solution and the updated cleaning head path solution are then assigned to the initial robot path solution and the initial cleaning head path solution, respectively. The iteration continues by fixing the initial cleaning head path solution and solving the upper-level optimization model. The Nash equilibrium solution algorithm, through alternating optimization of the upper and lower-level models, gradually converges the robot path and the cleaning head path to a stable strategy combination under mutual influence. This strategy combination satisfies the equilibrium condition that neither side can obtain a better result by unilaterally changing its strategy. The purpose of this step is to solve the Nash equilibrium solution of the two-level game optimization model and obtain the initial solutions for the robot path and the cleaning head path.

[0050] The specific implementation of step S05 involves calculating the total resource consumption based on the predicted energy consumption value and determining whether it exceeds the resource carrying capacity threshold. First, the total power consumption is calculated, which equals the sum of the predicted pose adjustment power consumption and the predicted cleaning power consumption, reflecting the total electrical energy required for the robot to complete the cleaning task. Then, the total cleaning water consumption is calculated, which equals the predicted cleaning water consumption, reflecting the water resource consumption during the cleaning process. Next, the total cleaning agent consumption is calculated, which equals the predicted cleaning agent consumption, reflecting the cleaning agent consumption during the cleaning process. The battery capacity threshold is determined based on the battery's rated capacity and safety margin coefficient. The safety margin coefficient is obtained experimentally according to the battery safety usage specifications and is defaulted to 0.8 to ensure the battery does not over-discharge. The cleaning water tank capacity threshold is determined based on the water tank volume and water density, with the water density set at 1000 kg / m³ under standard temperature and pressure conditions. The cleaning agent tank capacity threshold is determined based on the tank volume and cleaning agent density, which is obtained from the cleaning agent product instructions. The total power consumption is compared with the battery capacity threshold, the total cleaning water consumption with the cleaning water tank capacity threshold, and the total cleaning agent consumption with the cleaning agent tank capacity threshold to determine if any resource consumption exceeds the corresponding capacity threshold. The purpose of this step is to verify the resource feasibility of the planned path and ensure that the resources carried by the robot can support the completion of the cleaning task.

[0051] The specific implementation of step S06 involves determining the output path or recalculating based on the resource consumption judgment result. If the total resource consumption does not exceed the resource carrying capacity threshold, it indicates that the initial solutions for the robot path and the initial solutions for the cleaning head path are feasible under resource constraints. The initial solutions for the robot path and the initial solutions for the cleaning head path are directly output as the final planned paths. The final robot path is used to control the movement trajectory of the robot chassis, and the final cleaning head path is used to control the working trajectory of the cleaning head. If the total resource consumption exceeds the resource carrying capacity threshold, it indicates that the current path solution does not meet resource constraints, and a resource constraint penalty term needs to be added to the two-layer game optimization model. The resource constraint penalty term is calculated as follows: the power excess penalty coefficient multiplied by the difference between the total power consumption and the battery capacity threshold, plus the water resource excess penalty coefficient multiplied by the difference between the total cleaning water consumption and the cleaning water tank capacity threshold, plus the cleaning agent excess penalty coefficient multiplied by the difference between the total cleaning agent consumption and the cleaning agent storage tank capacity threshold. The penalty coefficients for exceeding power limits are obtained from energy management strategy experiments and have a default value of 1.5; the penalty coefficients for exceeding water resource limits are obtained from water resource management strategy experiments and have a default value of 1.2; and the penalty coefficients for exceeding cleaning agent limits are obtained from cleaning agent management strategy experiments and have a default value of 1.3. Resource constraint penalties are applied to the objective functions of both the upper-level and lower-level optimization models, increasing the optimization cost of paths exceeding resource constraints. This prompts the Nash equilibrium algorithm to adjust the initial solutions for the robot path and the cleaning head path towards satisfying resource constraints during iteration. Then, the algorithm returns to step S04 and re-executes the Nash equilibrium algorithm iteratively until a path solution satisfying resource constraints is obtained. The purpose of this step is to ensure that the final planned path output satisfies both energy consumption and time optimization objectives, as well as the capacity constraint of the resources carried by the robot.

[0052] It should be noted that the key technical ideas of this invention include the introduction of a probabilistic graphical structured prediction branch, the establishment of a two-layer game optimization model, and a dynamic adjustment mechanism for resource constraint penalty terms. The probabilistic graphical structured prediction branch models the stain recognition task as a conditional random field, using a mean-field variational inference algorithm to propagate probabilistic information between graph nodes. It explicitly encodes stain category relationships, spatial adjacency relationships, and illumination consistency relationships. Compared to traditional independent pixel classification methods, this effectively solves the problems of isolated misclassification points and blurred boundaries in the recognition results, improving the stain boundary positioning accuracy and overall recognition accuracy, and providing more accurate stain distribution information input for cleaning path planning. The two-layer game optimization model, through the collaborative optimization of minimizing the upper-layer robot movement energy consumption and minimizing the lower-layer cleaning head operation time, introduces the robotic arm posture adjustment energy consumption as a coupling term. This captures the mutual influence between the robot chassis movement path and the cleaning head operation path. Compared to traditional step-by-step planning methods, this avoids the global performance degradation caused by local optima and achieves overall coordinated optimization of the robot path and the cleaning head path. The dynamic adjustment mechanism of the resource constraint penalty term softens the resource capacity constraint into the optimization objective by adding a weighted penalty for resource excess in the two-level game optimization model. Compared with traditional hard constraint methods, this improves the convergence and robustness of the solution algorithm, ensuring that the planned path achieves the optimization goals of energy consumption and time while satisfying resource constraints. The synergistic effect of these three key technical ideas lies in the high-quality stain distribution input provided by the probabilistic graphical structured prediction branch, the overall coordination of the robot path and the cleaning head path achieved by the two-level game optimization model, and the dynamic adjustment mechanism of the resource constraint penalty term ensuring the resource feasibility of path planning. The three form a complete closed loop from perception to planning to constraint satisfaction. Compared with existing technologies that separate perception and planning and do not consider resource constraints in path planning, this significantly improves the operating efficiency and reliability of the glass curtain wall cleaning robot.

[0053] It should be noted that this invention also solves the following technical problems: poor spatial continuity and blurred boundaries in stain recognition results. Traditional independent pixel classification methods treat the stain category judgment of each pixel as an independent event, ignoring the correlation between stain types in adjacent areas and the similarity of stain features under the same illumination conditions. This leads to a large number of isolated misclassification points in the recognition results, jagged stain boundaries, and inaccurate localization. This invention introduces a probabilistic graphical structured prediction branch, models the stain recognition task as a conditional random field, defines a univariate potential function to describe the single-pixel classification probability, a pairwise potential function to describe the consistency constraint of adjacent pixel categories, and a higher-order potential function to describe the feature similarity constraint of the same illumination area. A mean-field variational inference algorithm is used to iteratively propagate probability information between graph nodes, so that the edge probability distributions of adjacent nodes influence each other and gradually reach a globally consistent state. This structured prediction method learns spatial constraint relationships during the model training phase. Through end-to-end optimization, the potential function parameters automatically adapt to the different stain distribution characteristics of curtain walls, avoiding information loss caused by the forced application of constraints in post-processing smoothing methods. This significantly improves the stain boundary positioning accuracy and regional segmentation continuity, providing more accurate and reliable stain distribution information input for subsequent path planning.

[0054] Specifically, the principle of this invention is as follows: The fundamental reason why this invention can solve the coupling problem between the robot chassis movement path and the cleaning head operation path lies in the fact that the two-layer game model transforms the solution of the coupled optimization problem into a process of finding game equilibrium. In traditional joint optimization, the chassis path and the cleaning head path are optimized simultaneously as joint decision variables. The objective function includes the interaction terms between the two, leading to complex gradient calculations and a tendency to get trapped in local optima. This invention treats the two paths as the strategy choices of two players in a game, with the energy consumption of the robotic arm posture adjustment as the cost of strategy interaction. The Nash equilibrium framework ensures that each player chooses its own optimal strategy given the opponent's strategy. The key advantage of this game theory perspective is that it decomposes the global optimization into an iterative process of local optimal responses. Each iteration only needs to solve a single-layer optimization problem under the condition of fixing the opponent's strategy, which greatly reduces the computational complexity. Furthermore, the Nash equilibrium point is stable, and any deviation by one player will increase its own cost. The logical rationality of the technical solution is reflected in the design of the coupling term for the energy consumption of the robotic arm posture adjustment. This term has a square relationship with both the chassis movement step size and the change in the position of the cleaning head, accurately characterizing the impact of the interaction between the two paths on the overall system performance. This makes the game optimization process naturally tend to reduce frequent adjustments and large movements, achieving coordinated optimization of energy consumption and time.

[0055] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0056] The specific implementation methods of steps S01 to S02 are the same as those described above, and will not be repeated in detail here.

[0057] The specific implementation of step S03 involves establishing a two-layer game optimization model for the robot path and the cleaning head path. The objective function of the upper-layer optimization model is expressed as follows:

[0058] ;

[0059] In the formula, Energy consumption for robot movement, unit: ; The distance traveled by the robot chassis, in units of ; For reference distance traveled, the unit is... Based on experiments conducted in typical operating scenarios of glass curtain wall cleaning robots, the empirical value is 100. ; Let be the coefficient of friction of the ground, which is dimensionless; The coefficient of friction is dimensionless and obtained from experiments on the working surface material of the glass curtain wall cleaning robot; the empirical value is 0.3. Chassis mass, unit: ; For reference quality, the unit is... Based on experimentally obtained design parameters of the glass curtain wall cleaning robot chassis, the empirical value is 300. ; This is the acceleration due to gravity, with units of 1. The value is 9.8 .in, It is obtained by accumulating the distances between adjacent path points in the initial solution of the robot path, and the specific calculation is described as follows:

[0060] ;

[0061] In the formula, The total number of path points in the initial solution of the robot path, dimensionless; The first initial solution for the robot path The location coordinates of each path point, in units of ; This is the Euclidean norm operator, used to calculate the straight-line distance between two position coordinates. The surface of the safety guide yellow line was measured using a friction tester. The weight is obtained by measuring the robot chassis using a weighing sensor. This formula achieves dimensionless processing by calculating the ratios of each physical quantity to its corresponding reference value, where the distance ratio... The ratio of friction coefficients reflects the scale of movement. Mass ratio reflects ground resistance level The energy consumption is calculated by multiplying the product of the three factors (load level, gravitational acceleration, and reference distance traveled) by the product of the gravitational acceleration and the reference distance traveled.

[0062] The objective function of the lower-level optimization model is expressed as follows:

[0063] ;

[0064] In the formula, The time for cleaning the head is in units of time. ; The total length of the cleaning head's movement trajectory, in units of... ; The reference trajectory length is expressed in units of... Based on experiments of typical movement paths of the cleaning head of a glass curtain wall cleaning robot, the empirical value is 50. ; The average moving speed of the cleaning head, in units of ; For reference movement speed, the unit is... Based on experiments on the motion performance of the cleaning head of the glass curtain wall cleaning robot, the empirical value is 0.5. ; For reference time, the unit is... Based on experiments using the reference travel time of the cleaning head of the glass curtain wall cleaning robot, the empirical value is 100. ; The total area of ​​the stained area is expressed in units of... ; For reference area, the unit is... Based on experiments with the glass curtain wall cleaning robot on the area covered in a single operation, the empirical value is 100. ; The cleaning time is per unit area, and the unit is 1. Based on experiments with glass curtain wall cleaning robots, the empirical value is 10. .in, The distance between adjacent path points in the initial solution of the cleaning head path is accumulated, and the specific calculation is described as follows:

[0065] ;

[0066] In the formula, The total number of path points in the initial solution of the cleaning head path, dimensionless; The first solution in the initial solution of the cleaning head path The location coordinates of each path point, in units of . The calculation is expressed as follows:

[0067] ;

[0068] In the formula, The total travel time corresponding to the initial solution of the cleaning head path, in units of . The calculation is obtained by summing the surface area of ​​all stained regions in the coordinate set of the stained distribution area. The specific calculation method is as follows:

[0069] ;

[0070] In the formula, The coordinates of the stain distribution area represent the total number of stain areas, which is dimensionless. For the first The area of ​​each stained area, in units of The first term of the formula is the travel time term, which is derived from the ratio of trajectory lengths. The reciprocal of the speed ratio The second term, calculated by multiplying the reference time, is the operation time, determined by the ratio of the stain area. The cleaning time is calculated by multiplying the unit cleaning time by the total cleaning time. The sum of the two is the total operation time.

[0071] The calculation of coupling terms in the two-level game optimization model is expressed as follows:

[0072] ;

[0073] In the formula, Energy consumption for robotic arm posture adjustment, in units of ; The robot's step length, in units of ; For reference step size, the unit is... Based on experiments with the chassis movement step distance of the glass curtain wall cleaning robot, the empirical value is 2. ; This represents the change in the position of the cleaning head, in units of... ; The change in reference position is expressed in units of... Based on experiments with the adjustment range of the cleaning head position of the glass curtain wall cleaning robot, the empirical value is 1. ; The number of joints in the robotic arm is dimensionless. Energy consumption is adjusted for a single joint, in units of Based on experiments on the energy consumption of the joint motors of the robotic arm in a glass curtain wall cleaning robot, the empirical value is 50. .in, This represents the distance between two adjacent path points in the initial solution of the robot path. Let be the straight-line distance between two adjacent path points in the initial solution of the cleaning head path. The formula is derived from the structure of the robotic arm of a glass curtain wall cleaning robot. This formula is obtained by using the square term of the moving step. and the square of the change in position The normalized ratio reflects the nonlinear growth characteristics of the attitude adjustment amplitude. Multiplying it by the number of joints and the energy consumption per joint yields the total adjustment energy consumption. The square term design reflects the physical law that a large attitude adjustment leads to a sharp increase in energy consumption.

[0074] The specific implementation of step S04 involves using the Nash equilibrium solution algorithm to iteratively calculate the two-layer game optimization model. First, the initial solutions for the robot path and the cleaning head path are initialized as greedy search results. Then, the initial solution for the cleaning head path is fixed, and the upper-level optimization model is solved to obtain the updated robot path solution. Next, the updated robot path solution is fixed, and the lower-level optimization model is solved to obtain the updated cleaning head path solution. Finally, the change in the objective function value of the upper-level optimization model is calculated. and the change in the objective function value of the lower-level optimization model ,like and All are less than the convergence threshold Then, the updated robot path solution and the updated cleaning head path solution are output as Nash equilibrium solutions and assigned to the initial robot path solution and the initial cleaning head path solution, respectively. or Greater than or equal to Then, the updated robot path solution and the updated cleaning head path solution are assigned to the initial robot path solution and the initial cleaning head path solution, respectively, and the process of solving the upper-level optimization model with the fixed initial cleaning head path solution continues iterating. The calculation is expressed as follows:

[0075] ;

[0076] In the formula, The updated robot mobility energy consumption is expressed in units of... ; The unit is the robot's mobile energy consumption before the update. . The calculation is expressed as follows:

[0077] ;

[0078] In the formula, The updated cleaning head operation time is in units of... ; The time for cleaning the head before the update is in units of... . For absolute value operators, The convergence threshold is dimensionless and was obtained experimentally based on the path planning accuracy requirements of the glass curtain wall cleaning robot. The empirical value is 0.01.

[0079] The specific implementation of step S05 involves calculating the total resource consumption based on the predicted energy consumption value and determining whether it exceeds the resource carrying capacity threshold. The calculation of total power consumption is described as follows:

[0080] ;

[0081] In the formula, Total electricity consumption, in units of ; Predicted power consumption for pose adjustment, in units of ; For predicted electricity consumption for cleaning, the unit is... The total cleaning water consumption is described below:

[0082] ;

[0083] In the formula, Total cleaning water consumption, in units of ; This is a predicted value for cleaning water consumption, in units of... The total cleaning agent consumption is described below:

[0084] ;

[0085] In the formula, Total cleaning agent consumption, in units of ; This is the predicted value for cleaning agent consumption, in units of... The calculation method for the battery capacity threshold is described below:

[0086] ;

[0087] In the formula, Battery capacity threshold, in units of ; Rated capacity of the battery, in units of This was obtained by reading the battery specifications of the glass curtain wall cleaning robot. The safety margin factor is dimensionless and obtained from experiments based on the safety usage guidelines for glass curtain wall cleaning robots; its empirical value is 0.8. The calculation method for the cleaning water tank capacity threshold is as follows:

[0088] ;

[0089] In the formula, This refers to the threshold capacity of the cleaning water tank, in units of... ; This refers to the volume of the water tank, in units of... This was obtained by measuring the size of the water tank of the glass curtain wall cleaning robot. This is the density of water, in units of... The value is taken as 1000 under standard temperature and pressure conditions. The calculation method for the capacity threshold of the cleaning agent storage tank is as follows:

[0090] ;

[0091] In the formula, This refers to the threshold capacity of the cleaning agent storage tank, in units of... ; The volume of the storage tank is expressed in units of 1000 liters. This was obtained by measuring the dimensions of the cleaning agent storage tank of the glass curtain wall cleaning robot. This refers to the density of the cleaning agent, in units of... According to the cleaning agent product instructions. (Comparison) and , and , and It determines whether the consumption of any resource exceeds the corresponding capacity threshold.

[0092] The specific implementation of step S06 involves determining the output path or recalculating based on the resource consumption judgment result. If the total resource consumption does not exceed the resource carrying capacity threshold, the initial solution for the robot path and the initial solution for the cleaning head path are output as the final planned path. If the total resource consumption exceeds the resource carrying capacity threshold, the resource constraint penalty term is calculated as follows:

[0093] ;

[0094] In the formula, This is a resource constraint penalty term, dimensionless. The penalty coefficient for exceeding the power limit is dimensionless and is obtained from experiments on the energy management strategy of glass curtain wall cleaning robots. The empirical value is 1.5. The water resource exceeding the penalty coefficient is dimensionless and is obtained from experiments on water resource management strategies for glass curtain wall cleaning robots. The empirical value is 1.2. The penalty coefficient for exceeding the cleaning agent limit is dimensionless and is obtained from experiments on the cleaning agent management strategy of glass curtain wall cleaning robots. The empirical value is 1.3. The function takes the larger value between the value inside the parentheses and zero, ensuring that a penalty is only incurred when resources are exceeded; This is the electricity normalization factor, in units of... Based on experiments with the battery capacity of glass curtain wall cleaning robots, the empirical value is 10,000. ; The water resources normalization coefficient is expressed in units of 1000 ppm. Based on experiments with the water tank capacity of the glass curtain wall cleaning robot, the empirical value is 100. ; This is the normalization factor for the cleaning agent, in units of... Based on experiments with the cleaning agent storage tank capacity of the glass curtain wall cleaning robot, the empirical value is 50. .Will The objective functions of the upper-level optimization model and the lower-level optimization model are applied respectively, and then the process returns to step S04 to re-execute the Nash equilibrium solution algorithm.

[0095] The calculation of the number of iterations in the mean-field variational inference algorithm of the stain identification energy consumption prediction model is described below:

[0096] ;

[0097] In the formula, The number of iterations for the mean-field variational inference algorithm is dimensionless. The surface area of ​​the curtain wall is expressed in units of... ; Standard curtain wall area, unit: Based on experiments conducted in typical operating scenarios of glass curtain wall cleaning robots, the empirical value is 1000. ; The working width of the cleaning head is expressed in units of... ; Standard cleaning width, unit: Based on experiments with the cleaning head size of glass curtain wall cleaning robots, the empirical value is 0.26. ; This represents the maximum extension length of the robotic arm, in units of... ; Standard stretch length, unit: Based on experiments with the specifications of the robotic arm of the glass curtain wall cleaning robot, the empirical value is 0.59. ; The iterative adjustment coefficient is dimensionless and takes values ​​in a closed interval from 1.2 to 1.8. The formula is a natural logarithmic function, with both input and output being dimensionless. By using a logarithmic term, the change in curtain wall area is transformed into a linear increase in the number of iterations, avoiding excessive increases in the number of iterations for large-area curtain walls and improving the width-to-width ratio. and the ratio of stretch length This reflects the impact of equipment specifications on recognition accuracy, and the adjustment coefficient is used to fine-tune the iteration depth under different application scenarios.

[0098] To better understand and implement this invention, a specific application scenario of the invention is provided below as Example 2: To verify the effectiveness of the invention, technicians built a numerical simulation analysis environment and conducted a complete simulation test on the cleaning head path planning method by simulating a high-rise building glass curtain wall cleaning task scenario. The total surface area of ​​the curtain wall in the simulation scenario was set to 1520. The curtain wall is 76m high and 20m wide. The robot chassis can move parallel to the curtain wall surface along the designated safety guide yellow line. The maximum extension length of the robotic arm is 10m, and the working width of the cleaning head is 0.6m. Technicians first configured the multispectral vision sensor parameters, including a visible light camera with a resolution of 1920×1080 pixels, a near-infrared camera with a wavelength range of 750–1400nm, and a polarization camera with a polarization angle resolution of 5°. The simulation environment simulated the lighting conditions of a sunny afternoon, with a solar altitude angle of 45°, an ambient temperature of 28℃, and a relative humidity of 62%.

[0099] In step S01, a multispectral vision sensor acquired image information of the curtain wall surface. Visible light images showed various types of stains on the surface, including dust accumulation, water stains, and oil films. Near-infrared images captured areas of uneven temperature distribution on the curtain wall surface, corresponding to the thermal radiation characteristics of organic stains. Polarized images identified the distribution of oil films on the glass surface, with polarization angles significantly different from the surrounding clean areas. The three images were stitched together according to spectral dimensions and input into a stain recognition energy consumption prediction model. The model's multi-scale feature extraction backbone network output feature maps at four scales, with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image. The probabilistic graphical structured prediction branch was based on the curtain wall surface area of ​​1520 mm. Based on a cleaning head working width of 0.6m and a robotic arm maximum extension length of 10m, the mean-field variational inference algorithm iterated 14 times. In this calculation, the standard curtain wall area was taken as 1000. The standard cleaning width is 0.26m, the standard extension length is 0.59m, and the iterative adjustment coefficient is 1.5. After 14 iterations, the coordinate set of the stain distribution area contains 278 stain areas with a total area of ​​386. As shown in Table 1, the prediction results from the energy consumption regression prediction branch output show that the predicted power consumption for pose adjustment is 58.6 kWh, the predicted power consumption for cleaning is 42.3 kWh, the predicted water consumption for cleaning is 1850 L, and the predicted consumption for cleaning agent is 92.5 L.

[0100] Table 1. Statistics on the distribution of major stain areas in the simulation scene.

[0101]

[0102] In step S02, based on the maximum extension length of the robotic arm and the movement range of the robot chassis, the curtain wall surface is divided into three cleaning operation sub-areas along longitudinal strips. The widths of each sub-area are 6.5m, 7.0m, and 6.5m, respectively, none exceeding twice the maximum extension length of the robotic arm. A cleaning coverage grid map is established for each cleaning operation sub-area, with the grid size set to 0.6m × 0.6m, corresponding to the working width of the cleaning head. The grid map of the first sub-area contains 8228 grids, of which 7456 are in an uncleaned state and 772 are in an obstacle state. The obstacle state corresponds to the window frames, decorative strips, and non-glass areas at the edges of the curtain wall. The grid maps of the second and third sub-areas contain 8953 and 8228 grids, respectively.

[0103] In step S03, a two-layer game optimization model was established for the robot path and the cleaning head path. The upper-layer optimization model aims to minimize the robot's movement energy consumption. The input parameters of its objective function include the robot chassis movement distance, the ground friction coefficient of 0.28, and the chassis mass of 520 kg. The reference movement distance is taken as 100 m, the reference friction coefficient as 0.3, the reference mass as 300 kg, and the gravitational acceleration as 9.8. The upper-level optimization model's constraints require the robot's movement path to be parallel to the curtain wall, the chassis to not deviate from the designated safety guide yellow line, and to stop at replenishment points to resupply resources. The lower-level optimization model aims to minimize the cleaning head's operation time. Its objective function's input parameters include the total length of the cleaning head's trajectory, the average speed of the cleaning head, and the total area of ​​the soiled region (386). The reference trajectory length is set at 50m, the reference moving speed at 0.5m / s, and the reference area at 100. The cleaning time per unit area is set to 10 seconds. The constraints of the lower-level optimization model require that the cleaning head's movement range must not exceed the robotic arm's workspace, the cleaning head path must cover all soiled area grids, and the same grid must not be cleaned more than once. The coupling term in the two-level game-theoretic optimization model is the robotic arm's posture adjustment energy consumption. This coupling term simultaneously affects the robot's movement energy consumption in the upper-level optimization model and the cleaning head's operation time in the lower-level optimization model. The reference step size is 2m, the reference position change is 1m, the robotic arm has 6 joints, and the energy consumption for single joint adjustment is 50J.

[0104] In step S04, the Nash equilibrium algorithm is used to iteratively calculate the two-layer game optimization model. The initial solutions for the robot path and the cleaning head path are initialized as greedy search results. The initial robot path solution contains 8 path points, and the initial cleaning head path solution contains 453 path points. In the first iteration, the upper-layer optimization model is solved with the fixed cleaning head path initial solution to obtain the updated robot path solution, which contains 7 path points, reducing the robot chassis movement distance to 17.8m. The lower-layer optimization model is then solved with the fixed robot path update solution to obtain the updated cleaning head path solution, which contains 428 path points, and the total length of the cleaning head's trajectory is 264.5m. The change in the objective function value of the upper-layer optimization model is calculated to be -12.3%, and the change in the objective function value of the lower-layer optimization model is -8.7%. Both absolute values ​​are greater than the convergence threshold of 0.01, therefore, the iteration continues. After six iterations, the absolute value of the change in the objective function value of the upper-level optimization model is 0.008, and the absolute value of the change in the objective function value of the lower-level optimization model is 0.006, both less than the convergence threshold. The Nash equilibrium solution algorithm outputs the initial solutions for the robot path and the cleaning head path. Figure 2 As shown, the stain distribution recognition results clearly demonstrate the spatial distribution characteristics of various stains on the curtain wall surface, and the probabilistic graph structured prediction branch effectively eliminates isolated misclassification points caused by independent pixel classification. Figure 3 As shown, the robot path and cleaning head path planning results show that the robot chassis's movement trajectory on the set safety guide yellow line and the cleaning head's working trajectory on the curtain wall surface are coordinated with each other, achieving dual optimization of energy consumption and time.

[0105] In step S05, the total resource consumption is calculated based on the predicted power consumption for pose adjustment (58.6 kWh), cleaning (42.3 kWh), cleaning water (1850 L), and cleaning agent (92.5 L). The total power consumption is 100.9 kWh, the total cleaning water consumption is 1850 L, and the total cleaning agent consumption is 92.5 L. The robot's battery has a rated capacity of 120 kWh, a safety margin factor of 0.8, and a battery capacity threshold of 96 kWh. The cleaning water tank volume is 2.5 kWh. The density of water is 1000 kg / m³. The cleaning water tank capacity threshold is 2500L. The cleaning agent storage tank volume is 0.15. The density of the cleaning agent is 1050 kg / m³. The cleaning agent storage tank capacity threshold is 150L. It was determined that the total power consumption of 100.9kWh exceeded the battery capacity threshold of 96kWh, an excess of 4.9kWh.

[0106] In step S06, since the total resource consumption exceeds the resource carrying capacity threshold, a resource constraint penalty term is added to the two-layer game optimization model, and the process returns to step S04 for recalculation. The power excess penalty coefficient is set to 1.5, and the resource constraint penalty term is 7.35. After applying the resource constraint penalty term to the objective functions of the upper and lower optimization models respectively, the Nash equilibrium solution algorithm is re-executed. After 5 iterations, new initial solutions for the robot path and the cleaning head path are obtained. The initial robot path solution contains 9 path points, of which the 5th path point is a replenishment point. The initial cleaning head path solution contains 476 path points. The total resource consumption is recalculated, and the total power consumption is reduced to 95.2 kWh, the total cleaning water consumption is 1820 L, and the total cleaning agent consumption is 91.0 L. All resource consumptions do not exceed the resource carrying capacity threshold. The initial solutions for the robot path and the cleaning head path are output as the final planned paths. The final robot path is used to control the movement trajectory of the robot chassis, and the final cleaning head path is used to control the working trajectory of the cleaning head. Figure 4 As shown, the trend of resource consumption during the iterative optimization process indicates that the introduction of the resource constraint penalty term effectively guides the path planning to adjust in a direction that satisfies resource constraints.

[0107] The main advancements of this invention compared to traditional cleaning path planning methods are reflected in the following aspects. Traditional methods typically employ independent pixel-level classification for stain identification, neglecting the spatial correlation between stain types in adjacent areas. This leads to isolated misclassifications and blurred boundaries in the identification results. This invention explicitly models the output spatial constraints of stain identification as a conditional random field through a probabilistic graphical structured prediction branch. It utilizes a mean-field variational inference algorithm to propagate probabilistic information between graph nodes, achieving joint inference between stain category, spatial location, and lighting conditions. This results in stain distribution predictions with better spatial continuity and clearer boundaries. Traditional methods often optimize the robot path and cleaning head path independently during path planning, failing to fully consider the coupling relationship between them. This makes it difficult to achieve global optimization in terms of overall energy consumption and operation time. This invention establishes a two-layer game optimization model for the robot path and cleaning head path. Through a Nash equilibrium solution algorithm, it achieves collaborative optimization of minimizing the upper-layer robot movement energy consumption and minimizing the lower-layer cleaning head operation time. The energy consumption of the robotic arm posture adjustment in the two-layer game optimization model, as a coupling term, simultaneously affects the objective functions of both layers, enabling the planning results to achieve a balance between energy consumption and time. Traditional methods often overlook the limited resources carried by the robot during path planning, which can easily lead to resource exhaustion and forced interruption of the cleaning task midway. This invention uses energy consumption regression prediction branch to predict pose adjustment of power consumption, cleaning power consumption and resource consumption during the stain recognition stage. During the path planning stage, the resource carrying capacity limit is incorporated into the optimization model through resource constraint penalty term to ensure that the final planned path is executable within the resource capacity range, thus avoiding interruption and repeated planning of the cleaning task.

[0108] It should be noted that the variables involved in this invention are explained in detail in Tables 2 and 3.

[0109] Table 2. Variable Explanation Table (Part 1)

[0110]

[0111] Table 3. Variable Explanation Table (Part Two)

[0112]

[0113] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for path planning of a cleaning head of a glass curtain wall cleaning robot, characterized in that, The glass curtain wall surface image information is collected by the multispectral vision sensor, and input into the stain identification energy consumption prediction model to obtain a stain distribution region coordinate set, a pose adjustment power consumption prediction value, a cleaning power consumption prediction value, a cleaning water consumption prediction value, and a cleaning agent consumption prediction value; the cleaning operation sub-regions are divided according to the stain distribution region coordinate set, and a cleaning coverage grid map is established for each cleaning operation sub-region; a double-layer game optimization model of the robot path and the cleaning head path is established, the double-layer game optimization model includes an upper-layer optimization model with the minimum robot movement energy consumption as the target and a lower-layer optimization model with the shortest cleaning head operation time as the target, and the pose adjustment energy consumption of the mechanical arm is taken as a coupling term to simultaneously affect the upper-layer optimization model and the lower-layer optimization model to realize path collaborative optimization; a Nash equilibrium solving algorithm is used to iteratively calculate the double-layer game optimization model to obtain an initial solution of the robot path and an initial solution of the cleaning head path; The total resource consumption is calculated according to the pose adjustment power consumption prediction value, the cleaning power consumption prediction value, the cleaning water consumption prediction value, and the cleaning agent consumption prediction value, and it is judged whether the total resource consumption exceeds the resource carrying capacity threshold; if the total resource consumption does not exceed the resource carrying capacity threshold, the initial solution of the robot path and the initial solution of the cleaning head path are output as the final planning path; if the total resource consumption exceeds the resource carrying capacity threshold, a resource constraint penalty term is added to the double-layer game optimization model and recalculation is performed.

2. The method of claim 1, wherein, The structure of the stain identification energy consumption prediction model is a three-branch architecture of a multiscale feature extraction backbone network, a probability graph structured prediction branch, and an energy consumption regression prediction branch.

3. The method of claim 2, wherein, The probability graph structured prediction branch models the stain identification task as a conditional random field, defines a unary potential function, a pairwise potential function, and a high-order potential function, and uses an average field variational inference algorithm to iteratively propagate probability information between graph nodes.

4. The method of claim 3, wherein, The iteration number of the average field variational inference algorithm is equal to the natural logarithm of the curtain wall surface area divided by the standard curtain wall area, multiplied by the product of the cleaning head working width divided by the standard cleaning width and the maximum extension length of the mechanical arm divided by the standard extension length, and finally multiplied by an iteration adjustment coefficient.

5. The method of claim 4, wherein, The standard curtain wall area is set as a default value of 1000 , the standard cleaning width is set as a default value of 0.26 m, and the standard extension length is set as a default value of 0.59 m.

6. The method of claim 5, wherein, The stain identification energy consumption prediction model training data set includes multiple groups of training sample pairs, each group of training sample pairs is composed of a multispectral image, a stain category label, a stain boundary coordinate label, a stain severity level label, a pose adjustment power consumption real value, a cleaning power consumption real value, a cleaning water consumption real value, and a cleaning agent consumption real value.

7. The method of claim 6, wherein, The stain identification energy consumption prediction model training uses a joint loss function, which is a weighted sum of the stain identification cross-entropy loss, the structured prediction potential energy loss, and the energy consumption regression mean square error loss.

8. The method of claim 7, wherein, The multispectral vision sensor includes a visible light camera, a near-infrared camera, and a polarization camera.

9. The method of claim 8, wherein, The division method of the cleaning operation sub-region is to divide the curtain wall surface into multiple cleaning operation sub-regions according to the maximum extension length of the mechanical arm and the movement range of the robot chassis, and the width of each cleaning operation sub-region is not more than twice the maximum extension length of the mechanical arm.

10. The method of claim 9, wherein, The grid size of the cleaning coverage grid map corresponds to the working width of the cleaning head. Each grid corresponds to a unit to be cleaned on the surface of the curtain wall. The state of the grid includes an uncleaned state, a cleaned state, and an obstacle state.

Images