Control method and system of window cleaning robot based on computer vision
By combining multimodal sensors with deep learning algorithms, adaptive cleaning of window cleaning robots has been achieved, solving the problems of low efficiency and resource waste in existing technologies, improving cleaning efficiency and reducing energy consumption.
Patent Information
- Application Number
- CN202510943125.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-09
- Publication Date
- 2025-11-18
AI Technical Summary
Existing window cleaning robots are inefficient, consume excessive resources, struggle to cope with complex stain patterns and environmental changes, and lack autonomous decision-making capabilities.
This method employs a multimodal sensor combined with deep learning algorithms. It uses computer vision to detect the shape, area, and reflectivity of a window image. By extracting the shape, area, and reflectivity of the area to be wiped, and using a computer vision-based method for detecting window images, it extracts the cleanliness index of the area to be wiped by extracting the two-dimensional grayscale image and three-dimensional depth image of the area to be wiped. Based on the principle of multi-objective optimization, it generates a set of candidate parameters and uses a deep reinforcement learning algorithm for adaptive adjustment.
It achieves efficient cleaning in dynamic environments, reduces energy consumption, enhances the autonomy and robustness of technology application, solves technical problems in existing technologies, improves cleaning efficiency and reduces energy consumption.
Smart Images
Figure CN120959611A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot control technology, specifically to a control method and system for a window cleaning robot based on computer vision. Background Technology
[0002] With rapid urban development, more and more high-rise buildings and commercial complexes are emerging. The windows of these buildings are often difficult to clean manually, leading to a growing market demand for automated window cleaning equipment. Traditional manual window cleaning requires a large workforce and corresponding safety measures. As labor costs rise, businesses and households are increasingly demanding efficient and economical cleaning solutions, driving the development of robotics technology. Advances in computer vision, artificial intelligence, and deep learning have made robotics possible.
[0003] However, most existing window cleaning robots use empirical parameters or simple rules to preset motion trajectories, ignoring power consumption, detergent consumption, and operating costs, resulting in low efficiency or excessive consumption. They are often based on threshold segmentation or manually set features, making it difficult to cope with complex stain patterns and reflection changes. When encountering insufficient resources or malfunctions, they cannot adjust their strategies in a timely manner and cannot update their models based on actual cleaning results and environmental changes. Summary of the Invention
[0004] (a) Technical problems to be solved
[0005] In view of the above-mentioned shortcomings of the existing technology, the present invention provides a control method and system for a window cleaning robot based on computer vision, which can effectively solve the problems of the existing technology.
[0006] (II) Technical Solution
[0007] To achieve the above objectives, the present invention provides the following technical solution:
[0008] This invention discloses a control method for a window cleaning robot based on computer vision, comprising the following steps:
[0009] Step 1: Simultaneously collect window images, ambient temperature, signal quality, and robot internal status data using multimodal sensors installed on the robot. Perform noise filtering, distortion correction, and region segmentation on the window images to extract the two-dimensional grayscale image and three-dimensional depth map of the area to be wiped.
[0010] Step 2: After preprocessing the image obtained in Step 1, input it into the pre-trained convolutional neural network to detect the shape, area and reflective characteristics of residual dirt on the window surface. Define the region cleanliness index based on the preset feature rules and divide it into several levels according to the cleanliness gradient.
[0011] Step 3: For different cleanliness levels, preset several action indicators, and generate several sets of candidate parameters according to the multi-objective optimization principle. Each set includes one or more action indicators.
[0012] Step 4: Real-time acquisition and updating of robot battery level, motor temperature, remaining cleaning dosage, and wireless signal strength; preset weighting coefficients; weighted sensitivity coefficients are then established as the current control status.
[0013] Step 5: Label and store the candidate parameter set, corresponding historical cleaning results, and operating costs. Train the prediction model using a deep reinforcement learning algorithm so that it can output the expected cleanliness and energy consumption cost when the current control state and a certain parameter set are input.
[0014] Step 6: Based on the expected cleanliness and energy consumption cost, select the optimal parameter set from several candidate parameter sets and send it to the robot drive in real time for adaptive adjustment of path and action;
[0015] Step 7: In the next running cycle, adjust the behavior and conduct an availability assessment based on actual cleaning feedback.
[0016] Furthermore, the multimodal sensors in step 1 include: a visible light camera for acquiring high-resolution two-dimensional images; a structured light or time-of-flight depth camera for acquiring three-dimensional depth maps; an ambient temperature sensor for measuring the real-time temperature around the window surface; and a wireless signal strength sensor, a built-in battery power sensor, and a motor temperature sensor for real-time monitoring of the robot's internal status.
[0017] Furthermore, the multi-objective optimization principles in step 3 include:
[0018] Maximizing the regional cleanliness index and minimizing energy consumption costs are considered as two conflicting optimization objectives.
[0019] A multi-objective evolutionary algorithm based on non-dominated sorting and crowding distance evaluation is used to construct a population. The candidate action indicators are combined as the initial population individuals. New individuals are generated through crossover and mutation operators. Non-dominated sorting and crowding comparison are performed on the parent and child individuals.
[0020] The multi-objective evolution is performed until the preset convergence condition is met or the maximum number of iterations is reached.
[0021] Extract several sets of candidate parameters located on the Pareto front from the final population as the output of step 3.
[0022] Furthermore, the action indicators in step 3 include: wiping speed, brush head pressure, cleaning agent spray volume, and path planning strategy.
[0023] Furthermore, the deep reinforcement learning algorithm used in step 5 is a deep deterministic network algorithm, whose reward function is composed of a weighted sum of the cleanliness improvement and energy consumption cost, and an experience replay pool is introduced during the training process.
[0024] Furthermore, the calculation formula for the reward function optimization prediction model process is as follows:
[0025] R(s t ,a t )=α·ΔC(s t ,a t )-β·(γ e ·ΔE+γ l ·ΔL)+λ·Ψ(s t+1 );
[0026] In the formula, R(s) t ,a t ) represents the state s t Next, execute action a t The reward value obtained, ΔC(s) t ,a t ) represents the action a. t The improvement in cleanliness after cleaning is calculated as C. new -C old C new and C old These represent the cleanliness indices before and after the action, respectively. e The normalization coefficient represents the power consumption, ΔE represents the power consumption during the action execution, and γ represents the power consumption. l The normalized coefficient represents the amount of cleaning agent consumed, ΔL represents the amount of cleaning agent consumed during the execution of the action, λ represents the discount factor, and Ψ(s) represents the amount of cleaning agent consumed. t+1 ) represents the state value function, α represents the reward weight coefficient for improving cleanliness, and β represents the global weight coefficient for energy consumption penalty.
[0027] Furthermore, the process of adjusting the usability assessment of behavior in step 7 is as follows:
[0028] Step 2 is used to reassess the local and global cleanliness, and the error is obtained by comparing it with the predicted value.
[0029] If the error exceeds the preset tolerance range, the currently collected data will be used as new training samples to incrementally update the prediction model.
[0030] When the number of error events reaches a preset limit within a preset period, an overall verification alarm for the prediction model is triggered.
[0031] A control system for a computer vision-based window cleaning robot includes:
[0032] The sensing module is used to simultaneously acquire window images, ambient temperature, signal quality, and internal robot status data;
[0033] The preprocessing module is used to perform noise filtering, distortion correction, and region segmentation on the window image, and to extract the two-dimensional grayscale image and three-dimensional depth image of the area to be wiped.
[0034] The cleanliness assessment module is used to input the grayscale image and depth image into a pre-trained convolutional neural network to detect the shape, area and reflective properties of residual stains on the window surface, define the area cleanliness index based on preset feature rules, and classify the level according to the cleanliness gradient.
[0035] The parameter generation module is used to generate several sets of candidate parameters based on the multi-objective optimization principle for different cleanliness levels. Each set of candidate parameters includes one or more motion and path control indicators.
[0036] The status monitoring module is used to acquire and update the robot's battery level, motor temperature, remaining cleaning dosage, and wireless signal strength in real time, and establish a sensitivity coefficient for the current control status accordingly.
[0037] The model prediction module is used to label and store the candidate parameter set and the corresponding historical cleaning results and operating costs, and to train the prediction model using a deep reinforcement learning algorithm so that it can output the expected cleanliness and energy consumption cost when the current operating state and parameter set are input.
[0038] An adaptive control module is used to select the optimal parameters from the candidate parameter set based on the expected cleanliness and energy consumption cost output by the prediction model, and send them to the robot drive system in real time to implement adaptive adjustments to the path and actions.
[0039] The feedback update module is used to re-evaluate the local and global cleanliness after execution through the stain detection and cleanliness assessment module, compare it with the predicted value, and update the prediction model by using the currently collected data as new training samples when the error exceeds the preset tolerance range. When the number of error events reaches the preset limit, an overall verification alarm for the prediction model is triggered.
[0040] Furthermore, a time-series loop is established between the adaptive control module and the feedback update module. After each window cleaning action is completed, the feedback update module synchronously sends the newly generated window image and cleanliness label back to the preprocessing module and the model prediction module for online incremental training and model fine-tuning.
[0041] Furthermore, the preprocessing module is interconnected with the sensing module and the cleanliness assessment module via a wireless network; the cleanliness assessment module is interconnected with the parameter generation module, the feedback update module, and the model prediction module via a wireless network; the status monitoring module is interconnected with the parameter generation module and the model prediction module via a wireless network; and the model prediction module is interconnected with the adaptive control module via a wireless network.
[0042] (III) Beneficial Effects
[0043] Compared with the known prior art, the technical solution provided by this invention has the following beneficial effects:
[0044] 1. By accurately extracting and evaluating the two-dimensional grayscale image and three-dimensional depth image of residual stains on the window surface, the robot can adjust the motion parameters in real time under the combined influence of dynamic ambient temperature, signal quality and robot internal state, thereby significantly improving cleaning efficiency and reducing energy consumption.
[0045] 2. This method is based on deep reinforcement learning algorithm to annotate and train the candidate parameter set, historical cleaning results and operating costs, so that the prediction model can give the optimal balance strategy between cleanliness and energy consumption cost when the current operation state is input, which effectively reduces the need for manual intervention and improves the system's autonomous decision-making ability and operational stability.
[0046] 3. Through feedback updates and model adaptation functions, the prediction model can be updated based on newly collected data increments after the actual window cleaning process is completed. At the same time, when the number of error event triggers reaches the preset limit, an overall verification alarm is actively issued, thereby ensuring the model accuracy and algorithm reliability in long-term operation and achieving a high degree of maintainability and scalability of the window cleaning robot. Attached Figure Description
[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0048] Figure 1 This is a flowchart illustrating the control method of the window cleaning robot in this invention;
[0049] Figure 2 This is a schematic diagram of the control system of the window cleaning robot in this invention.
[0050] The labels in the diagram represent: 1. Sensing module; 2. Preprocessing module; 3. Cleanliness assessment module; 4. Parameter generation module; 5. Status monitoring module; 6. Model prediction module; 7. Adaptive control module; 8. Feedback update module. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0052] The present invention will be further described below with reference to embodiments.
[0053] Example 1
[0054] This embodiment presents a control method for a window cleaning robot based on computer vision, such as... Figure 1 As shown, it includes the following steps:
[0055] Step 1: Simultaneously acquire window images, ambient temperature, signal quality, and robot internal status data using multimodal sensors installed on the robot. Perform noise filtering, distortion correction, and region segmentation on the window images to extract the 2D grayscale image and 3D depth map of the area to be wiped. The multimodal sensors include: a visible light camera for acquiring high-resolution 2D images; a structured light or time-of-flight depth camera for acquiring 3D depth maps; an ambient temperature sensor for measuring the real-time temperature around the window; and a wireless signal strength sensor, a built-in battery power sensor, and a motor temperature sensor for real-time monitoring of the robot's internal status.
[0056] Step 2: After preprocessing the image obtained in Step 1, input it into the pre-trained convolutional neural network to detect the shape, area and reflective characteristics of residual dirt on the window surface. Define the region cleanliness index based on the preset feature rules and divide it into several levels according to the cleanliness gradient.
[0057] Step 3: For different cleanliness levels, preset several action indicators, and generate several sets of candidate parameters according to the multi-objective optimization principle. Each set includes one or more action indicators; the action indicators include: wiping speed, brush head pressure, detergent spray volume, and path planning strategy; the multi-objective optimization principle includes:
[0058] Maximizing the regional cleanliness index and minimizing energy consumption costs are considered as two conflicting optimization objectives.
[0059] A multi-objective evolutionary algorithm based on non-dominated sorting and crowding distance evaluation is used to construct a population. The candidate action indicators are combined as the initial population individuals. New individuals are generated through crossover and mutation operators. Non-dominated sorting and crowding comparison are performed on the parent and child individuals.
[0060] The multi-objective evolution is performed until the preset convergence condition is met or the maximum number of iterations is reached.
[0061] Extract several sets of candidate parameters located on the Pareto front from the final population as the output of step 3;
[0062] Step 4: Real-time acquisition and updating of robot battery level, motor temperature, remaining cleaning dosage, and wireless signal strength; preset weighting coefficients; weighted sensitivity coefficients are then established as the current control status.
[0063] Step 5: Label and store the candidate parameter set, corresponding historical cleaning results, and operating costs. Use a deep reinforcement learning algorithm to train a prediction model so that it can output the expected cleanliness and energy consumption cost when given the current operating state and a certain parameter set.
[0064] Step 6: Based on the expected cleanliness and energy consumption cost, select the optimal parameter set from several candidate parameter sets and send it to the robot drive in real time for adaptive adjustment of path and action;
[0065] Step 7: In the next operating cycle, adjust the behavioral usability assessment based on actual cleaning feedback. The process for adjusting the behavioral usability assessment is as follows:
[0066] Step 2 is used to reassess the local and global cleanliness, and the error is obtained by comparing it with the predicted value.
[0067] If the error exceeds the preset tolerance range, the currently collected data will be used as new training samples to incrementally update the prediction model.
[0068] When the number of error events reaches a preset limit within a preset period, an overall verification alarm for the prediction model is triggered.
[0069] Compared with existing technologies, this method achieves accurate stain detection of two-dimensional grayscale and three-dimensional depth information on the window surface by fusing environmental and robot internal state data through multimodal sensors; it uses a pre-trained convolutional neural network to evaluate the cleanliness of the area and balances cleaning efficiency and energy consumption costs based on a multi-objective evolutionary algorithm; and it combines a deep reinforcement learning model for adaptive parameter selection and real-time feedback incremental updates, which not only improves the quality of window cleaning and operational reliability, but also significantly optimizes energy consumption performance, overcoming the shortcomings of existing technologies in terms of dynamic environment adaptability and self-learning ability.
[0070] Example 2
[0071] At other levels, this embodiment also provides another optimization mechanism based on embodiment 1, specifically a control system for a computer vision-based window cleaning robot, such as... Figure 2 As shown, it includes:
[0072] Sensing module 1 is used to synchronously collect window images, ambient temperature, signal quality, and robot internal status data;
[0073] Preprocessing module 2 is used to perform noise filtering, distortion correction and region segmentation on the window image, and extract the two-dimensional grayscale image and three-dimensional depth image of the area to be wiped.
[0074] The cleanliness assessment module 3 is used to input grayscale images and depth images into a pre-trained convolutional neural network to detect the shape, area and reflective properties of residual stains on the window surface, define the area cleanliness index based on preset feature rules, and classify the level according to the cleanliness gradient.
[0075] The parameter generation module 4 is used to generate several sets of candidate parameters based on the multi-objective optimization principle for different cleanliness levels. Each set of candidate parameters includes one or more motion and path control indicators.
[0076] The status monitoring module 5 is used to acquire and update the robot's battery level, motor temperature, remaining cleaning dosage and wireless signal strength in real time, and establish the sensitivity coefficient of the current control status accordingly.
[0077] Model prediction module 6 is used to label and store the candidate parameter set and the corresponding historical cleaning results and operating costs, and to train the prediction model using a deep reinforcement learning algorithm so that it can output the expected cleanliness and energy consumption cost when the current control state and parameter set are input.
[0078] The adaptive control module 7 is used to select the optimal parameters from the candidate parameter set based on the expected cleanliness and energy consumption cost output by the prediction model, and send them to the robot drive system in real time to implement adaptive adjustment of path and action.
[0079] The feedback update module 8 is used to re-evaluate the local and global cleanliness after execution through the stain detection and cleanliness assessment module, and compare it with the predicted value. When the error exceeds the preset tolerance range, the currently collected data is used as a new training sample to incrementally update the prediction model. When the number of error event triggers reaches the preset upper limit, an overall verification alarm for the prediction model is triggered. A time-series loop is established between the adaptive control module 7 and the feedback update module 8. After each window cleaning action is completed, the feedback update module 8 synchronously sends the newly generated window image and cleanliness label back to the preprocessing module 2 and the model prediction module 6 for online incremental training and model fine-tuning.
[0080] The preprocessing module 2 is interconnected with the sensing module 1 and the cleanliness assessment module 3 via a wireless network. The cleanliness assessment module 3 is interconnected with the parameter generation module 4, the feedback update module 8, and the model prediction module 6 via a wireless network. The status monitoring module 5 is interconnected with the parameter generation module 4 and the model prediction module 6 via a wireless network. The model prediction module 6 is interconnected with the adaptive control module 7 via a wireless network.
[0081] Example 3
[0082] In this embodiment, the deep reinforcement learning algorithm is a deep deterministic network algorithm. Its reward function is composed of a weighted sum of the cleanliness improvement and energy consumption cost. An experience replay pool is introduced during the training process. The calculation formula for the reward function to optimize the prediction model is as follows:
[0083] R(s t ,a t )=α·ΔC(s t ,a t )-β·(γ e ·ΔE+γ l ·ΔL)+λ·Ψ(s t+1 );
[0084] In the formula, R(s) t ,a t ) represents the state s t Next, execute action a t The reward value obtained, ΔC(s) t ,a t ) represents the action a. t The improvement in cleanliness after cleaning is calculated as C. new -C old C new and C old These represent the cleanliness indices before and after the action, respectively. e The normalization coefficient represents the power consumption, ΔE represents the power consumption during the action execution, and γ represents the power consumption. l The normalized coefficient represents the amount of cleaning agent consumed, ΔL represents the amount of cleaning agent consumed during the execution of the action, λ represents the discount factor, and Ψ(s) represents the amount of cleaning agent consumed. t+1 ) represents the state value function, α represents the reward weight coefficient for improving cleanliness, and β represents the global weight coefficient for energy consumption penalty.
[0085] Compared with existing technologies, this technology integrates immediate rewards and long-term benefits, incorporating three dimensions—cleaning effect, resource consumption, and equipment health—into a unified decision-making framework to form a control logic with self-evolution capabilities. Its dynamic weighting mechanism and state transition modeling significantly improve the robustness of traditional window cleaning algorithms in complex scenarios.
[0086] In summary, by combining multimodal sensor fusion, deep learning, and multi-objective evolutionary algorithms, high-precision detection and localization of stains on window surfaces were achieved. Path and motion parameters were optimized based on a dynamic trade-off between cleanliness and energy consumption, and the model was updated online adaptively using deep reinforcement learning. This not only significantly improved cleaning efficiency and effectiveness but also effectively reduced energy consumption and enhanced the system's intelligence, autonomy, and robustness.
[0087] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A control method for a window cleaning robot based on computer vision, characterized in that, Includes the following steps: Step 1: Simultaneously collect window images, ambient temperature, signal quality, and robot internal status data using multimodal sensors installed on the robot. Perform noise filtering, distortion correction, and region segmentation on the window images to extract the two-dimensional grayscale image and three-dimensional depth map of the area to be wiped. Step 2: After preprocessing the image obtained in Step 1, input it into the pre-trained convolutional neural network to detect the shape, area and reflective characteristics of residual dirt on the window surface. Define the region cleanliness index based on the preset feature rules and divide it into several levels according to the cleanliness gradient. Step 3: For different cleanliness levels, preset several action indicators, and generate several sets of candidate parameters according to the multi-objective optimization principle. Each set includes one or more action indicators. Step 4: Real-time acquisition and updating of robot battery level, motor temperature, remaining cleaning dosage, and wireless signal strength; preset weighting coefficients; weighted sensitivity coefficients are then established as the current control status. Step 5: Label and store the candidate parameter set, corresponding historical cleaning results, and operating costs. Train the prediction model using a deep reinforcement learning algorithm so that it can output the expected cleanliness and energy consumption cost when the current control state and a certain parameter set are input. Step 6: Based on the expected cleanliness and energy consumption cost, select the optimal parameter set from several candidate parameter sets and send it to the robot drive in real time for adaptive adjustment of path and action; Step 7: In the next running cycle, adjust the behavior and conduct an availability assessment based on actual cleaning feedback.
2. The control method for a window cleaning robot based on computer vision according to claim 1, characterized in that, The multimodal sensors in step 1 include: a visible light camera for acquiring high-resolution two-dimensional images; a structured light or time-of-flight depth camera for acquiring three-dimensional depth maps; an ambient temperature sensor for measuring the real-time temperature around the window; and a wireless signal strength sensor, a built-in battery power sensor, and a motor temperature sensor for real-time monitoring of the robot's internal status.
3. The control method for a window cleaning robot based on computer vision according to claim 1, characterized in that, The multi-objective optimization principles in step 3 include: Maximizing the regional cleanliness index and minimizing energy consumption costs are considered as two conflicting optimization objectives. A multi-objective evolutionary algorithm based on non-dominated sorting and crowding distance evaluation is used to construct a population. The candidate action indicators are combined as the initial population individuals. New individuals are generated through crossover and mutation operators. Non-dominated sorting and crowding comparison are performed on the parent and child individuals. The multi-objective evolution is performed until the preset convergence condition is met or the maximum number of iterations is reached. Extract several sets of candidate parameters located on the Pareto front from the final population as the output of step 3.
4. The control method for a window cleaning robot based on computer vision according to claim 1, characterized in that, The action indicators in step 3 include: wiping speed, brush head pressure, cleaning agent spray volume, and path planning strategy.
5. The control method for a window cleaning robot based on computer vision according to claim 1, characterized in that, The deep reinforcement learning algorithm used in step 5 is a deep deterministic network algorithm. Its reward function consists of a weighted sum of the cleanliness improvement and energy consumption cost, and an experience replay pool is introduced during the training process.
6. The control method for a window cleaning robot based on computer vision according to claim 5, characterized in that, The calculation formula for the reward function optimization prediction model process is as follows: R(s t ,a t )=α·ΔC(s t ,a t )-b·(c e ·ΔE+γ l ·ΔL)+λ·Ψ(s t+1 ); In the formula, R(s) t ,a t ) represents the state s t Next, execute action a t The reward value obtained, ΔC(s) t ,a t ) represents the action a. t The improvement in cleanliness after cleaning is calculated as C. new -C old C new and C old These represent the cleanliness indices before and after the action, respectively. e The normalization coefficient represents the power consumption, ΔE represents the power consumption during the action execution, and γ represents the power consumption. l The normalized coefficient represents the amount of cleaning agent consumed, ΔL represents the amount of cleaning agent consumed during the execution of the action, λ represents the discount factor, and Ψ(s) represents the amount of cleaning agent consumed. t+1 ) represents the state value function, α represents the reward weight coefficient for improving cleanliness, and β represents the global weight coefficient for energy consumption penalty.
7. The control method for a window cleaning robot based on computer vision according to claim 1, characterized in that, The process of adjusting the usability assessment of behavior in step 7 is as follows: Step 2 is used to reassess the local and global cleanliness, and the error is obtained by comparing it with the predicted value. If the error exceeds the preset tolerance range, the currently collected data will be used as new training samples to incrementally update the prediction model. When the number of error events reaches a preset limit within a preset period, an overall verification alarm for the prediction model is triggered.
8. A control system for a computer vision-based window cleaning robot, said system being an implementation system of a control method for a computer vision-based window cleaning robot according to any one of claims 1-7, characterized in that, include: The sensing module (1) is used to synchronously collect window images, ambient temperature, signal quality and robot internal state data; The preprocessing module (2) is used to perform noise filtering, distortion correction and region segmentation on the window image, and extract the two-dimensional grayscale image and three-dimensional depth image of the area to be wiped. The cleanliness assessment module (3) is used to input the grayscale image and depth image into a pre-trained convolutional neural network to detect the shape, area and reflective properties of residual stains on the window surface, define the area cleanliness index based on preset feature rules, and classify the level according to the cleanliness gradient. The parameter generation module (4) is used to generate several sets of candidate parameters based on the multi-objective optimization principle for different cleanliness levels. Each set of candidate parameters includes one or more motion and path control indicators. The status monitoring module (5) is used to acquire and update the robot's battery power, motor temperature, remaining cleaning dosage and wireless signal strength in real time, and establish the sensitivity coefficient of the current control status accordingly. The model prediction module (6) is used to label and store the candidate parameter set and the corresponding historical cleaning results and operating costs, and to train the prediction model using a deep reinforcement learning algorithm so that it outputs the expected cleanliness and energy consumption cost when the current control state and parameter set are input. The adaptive control module (7) is used to select the optimal parameters from the candidate parameter set based on the expected cleanliness and energy consumption cost output by the prediction model, and send them to the robot drive system in real time to implement adaptive adjustment of path and action. The feedback update module (8) is used to re-evaluate the local and global cleanliness through the stain detection and cleanliness assessment module after execution, compare it with the predicted value, and update the prediction model by using the currently collected data as a new training sample increment when the error exceeds the preset tolerance range, and trigger the overall verification alarm of the prediction model when the number of error event triggers reaches the preset upper limit.
9. The control system for a computer vision-based window cleaning robot according to claim 8, characterized in that, A time-series loop is established between the adaptive control module (7) and the feedback update module (8). After each window cleaning action is completed, the feedback update module (8) synchronously sends the newly generated window image and cleanliness label back to the preprocessing module (2) and the model prediction module (6) for online incremental training and model fine-tuning.
10. The control system for a computer vision-based window cleaning robot according to claim 8, characterized in that, The preprocessing module (2) is interconnected with the sensing module (1) and the cleanliness assessment module (3) via a wireless network. The cleanliness assessment module (3) is interconnected with the parameter generation module (4), the feedback update module (8), and the model prediction module (6) via a wireless network. The status monitoring module (5) is interconnected with the parameter generation module (4) and the model prediction module (6) via a wireless network. The model prediction module (6) is interconnected with the adaptive control module (7) via a wireless network.