A method for cleaning an internal cavity of a GIS device and a cleaning robot

Abstract

The application relates to the technical field of cleaning robots and discloses a GIS device internal cavity cleaning method and a cleaning robot. The method comprises the following steps: collecting image information of a GIS device internal cavity, separating a foreign matter area from a background area of the image information, generating a binary mask image, and calculating three-dimensional coordinates of the foreign matter area; converting joint angle sequences of discrete points on a smooth track into driving signals of servo motors through an inverse kinematics solver to drive a flexible joint arm to approach the foreign matter area point by point; after an end of a mechanical arm reaches a target position, a roll-type dust-free cloth and a flexible gripper work in parallel to clean the foreign matter area; new image information is generated, the new image information is input into a U-Net network, a new mask output by the U-Net network is calculated with an initial mask, a re-calculation of a supplementary scanning track is continued, and cleaning is continued until an intersection and union ratio is greater than or equal to a preset threshold. The application improves the cleaning efficiency and accuracy of the GIS device internal cavity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cleaning robots, and in particular to a cleaning method for an internal cavity of a GIS device and a cleaning robot. BACKGROUND

[0002] With the continuous development of automation technology and intelligent equipment, cleaning robots have been widely used in industrial, medical, home and other fields, especially in environments that require high precision and efficient cleaning, the demand for automated cleaning equipment is increasing. In particular in the field of cleaning GIS (Gas Insulated Switchgear) devices, the internal cavity of the cleaning equipment is usually narrow and complex, traditional manual cleaning not only has the problem of low efficiency, but also has high risk and the hidden danger of not being able to achieve complete cleaning.

[0003] The existing cleaning method mainly relies on manual operation or simple automated equipment, and usually faces technical problems such as inaccurate foreign object identification, lack of intelligent optimization in the cleaning process, inaccurate path planning and motion control, and difficulty in verifying cleaning effect.

[0004] With the continuous progress of deep learning, computer vision, path planning and adaptive control technology, new technical methods have been proposed to solve the above problems. For example, the U-Net convolutional neural network has been widely used in image segmentation, which can effectively identify foreign object areas with high precision; at the same time, deep reinforcement learning and other methods are used for dynamic path planning and cleaning strategy; the adaptive control system can automatically adjust the cleaning force and adsorption force according to the physical characteristics of foreign objects and environmental changes.

[0005] However, the existing technical solutions still have some limitations, especially in dealing with complex cleaning tasks, the high precision of foreign object identification, the efficiency of path planning, and the self-verification and optimization of cleaning effect have not been achieved. Therefore, developing a full-automatic cleaning method that can automatically identify foreign objects, dynamically adjust cleaning strategies and ensure efficient and non-destructive cleaning is still an important technical problem to be solved in this field. SUMMARY

[0006] The present application provides a cleaning method for an internal cavity of a GIS device and a cleaning robot to improve the cleaning efficiency and precision of the internal cavity of the GIS device.

[0007] In a first aspect, the present application provides a cleaning method for an internal cavity of a GIS device, which comprises:

[0008] The cleaning robot collects image information of an internal cavity of a GIS device by using a binocular vision system, performs semantic segmentation on the image information by using a U-Net network, separates a foreign matter region from a background region of the image information, generates a binary mask image, and calculates three-dimensional coordinates of the foreign matter region;

[0009] A collision-free smooth trajectory is generated based on the three-dimensional coordinates, a joint angle sequence corresponding to discrete points on the smooth trajectory is converted into a driving signal of a servo motor by using an inverse kinematics solver, and the flexible joint arm is driven to approach the foreign matter region point by point based on the driving signal;

[0010] After the end of the mechanical arm reaches the target position, the roll-type dust-free cloth and the flexible gripper work in parallel to clean the foreign matter region;

[0011] After the cleaning action is completed, the foreign matter region is photographed again to generate new image information, the new image information is input into the U-Net network, the output new mask is calculated with the initial mask, and if the intersection over union is less than a preset threshold, the re-computed scanning trajectory is continued to be cleaned until the intersection over union is greater than or equal to the preset threshold.

[0012] In combination with the first aspect, the separating of the foreign matter region from the background region of the image information comprises:

[0013] A first image set of internal cavities of a plurality of GIS devices is collected, a second image is generated after a region label is marked on any first image in the first image set and image pre-processing is performed;

[0014] A U-Net network is constructed by using a deep learning framework, all the second images are used as input, a region label corresponding to a binary classification result of each pixel point is used as output, and a cross-entropy loss function is used to calculate a difference value between network output and a real label;

[0015] The second image set corresponding to the second image is input into the U-Net network for training, and the network hyperparameters are adjusted based on the difference value;

[0016] All the image information is input into the adjusted U-Net network, the U-Net network predicts any pixel point and outputs a probability value, a first preset value is set, if the probability value is greater than the first preset value, the pixel point is marked as a foreign matter pixel, otherwise, it is marked as a background pixel;

[0017] The image information is segmented according to different label categories, all the foreign matter pixels are combined and set as the foreign matter region, all the background pixels are combined and set as the background region, and the probability value of any pixel point is binarized to generate the binary mask image.

[0018] In combination with the first aspect, the calculating the three-dimensional coordinates of the foreign matter region comprises:

[0019] extracting, based on the binocular vision system, left-view images and right-view images corresponding to the same image stream in all the image information;

[0020] extracting, based on the binary mask image, matching pixel points of the foreign matter region in the left-view images and the right-view images respectively, and calculating disparity values between each pair of matching pixel points through a stereo matching algorithm;

[0021] obtaining a parameter matrix of the binocular vision system, and converting two-dimensional coordinates of the foreign matter region into the three-dimensional coordinates based on the parameter matrix and the disparity values.

[0022] In combination with the first aspect, the generating the smooth trajectory without collision based on the three-dimensional coordinates comprises:

[0023] obtaining pose data of the cleaning robot, the pose data comprising motion data and IMU data, fusing all the pose data through an extended Kalman filter to generate real-time position and attitude data;

[0024] calculating a relative pose error between a robot base and the foreign matter region based on the three-dimensional coordinates, the real-time position and the attitude data;

[0025] generating a first path of the mechanical arm from a current position to the foreign matter region in combination with the relative pose error and using a trajectory planning algorithm;

[0026] smoothing the first path through a quintic polynomial interpolation to generate the smooth trajectory.

[0027] In combination with the first aspect, in the process of approaching the foreign matter region, the vacuum pump adjusts the rotating speed according to an expected negative pressure value output by an adsorption force model, the adsorption force model establishes a three-element mapping relationship through Bernoulli equation and gap flow equation based on a current load mass, a sealing gap height and a friction coefficient, and controls the adsorption force of the cleaning robot based on the three-element mapping relationship.

[0028] In combination with the first aspect, the calculating the intersection-union ratio between the output new mask and the initial mask comprises:

[0029] determining whether the mask values corresponding to any pixel points in the new mask and the initial mask are the same, calculating the intersection area and the union area between the new mask and the initial mask based on the determination result, and setting the ratio of the intersection area to the union area as the intersection-union ratio.

[0030] In combination with the first aspect, the recalculating the rescan trajectory and continuing cleaning comprises:

[0031] Position a residual area of the foreign matter area based on the new mask, regenerate a cleaning track of the mechanical arm based on a coordinate position and shape of the residual area, smooth the cleaning track using a quintic polynomial interpolation, and generate the supplementary cleaning track.

[0032] In combination with the first aspect, the cleaning robot continues cleaning, and further includes:

[0033] Collect historical record data of all cleaning tasks, the historical record data including task data, robot performance data and cleaning effect data, and perform data preprocessing on the historical record data based on data types;

[0034] Set optimization targets, the optimization targets including path planning, cleaning strategies and device state evaluation, select a machine learning algorithm based on the optimization targets to establish an optimization model;

[0035] Extract an associated data set corresponding to the same optimization target from all the processed historical record data, train the optimization model using the associated data set, and output an optimization result;

[0036] Summarize all the optimization results based on the optimization targets, generate a cleaning strategy, and cause the cleaning robot to perform a new cleaning task based on the cleaning strategy.

[0037] In combination with the first aspect, the cleaning of the foreign matter area includes:

[0038] Real-time detect physical properties of the foreign matter in the foreign matter area based on a sensor, use a support vector machine algorithm to classify and identify the foreign matter, and obtain a foreign matter type;

[0039] Adjust an adsorption force and a cleaning intensity based on the foreign matter type and the physical properties, the adsorption force model adjusting a negative pressure value according to a current load quality, a sealing gap height and a friction coefficient;

[0040] Calculate a negative pressure by a Bernoulli equation on a flow rate and a pressure difference of a suction disc area, calculate a flow rate of a fluid passing through a tiny gap between the suction disc and a device surface according to a gap flow equation, construct a mathematical expression based on the negative pressure and the flow rate, and set the three-element mapping relationship;

[0041] In a flexible joint arm operation process, adaptively adjust the adsorption force based on the three-element mapping relationship, and clean the foreign matter area based on the adjusted adsorption force.

[0042] The second aspect provides a cleaning robot, including:

[0043] The collection module is used for collecting image information of an internal cavity of a GIS device by the cleaning robot using a binocular vision system, performing semantic segmentation on the image information using a U-Net network, separating a foreign matter area from a background area of the image information, generating a binary mask image, and calculating three-dimensional coordinates of the foreign matter area;

[0044] The driving module is used for generating a collision-free smooth trajectory according to the three-dimensional coordinates, converting a joint angle sequence corresponding to discrete points on the smooth trajectory into a driving signal of a servo motor through an inverse kinematics solver, and driving the flexible joint arm to approach the foreign matter area point by point based on the driving signal;

[0045] The cleaning module is used for cleaning the foreign matter area by the roll-type dust-free cloth and the flexible gripper in parallel after the end of the mechanical arm reaches the target position;

[0046] The verification module is used for photographing the foreign matter area again after the cleaning action is completed, generating new image information, inputting the new image information into the U-Net network, calculating an intersection-over-union of an output new mask and an initial mask, and if the intersection-over-union is less than a preset threshold, recalculating a supplementary scanning trajectory and continuing cleaning until the intersection-over-union is greater than or equal to the preset threshold.

[0047] In the technical scheme provided in the application, firstly, the accurate identification of the foreign matter area is realized by combining the binocular vision system and the U-Net network. The image is segmented with high precision using the deep learning framework, which can effectively separate the foreign matter area from the background area, thereby accurately positioning the foreign matter position and generating an accurate cleaning path. Then, the foreign matter classification and identification technology based on the support vector machine algorithm can intelligently adjust the robot adsorption force and cleaning intensity by detecting the physical properties of the foreign matter in real time, ensuring that the adsorption force and cleaning intensity can be optimized and adjusted in the cleaning process of different types of foreign matter, avoiding the problems of device damage or incomplete cleaning caused by excessive or insufficient adsorption force in the traditional cleaning method. Next, the real-time feedback and verification of the cleaning effect are realized by combining the U-Net network and the intersection-over-union calculation. After the robot completes the cleaning task, the cleaning area is photographed again and the intersection-over-union is calculated to evaluate the cleaning effect in real time, and the closed-loop feedback mechanism ensures the efficiency and accuracy of the cleaning process, avoiding the defects of difficult control and verification of the cleaning effect in the traditional cleaning method. Finally, the cleaning robot can make optimization decisions based on historical data, automatically select the best path planning, cleaning strategy and device state evaluation method, and improve the long-term operation performance of the robot.

[0048] The application can also provide accurate pose information of the robot in a complex environment in real time by applying extended Kalman filtering to the pose data fusion. In combination with real-time feedback of the cleaning task, the robot can dynamically adjust the path planning and cleaning strategy to adapt to different types of cleaning tasks and environments. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings.

[0050] Figure 1 An embodiment schematic diagram of a GIS device internal cavity cleaning method in the embodiment of the present application;

[0051] Figure 2 An embodiment schematic diagram of a cleaning robot overall architecture in the embodiment of the present application;

[0052] Figure 3 An embodiment schematic diagram of a foreign matter identification process in the embodiment of the present application;

[0053] Figure 4 An embodiment schematic diagram of a cleaning robot module distribution in the embodiment of the present application. DETAILED DESCRIPTION

[0054] The embodiment of the present application provides a GIS device internal cavity cleaning method and a cleaning robot. The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "comprising" or "having" and any variation thereof is intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0055] For the convenience of understanding, the specific process of the embodiment of the present application will be described below. Please refer to Figure 1 An embodiment of the GIS device internal cavity cleaning method in the embodiment of the present application includes:

[0056] In step S101, the cleaning robot collects image information of the internal cavity of the GIS device by using a binocular vision system, performs semantic segmentation on the image information by using a U-Net network, separates the foreign matter area from the background area of the image information, generates a binary mask image, and calculates the three-dimensional coordinates of the foreign matter area.

[0057] It can be understood that the execution subject of the present application can be a cleaning device for the internal cavity of the GIS device, and can also be a terminal or a server, which is not limited here. The server is taken as an example for description of the embodiments of the present application.

[0058] Specifically, Figure 2 The overall architecture of the cleaning robot is shown in FIG. 1. The binocular vision system is composed of two cameras, which capture two slightly different image information by simultaneously shooting the same scene from different angles. The U-Net is a deep learning network specially used for image segmentation. The network performs efficient pixel-level processing on the input image information through an encoding-decoding structure, and separates the target area (such as the foreign matter area) from the background area. The foreign matter area refers to the part of the foreign matter to be cleaned, and the background area refers to the part of the internal cavity of the GIS device. The foreign matter area is marked as “1” and the background area is marked as “0” in each pixel point, so as to generate a binary mask image. The three-dimensional coordinates refer to the spatial coordinates of the foreign matter area.

[0059] In step S102, a smooth trajectory without collision is generated based on the three-dimensional coordinates, and the joint angle sequence corresponding to the discrete points on the smooth trajectory is converted into a driving signal of a servo motor by using an inverse kinematics solver, and the flexible joint arm is driven to approach the foreign matter area point by point based on the driving signal.

[0060] Specifically, the smooth trajectory refers to the running route of the cleaning robot to the foreign matter area. Based on the generated smooth trajectory, the angle of each joint of the mechanical arm, i.e., the joint angle sequence, is calculated by using the inverse kinematics method. The inverse kinematics solver converts each discrete point in the trajectory into the joint angle of the mechanical arm, and controls the movement of each joint of the mechanical arm by using a servo motor. The joint angle obtained by the inverse kinematics is converted into a PWM duty cycle signal, i.e., a driving signal, and then the precise movement of the mechanical arm is realized by using a motor drive control system. The calculation and control instructions of each joint angle are sent to the mechanical arm control system in real time, so that the cleaning robot approaches the foreign matter area point by point.

[0061] In step S103, after the end of the mechanical arm reaches the target position, the roll-type dust-free cloth and the flexible gripper work in parallel to clean the foreign matter area.

[0062] Specifically, when the cleaning task is executed, the end of the mechanical arm is equipped with two cleaning tools, i.e., a roll-type dust-free cloth and a flexible gripper. The dust-free cloth is used for soft wiping and cleaning, and the flexible gripper is suitable for adsorbing larger and heavier foreign matters.

[0063] Step S104, after the cleaning action is completed, the foreign matter area is photographed again, new image information is generated, the new image information is input into the U-Net network, the output new mask is calculated with the initial mask, and if the intersection over union is less than a preset threshold, the supplementary scanning track is recalculated and the cleaning is continued until the intersection over union is greater than or equal to the preset threshold.

[0064] Specifically, the intersection over union (IoU) is a commonly used evaluation index in target detection and segmentation tasks, which is used to measure the degree of coincidence of two regions. If the intersection over union is less than a preset threshold, it means that the cleaning effect does not meet the requirements, and the supplementary scanning track needs to be recalculated and the cleaning needs to be continued. Until the intersection over union is greater than or equal to the preset threshold, it means that the cleaning task is completed.

[0065] In a specific embodiment, the foreign matter area and the background area of the image information are separated, comprising:

[0066] (1) Collect a first image set of the internal cavities of a plurality of GIS devices, and perform image preprocessing on any first image in the first image set after region labeling, to generate a second image.

[0067] (2) Use a deep learning framework to build a U-Net network, use all second images as input, and use the region label corresponding to the binary classification result of each pixel point as output, and use a cross-entropy loss function to calculate the difference value between the network output and the true label.

[0068] (3) Input the second image set corresponding to the second image into the U-Net network for training, and adjust the network hyperparameters based on the difference value.

[0069] (4) Input all image information into the adjusted U-Net network, the U-Net network predicts any pixel point and outputs a probability value, sets a first preset value, if the probability value is greater than the first preset value, the pixel point is marked as a foreign matter pixel, otherwise, it is marked as a background pixel.

[0070] (5) Segment the image information according to different label categories, combine all foreign matter pixels and set them as a foreign matter area, combine all background pixels and set them as a background area, and generate a binary mask image after binarizing the probability value of any pixel point.

[0071] Specifically, Figure 3The flowchart for foreign matter recognition is shown in FIG. 1. The first image set refers to a set of first images collected by a vision system (such as a binocular vision system) equipped on a cleaning robot in multiple different internal cavities of GIS equipment. The first images contain various objects and foreign matters in the internal cavities of the equipment, such as dust, metal particles, debris, etc. The region labeling is a process of manually labeling the foreign matter regions in the images. In each image, the foreign matter regions (such as dust, debris) and the background regions (such as the surface of the equipment) are manually labeled. After the labeling is completed, necessary image preprocessing operations (such as size adjustment, denoising processing, standardization, etc.) are performed, and a second image is obtained, which will be used as the label data for network training.

[0072] The U-Net can accurately separate the target (foreign matter region) from the background region in the image through its encoder-decoder structure. The cross-entropy loss function is commonly used in binary classification problems to measure the difference value between the network output and the true label. For example, the calculation formula of the difference value L output by the cross-entropy loss function is as follows: wherein, is the i-th true label, is the i-th probability predicted by the U-Net network, and N is the number of labels. In this application, the output is the classification result (foreign matter or background) of each pixel in the image, and the true label is the manually labeled foreign matter region and background region.

[0073] During the training process, the error calculated by the loss function is used to adjust the hyperparameters (such as learning rate, batch size, network layer number, etc.) of the model, so that the model can better fit the training data, and finally achieve high accuracy and generalization ability.

[0074] The trained U-Net network can predict the information of each input image, and output whether each pixel point is a “foreign matter pixel” or a “background pixel” through a probability value.

[0075] Through the segmentation result of the U-Net network, a final binary mask image is generated, in which the pixel value of the foreign matter region is 1 and the pixel value of the background region is 0.

[0076] In a specific embodiment, the three-dimensional coordinates of the foreign matter region are calculated, including:

[0077] (1) Extract the left-view image and the right-view image corresponding to the same image stream in all image information based on the binocular vision system.

[0078] (2) Extract the matching pixel points of the foreign matter region in the left-view image and the right-view image based on the binary mask image, and calculate the disparity value between each pair of matching pixel points through a stereo matching algorithm.

[0079] (3) Obtain the parameter matrix of the binocular vision system, and convert the two-dimensional coordinates of the foreign object area into three-dimensional coordinates based on the parameter matrix and the parallax value.

[0080] Specifically, the binocular vision system is composed of two cameras, and the two cameras shoot the same scene at a fixed angle, i.e., left and right angle images, respectively. By comparing the differences (parallax values) between the two images, the depth information of the objects in the scene can be calculated.

[0081] The parallax value is the offset of the corresponding pixel points in the two images in the X-axis direction, which represents the distance of the object from the camera. The larger the parallax value, the closer the object is to the camera, and vice versa. The stereo matching algorithm includes but is not limited to block matching or other matching methods.

[0082] The parameter matrix includes the internal parameter matrix (focal length, optical center coordinates, etc.) of the camera in the binocular vision system and the external parameter matrix (position and rotation angle of the two cameras) between the cameras. According to the pinhole imaging principle and the triangulation method, the two-dimensional image coordinates are converted into three-dimensional space coordinates. For example, the pinhole imaging principle formula is: where f is the focal length of the camera, B is the baseline distance between the two cameras in the binocular vision system, d is the parallax value of the pixel point, and Z is the depth information of the object in the three-dimensional space (i.e., the distance between the camera and the object). The three-dimensional coordinates of the pixel points in the image are mapped to the three-dimensional coordinates in the three-dimensional space using the triangulation principle . where , The exact position of each foreign object pixel in the three-dimensional space can be determined.

[0083] In a specific embodiment, a collision-free smooth trajectory is generated based on the three-dimensional coordinates, including:

[0084] (1) Obtain the pose data of the cleaning robot, including motion data and IMU data. All pose data is fused by extended Kalman filtering to generate real-time position and attitude data.

[0085] (2) Calculate the relative pose error between the robot base and the foreign object area based on the three-dimensional coordinates, real-time position and attitude data.

[0086] (3) Generate the first path of the robot arm from the current position to the foreign object area by combining the relative pose error and using the trajectory planning algorithm.

[0087] (4) Smooth the first path by quintic polynomial interpolation to generate a smooth trajectory.

[0088] Specifically, the motion data refers to the displacement, velocity and acceleration information obtained by the robot through motion sensors (such as encoders). The IMU data is used to measure the acceleration and angular velocity of the robot in space, estimate its rotation angle and linear velocity. The Extended Kalman Filter (EKF) is a classic filtering algorithm specifically designed for state estimation problems in nonlinear systems, which can provide more accurate pose estimation based on data from multiple sensors. The fused data can provide the real-time pose of the robot (including real-time position and attitude data), i.e., the position, angle and motion state of the robot in space, for example, the fusion formula is: wherein, is the pose estimation at the current time, is the state transition function (estimating the pose by odometer and IMU data), is the noise coefficient.

[0089] The relative pose error refers to the pose error between the current position of the cleaning robot and the target foreign object region, which represents the distance and angle difference between the robot and the foreign object region, including position error and attitude error. The position error is the distance difference between the current position of the robot and the target position, and the attitude error is the angle difference between the current orientation of the robot and the target direction.

[0090] The trajectory planning algorithm (such as the RRT algorithm) is the process of calculating the path of the robot from the starting position to the target position, wherein the RRT algorithm is to explore a tree in the configuration space (i.e., the space where the robot can move) by random sampling, thereby generating a path from the starting point to the target point, i.e., the first path, which is collision-free and calculated according to the real-time environment and obstacle avoidance.

[0091] The quintic polynomial interpolation is a commonly used method for smoothing the path. Through quintic polynomial interpolation, the path can be made smoother and avoid sudden turns or discontinuities in the path.

[0092] In a specific embodiment, during the approach of the flexible joint arm to the foreign object region, the vacuum pump adjusts the rotational speed according to the expected negative pressure value output by the adsorption force model, and the adsorption force model establishes a three-dimensional mapping relationship based on the current load mass, sealing gap height and friction coefficient, and through Bernoulli equation and gap flow equation, and controls the adsorption force of the cleaning robot based on the three-dimensional mapping relationship.

[0093] Specifically, the suction force model refers to a mathematical model that can calculate the required negative pressure value based on the actual conditions of the robot suction system, such as load mass, sealing gap height, friction coefficient, etc. Negative pressure control is the core part of the cleaning robot suction process, as too high suction force can cause damage to the device surface, and too low suction force may not be able to firmly attach. Negative pressure is a core variable in the cleaning robot suction system, directly affecting the suction force between the robot and the surface. The vacuum pump is one of the core components of the cleaning robot suction system, responsible for generating the required negative pressure. By adjusting the speed of the vacuum pump, the size of the negative pressure can be changed in real time, thereby adjusting the suction force. The ternary mapping relationship refers to the "structure parameter-leakage flow-negative pressure" relationship. Bernoulli's equation describes the principle of energy conservation of fluids, and in the cleaning robot suction system, Bernoulli's equation can be used to describe the pressure change of air flow between the suction cup and the device surface, thereby deriving the relationship between negative pressure and flow. The gap flow equation is mainly used to describe the flow behavior of fluids in a small gap (e.g., the gap between the suction cup and the surface), and there is a close relationship between fluid flow and negative pressure, which can be described by fluid mechanics equations. According to the combination of Bernoulli's equation and the gap flow equation, the ternary mapping relationship of "structure parameter-leakage flow-negative pressure" is established. Specifically, the structure parameters of the suction cup (such as size, sealing gap, friction coefficient) affect the flow speed and pressure of air, thereby affecting the size of the negative pressure. Through experiments and numerical analysis, the relationship between the structure parameters of the suction cup, the leakage flow, and the negative pressure is determined, and finally the mapping relationship is established. This mapping relationship can help the system adjust the suction force according to the target negative pressure, thereby accurately controlling the contact between the robot and the device surface.

[0094] Through the ternary mapping relationship, the system can optimize the suction force in real time according to the relationship between the structure parameters, the leakage flow, and the negative pressure. If different types of foreign matter are encountered in the cleaning task, the system will automatically adjust the suction force (negative pressure) according to the different physical characteristics.

[0095] In a specific embodiment, the output new mask is calculated with the initial mask to obtain an intersection ratio, including:

[0096] In the new mask and the initial mask, it is judged whether the mask values corresponding to any pixel points are the same, and based on the judgment result, the intersection area and the union area between the new mask and the initial mask are calculated, and the ratio of the intersection area to the union area is set as the intersection ratio.

[0097] Specifically, the new mask refers to the binary image generated after the image taken by the end camera of the cleaning robot after the completion of the cleaning task is processed by the U-Net network, indicating whether there are foreign matter parts in the cleaning area. In the new mask, "1" represents the foreign matter area, and "0" represents the background area. The initial mask is a binary image generated after the pre-cleaning image is processed by the U-Net network before the start of the cleaning task, identifying the foreign matter area in the pre-cleaning image. Each pixel point in the mask image represents the classification result of the corresponding position in the image, i.e., whether the position belongs to the foreign matter area (1) or the background area (0). In order to calculate the intersection over union, it is necessary to compare whether the values of each pixel point in the new mask and the initial mask are consistent. The intersection area refers to the area of the region corresponding to the pixel points marked as "1" in both the new mask and the initial mask, i.e., the area of the foreign matter region overlapping before and after cleaning. The union area refers to the area of the region marked as "1" in at least one of the new mask and the initial mask, i.e., the total area of the foreign matter region before and after cleaning. The intersection over union is a commonly used indicator to measure the overlap of two regions, and is usually used in image segmentation tasks to evaluate the matching degree between the model output and the true label. The higher the value, the better the removal effect of the foreign matter area after cleaning, and the higher the cleaning quality.

[0098] In a specific embodiment, the recalculated supplementary cleaning trajectory and the continued cleaning include:

[0099] Based on the new mask, the residual area of the foreign matter area is located, the cleaning trajectory of the mechanical arm is regenerated based on the coordinate position and shape of the residual area, the cleaning trajectory is smoothed using a quintic polynomial interpolation, and a supplementary cleaning trajectory is generated.

[0100] Specifically, the residual area refers to the area where foreign matter still exists after the initial cleaning. By comparing the mask images before and after cleaning (initial mask and new mask), the area that is not completely cleaned after cleaning, i.e., the residual area, can be obtained. The cleaning trajectory is the path that the mechanical arm must follow when performing the cleaning task. When calculating the supplementary cleaning trajectory, the path is re-planned according to the position and shape of the residual area. Similarly, the path is smoothed using a quintic polynomial interpolation algorithm to generate a supplementary cleaning trajectory.

[0101] In a specific embodiment, after the cleaning robot continues to clean, it further includes:

[0102] (1) Collecting all historical record data of cleaning tasks, the historical record data including task data, robot performance data and cleaning effect data, and performing data preprocessing on the historical record data based on the data type.

[0103] (2) Setting optimization goals, the optimization goals including path planning, cleaning strategy and equipment state evaluation, and establishing an optimization model based on the optimization goals and selecting a machine learning algorithm.

[0104] (3) Extract the associated data set corresponding to the same optimization target from all the processed historical record data, train the optimization model using the associated data set, and output the optimization result.

[0105] (4) Based on the optimization target, all optimization results are summarized to generate a cleaning strategy, and the cleaning robot performs a new cleaning task based on the cleaning strategy.

[0106] Specifically, the task data includes the time of the task, the size of the cleaning area, the cleaning target, etc. The robot performance data includes the motion trajectory of the robot, the cleaning time, the speed, the energy consumption, the motion data of the mechanical arm, etc. The cleaning effect data includes the final effect evaluation data of the cleaning task, such as the accuracy of the cleaning effect (calculated by the intersection over union), the task completion degree, and the residual foreign matter. In order to enable the machine learning model to be trained efficiently, the collected historical data needs to undergo data preprocessing operations such as data cleaning, standardization, and normalization.

[0107] Path planning refers to optimizing the path of the robot during cleaning to ensure efficient and collision-free task completion. Cleaning strategy refers to the specific strategy for the robot to execute the cleaning task, including the selection of cleaning tools, the adjustment of cleaning intensity and speed, etc. Equipment state evaluation refers to evaluating the state of the equipment and making optimization suggestions through machine learning algorithms to improve the long-term performance and stability of the equipment. Different optimization targets require different machine learning algorithms to build optimization models. For example, reinforcement learning algorithms can be used in path planning and cleaning strategy optimization to gradually optimize the strategy through a reward mechanism. Cluster analysis of historical record data through clustering algorithms automatically identifies patterns of different equipment, cleaning tasks, and equipment states. Supervised learning algorithms are used to predict the relationship between cleaning effect and task data (such as regression analysis, decision trees, etc.).

[0108] If the optimization target is path planning, task data related to path planning such as path points, cleaning areas, and cleaning times need to be extracted. Through the selected machine learning algorithm and the extracted associated data set, the optimization model is trained to learn the relationship between various factors in the historical data, thereby achieving optimization of the cleaning task. Through training of the optimization model, the output result is the cleaning task parameters optimized according to the historical data and training target, such as the best path, the best cleaning strategy, and appropriate equipment state evaluation.

[0109] The cleaning strategy includes the path, cleaning method, and cleaning intensity that the cleaning robot needs to follow when executing the task, enabling the cleaning robot to achieve efficient and accurate operation in subsequent cleaning tasks.

[0110] In a specific embodiment, the foreign object area is cleaned, including:

[0111] (1) Based on real-time detection of the physical properties of foreign matter in the foreign matter area, the support vector machine algorithm is used to classify and identify the foreign matter, and the type of foreign matter is obtained.

[0112] (2) Based on the type and physical properties of the foreign matter, the adsorption force and cleaning strength are adjusted. The adsorption force model adjusts the negative pressure value according to the current load mass, sealing gap height, and friction coefficient.

[0113] (3) The flow rate and pressure difference of the suction cup area are calculated by the Bernoulli equation, the flow rate of the fluid through the small gap between the suction cup and the device surface is calculated according to the gap flow equation, and a mathematical expression is constructed based on the negative pressure and flow rate, and is set as a ternary mapping relationship.

[0114] (4) During the operation of the flexible joint arm, the adsorption force is adjusted based on the ternary mapping relationship, and the foreign matter area is cleaned based on the adjusted adsorption force.

[0115] Specifically, foreign matter refers to the material in the foreign matter area that needs to be cleaned, and physical properties include hardness, viscosity, friction coefficient, surface roughness, etc. The physical properties of the foreign matter are collected in real time using appropriate sensors (such as force sensors, pressure sensors, etc.). Through pre-labeled training data, the support vector machine (SVM) algorithm can classify the foreign matter into different types (such as metal, glass, plastic, dust, etc.), i.e. the type of foreign matter.

[0116] The adsorption force model calculates the required negative pressure value based on the physical properties of the foreign matter (such as mass, sealing gap height, friction coefficient, etc.), and then controls the adsorption force of the suction cup. For different types of foreign matter, the adsorption force needs to be adjusted appropriately.

[0117] According to the detected mass and friction coefficient of the foreign matter, the adsorption force (i.e. negative pressure value) is adjusted. For example, heavier foreign matter requires stronger adsorption force to ensure stable adsorption, while lighter foreign matter does not require too strong adsorption force. According to the detected mass and friction coefficient of the foreign matter, the adsorption force (i.e. negative pressure value) is adjusted. For example, heavier foreign matter requires stronger adsorption force to ensure stable adsorption, while lighter foreign matter does not require too strong adsorption force. By adjusting the contact pressure of the suction cup and the movement speed of the robot flexible joint arm, the cleaning strength is controlled. For soft foreign matter, reduce the pressure; for hard foreign matter, increase the cleaning strength.

[0118] Similarly, according to the Bernoulli equation and the fluid flow between the robot suction cup and the surface, the required negative pressure value of the suction cup is calculated. The greater the negative pressure, the stronger the adsorption force, and vice versa. According to the gap between the suction cup and the device surface and the viscosity of the fluid, the flow rate of the fluid (i.e. leakage flow rate) is calculated. According to the relationship between flow rate and negative pressure, the negative pressure is adjusted, and then the adsorption force is controlled.

[0119] By combining the Bernoulli equation and the gap flow equation, a mathematical model is established to relate the structural parameters (such as the size of the suction cup, the height of the gap, etc.), the leakage flow rate, and the negative pressure. This mapping relationship can help the control system adjust the suction force based on the current load mass, friction coefficient, and sealing gap height.

[0120] According to the ternary mapping relationship, the robot can adaptively adjust the suction force (negative pressure) in real time. If the physical properties of the foreign matter change (such as mass, friction coefficient, surface smoothness, etc.), the system will automatically adjust the negative pressure to ensure that the suction force is always within the optimal range, avoiding excessive suction or insufficient suction.

[0121] The above describes the cleaning method of the internal cavity of the GIS device in the embodiments of the present application. Next, a cleaning robot in the embodiments of the present application is described. Referring to Figure 4 , one embodiment of the cleaning robot in the embodiments of the present application includes:

[0122] The acquisition module 201 is configured to collect image information of the internal cavity of the GIS device by the cleaning robot using a binocular vision system, perform semantic segmentation on the image information using a U-Net network, separate the foreign matter area from the background area of the image information, generate a binary mask image, and calculate the three-dimensional coordinates of the foreign matter area.

[0123] The driving module 202 is configured to generate a collision-free smooth trajectory based on the three-dimensional coordinates, convert the joint angle sequence corresponding to the discrete points on the smooth trajectory into a driving signal of a servo motor through an inverse kinematics solver, and drive the flexible joint arm to approach the foreign matter area point by point based on the driving signal.

[0124] The cleaning module 203 is configured to work in parallel with the flexible gripper to clean the foreign matter area after the end of the mechanical arm reaches the target position.

[0125] The verification module 204 is configured to, after the cleaning action is completed, take a new picture of the foreign matter area, generate new image information, input the new image information into the U-Net network, calculate the intersection-over-union of the output new mask and the initial mask, and if the intersection-over-union is less than a preset threshold, recalculate the supplementary scanning trajectory and continue cleaning until the intersection-over-union is greater than or equal to the preset threshold.

[0126] Through the cooperation of the above components, first, the precise identification of foreign object regions is realized by combining the binocular vision system with the U-Net network. Using the deep learning framework for high-precision image segmentation can effectively separate the foreign object region from the background region, thereby accurately positioning the foreign object location and generating an accurate cleaning path. Then, based on the foreign object classification and identification technology of the support vector machine algorithm, the physical properties of the foreign object are detected in real time, and the robot's adsorption force and cleaning intensity are intelligently adjusted to ensure that the adsorption force and cleaning intensity can be optimally adjusted during the cleaning process of different types of foreign objects, avoiding the problems of equipment damage or incomplete cleaning caused by excessive or insufficient adsorption force in traditional cleaning methods. Next, by combining the U-Net network with the intersection-over-union calculation, real-time feedback and verification of the cleaning effect are realized. After completing the cleaning task, the robot re-shoots the cleaning area and calculates the intersection-over-union to evaluate the cleaning effect in real time, and the closed-loop feedback mechanism ensures the efficiency and accuracy of the cleaning process, avoiding the defects of difficult control and verification of cleaning effect in traditional cleaning methods. Finally, based on the data of historical cleaning tasks, adaptive optimization can be performed to continuously improve the efficiency and accuracy of subsequent tasks. The cleaning robot can make optimal decisions based on historical data, automatically select the best path planning, cleaning strategy and equipment state evaluation method, and improve the long-term operation performance of the robot.

[0127] The application also applies extended Kalman filtering to pose data fusion to provide accurate pose information of the robot in complex environments in real time. Combined with real-time feedback of the cleaning task, the robot can dynamically adjust the path planning and cleaning strategy to adapt to different types of cleaning tasks and environments.

[0128] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, system and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0129] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0130] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for cleaning the internal cavity of a GIS device, characterized in that, The cleaning method for the internal cavity of the GIS equipment includes: The cleaning robot uses a binocular vision system to collect image information of the internal cavity of the GIS equipment, uses a U-Net network to perform semantic segmentation on the image information, separates the foreign object area from the background area of ​​the image information, generates a binary mask image, and calculates the three-dimensional coordinates of the foreign object area. Based on the three-dimensional coordinates, a collision-free smooth trajectory is generated, and the joint angle sequence corresponding to the discrete points on the smooth trajectory is converted into the drive signal of the servo motor through the inverse kinematics solver. Based on the drive signal, the flexible articulated arm is driven to approach the foreign object region point by point. After the robotic arm reaches the target position, the roll-up cleanroom cloth and the flexible gripper work in parallel to clean the area containing foreign objects. After the cleaning process is completed, the foreign object area is photographed again to generate new image information. The new image information is input into the U-Net network, and the cross-over ratio (COP) of the output new mask and the initial mask is calculated. If the COP is less than a preset threshold, the re-scanning trajectory is recalculated and cleaning continues until the COP is greater than or equal to the preset threshold.

2. The cleaning method for the internal cavity of GIS equipment according to claim 1, characterized in that, The step of separating the foreign object region from the background region of the image information includes: A first image set of internal cavities of various GIS devices is collected. After region marking of any first image in the first image set, image preprocessing is performed to generate a second image. The U-Net network is constructed using a deep learning framework. All the second images are used as input, and the region label corresponding to the binary classification result of each pixel is used as output. The difference between the network output and the true label is calculated using the cross-entropy loss function. The second image set corresponding to the second image is input into the U-Net network for training, and the network hyperparameters are adjusted based on the difference value. All the image information is input into the adjusted U-Net network. The U-Net network predicts any pixel and outputs a probability value. A first preset value is set. If the probability value is greater than the first preset value, the pixel is marked as a foreign object pixel; otherwise, it is marked as a background pixel. The image information is segmented according to different marker categories, all foreign object pixels are combined and set as the foreign object region, all background pixels are combined and set as the background region, and the probability value of any pixel is binarized and then combined to generate the binary mask image.

3. The cleaning method for the internal cavity of GIS equipment according to claim 2, characterized in that, The calculation of the three-dimensional coordinates of the foreign object region includes: Based on the binocular vision system, the left-view image and the right-view image corresponding to the same image stream are extracted from all the image information; Based on the binary mask image, matching pixels of the foreign object region are extracted in the left and right view images respectively, and the disparity value between each pair of matching pixels is calculated by a stereo matching algorithm. Obtain the parameter matrix of the binocular vision system, and convert the two-dimensional coordinates of the foreign object region into the three-dimensional coordinates based on the parameter matrix and the disparity value.

4. The cleaning method for the internal cavity of GIS equipment according to claim 1, characterized in that, The generation of a collision-free smooth trajectory based on the three-dimensional coordinates includes: The pose data of the cleaning robot is acquired, including motion data and IMU data. All pose data are fused by extended Kalman filtering to generate real-time position and attitude data. The relative pose error between the robot base and the foreign object region is calculated based on the three-dimensional coordinates, the real-time position, and the attitude data. Based on the relative pose error, a trajectory planning algorithm is used to generate the first path for the robotic arm from its current position to the foreign object region; The smoothed trajectory is generated by smoothing the first path using fifth-order polynomial interpolation.

5. The method for cleaning the internal cavity of a GIS device according to claim 1, characterized in that, As the flexible articulated arm approaches the foreign object area, the vacuum pump adjusts its rotation speed according to the desired negative pressure value output by the adsorption force model. The adsorption force model establishes a ternary mapping relationship based on the current load mass, sealing gap height, and friction coefficient, and through Bernoulli's equation and the gap flow equation. The adsorption force of the cleaning robot is controlled based on the ternary mapping relationship.

6. The cleaning method for the internal cavity of GIS equipment according to claim 1, characterized in that, The step of calculating the intersection-union ratio (IUU) between the output new mask and the initial mask includes: In the new mask and the initial mask, it is determined whether the mask value corresponding to any pixel point is the same. Based on the determination result, the intersection area and the union area between the new mask and the initial mask are calculated, and the ratio of the intersection area to the union area is set as the intersection-union ratio.

7. The cleaning method for the internal cavity of GIS equipment according to claim 6, characterized in that, The process of recalculating the cleaning trajectory and continuing cleaning includes: Based on the residual area of ​​the foreign object region located by the new mask, the cleaning trajectory of the robotic arm is regenerated based on the coordinate position and shape of the residual area. The cleaning trajectory is smoothed using fifth-order polynomial interpolation to generate the supplementary cleaning trajectory.

8. The method for cleaning the internal cavity of a GIS device according to claim 1, characterized in that, After the cleaning robot continues cleaning, it also includes: Collect historical data of all cleaning tasks, including task data, robot performance data, and cleaning effect data, and perform data preprocessing on the historical data based on the data type; Set optimization objectives, including path planning, cleaning strategies, and equipment status assessment, and select machine learning algorithms to build an optimization model based on the optimization objectives; Extract the associated datasets corresponding to the same optimization objective from all processed historical data, train the optimization model using the associated datasets, and output the optimization results; Based on the optimization objective, all optimization results are summarized to generate a cleaning strategy, and the cleaning robot executes a new cleaning task based on the cleaning strategy.

9. The cleaning method for the internal cavity of GIS equipment according to claim 5, characterized in that, The cleaning of the foreign object area includes: Based on the real-time detection of the physical characteristics of foreign objects in the foreign object area by sensors, the support vector machine algorithm is used to classify and identify the foreign objects and obtain the type of foreign object; The adsorption force and cleaning power are adjusted based on the type of foreign matter and the physical properties. The adsorption force model adjusts the negative pressure value according to the current load mass, sealing gap height, and friction coefficient. The negative pressure in the suction cup region is calculated using Bernoulli's equation based on the flow velocity and pressure difference. The flow rate of fluid passing through the tiny gap between the suction cup and the device surface is calculated based on the gap flow equation. A mathematical expression is constructed based on the negative pressure and the flow rate and set as the ternary mapping relationship. During the operation of the flexible articulated arm, the adsorption force is adaptively adjusted based on the ternary mapping relationship, and the foreign matter area is cleaned based on the adjusted adsorption force.

10. A cleaning robot, characterized in that, include: The acquisition module is used by the cleaning robot to acquire image information of the internal cavity of the GIS equipment using a binocular vision system, use the U-Net network to perform semantic segmentation on the image information, separate the foreign object area from the background area of ​​the image information, generate a binary mask image, and calculate the three-dimensional coordinates of the foreign object area. The driving module is used to generate a collision-free smooth trajectory based on the three-dimensional coordinates, and convert the joint angle sequence corresponding to the discrete points on the smooth trajectory into a servo motor driving signal through an inverse kinematics solver, and drive the flexible articulated arm to approach the foreign object region point by point based on the driving signal. The cleaning module is used to clean the foreign object area by having a roll-up cleanroom cloth and a flexible gripper work in parallel after the end of the robotic arm reaches the target position. The verification module is used to re-capture the foreign object area after the cleaning action is completed, generate new image information, input the new image information into the U-Net network, calculate the intersection-over-union ratio (IoU) between the output new mask and the initial mask, and if the IoU is less than a preset threshold, recalculate the re-scanning trajectory and continue cleaning until the IoU is greater than or equal to the preset threshold.

Images