Robot-based workpiece cleaning method, electronic device, and storage medium

Abstract

The present disclosure provides a workpiece cleaning method based on a robot, an electronic device and a storage medium. The workpiece cleaning method comprises: driving the robot to move the image acquisition device, acquiring initial images of the workpiece to be cleaned at different photographing points and different robot poses, determining at least one target photographing point and the corresponding photographing pose based on the initial images, and constructing a photographing pose matrix; driving the robot to move the image acquisition device to the corresponding target photographing point at the photographing pose, and acquiring a plurality of local images of the workpiece to be cleaned at the target photographing point; detecting the local images to obtain the position information and shape information of the cleaning area in the workpiece to be cleaned; determining a matching target cleaning strategy from a plurality of preset cleaning strategies based on the shape information, and driving the end effector of the robot to perform cleaning work on the cleaning area based on the target cleaning strategy according to the position information. The present disclosure can improve the detection rate and classification accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of industrial manufacturing and maintenance technology, and in particular to a robot-based workpiece cleaning method, electronic equipment, and storage medium. Background Technology

[0002] In industrial manufacturing and equipment maintenance scenarios, cleaning and inspection of workpiece surfaces or internal hole structures typically employ two traditional methods: manual positioning cleaning and automated robotic cleaning using machine vision.

[0003] Manual positioning cleaning offers high flexibility, but the quality of the work heavily depends on the operator's experience, and generally suffers from low efficiency and poor cleaning consistency. In particular, it is difficult to guarantee stable cleaning quality for complex hole structures such as deep holes, irregular holes, or dense hole groups.

[0004] While machine vision-based robotic automatic cleaning improves the consistency and efficiency of hole cleaning to some extent, this method typically uses fixed poses to acquire images, without considering the actual working conditions such as the size, distribution, and occlusion of holes in the image. This can easily lead to problems such as some holes being too small in the image or mismatch in the shooting pose, which significantly reduces the accuracy of hole location recognition and the success rate of cleaning. Furthermore, the cleaning operation uses a single fixed cleaning mode, which cannot adapt to the differentiated cleaning needs of different hole types, resulting in poor overall cleaning effect. Summary of the Invention

[0005] The technical problem to be solved by this disclosure is to overcome the shortcomings of low success rate and poor cleaning effect of hole cleaning in the prior art, and to provide a robot-based workpiece cleaning method, electronic device and storage medium.

[0006] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0007] A first aspect of this disclosure provides a robot-based workpiece cleaning method, wherein the robot is equipped with an image acquisition device, and the workpiece cleaning method includes:

[0008] The robot is driven to move the image acquisition device to acquire initial images of the workpiece to be cleaned at different shooting points and different robot poses. Based on the initial images, at least one target shooting point and the corresponding shooting pose are determined, and a shooting pose matrix is ​​constructed.

[0009] The robot is driven to move the image acquisition device to the corresponding target image point in the image-taking pose, and to acquire several partial images of the workpiece to be cleaned at the target image point.

[0010] The local image is subjected to cleaning area detection to obtain the position and shape information of the cleaning area in the workpiece to be cleaned; wherein, the cleaning area includes hole area and / or groove area;

[0011] Based on the shape information, a matching target cleaning strategy is determined from multiple preset cleaning strategies, and the robot's end effector is driven to perform cleaning operations on the area to be cleaned based on the target cleaning strategy according to the position information.

[0012] Optionally, before the step of acquiring several partial images of the workpiece to be cleaned at the target imaging point, the workpiece cleaning method further includes:

[0013] The image acquisition device is driven to adjust to the preset target aperture value and target focal length value to acquire the first low-exposure image and the first high-exposure image of the workpiece to be cleaned;

[0014] The target aperture value, the target focal length value, the first low-exposure image, and the first high-exposure image are input into a pre-trained exposure time prediction model to obtain the target exposure time.

[0015] The step of acquiring several local images of the workpiece to be cleaned at the target imaging point includes:

[0016] The local image of the workpiece to be cleaned is acquired using the target aperture value, the target focal length value, and the target exposure time.

[0017] Optionally, prior to the step of detecting the area to be cleaned in the local image, the workpiece cleaning method further includes:

[0018] Obtain the quality evaluation parameters of the local image, and determine the image quality score based on the quality evaluation parameters;

[0019] In response to the image quality score being less than a preset threshold, the image acquisition device is driven to adjust to the target aperture value and the target focal length value, and acquire multiple second low-exposure images and second high-exposure images of the workpiece to be cleaned;

[0020] In response to receiving the exposure time labels corresponding to the second low-exposure image and the second high-exposure image, the exposure time prediction model is fine-tuned and trained using the second low-exposure image and the second high-exposure image as input and the corresponding exposure time labels as output. Then, the step of driving the image acquisition device to adjust to the preset target aperture value and target focal length value to acquire the first low-exposure image and the first high-exposure image of the workpiece to be cleaned is returned to be executed.

[0021] Optionally, the step of detecting the area to be cleaned in the local image to obtain the location and shape information of the area to be cleaned in the workpiece includes:

[0022] Each of the aforementioned local images is stitched together to obtain a global image, and the global image is then converted into a global grayscale image.

[0023] The global grayscale image is binarized using a root mean square error adaptive threshold to obtain candidate regions to be cleaned.

[0024] A pre-trained semantic segmentation model is used to perform pixel-level classification of the candidate regions to be cleaned, thereby obtaining a mask of the regions to be cleaned.

[0025] Morphological processing and shape analysis are performed on the mask of the area to be cleaned to obtain the location information and shape information of the area to be cleaned.

[0026] Optionally, the step of performing morphological processing and shape analysis on the mask of the area to be cleaned to obtain the location information and shape information of the area to be cleaned includes:

[0027] The target mask is obtained by sequentially performing adaptive erosion, directional closing operation, and conditional opening operation on the mask of the area to be cleaned.

[0028] The target mask is marked with connected components to obtain multiple connected regions;

[0029] Obtain the geometric features of each connected region, and select the connected regions that meet the preset morphological conditions as the regions to be cleaned based on the geometric features;

[0030] Extract the outline of each of the areas to be cleaned, and determine the shape of each area to be cleaned and the center coordinates of each area to be cleaned in the global image based on the outline;

[0031] Based on the transformation relationship between the coordinate system of the image acquisition device and the robot base coordinate system, the center coordinates are converted into the target three-dimensional coordinates under the robot base coordinate system.

[0032] Optionally, the step of converting the center coordinates into target three-dimensional coordinates in the robot base coordinate system based on the transformation relationship between the image acquisition device coordinate system and the robot base coordinate system includes:

[0033] Obtain the intrinsic parameter matrix and distortion coefficients of the image acquisition device;

[0034] The center coordinates are subjected to distortion correction based on the distortion coefficients to obtain normalized image coordinates.

[0035] The depth value of the area to be cleaned is obtained, and the normalized image coordinates are back-projected based on the depth value and the intrinsic parameter matrix to obtain the initial three-dimensional coordinates in the coordinate system of the image acquisition device.

[0036] Based on the transformation relationship, the initial three-dimensional coordinates are converted into the target three-dimensional coordinates.

[0037] Optionally, the step of determining a matching target cleaning strategy from multiple preset cleaning strategies based on the shape information includes:

[0038] In response to receiving a cleaning strategy setting instruction, the target cleaning strategy is determined according to the cleaning strategy setting instruction;

[0039] or,

[0040] In response to the absence of the cleaning strategy setting instruction, the diameter parameter of each area to be cleaned is determined based on the shape information;

[0041] In response to the diameter parameter being greater than a preset diameter, the target cleaning strategy is determined to be a spiral progressive cleaning strategy or a composite cleaning strategy; or, in response to the diameter parameter being less than or equal to the preset diameter, the target cleaning strategy is determined to be a straight-in, straight-out rapid cleaning strategy.

[0042] A second aspect of this disclosure provides a robot-based workpiece cleaning system, wherein the robot is equipped with an image acquisition device, and the workpiece cleaning system includes:

[0043] The module is used to drive the robot to move the image acquisition device, acquire initial images of the workpiece to be cleaned at different shooting points and different robot poses, and determine at least one target shooting point and corresponding shooting pose based on the initial images to construct a shooting pose matrix.

[0044] The acquisition module is used to drive the robot to move the image acquisition device to the corresponding target image point in the image-taking pose, and acquire several local images of the workpiece to be cleaned at the target image point;

[0045] The detection module is used to detect the area to be cleaned in the local image to obtain the position and shape information of the area to be cleaned in the workpiece; wherein, the area to be cleaned includes a hole area and / or a groove area;

[0046] The cleaning module is used to determine a matching target cleaning strategy from multiple preset cleaning strategies based on the shape information, and drive the end effector of the robot to perform cleaning operations on the area to be cleaned based on the target cleaning strategy according to the position information.

[0047] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein the processor executes the computer program to implement the robot-based workpiece cleaning method described in the first aspect of this disclosure.

[0048] A fourth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robot-based workpiece cleaning method described in the first aspect of this disclosure.

[0049] A fifth aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the robot-based workpiece cleaning method described in the first aspect of this disclosure.

[0050] Based on common knowledge in the field, the above optional conditions can be combined arbitrarily to obtain the optional examples of this disclosure.

[0051] The positive advancements of this disclosure are as follows: By systematically determining the imaging points and poses, the consistency of image imaging and positioning is improved across different batches and under different lighting conditions. Furthermore, morphological processing of the area to be cleaned effectively suppresses interference such as edge breakage, adhesion, and reflection, thereby improving detection and classification accuracy. Simultaneously, cleaning strategies can be dynamically selected based on hole shape and contamination level, avoiding inefficiency or incomplete cleaning caused by a single cleaning strategy. The entire process, from visual perception, parameter optimization, hole recognition and positioning to cleaning execution, is automated, reducing manual intervention, improving efficiency and consistency, and lowering labor costs and operational risks. In addition, the imaging points, image acquisition parameters, and models can all be switched according to the workpiece, making it applicable to various robots and possessing good versatility and scalability. Attached Figure Description

[0052] Figure 1 This is a flowchart of the robot-based workpiece cleaning method disclosed herein;

[0053] Figure 2 This is a schematic diagram of the dotted circle plate disclosed herein;

[0054] Figure 3 A physical schematic diagram of a specific example of this disclosure;

[0055] Figure 4 A physical schematic diagram for another specific example of this disclosure;

[0056] Figure 5 This is a schematic diagram of the modules of the robot-based workpiece cleaning system disclosed herein;

[0057] Figure 6 This is a schematic diagram of the structure of the electronic device disclosed herein. Detailed Implementation

[0058] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0059] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not be construed as an unnecessary limitation. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0060] Traditional methods mainly employ two approaches: manual positioning and cleaning, and automated cleaning by robots based on machine vision.

[0061] Manual positioning cleaning involves operators using magnifying glasses, endoscopes, or simple optical equipment to observe the position and shape of the hole, and then manually operating cleaning tools (such as brushes, air guns, spray devices, etc.) to clean it. This method is highly flexible, but it heavily relies on the operator's experience, is inefficient and inconsistent, and its cleaning effect is difficult to guarantee for deep holes, irregularly shaped holes, or densely packed hole groups.

[0062] The automatic cleaning method of robots based on machine vision achieves hole positioning and cleaning through pre-teaching or offline programming. By fixing a camera at a certain place in the workspace, a global image of the workpiece is captured. The position of the hole center is identified by template matching or simple edge detection. The robot generates a motion trajectory based on the identification result and performs cleaning.

[0063] However, camera position is usually determined based on experience or a single teaching session, without considering the size, distribution, and occlusion of holes in the image, nor is a posture verification process introduced. This can easily lead to some holes being imaged too small or having mismatched postures, affecting the success rate of recognition and cleaning. Furthermore, since camera parameters (e.g., aperture, focal length, exposure) are mostly adjusted manually, no automatic optimization mechanism adapted to workpiece characteristics and lighting conditions has been established, resulting in fluctuations in image clarity across different batches or working conditions, directly affecting subsequent recognition accuracy. Hole recognition algorithms often use simple thresholds or rigid template matching, usually designed only for circular holes or a few fixed hole types. A single template or fixed threshold is insufficient to cover the differences in projection features of different hole shapes, and is inadequate for edge breaks. Interference from cracks, adhesion between adjacent holes, and reflections lacks effective morphological repair and pseudo-region filtering mechanisms. It also lacks layered discrimination logic for hole shapes and cannot automatically classify and extract corresponding size and orientation parameters based on contour geometric features. This leads to false detection, missed detection, or shape misjudgment in scenarios with a mixture of various shapes, such as elliptical holes, polygonal holes, and irregular holes, which are common in actual production. The recognition rate drops significantly, seriously affecting the accuracy of subsequent robot positioning and cleaning. In addition, existing automatic cleaning robots mostly use fixed linear insertion and removal actions, without dynamically selecting or combining cleaning modes according to the shape (e.g., round holes, elliptical holes, polygonal holes), size, and degree of contamination. This makes it difficult to balance efficiency and effectiveness and poses a risk of collision.

[0064] In view of this, this disclosure provides a robot-based workpiece cleaning method, electronic device, and storage medium to solve the shortcomings of existing robot automatic cleaning methods, such as low success rate of hole cleaning and poor cleaning effect.

[0065] Example 1

[0066] In one specific embodiment of this disclosure, a robot-based workpiece cleaning method is provided, wherein the robot is equipped with an image acquisition device, such as... Figure 1 As shown, the workpiece cleaning method includes:

[0067] S1. Drive the robot to move the image acquisition device to acquire initial images of the workpiece to be cleaned at different shooting points and different robot poses, and determine at least one target shooting point and corresponding shooting pose based on the initial images to construct a shooting pose matrix.

[0068] S2. Drive the robot to move the image acquisition device to the corresponding target image point in the photo-taking pose, and acquire several local images of the workpiece to be cleaned at the target image point.

[0069] S3. Detect the area to be cleaned in the local image to obtain the position and shape information of the area to be cleaned in the workpiece; wherein, the area to be cleaned includes hole areas and / or groove areas;

[0070] S4. Based on shape information, determine the matching target cleaning strategy from multiple preset cleaning strategies, and drive the robot's end effector to perform cleaning operations on the area to be cleaned based on the target cleaning strategy according to the position information.

[0071] Specifically, the robot's end effector is equipped with cleaning tools, such as a cleaning rod. An image acquisition device (e.g., a camera) is rigidly mounted on the robot's end effector, ensuring that the camera's optical axis is aligned with the axis of the hole to be cleaned and the orientation of the cleaning rod. When the robot is required to perform a cleaning operation, the camera's real-time image acquisition function is activated to confirm that it can acquire and display the real-time image captured by the camera. For example, the robot is controlled to move in a straight line towards the workpiece to be cleaned at a normal speed for 3 seconds, ensuring that the movement does not exceed the robot's joint limits. During the movement, the camera's real-time image is observed to see if there are any dynamic changes. If the image updates with the robot's movement, the camera's real-time acquisition is considered normal.

[0072] In step S1, the robot's end effector position is fine-tuned along multiple directions (e.g., up / down, left / right, front / back, and combined directions) to move the image acquisition device, acquiring initial images of the workpiece to be cleaned from different camera points and in different robot poses. If the initial image completely covers the entire workpiece area, all holes or grooves within the area are clearly visible, and the workpiece is located in the central area of ​​the initial image, then the position coordinates of each joint of the robot at this time are recorded as a target camera point. If the initial image covers the entire workpiece area, but the holes or grooves appear too small in the image, the workpiece to be cleaned can be divided into several sub-regions, and a suitable target camera point can be determined in each sub-region. Subsequently, a complete workpiece view is obtained through image stitching. Thus, one or more target camera points are recorded.

[0073] Move the robot to the recorded target image point and fine-tune the end effector posture. For example, establish a robot end effector coordinate system T, where the Z-axis direction is consistent with the axis of the end effector cleaning rod, pointing outward from the end of the robotic arm; the X-axis direction is consistent with the X-axis direction of the robot base coordinate system; and the Y-axis is determined by the X and Z axes according to the right-hand screw rule. Define the end of the cleaning rod away from the end of the robotic arm as the cleaning rod tip, and set the origin of the robot end effector coordinate system T at this tip. Control the robot's movement through the robot end effector coordinate system T, aligning the cleaning rod tip with any hole or groove to be cleaned. Then, advance the cleaning rod along the positive Z-axis, maintaining the robot end effector posture during the advancement. If the cleaning rod enters the hole or groove without interference, the current posture is recorded as the image posture corresponding to the target image point. If interference occurs, withdraw the cleaning rod from the hole along the negative Z-axis, adjust the cleaning rod posture appropriately, and try inserting it again until there is no interference. Record the final posture as the image posture corresponding to the target image point.

[0074] To further ensure image accuracy, the orientation of the workpiece to be cleaned and the holes or grooves in the initial image can be checked again to ensure they meet expectations (e.g., no skew). If there is an orientation deviation, the robot is rotated around the end joint to restore the image to the expected orientation, and the target image point and corresponding image posture record are updated.

[0075] The final determined target shooting points and their corresponding shooting postures are combined into a shooting pose matrix, denoted as . ,in, This represents the pose matrix for all target shooting points. A two-dimensional matrix with 6 rows and 6 columns, the first 3 columns are the target shooting point positions, and the last 3 columns are the shooting pose (represented by Euler angles). The number of photo spots, It is a positive integer.

[0076] After obtaining the photographic pose matrix, step S2 drives the robot to move sequentially to the corresponding target photographic points, and uses the image acquisition device to acquire local images of the workpiece to be cleaned at the target photographic point in the photographic pose corresponding to that target photographic point, thereby acquiring several local images. Of course, if there is only one target photographic point, the image covering the entire workpiece area acquired at that target photographic point is the local image.

[0077] Step S3 involves detecting the areas to be cleaned in the acquired local images, for example, detecting the hole areas and / or grooves in the local images, to obtain the position and shape information of all hole areas and groove areas to be cleaned in the workpiece, so that the cleaning rod or detection probe can be accurately positioned subsequently.

[0078] To ensure the cleaning efficiency and effectiveness of holes / grooves of different shapes, sizes and levels of contamination, cleaning strategies are pre-set for different types of holes and their cleaning needs. Step S4 determines the matching target cleaning strategy from multiple preset cleaning strategies based on the shape information of the detected area to be cleaned, and generates an executable trajectory based on the position information of each area to be cleaned. This drives the robot's end effector to perform the cleaning operation of the target cleaning strategy on each area to be cleaned in sequence, avoiding the inefficiency or incomplete cleaning caused by a single mode, and reducing the risk of collision.

[0079] This specific implementation method determines the optimal image-taking position by fine-tuning the robot's end effector posture in multiple directions, combined with image coverage and hole / groove clarity. It also introduces a robot end effector coordinate system and cleaning rod insertion experiments to verify and optimize the image-taking posture, avoiding interference in subsequent cleaning processes and improving the consistency of imaging and positioning. This systematically determines the image-taking point and pose, improving the consistency of image imaging and positioning under different batches and lighting conditions. Furthermore, morphological processing of the area to be cleaned effectively suppresses interference such as edge breakage, adhesion, and reflection, improving detection rate and classification accuracy. Simultaneously, it can dynamically select cleaning strategies based on hole type and contamination level, avoiding the inefficiency or incomplete cleaning caused by a single cleaning strategy. The entire process, from visual perception, parameter optimization, hole recognition and positioning to cleaning execution, is automated, reducing manual intervention, improving efficiency and consistency, and lowering labor costs and operational risks.

[0080] In one specific embodiment, prior to step S2, the workpiece cleaning method further includes:

[0081] S01. Drive the image acquisition device to the preset target aperture value and target focal length value to acquire the first low exposure image and the first high exposure image of the workpiece to be cleaned;

[0082] S02. Input the target aperture value, target focal length value, first low exposure image and first high exposure image into the pre-trained exposure time prediction model to obtain the target exposure time;

[0083] Step S2 includes:

[0084] S21. Acquire a local image of the workpiece to be cleaned using the target aperture value, target focal length value, and target exposure time.

[0085] Specifically, to ensure that all images acquired under different batches or conditions have consistent optical characteristics, optimal camera parameters can be preset. For example, such as Figure 2 As shown, a dotted circle plate is used as a reference to assist in adjusting the camera's aperture and focal length. This plate includes common hole sizes and shapes, such as circles, squares, triangles, pentagons, hexagons, and octagons. The size of the holes is basically consistent with the size of the holes on the workpiece to be cleaned, and the hole diameter range covers ±20% of the hole diameter on the workpiece to be cleaned, so as to ensure that the largest and smallest holes can be clearly seen at the same time when adjusting the aperture / focal length. In addition, the dotted circle plate also has at least three Chinese characters, and the Chinese characters are arranged in a tiered manner from small to large. Among them, the smallest Chinese character in the size gradient can still be clearly distinguished at the selected focal length.

[0086] Securely mount the dotted image plate on the worktable or a dedicated adjustable bracket, ensuring that the plate's plane is parallel to a reference plane in the robot's base coordinate system or positioned at a predetermined angle. The plate should spatially cover the camera's likely field of view for subsequent multi-angle image acquisition. Ensure the plate does not shift or vibrate during camera acquisition; if necessary, use clamps or suction devices for reinforcement. After installation, record the plate's spatial position and orientation in the robot's base coordinate system as a reference for subsequent camera use.

[0087] By connecting the camera and enabling real-time streaming, confirm that the camera and robot control system are communicating normally, and open the real-time image preview window. Then, control the robot to move so that the dot pattern is centered in the camera's field of view. Use the camera's built-in or external focus adjustment mechanism (e.g., manual or motorized zoom lens) to initially adjust the focus, ensuring that the feature patterns on the dot pattern are clearly defined and without significant blur in the image. Adjust the camera aperture according to the ambient lighting conditions. In sufficient light, appropriately reduce the aperture to increase the depth of field; in insufficient light, moderately increase the aperture to increase the amount of light entering the camera. By observing the real-time image, ensure that the overall brightness of the dot pattern is uniform, avoid local overexposure or underexposure, and minimize the shallow depth of field caused by an excessively large aperture, which could lead to some features being out of focus.

[0088] After determining the aperture, fine-tune the focus again to ensure that both the nearest and farthest feature points on the dot matrix remain sharp in the image. This can be achieved by acquiring multiple images and comparing feature point extraction errors to determine if optimal focus has been achieved. The target aperture value after adjustment is then set. With the target focal length value Write it into the camera configuration file or robot control program.

[0089] When it is necessary to acquire several partial images of the workpiece to be cleaned at the target shooting point, the camera's aperture and focal length are adjusted to the predetermined target aperture value. and target focal length value And acquire the first low-exposure image of the workpiece to be cleaned at the work site. And the first high-exposure image Set the target aperture value Target focal length value First low-exposure image And the first high-exposure image Input the pre-trained exposure time prediction model to obtain the optimal target exposure time based on the actual lighting conditions at the work site. .

[0090] Control the camera to the target aperture value Target focal length value and target exposure time Image capture is performed to obtain clear local images of the workpiece to be cleaned and the holes / grooves to be cleaned.

[0091] The exposure time prediction model is constructed using a multimodal fusion neural network, which includes a dual-path convolutional neural network layer, a fully connected network layer, a feature fusion layer, and a regression prediction layer.

[0092] During the pre-training of the exposure time prediction model, a training set is constructed by collecting sample image data of different workpieces under various lighting conditions. Each sample image data includes the camera aperture value. Focal length value Current exposure time Low-exposure images of the same scene With high-exposure images and the optimal exposure time determined by expert evaluation. As a label.

[0093] For each sample image data in the training set , Perform size standardization and normalization processing. , , Numerical normalization or logarithmic transformation is performed, along with enhancement operations such as random brightness / contrast adjustment and noise addition, to improve the model's generalization ability.

[0094] Preprocessed , , , , Input to a multimodal fusion neural network, and use dual-path convolutional neural network layers to respectively... , Feature encoding is performed to extract high-dimensional visual features from the image; fully connected network layers are then used to... , , The system encodes visual features to obtain numerical feature vectors. These features are then concatenated using a feature fusion layer or fused via an attention mechanism to obtain fused features. Finally, a regression prediction layer outputs the predicted exposure time. .

[0095] Meanwhile, a clarity-related loss function (Clarity Loss) and an exposure regression loss (MSE Loss) are introduced to jointly optimize the multimodal fusion neural network, as shown in Equation (1):

[0096] (1)

[0097] in, This indicates that the regression prediction layer outputs the predicted exposure time. This indicates the optimal exposure time determined by expert evaluation. It is the mean square error between the predicted exposure time and the optimal exposure time, used to measure the accuracy of the exposure time prediction. It is a sharpness-related loss, used to measure the difference between the sharpness of an image captured at the predicted exposure time and the ideal sharpness. It is a weighting factor ( This is used to balance the relative importance of the two losses.

[0098] During pre-training, the exposure time prediction model can be optimized by jointly optimizing the sharpness-related loss function and the exposure regression loss, so that it can both predict reasonable exposure time and improve the image sharpness under the predicted exposure.

[0099] It should be noted that during the training process, the exposure time prediction model can be pre-trained on a large-scale general industrial image or natural image dataset. Then, on a small sample dataset of the target workpiece, the first few layers of the backbone network can be frozen, and only the feature fusion layer and regression prediction layer can be fine-tuned to reduce the risk of overfitting.

[0100] This specific implementation method uses a dot-circle plate to perform multi-feature optical adjustment on the image acquisition device and uses an exposure time prediction model to automatically optimize the parameters of the image acquisition device, overcoming the imaging fluctuations caused by traditional manual adjustment and ensuring the image clarity and uniformity of different batches and under different lighting conditions.

[0101] In one specific embodiment, prior to step S3, the workpiece cleaning method further includes:

[0102] Obtain the quality assessment parameters of the local image, and determine the image quality score based on the quality assessment parameters;

[0103] In response to an image quality score being less than a preset threshold, the image acquisition device is driven to adjust to the target aperture value and target focal length value, and multiple second low-exposure images and second high-exposure images of the workpiece to be cleaned are acquired.

[0104] In response to receiving the exposure time labels corresponding to the second low-exposure image and the second high-exposure image, the exposure time prediction model is fine-tuned and trained using the second low-exposure image and the second high-exposure image as input and the corresponding exposure time labels as output, and then the process returns to step S01.

[0105] Specifically, to enable the exposure time prediction model to accurately predict the exposure time for new workpiece types, after capturing local images on-site, manual visual inspection or image quality detection algorithms can be used to determine whether the imaging quality of the local images meets the requirements. For example, an image quality score for the local image can be calculated based on indicators such as hole edge sharpness, overall brightness uniformity, and detail visibility. If the image quality score is lower than a set threshold, it is determined that the imaging quality of the local image is substandard, and a rapid fine-tuning process is triggered.

[0106] For new workpiece types, a pre-set target aperture value can be used under the same or similar lighting conditions. and target focal length value Take s images of the second low exposure. Second highest exposure image Typically, 3-10 images are sufficient, and the corresponding current exposure time should be recorded. Then, each second low-exposure image was determined by expert evaluation. Second highest exposure image The corresponding optimal exposure time , as labels for fine-tuning training, and divide 1-2 groups of samples from the s groups of samples as validation sets, and the other groups of samples constitute the fine-tuning training set.

[0107] The deployed exposure time prediction model is fine-tuned using a fine-tuning training set and optimized using the same loss function combination as the pre-training process. The learning rate can be appropriately increased to accelerate convergence. The training is iterated for several epochs (rounds), such as 5–20 times. During the iterative training, the deviation between the predicted exposure time and the optimal exposure time on the validation set is monitored. Training can be stopped once the deviation decreases to an acceptable range or the loss tends to stabilize.

[0108] It should be noted that during fine-tuning training, depending on the available computing resources, you can choose to freeze the first few layers of the backbone network, keep the weights of the first few layers unchanged, and only unfreeze the feature fusion layer and regression prediction layer for fine-tuning training; alternatively, you can choose to appropriately unfreeze a few layers close to the output based on the differences in the features of the new workpiece, in order to balance adaptability and prevent overfitting.

[0109] Of course, in the second low-exposure image Second highest exposure image Current exposure time Before inputting the exposure time prediction model, the same preprocessing operations as the pre-training process can be performed, including size unification, normalization, brightness / contrast adjustment, and noise addition. Simultaneously, the target aperture value can be adjusted. Target focal length value and optimal exposure time Perform the same numerical normalization or logarithmic transformation as in the pre-training process to maintain data distribution consistency.

[0110] After completing the fine-tuning training, immediately apply the target aperture value on-site. and target focal length value Re-capture the first low-exposure image and the first high-exposure image, and use the finely tuned exposure time prediction model to predict the target exposure time, with the target aperture value. Target focal length value Local images are acquired at the target exposure time to check if the imaging quality of the local images has improved. If the requirements are still not met, a second low-exposure image and a second high-exposure image are acquired, or the freeze strategy is adjusted to fine-tune the exposure time prediction model until the quality of the acquired local images meets the requirements.

[0111] For fine-tuned models that have proven effective in the field, they can be saved as a dedicated version for that workpiece type, with the workpiece model, lighting conditions, and fine-tuning date noted. In subsequent applications, different versions of the model can be switched to adapt to production scenarios with multiple workpiece types.

[0112] This specific implementation design incorporates a dot-circle plate with various aperture shapes and Chinese character gradients to achieve rapid matching of optical characteristics such as aperture and focal length. It also combines a multimodal fusion network of dual-path image features and numerical parameters to predict the optimal exposure time under small sample conditions, supporting rapid on-site fine-tuning and model version management, and ensuring stable imaging quality under different lighting conditions and new workpiece types.

[0113] In one specific implementation, step S3 includes:

[0114] S31. Stitch together each local image to obtain a global image, and convert the global image into a global grayscale image;

[0115] S32. The global grayscale image is binarized using the root mean square error adaptive threshold to obtain the candidate region to be cleaned.

[0116] S33. A pre-trained semantic segmentation model is used to perform pixel-level classification of the candidate regions to be cleaned, and a mask of the region to be cleaned is obtained.

[0117] S34. Perform morphological processing and shape analysis on the mask of the area to be cleaned to obtain the location and shape information of the area to be cleaned.

[0118] Specifically, after acquiring local images that meet the image quality requirements, each local image is stitched together to obtain a global image of the complete workpiece view. The global image is then converted into a global grayscale image for subsequent shape and position recognition of the area to be cleaned.

[0119] A two-stage object detection algorithm is employed to extract the mask of the region to be cleaned. First, an adaptive threshold based on root mean square error is used to separate regions in the global grayscale image whose brightness is significantly lower or higher than the background into binary images, initially obtaining candidate regions for cleaning, such as hole candidate regions. Then, a pre-trained semantic segmentation model (such as U-Nek (a convolutional neural network for image segmentation), Mask R-CNN (an instance segmentation algorithm), etc.) is used for pixel-level classification, outputting the mask of the region to be cleaned. This two-stage object detection algorithm, through coarse and fine steps to extract the mask of the region to be cleaned, accelerates the entire extraction process while ensuring that regions of different sizes can be accurately distinguished and extracted.

[0120] After generating the mask for the area to be cleaned (taking a hole region as an example), due to factors such as uneven lighting in industrial environments, workpiece surface reflection, and image noise, isolated small noise points, broken hole edges, and adhesion between adjacent holes often appear in the mask, affecting subsequent contour extraction and shape analysis. Therefore, based on the shape characteristics, size distribution, and projection patterns of the area to be cleaned in the image, a morphological processing method optimized for hole / groove recognition is formed by using specific structural element shapes and sizes and a conditional combination strategy, effectively improving mask quality and subsequent recognition accuracy.

[0121] Among them, isolated small noise spots refer to falsely detected patches with an area much smaller than the actual hole size, which are easily misjudged as microholes during the contour analysis stage; hole edge breakage refers to the fragmentation of the hole contour in the image due to discontinuous edge detection response or loss of weak edges by threshold segmentation; adjacent hole adhesion refers to the mask incorrectly merging multiple holes into a single connected region when the hole spacing is small or the projection overlaps.

[0122] After morphological processing of the masked area to be cleaned, some pseudo-regions may still exist, which are not connected regions of real holes / grooves. These are closed regions formed by background textures, lighting spots, surface scratches, or defects with abnormal shapes. If these pseudo-regions enter the subsequent contour extraction and shape analysis stages, they will lead to misjudgments of the number, location, and shape of holes / grooves, thus affecting the accuracy of robot cleaning or inspection. Therefore, a pseudo-region filtering method based on hole morphology priors is used. By quantitatively calculating the geometric features of connected regions and comparing them with workpiece design parameters or empirical statistical ranges, pseudo-regions that do not conform to the characteristics of holes / grooves are eliminated, thereby obtaining a high-confidence set of hole / groove regions.

[0123] Based on the geometric features obtained from the analysis, the image center coordinates of each hole / groove area can be determined, thereby determining the position and shape information of each hole / groove area, enabling the robot to accurately locate and perform operations on the area to be cleaned.

[0124] This specific implementation adopts a two-stage target detection algorithm, which extracts candidate regions in two steps: coarse (coarse screening with grayscale threshold) and fine (fine screening with semantic segmentation). This can accelerate the entire extraction process while ensuring that regions of different sizes can be accurately distinguished and extracted, thereby improving the extraction efficiency and accuracy of candidate regions.

[0125] In one specific embodiment, step S34 includes:

[0126] S341. Perform adaptive erosion, directional closing operation, and conditional opening operation sequentially on the mask of the area to be cleaned to obtain the target mask;

[0127] S342. Mark the target mask with connected components to obtain multiple connected regions;

[0128] S343. Obtain the geometric features of each connected region, and select connected regions that meet the preset morphological conditions as regions to be cleaned based on the geometric features.

[0129] S344. Extract the outline of each area to be cleaned, and determine the shape of each area to be cleaned and the center coordinates of each area to be cleaned in the global image based on the outline.

[0130] S345. Based on the transformation relationship between the coordinate system of the image acquisition device and the robot base coordinate system, the center coordinates are converted into the target three-dimensional coordinates under the robot base coordinates.

[0131] Specifically, morphological processing includes adaptive erosion for noise removal, directional closing operation for edge fracture repair, and conditional opening operation for adjacent hole separation.

[0132] In adaptive erosion, the pixel area of ​​each connected region in the mask of the area to be cleaned is statistically analyzed, and the average area is calculated. with standard deviation The upper limit of the noise area is determined based on the mean and standard deviation of the area. ,in, This is a preset coefficient, usually set to 1.5~2, to select pixels with an area smaller than... The connected components are treated as isolated noise points, and then erosion is performed using a structuring element that matches the area to be cleaned. After erosion, pixels with an area smaller than [a certain value] are removed. The first mask can be obtained by identifying the connected components. For example, a circular structuring element can be used for the hole region. Corrosion occurs, where the radius... Based on the radius of the minimum real hole Set proportionally, such as setting To avoid eroding the main area of ​​the actual hole, pixels smaller than [a certain size] are removed after etching. The connected components are used to obtain the first mask corresponding to the hole region mask.

[0133] In directional closure operations, the major axis orientation angle of the hole / groove is determined using principal component analysis (PCA). Construct elliptical structural elements aligned with the major axis. ,in, The average major and minor axis pixel lengths of the holes / grooves are respectively used, and then the first mask is applied. Perform a directional closing operation (dilation followed by erosion), using elliptical structuring elements to fill edge fractures along the major axis while maintaining boundary sharpness along the minor axis, to obtain a second mask. It's understandable that if the area to be cleaned is a circular hole, then... , .

[0134] In the conditional decomposition operation, the second mask is calculated. The centroid spacing of the connected regions, and the lower limit of the preset hole / groove spacing. Compare; if the spacing of connected components is less than Therefore, slender strip-shaped structural elements are used. An opening operation is performed to sever the adherent neck, wherein the length Slightly larger than the hole / groove spacing, width W is much smaller than the hole / groove diameter, direction Perpendicular to the direction of the hole connection; after completing the opening operation, again based on the upper limit of the noise area. Area filtering is performed to remove tiny residual fragments caused by cutting, resulting in the target mask. .

[0135] After obtaining the target mask Next, target mask Perform connected component labeling to obtain m connected regions. .

[0136] For the i-th connected region Extract its target mask Combined with two-dimensional boundary information, and calculation of each connected region. area The aspect ratio of the minimum bounding box Iso-geometric features. Among them, the target mask. For a binary image, only pixels within the connected region are retained as the foreground; the 2D boundary information is a sequence of 2D boundary points or the minimum bounding rectangle. Area Based on connected regions The total number of pixels is calculated based on the camera calibration results, converted to the actual physical area (e.g., in square millimeters mm²); aspect ratio. Based on the length of the longest side of the smallest bounding rectangle With the length of the shorter side The calculation yielded: .

[0137] Based on a pre-set area range Aspect Ratio Range For each connected region in turn Perform filtering. If... or Then determine the connected region. If it is a pseudo-region, it will be removed; ,or Then determine the connected region. False regions are removed.

[0138] If the area to be cleaned in the workpiece is a circular hole / groove, each connected region can also be calculated. roundness Roundness Based on area and connected regions Perimeter of the area boundary The calculation yielded: A roundness close to 1 indicates a connected region. Approximately circular. Based on a preset roundness range. For each connected region in turn Perform filtering, if Then determine the connected region. Non-circular holes or irregular regions are removed. Of course, connected regions can also be calculated. If the eccentricity exceeds the design range of the hole type, or if the unevenness of the area boundary is abnormal, it will be rejected.

[0139] Among them, area range The actual physical area of ​​the region to be cleaned can be determined based on the workpiece design drawings or historical data. And convert it into the image pixel area range. This area range excludes regions significantly smaller than micropores or larger than non-porous structures (such as solid stains); aspect ratio range It can be designed according to the shape of the area to be cleaned (e.g., a circular hole). elliptical hole By setting a reasonable aspect ratio, this range can exclude fine cracks, scratches, or blocky reflective spots. Area range Aspect Ratio Range It can be pre-stored in the workpiece configuration file and automatically loaded by the host computer or robot controller when switching workpiece types; for new workpieces being processed for the first time, a small number of samples can be collected on-site, and the results can be automatically estimated through statistical analysis. , , , These parameters enable rapid adaptation and improve versatility.

[0140] Connected regions that pass all filtering conditions Forming a collection of areas to be cleaned ,in, This ensures that subsequent processing targets only high-confidence genuine hole / recess areas, for each area to be cleaned. Includes target mask Two-dimensional boundary point sequences or contour information, geometric feature parameters , , , , wait.

[0141] A collection of areas to be cleaned with high confidence. Next, each area to be cleaned is cleaned first. Perform outer boundary contour extraction to obtain the pixel-level geometric contours of the holes / grooves in the local image. This is achieved by... (The sentence is incomplete and requires more context to translate accurately.) Target mask As input, a contour extraction algorithm (e.g., findContours in OpenCV, a core function for extracting contours from binary images) is invoked to perform boundary tracing, and the extracted sequence of outer boundary points is denoted as... ,in, This represents the pixel coordinates of the j-th contour point in the global image coordinate system. This represents the number of points on the contour. The outer boundary point sequence is then ordered to ensure the point order is continuous along the boundary in a counter-clockwise (or clockwise) direction, facilitating subsequent geometric fitting and analysis.

[0142] Next, the sequence of outer boundary points Geometric fitting and feature analysis are performed to determine the specific shape category of the hole / groove. Different discrimination strategies are used for different hole / groove shapes.

[0143] The discrimination strategy for circular / near-circular holes / grooves: outer boundary point sequence Perform least-squares ellipse fitting to obtain the length of the major axis of the ellipse. minor axis length and major axis direction angle Calculate the ratio of major to minor axis. ,like If the area to be cleaned is determined to be a circular or near-circular hole / groove, then the orientation angle is set to 0. This discrimination strategy avoids misclassifying a slightly elliptical hole as an elliptical one by tolerating a certain degree of imaging distortion or edge extraction error.

[0144] The discrimination strategy for elliptical holes / grooves: If If the area to be cleaned is determined to be an elliptical hole / groove, the length of its major axis is recorded. minor axis length and major axis direction angle It is used to calculate the direction of three-dimensional axes.

[0145] The discrimination strategy for polygons / irregular holes / grooves: sequence of outer boundary points. Calculate the convex hull and count the number of vertices in the convex hull. ,like Furthermore, if the distribution of the inner angles of the contour does not conform to the smoothness characteristics of an elliptical model, it is initially determined to be a polygonal or free-form hole / groove. Based on the matching results of the convex hulls of each shape, the area to be cleaned is further subdivided into triangular holes / grooves, rectangular holes / grooves, polygonal holes / grooves, or irregular irregular holes / grooves.

[0146] After the shape category is determined, each area to be cleaned... Calculate a set of geometric parameters for positioning and operation, including for each area to be cleaned. Center coordinates / centroid, diameter / major and minor axis lengths, orientation angle ,area Roundness wait.

[0147] Wherein, center coordinates / centroid In the global image coordinate system, through the target mask The result is shown in formula (2):

[0148] (2)

[0149] In formula (2) This represents the pixel coordinates of any contour point in the global image coordinate system. Represents the x-coordinate of the contour points. Represents the ordinate of the contour point.

[0150] For circular or near-circular holes / grooves, the diameter / major and minor axis lengths are taken as the average diameter. .

[0151] For elliptical holes / grooves, the major axis a and minor axis b are recorded respectively to characterize the size and projected deformation of the hole / groove.

[0152] Clean each area The analysis results are organized as structured data, including shape categories (circles, ellipses, polygons, irregular shapes, etc.), center coordinates / centroid. Dimensions Direction angle Roundness Compared with other shape metrics (such as eccentricity).

[0153] After obtaining each area to be cleaned After obtaining the structured data, the centroid and shape information of the area to be cleaned on the global image plane are mapped to the robot's base coordinate system to obtain the target three-dimensional coordinates and direction vectors that can be directly used to control the robot's movement, thereby enabling the robot to accurately locate and perform tasks (such as cleaning, inspection, or assembly) on the area to be cleaned.

[0154] This specific implementation first separates circular / near-circular holes from elliptical holes using least-squares ellipse fitting and major / minor axis ratio. Then, it calculates the number of convex hull vertices and interior angle distribution for suspected polygonal holes to identify polygonal features. Finally, it combines shape descriptors such as contour angle sequences with template matching to further subdivide polygonal and free-form holes. It also introduces morphological processing and shape analysis, such as cross-validation of roundness and eccentricity, to stably identify circular holes, elliptical holes, polygonal holes, and irregular holes. It makes full use of hole geometry priors and maintains high recognition and classification accuracy even under complex working conditions such as noise, reflection, edge breakage, and adhesion. It is significantly better than traditional single-step discrimination methods. It solves the problem that traditional hole recognition methods are mostly based on single ellipse fitting or fixed template matching, which can only identify circular or very few hole types and are difficult to adapt to the mixing of multiple hole types in industrial sites. It can also adjust the insertion strategy according to the roundness of the hole in subsequent robot operation planning.

[0155] In one specific embodiment, step S345 includes:

[0156] S3451. Obtain the intrinsic parameter matrix and distortion coefficients of the image acquisition device;

[0157] S3452. Perform distortion correction on the center coordinates based on the distortion coefficients to obtain normalized image coordinates;

[0158] S3453. Obtain the depth value of the area to be cleaned, and perform back projection processing on the normalized image coordinates based on the depth value and the intrinsic parameter matrix to obtain the initial three-dimensional coordinates in the coordinate system of the image acquisition device.

[0159] S3454. Based on the transformation relationship, convert the initial three-dimensional coordinates into the target three-dimensional coordinates.

[0160] Specifically, in the two-dimensional coordinates Before converting to 3D coordinates, it is necessary to obtain the depth value of the centroid in the coordinate system of the image acquisition device. Among them, depth value Different acquisition methods can be used depending on the type of image acquisition device. For example, with binocular stereo vision or structured light cameras, the depth value corresponding to the center pixel position can be directly read from the depth map output by the camera. (Unit: mm) This value has been calibrated in the coordinate system of the image acquisition device, with an accuracy of sub-millimeter level. This method offers strong real-time performance and high precision, making it suitable for dynamic scenes. However, for monocular cameras, depth information cannot be directly obtained. Therefore, if the vertical distance between the workpiece surface and the camera's optical center is known (determined by the workpiece design drawings or installation positioning), this distance can be directly used as the depth value. Otherwise, the robot's end effector drives the camera to capture images of the same hole / groove from at least two different poses. These two poses are translated a certain distance in the base coordinate system along a direction perpendicular to the hole / groove axis to ensure sufficient parallax. Then, using the pixel displacement of the hole / groove center in the image and the robot's pose change, the depth value is calculated using triangulation formulas. This method is suitable for static workpieces and allows for a certain amount of data acquisition time.

[0161] Obtaining depth values Next, the image pixel coordinates of the hole / groove center need to be... Convert to initial 3D coordinates in the image acquisition device coordinate system The process includes distortion correction and pinhole model backprojection using the pre-acquired intrinsic parameter matrix K and distortion coefficients distCoeffs of the image acquisition device. The intrinsic parameter matrix K and distortion coefficients distCoeffs of the image acquisition device can be pre-obtained by selecting a checkerboard calibration board of known size (e.g., 9×6 with known corner points and square side lengths). The checkerboard calibration board is stably placed in multiple different poses and positions within the camera's field of view, ensuring coverage of different depths and angles of the workspace. The robot is controlled to move or fix the camera, acquiring at least 15 clear images containing the checkerboard calibration board, where the checkerboard calibration board is fully visible in the acquired images and the corner points are evenly distributed. Corner detection algorithms (e.g., OpenCV's findChessboardCorners (a function for detecting internal corner points in checkerboard images)) are used to extract the checkerboard corner coordinates of each image, and subpixel optimization (e.g., cornerSubPix (a computer vision function for detecting subpixel-level corners)) is used to improve corner localization accuracy. Call the calibration function (such as OpenCV's calibrateCamera, a function for camera calibration), input the world coordinates and pixel coordinates of the corners of each image, and calculate the camera's intrinsic parameter matrix. And the distortion coefficient distCoeffs, as shown in formulas (3)-(4):

[0162] (3)

[0163] in, This represents the focal length in the x-direction. This represents the focal length in the y-direction. This indicates the principal point coordinates. The principal point refers to the intersection of the camera's optical axis and the image plane.

[0164] (4)

[0165] in, Represents the radial distortion coefficient. This represents the tangential distortion coefficient.

[0166] After obtaining the intrinsic parameter matrix After calculating the distortion coefficients (distCoeffs), the calibration accuracy can be evaluated by calculating the reprojection error. If the average reprojection error is below a preset threshold (e.g., 0.2 pixels), the calibration result is considered reliable; otherwise, a clear image containing the checkerboard calibration board is reacquired or the quality of the calibration board is checked until the average reprojection error is below the preset threshold. The final intrinsic parameter matrix is ​​then used. The distortion coefficients distCoeffs are written into the camera configuration file or robot control program to ensure that uniform camera model parameters are used throughout the operation.

[0167] Then, based on the pinhole camera model and the Brown-Conrady distortion model, Distortion correction is performed to obtain normalized image coordinates. As shown in formula (5):

[0168] (5)

[0169] Based on the intrinsic parameter matrix K, the three-dimensional coordinates of the center point in the coordinate system of the image acquisition device are obtained by back projection using the pinhole model. As shown in formula (6):

[0170] (6)

[0171] Based on the pose calibration of the image acquisition device and the robot in their respective base coordinate systems, the transformation matrix between the image acquisition device coordinate system and the robot's base coordinate system can be obtained. The transformation matrix It can be a 4×4 homogeneous transformation matrix, including rotation matrices. With translation vector Next Constructed as homogeneous coordinates Then, the center coordinates of the hole / groove in the robot's base coordinate system are as shown in formula (7):

[0172] (7)

[0173] Pick The first three elements serve as the target 3D coordinates of the hole / groove center in the robot's base coordinate system. This allows for the transformation between the initial three-dimensional coordinates and the target three-dimensional coordinates, enabling subsequent path planning and motion control of the robot.

[0174] It should be noted that the target 3D coordinates and shape information of each hole / groove can be organized into structured data (e.g., array, matrix, or JSON object), and include the shape category: circular / elliptical / polygonal / irregular hole; image center coordinates: Target 3D coordinates: Direction vector: Dimensional parameters: diameter or major and minor axis lengths, area, roundness, eccentricity, etc.; confidence score (which can be comprehensively evaluated based on fitting residuals, filtering scores, etc.). The above structured data is displayed on the human-machine interface image for manual verification and transmitted to the robot controller via the communication system as input for path planning and task execution.

[0175] This specific implementation ensures that uniform camera model parameters are used throughout the entire operation by pre-calculating the intrinsic parameter matrix and distortion coefficients of the image acquisition device.

[0176] In one specific implementation, step S4 includes: in response to receiving a cleaning strategy setting instruction, determining a matching target cleaning strategy according to the cleaning strategy setting instruction;

[0177] or,

[0178] Step S4 includes: in response to not receiving a cleaning strategy setting instruction, determining the diameter parameter of each area to be cleaned based on the shape information;

[0179] If the diameter parameter is greater than the preset diameter, the target cleaning strategy is determined to be a spiral progressive cleaning strategy or a composite cleaning strategy; or, if the diameter parameter is less than or equal to the preset diameter, the target cleaning strategy is determined to be a straight-in, straight-out rapid cleaning strategy.

[0180] Specifically, to ensure that different hole types have suitable cleaning modes to guarantee overall cleaning effectiveness, multiple cleaning modes can be preset according to the cleaning needs and goals of different hole types (such as cleaning speed, cleaning effect, and balancing cleaning speed and effect). For example, three cleaning modes can be preset: a straight-in-straight-out rapid cleaning strategy, a spiral progressive cleaning strategy, and a composite cleaning strategy.

[0181] The straight-in, straight-out rapid cleaning strategy is designed for scenarios with high cleaning speed requirements, regular hole shapes, and light contamination, such as batch processing of standard through holes or maintenance tasks after pre-cleaning. In this cleaning mode, a straight path is planned based on the determined three-dimensional coordinates and direction vector of the area to be cleaned, as shown in formula (8):

[0182] (8)

[0183] in, This represents the target's three-dimensional coordinates in the area to be cleaned. Let t represent the unit direction vector and t represent the time step. This indicates the preset cleaning depth.

[0184] Based on the planned straight path and constant feed rate The cleaning rod mounted on the end of the drive robot moves linearly along the axis of the hole into the opening, reaching the preset cleaning depth. Then, it exits along the original path, and its trajectory in the robot's base coordinate system is a straight line segment. The constant feed rate... It can achieve a higher cleaning speed than usual, thereby shortening the single-hole operation time, and keeping the tool perpendicular to the end face of the hole during entry and exit, reducing lateral friction and the time spent on posture adjustment.

[0185] The straight-in-straight-out rapid cleaning strategy can significantly improve cleaning efficiency, reduce the overall operation cycle, and is simple to control. It is suitable for rapid processing of large batches of low-complexity holes. In addition, it has low requirements for robot motion control and is easy to achieve stable and repetitive operation.

[0186] The spiral progressive cleaning strategy is designed for scenarios with high requirements for cleaning effect, where there may be stubborn contaminants or irregular hole shapes (such as ellipses or polygons) inside the hole, such as precision parts maintenance or pre-assembly treatment with high cleanliness requirements. In this mode, based on the diameter D and depth of the area to be cleaned, the upper limit of the spiral radius Rmax ≤ D / 2 is set to generate a spiral trajectory (cylindrical coordinate system), as shown in formula (9):

[0187] (9)

[0188] in, Indicates the starting radius (can be zero to indicate entry from the center). Indicates the control of radial spread rate, This indicates the control of the axial feed rate. This indicates the rotation angle around the axis of the hole.

[0189] Based on the screw feed speed The robot's end effector moves in a spiral trajectory, gradually penetrating deeper into the hole, allowing the cleaning tool to cover a larger area of ​​the hole's inner wall and progressively remove the attached material. The spiral feed speed... A rapid cleaning strategy with a straight-in, straight-out flow can be used to ensure media action time and coverage.

[0190] The spiral progressive cleaning strategy can prevent collisions with the hole wall, and the tool posture can be controlled synchronously during the spiral process to ensure that its axis always coincides with the hole axis, thereby improving the uniformity of cleaning.

[0191] The composite cleaning strategy is designed for scenarios that require both a certain cleaning speed and a high level of cleaning quality, such as workpieces with a mixture of different pore sizes, or production lines that have requirements for both cycle time and cleanliness. In this mode, a combination of a straight-in, straight-out rapid cleaning strategy and a spiral progressive cleaning strategy can be achieved according to a predetermined sequence or conditions to balance speed and effectiveness.

[0192] For example, a rapid cleaning strategy of straight in and straight out can be used to quickly reach the shallow cleaning depth. First, loose contaminants are removed; then, a spiral, progressive cleaning strategy is adopted to reach the final depth. It thoroughly removes stubborn deposits; finally, it exits using a straight-in, straight-out rapid cleaning strategy to reduce ineffective dwell time.

[0193] Specifically, it can automatically generate composite trajectory parameters based on the size and contamination level of the area to be cleaned before operation, and smoothly transition at trajectory switching points to avoid vibration or tool deviation caused by sudden changes in robot acceleration. Furthermore, it can adaptively adjust the spiral coverage range for different hole types to ensure full coverage.

[0194] Once the target three-dimensional coordinates and shape information of each hole / groove are obtained and organized into structured data, corresponding cleaning strategies can be executed for different types of holes and their cleaning requirements.

[0195] The corresponding cleaning strategy can be automatically selected based on the shape and size of the area to be cleaned. When the diameter parameter of the area to be cleaned is greater than the preset diameter, the target cleaning strategy is determined to be a spiral progressive cleaning strategy or a composite cleaning strategy. When the diameter parameter is less than or equal to the preset diameter, the target cleaning strategy is determined to be a straight-line in and straight-line out rapid cleaning strategy.

[0196] Of course, users can also manually select the corresponding cleaning strategy through cleaning strategy setting instructions. For example, users can manually select a straight-in-straight-out cleaning strategy based on the cleaning speed priority, or a spiral progressive cleaning strategy based on the cleaning effect priority, or a compound cleaning strategy based on the cleaning effect balance.

[0197] After determining the cleaning strategy, an executable trajectory can be generated based on the target's three-dimensional coordinates and direction vector, and collision detection and safety verification can be performed before driving the actuator to complete the cleaning operation.

[0198] This specific implementation method is based on the selection and execution of intelligent cleaning strategies according to hole shape and working conditions, and provides three cleaning modes: linear fast, spiral progressive, and compound, which are respectively suitable for speed-priority, effect-priority, and balanced scenarios. It can automatically select or combine cleaning strategies according to the shape of the hole and the user-set goals.

[0199] In a specific example of cleaning deep circular holes using a 6-axis collaborative robotic arm and a monocular camera, a 6-axis collaborative robotic arm is used. The robot's end effector rigidly mounts a monocular industrial camera (1920×1080 resolution, 30fps) and a cleaning rod (12mm diameter, 300mm length) via a flange. The workpiece to be cleaned has 12 circular holes, approximately 200mm deep and 16mm in diameter. The hole axes are perpendicular to the workpiece surface and are concentrated in one area, allowing for complete coverage of all holes from a single camera position. Figure 3 As shown.

[0200] First, camera installation and real-time data acquisition verification were completed in step S1. Through multi-directional fine-tuning of the robot's end effector, it was found that placing the end effector in a fixed position was sufficient to ensure the camera's field of view completely covered the workpiece, and all 12 holes were clearly visible in the image and located in the center of the frame, eliminating the need for sub-regions. Therefore, this position was recorded as the unique target image point. Next, a robot end effector coordinate system was established (Z-axis outward along the cleaning rod axis), and the front end of the cleaning rod was aligned with one of the holes. It was advanced approximately 200mm along the Z-axis, ensuring no interference during insertion. This posture was recorded as the image-taking posture. Simultaneously, the image orientation was checked for skewness, and the image pose matrix was output. .

[0201] Next, in step S2, use a dot plate (including circular, square, and other hole shapes, with varying Chinese character sizes) to install and adjust the aperture and focal length, ensuring that both the nearest and farthest feature points on the dot plate are sharp, and save the target aperture value. With the target focal length value The system sets camera parameters and acquires low-exposure / high-exposure images under different lighting conditions. It then labels the optimal exposure time (expert-defined) and trains a multimodal fusion network to achieve small-sample exposure presets. Finally, it drives the robot back to the single target image point and captures images of the workpiece using the determined camera parameters.

[0202] Step S3 uses grayscale thresholding for initial screening and U-Net semantic segmentation to obtain 12 candidate hole regions. Morphological processing is then applied to these candidate regions to remove noise and repair edges. Ellipse fitting yields a major-to-minor axis ratio of ≈1 and a roundness of [missing information]. If the value is close to 1, all holes are determined to be circular, and the centroid and diameter of the holes are calculated. Since the monocular camera knows the workpiece surface height, it is directly used as the depth value. The centroid is distorted and back-projected using the pre-calibrated camera intrinsic parameter matrix K and distortion coefficients to obtain the initial three-dimensional coordinates. Then, the coordinates are converted to the robot base coordinates through hand-eye calibration, and the target three-dimensional coordinates and orientations of the 12 holes are output.

[0203] Because of the regular hole shape and light contamination, step S4 adopts a straight-line inlet and straight-line outlet rapid cleaning strategy, and sets the feed rate. The preset cleaning depth is 200mm. It enters and exits in a vertical position to complete batch cleaning.

[0204] This specific example demonstrates that when a single image point can cover all target holes, image point selection, parameter optimization, multi-hole type identification, and efficient cleaning can be completed quickly, making it suitable for small and medium-sized workpieces with concentrated hole distribution.

[0205] In another specific example of cleaning a large workpiece with multiple holes using a dual-arm humanoid robot and an RGB-D binocular camera, a dual-arm humanoid robot is used. Each arm is a 7-axis collaborative robotic arm, with a gripper at the end holding a cleaning rod (15mm in diameter and 300mm in length). The vision system is an RGB-D binocular camera (1280×720 resolution). The workpiece to be cleaned is large, requiring the cleaning of 30 circular holes, approximately 400mm deep and 20mm in diameter. The holes are distributed in different areas of the workpiece, and a single camera position cannot cover all the holes. Figure 4 As shown.

[0206] First, step S1 completes the installation and flow verification of two cameras (mounted at the ends of the left and right arms). Due to the large size of the workpiece, after fine-tuning the robot's end effector in multiple directions, it was found that four imaging positions are needed to fully cover the 30 holes. Each position's field of view covers a portion of the holes for subsequent image stitching. Specifically, the left arm camera captures the left half of the workpiece, requiring two imaging positions; the right arm camera captures the right half of the workpiece, requiring two imaging positions. Therefore, the joint coordinates of each position are recorded as the target imaging point location, and a cleaning rod is inserted to verify the holes (due to the depth of the holes, it is pushed in by approximately 80mm to ensure time efficiency) to ensure no posture interference. This is recorded as the imaging posture, resulting in the imaging pose matrix. .

[0207] Next, in step S2, aperture, focal length, and exposure presets are performed using a dot-matrix plate and a multimodal network. Since dual-arm operation requires consistent imaging between the left and right arms, calibration and parameter synchronization are performed on the two end-effector cameras. The intrinsic parameter matrix K and distortion coefficients are calibrated independently for each camera to ensure depth map accuracy. The robot's left and right arms are then moved sequentially to four target image points to capture images, resulting in four local images.

[0208] Step S3 involves a two-stage target detection algorithm to extract the area to be cleaned and perform morphological processing on each image, resulting in local hole candidate regions. Ellipse fitting and roundness checks are then used to determine that all holes are circular, and the centroid and diameter are extracted. Since the RGB-D camera directly provides the depth map, the depth value at the hole center is read. The initial three-dimensional coordinates are obtained by distortion removal and back projection, and then converted to robot base coordinates by their respective hand-eye calibration. The four sets of hole position coordinates are converted to a unified base coordinate system and stitched together to form a complete set of three-dimensional coordinates and orientations of the 30-hole target.

[0209] Due to the depth of the hole and the potential presence of stubborn contaminants, step S4 employs a spiral progressive cleaning strategy, setting the upper limit of the spiral radius Rmax ≤ 10mm, and the feed rate... This ensures complete cleaning of the inner wall. The dual arms work in tandem, with the left arm cleaning one side of the holes and the right arm cleaning the other, improving overall efficiency.

[0210] This specific example demonstrates that in scenarios involving large workpieces and widely distributed holes, the present invention can achieve complete hole recognition through the selection of multiple shooting points and image stitching, and directly acquire depth information using an RGB-D binocular camera to improve positioning accuracy, while the dual-arm collaboration further enhances cleaning efficiency.

[0211] This embodiment improves the consistency of image imaging and positioning by systematically determining the imaging points and poses under different batches and lighting conditions. Morphological processing of the area to be cleaned effectively suppresses interference such as edge breakage, adhesion, and reflection, improving detection and classification accuracy. Simultaneously, it can dynamically select cleaning strategies based on hole shape and contamination level, avoiding inefficiency or incomplete cleaning caused by a single cleaning strategy. The entire process, from visual perception, parameter optimization, hole recognition and positioning to cleaning execution, is automated, reducing manual intervention, improving efficiency and consistency, and lowering labor costs and operational risks. Furthermore, the imaging points, image acquisition parameters, and models can all be switched according to the workpiece, making it suitable for various robots and possessing good versatility and scalability.

[0212] Example 2

[0213] In one specific embodiment of this disclosure, a robot-based workpiece cleaning system is provided, wherein the robot is equipped with an image acquisition device, such as... Figure 5 As shown, the workpiece cleaning system includes:

[0214] The construction module 100 is used to drive the robot to move the image acquisition device, acquire initial images of the workpiece to be cleaned at different shooting points and different robot poses, and determine at least one target shooting point and corresponding shooting pose based on the initial images to construct a shooting pose matrix.

[0215] The acquisition module 200 is used to drive the robot to move the image acquisition device to the corresponding target image point in the image-taking pose, and to acquire several local images of the workpiece to be cleaned at the target image point.

[0216] The detection module 300 is used to detect the area to be cleaned in a local image and obtain the position and shape information of the area to be cleaned in the workpiece; wherein, the area to be cleaned includes hole areas and / or groove areas;

[0217] The cleaning module 400 is used to determine the matching target cleaning strategy from multiple preset cleaning strategies based on shape information, and drive the robot's end effector to perform cleaning operations on the area to be cleaned based on the target cleaning strategy according to the position information.

[0218] In one specific embodiment, the workpiece cleaning system further includes:

[0219] The adjustment module is used to drive the image acquisition device to adjust to the preset target aperture value and target focal length value in order to acquire the first low-exposure image and the first high-exposure image of the workpiece to be cleaned;

[0220] The prediction module is used to input the target aperture value, target focal length value, first low exposure image and first high exposure image into a pre-trained exposure time prediction model to obtain the target exposure time;

[0221] The acquisition module 200 is specifically used to acquire local images of the workpiece to be cleaned using the target aperture value, target focal length value, and target exposure time.

[0222] In one specific embodiment, the workpiece cleaning system further includes:

[0223] The acquisition module is used to acquire quality evaluation parameters of local images and determine the image quality score based on the quality evaluation parameters;

[0224] The sample acquisition module 200 is used to drive the image acquisition device to adjust to the target aperture value and target focal length value in response to the image quality score being less than a preset threshold, and to acquire multiple second low-exposure images and second high-exposure images of the workpiece to be cleaned.

[0225] The fine-tuning training module is used to respond to the received exposure time labels corresponding to the second low-exposure image and the second high-exposure image, to fine-tune the exposure time prediction model with the second low-exposure image and the second high-exposure image as input and the corresponding exposure time labels as output, and then return to call the adjustment module.

[0226] In one specific embodiment, the detection module 300 includes a splicing unit, a processing unit, and an analysis unit;

[0227] The stitching unit is used to stitch together each local image to obtain a global image, and then convert the global image into a global grayscale image.

[0228] The processing unit is used to perform binarization processing on the global grayscale image using an adaptive threshold of root mean square error to obtain candidate regions to be cleaned.

[0229] The processing unit is used to perform pixel-level classification of the candidate regions to be cleaned using a pre-trained semantic segmentation model to obtain a mask of the regions to be cleaned.

[0230] The analysis unit is used to perform morphological processing and shape analysis on the mask of the area to be cleaned, and to obtain the location and shape information of the area to be cleaned.

[0231] In one specific implementation, the analysis unit is specifically used to sequentially perform adaptive erosion, directional closing operation, and conditional opening operation on the mask of the area to be cleaned to obtain a target mask; mark the target mask with connected components to obtain multiple connected regions; acquire the geometric features of each connected region, and select connected regions that meet preset morphological conditions as the areas to be cleaned based on the geometric features; extract the contour of each area to be cleaned, and determine the shape of each area to be cleaned and the center coordinates of each area to be cleaned in the global image based on the contour; and convert the center coordinates into the target three-dimensional coordinates under the robot base coordinates based on the transformation relationship between the image acquisition device coordinate system and the robot base coordinate system.

[0232] In one specific implementation, the analysis unit is further configured to obtain the intrinsic parameter matrix and distortion coefficients of the image acquisition device; perform distortion correction processing on the center coordinates based on the distortion coefficients to obtain normalized image coordinates; obtain the depth value of the area to be cleaned, and perform back projection processing on the normalized image coordinates based on the depth value and the intrinsic parameter matrix to obtain the initial three-dimensional coordinates in the coordinate system of the image acquisition device; and convert the initial three-dimensional coordinates into target three-dimensional coordinates based on the transformation relationship.

[0233] In one specific implementation, the cleaning module 400 is specifically configured to determine a matching target cleaning strategy in response to receiving a cleaning strategy setting instruction.

[0234] or,

[0235] The cleaning module 400 is specifically used to determine the diameter parameter of each area to be cleaned based on the shape information in response to not receiving a cleaning strategy setting instruction; in response to the diameter parameter being greater than the preset diameter, determine the target cleaning strategy as a spiral progressive cleaning strategy or a composite cleaning strategy; or, in response to the diameter parameter being less than or equal to the preset diameter, determine the target cleaning strategy as a straight-line in-straight-out rapid cleaning strategy.

[0236] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0237] This embodiment improves the consistency of image imaging and positioning by systematically determining the imaging points and poses under different batches and lighting conditions. Morphological processing of the area to be cleaned effectively suppresses interference such as edge breakage, adhesion, and reflection, improving detection and classification accuracy. Simultaneously, it can dynamically select cleaning strategies based on hole shape and contamination level, avoiding inefficiency or incomplete cleaning caused by a single cleaning strategy. The entire process, from visual perception, parameter optimization, hole recognition and positioning to cleaning execution, is automated, reducing manual intervention, improving efficiency and consistency, and lowering labor costs and operational risks. Furthermore, the imaging points, image acquisition parameters, and models can all be switched according to the workpiece, making it suitable for various robots and possessing good versatility and scalability.

[0238] Example 3

[0239] Figure 6 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the robot-based workpiece cleaning method described in any of the above embodiments. Figure 6 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0240] like Figure 6 As shown, the electronic device 30 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0241] Bus 33 includes a data bus, an address bus, and a control bus.

[0242] The memory 32 may include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and may further include read-only memory (ROM) 323.

[0243] The memory 32 may also include a program tool 325 (or utility) having a set (at least one) program module 324, such program module 324 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0244] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the robot-based workpiece cleaning method provided in any of the above embodiments.

[0245] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 35. Furthermore, electronic device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 36. Figure 6 As shown, network adapter 36 communicates with other modules of electronic device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with electronic device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0246] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0247] Example 4

[0248] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robot-based workpiece cleaning method provided in any of the above embodiments.

[0249] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0250] Example 5

[0251] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the robot-based workpiece cleaning method described in any of the preceding embodiments.

[0252] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0253] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. A robot-based workpiece cleaning method, characterized in that, The robot is equipped with an image acquisition device, and the workpiece cleaning method includes: The robot is driven to move the image acquisition device to acquire initial images of the workpiece to be cleaned at different shooting points and different robot poses. Based on the initial images, at least one target shooting point and the corresponding shooting pose are determined, and a shooting pose matrix is ​​constructed. The robot is driven to move the image acquisition device to the corresponding target image point in the image-taking pose, and to acquire several partial images of the workpiece to be cleaned at the target image point. The local image is subjected to cleaning area detection to obtain the position and shape information of the cleaning area in the workpiece to be cleaned; wherein, the cleaning area includes hole area and / or groove area; Based on the shape information, a matching target cleaning strategy is determined from multiple preset cleaning strategies, and the robot's end effector is driven to perform cleaning operations on the area to be cleaned based on the target cleaning strategy according to the position information.

2. The workpiece cleaning method according to claim 1, characterized in that, Before the step of acquiring several local images of the workpiece to be cleaned at the target imaging point, the workpiece cleaning method further includes: The image acquisition device is driven to adjust to the preset target aperture value and target focal length value to acquire the first low-exposure image and the first high-exposure image of the workpiece to be cleaned; The target aperture value, the target focal length value, the first low-exposure image, and the first high-exposure image are input into a pre-trained exposure time prediction model to obtain the target exposure time. The step of acquiring several local images of the workpiece to be cleaned at the target imaging point includes: The local image of the workpiece to be cleaned is acquired using the target aperture value, the target focal length value, and the target exposure time.

3. The workpiece cleaning method according to claim 2, characterized in that, Prior to the step of detecting the area to be cleaned in the local image, the workpiece cleaning method further includes: Obtain the quality evaluation parameters of the local image, and determine the image quality score based on the quality evaluation parameters; In response to the image quality score being less than a preset threshold, the image acquisition device is driven to adjust to the target aperture value and the target focal length value, and acquire multiple second low-exposure images and second high-exposure images of the workpiece to be cleaned; In response to receiving the exposure time labels corresponding to the second low-exposure image and the second high-exposure image, the exposure time prediction model is fine-tuned and trained using the second low-exposure image and the second high-exposure image as input and the corresponding exposure time labels as output. Then, the step of driving the image acquisition device to adjust to the preset target aperture value and target focal length value to acquire the first low-exposure image and the first high-exposure image of the workpiece to be cleaned is returned to be executed.

4. The workpiece cleaning method according to claim 1, characterized in that, The step of detecting the area to be cleaned in the local image to obtain the position and shape information of the area to be cleaned in the workpiece includes: Each of the aforementioned local images is stitched together to obtain a global image, and the global image is then converted into a global grayscale image. The global grayscale image is binarized using a root mean square error adaptive threshold to obtain candidate regions to be cleaned. A pre-trained semantic segmentation model is used to perform pixel-level classification of the candidate regions to be cleaned, thereby obtaining a mask of the regions to be cleaned. Morphological processing and shape analysis are performed on the mask of the area to be cleaned to obtain the location information and shape information of the area to be cleaned.

5. The workpiece cleaning method according to claim 4, characterized in that, The step of performing morphological processing and shape analysis on the mask of the area to be cleaned to obtain the location information and shape information of the area to be cleaned includes: The target mask is obtained by sequentially performing adaptive erosion, directional closing operation, and conditional opening operation on the mask of the area to be cleaned. The target mask is marked with connected components to obtain multiple connected regions; Obtain the geometric features of each connected region, and select the connected regions that meet the preset morphological conditions as the regions to be cleaned based on the geometric features; Extract the outline of each of the areas to be cleaned, and determine the shape of each area to be cleaned and the center coordinates of each area to be cleaned in the global image based on the outline; Based on the transformation relationship between the coordinate system of the image acquisition device and the robot base coordinate system, the center coordinates are converted into the target three-dimensional coordinates under the robot base coordinate system.

6. The workpiece cleaning method according to claim 5, characterized in that, The step of converting the center coordinates into target three-dimensional coordinates in the robot base coordinate system based on the transformation relationship between the image acquisition device coordinate system and the robot base coordinate system includes: Obtain the intrinsic parameter matrix and distortion coefficients of the image acquisition device; The center coordinates are subjected to distortion correction based on the distortion coefficients to obtain normalized image coordinates. The depth value of the area to be cleaned is obtained, and the normalized image coordinates are back-projected based on the depth value and the intrinsic parameter matrix to obtain the initial three-dimensional coordinates in the coordinate system of the image acquisition device. Based on the transformation relationship, the initial three-dimensional coordinates are converted into the target three-dimensional coordinates.

7. The workpiece cleaning method according to any one of claims 1 to 6, characterized in that, The step of determining the matching target cleaning strategy from multiple preset cleaning strategies based on the shape information includes: In response to receiving a cleaning strategy setting instruction, the target cleaning strategy is determined according to the cleaning strategy setting instruction; or, In response to the absence of the cleaning strategy setting instruction, the diameter parameter of each area to be cleaned is determined based on the shape information; In response to the diameter parameter being greater than a preset diameter, the target cleaning strategy is determined to be a spiral progressive cleaning strategy or a composite cleaning strategy; or, in response to the diameter parameter being less than or equal to the preset diameter, the target cleaning strategy is determined to be a straight-in, straight-out rapid cleaning strategy.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the robot-based workpiece cleaning method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the robot-based workpiece cleaning method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot-based workpiece cleaning method according to any one of claims 1 to 7.

Images