Double-arm clothes folding robot control method, device and equipment and storage medium

Abstract

The invention discloses a double-arm clothes folding robot control method and device, equipment and a storage medium. The method comprises the following steps: determining three-dimensional point cloud data of a target workbench according to a collected image through a visual perception assembly; determining a scene label graph and a desktop edge line set through a scene division assembly according to the three-dimensional point cloud data, determining a clothing mask of the to-be-processed clothing according to the scene label graph through a clothing classification assembly, and determining a clothing description vector according to the clothing mask; determining a low-risk action set and a protection area mask from the candidate clothes folding action set through a safety protection component according to the scene label graph, the desktop edge line set and the clothes mask; determining a target clothes folding action sequence through an action planning component according to the clothes description vector, the low-risk action set and the protection area mask; and folding the to-be-processed clothes through the robot control assembly according to the target clothes folding action sequence. According to the scheme, the clothes folding efficiency and reliability of the robot are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to the field of artificial intelligence technology, specifically to a control method, apparatus, device, and storage medium for a dual-arm folding robot. Background Technology

[0002] Current robot clothing folding tests are generally conducted in an ideal environment of "single garment + smooth large table + no clutter" to complete the flattening and folding: the dual-arm robot performs a series of pulling, dragging and swinging actions on the table to flatten a garment from a wrinkled state, and then folds it according to the preset fold line. The goal is to achieve a high success rate and high speed in this controlled environment; the garment can slide freely on the table, there are no mobile phones, water cups or other piles of clothes around, and there is no problem of the table size being too small.

[0003] Therefore, conventional dual-arm robot folding algorithms only optimize indicators such as "flattening rate and speed" during design, without incorporating limited space and collisions with clutter as hard constraints or penalty targets into the decision-making process. As a result, in daily actual use, when faced with scenarios of "small table, many things, and piles of clothes", the clothes are easily dragged to the edge of the table or even slipped off, or swept into surrounding objects when dragged in large quantities. Summary of the Invention

[0004] This application provides a control method, apparatus, device, and storage medium for a dual-arm folding robot to improve the efficiency and reliability of folding clothes in complex scenarios.

[0005] According to one aspect of this application, a control method for a dual-arm folding robot is provided. This method is applied to an intelligent folding control system for a target folding robot. The intelligent folding control system includes a visual perception component, a scene segmentation component, a clothing classification component, a safety protection component, a motion planning component, and a robot control component. The components in the intelligent folding control system are communicatively connected. The method includes:

[0006] The visual perception component acquires images of the target workbench where the target folding robot is currently located based on a preset acquisition frequency, thereby obtaining the current color image and current depth image of the target workbench, and determining the three-dimensional point cloud data of the target workbench based on the current color image and the current depth image.

[0007] The scene segmentation component performs planar fitting on the 3D point cloud data to obtain the desktop plane point set of the target workbench. Based on the 3D point cloud data and the desktop plane point set, the scene label map and desktop edge line set of the target workbench are determined.

[0008] The clothing classification component determines the clothing mask of the clothing to be processed in the target workbench based on the scene label map, and determines the clothing description vector of the clothing to be processed based on the clothing mask.

[0009] The security protection component determines a set of low-risk actions from a set of candidate folding actions based on the scene label map, the set of desktop edge lines, and the clothing mask; and determines the protective area mask of the target workbench based on the scene label map.

[0010] The action planning component determines the target folding action sequence for the clothing to be processed based on the clothing description vector, the low-risk action set, and the protective area mask.

[0011] The robot control component controls the arms of the target folding robot and the suction tool on the target workbench to fold the clothes according to the target folding action sequence.

[0012] According to another aspect of this application, a control device for a dual-arm folding robot is provided, which is configured in the intelligent folding control system of the target folding robot; the intelligent folding control system includes a visual perception component, a scene segmentation component, a clothing classification component, a safety protection component, a motion planning component, and a robot control component; the components in the intelligent folding control system are communicatively connected; the device includes:

[0013] The image acquisition module is used to acquire images of the target workbench where the target folding robot is currently located based on a preset acquisition frequency through the visual perception component, to obtain the current color image and current depth image of the target workbench, and to determine the three-dimensional point cloud data of the target workbench based on the current color image and the current depth image.

[0014] The point cloud analysis module is used to perform planar fitting on the 3D point cloud data through the scene segmentation component to obtain the desktop plane point set of the target workbench, and to determine the scene label map and desktop edge line set of the target workbench based on the 3D point cloud data and the desktop plane point set.

[0015] The clothing classification module is used to determine the clothing mask of the clothing to be processed in the target workbench based on the scene label map by the clothing classification component, and to determine the clothing description vector of the clothing to be processed based on the clothing mask.

[0016] The action evaluation module is used to determine a low-risk action set from the candidate folding action set based on the scene label map, the desktop edge line set, and the clothing mask, and to determine the protective area mask of the target workbench based on the scene label map.

[0017] The action planning module is used to determine the target folding action sequence of the clothing to be processed based on the clothing description vector, the low-risk action set, and the protective area mask through the action planning component;

[0018] The clothing folding module is used to control the arms of the target folding robot and the suction tool on the target workbench to fold the clothing to be processed according to the target folding action sequence by the robot control component.

[0019] According to another aspect of this application, an electronic device is provided, the electronic device comprising:

[0020] One or more processors;

[0021] Memory, used to store one or more programs;

[0022] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the dual-arm folding robot control methods provided in the embodiments of this application.

[0023] According to another aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements any of the dual-arm folding robot control methods provided in the embodiments of this application.

[0024] According to another aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the dual-arm folding robot control methods provided in the embodiments of this application.

[0025] This application uses a visual perception component to acquire images of the target workbench where the target folding robot is currently located, based on a preset acquisition frequency. This results in the current color image and current depth image of the target workbench. Based on these images, the 3D point cloud data of the target workbench is determined. A scene segmentation component performs planar fitting on the 3D point cloud data to obtain the desktop plane point set of the target workbench. Based on the 3D point cloud data and the desktop plane point set, a scene label map and a set of desktop edge lines are determined. Finally, a clothing classification component uses the scene label map to determine the clothing mask of the clothing to be processed on the target workbench. The system employs a membrane and, based on the clothing mask, determines the clothing description vector of the clothing to be processed. A safety protection component, based on the scene label map, the set of desktop edge lines, and the clothing mask, identifies a low-risk action set from the candidate folding action set and determines the protective area mask of the target workbench based on the scene label map. An action planning component, based on the clothing description vector, the low-risk action set, and the protective area mask, determines the target folding action sequence for the clothing to be processed. A robot control component, based on the target folding action sequence, controls the arms of the target folding robot and the suction tool on the target workbench to fold the clothing. This technical solution, while maintaining the basic framework of the dual-arm robot folding system—"scene recognition + arbitrary shape clothing classification + dual-arm flattening and folding"—introduces a safety boundary prediction and local fixation collaborative control mechanism. This suppresses or even completely avoids unrestrained sliding and out-of-bounds drifting of clothing during the flattening stage, preventing interference with already folded clothing and desktop clutter. This allows the folding robot to stably and continuously complete the automated operation from pile of clothes to folded finished product in a real home environment with limited space and cluttered desktops. Attached Figure Description

[0026] Figure 1 This is a flowchart of a control method for a dual-arm folding robot according to Embodiment 1 of this application;

[0027] Figure 2 This is a flowchart of a control method for a dual-arm folding robot according to Embodiment 2 of this application;

[0028] Figure 3 This is a schematic diagram of a control device for a dual-arm folding clothes robot according to Embodiment 3 of this application;

[0029] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the control method of the dual-arm folding robot of Embodiment 4 of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Furthermore, it should be noted that the preset acquisition frequency and the collection, storage, use, processing, transmission, provision, and disclosure of relevant data such as current color images involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0033] Example 1

[0034] Figure 1 This is a flowchart of a dual-arm folding robot control method according to Embodiment 1 of this application. This embodiment is applicable to complex scenarios such as multiple items being stacked, limited table space, and surrounding objects, where a dual-arm robot can efficiently fold different types of clothing. The method can be executed by a dual-arm folding robot control device, which can be implemented in hardware and / or software. This control device can be configured in a computer device, such as the intelligent folding control system of the target folding robot. The intelligent folding control system includes a visual perception component, a scene segmentation component, a clothing classification component, a safety protection component, a motion planning component, and a robot control component. The components in the intelligent folding control system are interconnected. Figure 1 As shown, the method includes:

[0035] S110. Using a visual perception component based on a preset acquisition frequency, images of the target workbench where the target folding robot is currently located are acquired, obtaining the current color image and current depth image of the target workbench, and determining the three-dimensional point cloud data of the target workbench based on the current color image and current depth image.

[0036] The visual perception component refers to the system component in the intelligent clothes-folding control system used to acquire and analyze images of the workbench where the dual-arm robot needs to perform related clothes-folding processing. The preset acquisition frequency is set based on actual conditions or empirical values. The target clothes-folding robot refers to a robot with two arms. The target workbench refers to the work area or platform that the robot is processing; in the application scenario of the clothes-folding robot, the workbench can be a surface for placing clothes, and the robot needs to process the items on the workbench through visual perception and motion planning. The current color image refers to an image containing the red, green, and blue channels. The current depth image refers to an image that contains not only color information but also the distance information between each pixel and the sensor. 3D point cloud data refers to a set of three-dimensional coordinate points, which represent the shape and position of certain objects or surfaces in three-dimensional space.

[0037] For example, a color image and depth map above the workbench are acquired at a fixed frame rate using a visual perception component to form a 3D point cloud dataset.

[0038] S120. The 3D point cloud data is fitted to a plane using the scene segmentation component to obtain the desktop plane point set of the target workbench. Based on the 3D point cloud data and the desktop plane point set, the scene label map and desktop edge line set of the target workbench are determined.

[0039] The scene segmentation component refers to a system component in the intelligent folding clothes control system used to divide scene data (such as 3D point cloud data) into different regions or levels for more detailed analysis and processing. The desktop plane point set refers to the set of all points representing the surface of the target workbench (or desktop) extracted from 3D point cloud data using a plane fitting algorithm; this point set contains all positional data on the desktop, used to describe the planar shape of the workbench. The scene label map refers to an image that assigns labels to different regions or objects in the scene based on scene data (such as 3D point clouds or images); these labels can represent different objects or surfaces, such as desktops, objects, walls, etc.; in the scene label map of the target workbench, each pixel (or point) is assigned a label indicating that it belongs to the desktop, object, background, or other regions. The desktop edge line set refers to the edge lines identified through further analysis of the desktop plane, used to characterize the edge contours of the workbench surface.

[0040] For example, the point cloud data is fitted to a plane by a scene segmentation component to obtain the desktop plane click. Then, using deep clustering and semantic segmentation models, the 3D point cloud data is divided into desktop area, clutter area, finished clothing area and clothing to be processed area according to the desktop plane point set, and the scene label map and desktop edge line set are output.

[0041] S130. Based on the scene label map, the clothing classification component determines the clothing mask of the clothing to be processed in the target workbench, and determines the clothing description vector of the clothing to be processed based on the clothing mask.

[0042] The clothing classification component in the intelligent clothing folding control system is used to identify and classify clothing on the target workbench. Typically using image processing, machine learning, or deep learning algorithms, the clothing classification component analyzes image data, identifies different types of clothing, and assigns labels to each garment according to its category. The clothing to be processed refers to the clothing located on the target workbench that the robot currently needs to fold. The clothing mask is a binary image generated based on the analysis results of the scene label map, used to clearly define the location of clothing in the image, helping the robot accurately identify and manipulate the clothing. The clothing description vector is a multi-dimensional vector representing the characteristics of the clothing; each garment's description vector contains multiple feature information, such as color, type, size, material, and texture.

[0043] Optionally, the clothing classification component determines the clothing area to be processed in the target workbench based on the scene label map, and performs binarization processing on the clothing area to be processed to obtain the clothing mask of the clothing to be processed in the target workbench; the clothing classification component determines the minimum bounding rectangle area, clothing type label and coarse pose vector of each clothing to be processed based on the clothing mask, and determines the clothing description vector of the clothing to be processed based on the minimum bounding rectangle area, clothing type label and coarse pose vector.

[0044] The "Clothing to be Processed" area refers to the region comprised of all pixels / points marked as "Clothing to be Processed" in the workbench, based on the scene label map. The minimum bounding rectangle area refers to the smallest rectangle that can completely enclose the clothing to be processed. This rectangle is not necessarily horizontal or vertical; it can be rotated according to the actual posture of the object to ensure a tight enclosure of the clothing. It describes the basic size and shape characteristics of the clothing. The clothing type label is the label assigned to each piece of clothing after classification; this label describes the type of clothing. The coarse pose vector refers to the approximate position or posture of the clothing on the workbench; it does not focus on the fine folding or rolling state of the clothing, but rather on its general placement posture, such as folded or stacked state.

[0045] For example, the clothing classification component determines the clothing area to be processed in the target workbench based on the clothing area label in the scene label image; the clothing classification component performs binarization processing on the clothing area to be processed, that is, converts each piece of clothing into a separate binary mask, where the clothing area is white (1) and the background is black (0), and uses morphological processing to optimize the binary mask to obtain the clothing mask of the clothing to be processed in the target workbench; the clothing classification component extracts the contour of each clothing mask to find the boundary of each clothing to be processed, and uses the minimum bounding rectangle algorithm to calculate the bounding matrix of each clothing to be processed to obtain the width and height of the minimum bounding matrix. The system first determines the minimum bounding rectangle area; then, using a clothing classification component, it takes the clothing mask as input and uses a convolutional neural network to classify each piece of clothing, outputting a clothing type label for each piece; using the clothing classification component, it takes the clothing mask as input and uses a convolutional neural network or other pose estimation model that can learn the basic pose features of clothing, such as the folding method and unfolding state, to roughly estimate the pose of the clothing, outputting a coarse pose vector representing the possible poses of the clothing (e.g., folded state, hanging state, stacked state, etc.); finally, the clothing classification component concatenates the minimum bounding rectangle area, clothing type label, and pose vector of each piece of clothing into a unified clothing description vector.

[0046] S140. Using the safety protection component, determine the low-risk action set from the candidate folding action set based on the scene label map, the desktop edge line set, and the clothing mask, and determine the protection area mask of the target workbench based on the scene label map.

[0047] Among them, the safety protection component refers to the system component in the intelligent folding clothes control system used to analyze risk factors in the environment and select low-risk operation steps to avoid potential dangers or accidents when automated tasks involving robots are performed. The candidate folding action set refers to the set of possible actions listed for clothing arrangement (such as folding, stacking, etc.) in the task; these actions can include different folding methods, stacking positions, etc. The low-risk action set refers to those actions that, after evaluation, have low risk and are unlikely to cause collisions or damage. The protection area mask is a binary image used to identify areas on the target workbench that need to be protected. These areas may include fragile items, table edges, and other areas that should not be disturbed or damaged, such as areas of finished clothing that have already been folded, or cluttered areas with a large amount of miscellaneous items.

[0048] Optionally, the safety protection component includes a risk prediction sub-component; accordingly, the risk prediction sub-component predicts the clothing pose of the clothing to be processed after each folding action in the candidate folding action set based on the outline of the finished clothing area and the clothing mask in the scene label image, thereby obtaining the clothing pose set corresponding to the candidate folding action set; the risk prediction sub-component determines the low-risk action set from the candidate folding action set based on the clothing pose set and the desktop edge line set.

[0049] The risk prediction subcomponent refers to the safety protection component responsible for predicting potential risks during the operation, particularly based on analysis of factors such as the movement, position, and posture of the garments to be processed. The finished garment area outline refers to the edge of the area of ​​neatly folded, completed garments. Garment pose refers to the position of the garment in three-dimensional space (e.g., its position on a table) and its orientation or posture (e.g., whether the garment is flat, folded, etc.). The garment pose set refers to the set of possible positions and postures of the garment on the workbench after each folding action, predicted based on different folding actions.

[0050] For example, the risk prediction subcomponent determines the safe distance from each discrete point of the clothing outline to the nearest edge of the table based on the set of discrete points of the clothing outline in the clothing pose set and the set of table edge lines. If the safe distance corresponding to the candidate folding action is less than or equal to the safe distance threshold, the candidate folding action is marked as a low-risk action. If the safe distance corresponding to the candidate folding action is greater than the safe distance threshold, the candidate folding action is marked as a high-risk action.

[0051] The set of discrete points representing the clothing outline refers to the set of discrete points along the edge of the clothing in the image, typically extracted using image processing techniques (such as edge detection algorithms); each point represents a specific location on the edge of the clothing. The safe distance refers to the minimum safe distance between the clothing to be processed and the edge of the table after performing a candidate folding action. The safe distance threshold is a set reference value used to determine whether the current safe distance meets safety standards; this threshold is preset based on the system's safety requirements, environmental conditions, and task type. Low-risk actions occur when the safe distance corresponding to a candidate folding action is less than or equal to the safe distance threshold, indicating that the operation is within the safe range and is considered low-risk; such actions are relatively safe to execute and are less likely to result in collisions or damage. High-risk actions occur when the safe distance corresponding to a candidate folding action is greater than the safe distance threshold, indicating that the operation exceeds the safe range and carries a high risk of collision or damage; such actions may lead to unnecessary risks.

[0052] Optionally, the safety protection component also includes an area protection sub-component; correspondingly, the area protection sub-component determines the outline of the finished clothing area and the outline of the desktop clutter area based on the scene label map; the area protection sub-component determines the protective area mask of the target workbench based on the outline of the finished clothing area and the outline of the desktop clutter area.

[0053] Among them, the area protection sub-component refers to the sub-component in the safety protection component that is responsible for determining the protected area and generating the constraints of the protected area. The desktop clutter area outline refers to the outline of the area where clutter (such as tools, papers, small objects, etc.) is located.

[0054] S150. The action planning component determines the target folding action sequence of the clothing to be processed based on the clothing description vector, the low-risk action set, and the protective area mask.

[0055] The motion planning component refers to the system component in the intelligent clothes folding control system responsible for generating a series of reasonable and effective motion sequences according to task requirements. The target clothes folding motion sequence refers to the sequence of actions planned to complete a specific clothes folding task; the target clothes folding motion sequence may include target flattening and folding motion sequences and target fixing motion sequences.

[0056] For example, the action planning component determines the candidate folding action sequence of the clothing to be processed based on the clothing description vector and the protective area mask, and determines the candidate folding action sequence as the target flattening and folding action sequence; the action planning component determines the folding action sequence among the candidate folding action sequences that has the risk of the clothing slipping as a whole during the execution process based on the low-risk action set, and determines the target fixing action sequence based on the folding action sequence.

[0057] S160. The robot control component controls the arms of the target folding robot and the suction tool on the target workbench according to the target folding action sequence to fold the clothes to be processed.

[0058] The robot control component refers to the system components in the intelligent clothes folding control system used to command and regulate the robot and workbench to perform various operations. The dual arms refer to the two robotic arms equipped on the target clothes folding robot; the dual-arm configuration allows the robot to more flexibly grasp, manipulate, and fold items. The suction tool refers to a tool used on the workbench to fix or grasp clothing during robot operation, such as a vacuum suction cup or flexible pressure plate on the workbench.

[0059] In one optional embodiment, the intelligent folding clothes control system further includes a status monitoring and feedback component; accordingly, the status monitoring and feedback component continuously collects the projected area of ​​the clothes to be processed after the flattening or folding action is completed, and divides the current projected area of ​​the clothes to be processed by the area of ​​the minimum bounding rectangle to obtain the flatness index; if the flatness index is less than a preset threshold, the non-compliance is fed back to the action planning unit so that the action planning unit can initiate a new round of action planning.

[0060] The status monitoring and feedback component refers to the system component in the intelligent garment folding control system used to monitor and collect the robot's status data in real time during task execution, and to feed this data back to other components of the system. The garment projection area refers to the size of the garment's projection on a plane (usually a horizontal plane), typically obtained by calculating the garment's two-dimensional outline. The flatness index is a value calculated by dividing the garment's projection area by the area of ​​its smallest bounding rectangle; this value reflects the flatness of the garment, i.e., the effectiveness of its folding.

[0061] In another alternative implementation, the intelligent clothes folding control system also includes a human-computer interaction and task management component; accordingly, the human-computer interaction and task management component responds to user control commands and adjusts the working modes and working parameters of each component in the intelligent clothes folding control system.

[0062] Among them, the human-computer interaction and task management component refers to the system component in the intelligent clothes folding control system used to receive user task instructions, manage task execution, and coordinate various related subsystems (such as motion planning, status monitoring, etc.) to complete the task. The working mode refers to the different operating states or modes of operation of the component when performing a task. The working parameters refer to the settings or configurations of the component for performing the task, such as speed, force, and path planning.

[0063] This application embodiment uses a visual perception component to acquire images of the target workbench where the target folding robot is currently located, based on a preset acquisition frequency. This yields the current color image and current depth image of the target workbench, and based on these images, determines the 3D point cloud data of the target workbench. A scene segmentation component performs planar fitting on the 3D point cloud data to obtain the desktop plane point set of the target workbench. Based on the 3D point cloud data and the desktop plane point set, a scene label map and a set of desktop edge lines are determined for the target workbench. Finally, a clothing classification component uses the scene label map to determine the clothing categories of the garments to be processed on the target workbench. The system employs a masking mechanism to determine the clothing description vector of the garments to be processed. A safety protection component identifies a set of low-risk actions from a candidate folding action set based on a scene label map, a set of desktop edge lines, and the clothing mask. It also determines the protective area mask for the target workbench based on the scene label map. An action planning component determines the target folding action sequence for the garments to be processed based on the clothing description vector, the low-risk action set, and the protective area mask. Finally, a robot control component controls the arms of the target folding robot and the suction tools on the target workbench according to the target folding action sequence to fold the garments. This technical solution, while maintaining the basic framework of the dual-arm robot folding system—"scene recognition + arbitrary shape clothing classification + dual-arm flattening and folding"—introduces a safety boundary prediction and local fixation collaborative control mechanism. This suppresses or even completely avoids unrestrained sliding and out-of-bounds drifting of clothing during the flattening stage, preventing interference with already folded clothing and desktop clutter. This allows the folding robot to stably and continuously complete the automated operation from piled-up clothes to folded finished products in a real home environment with limited space and cluttered desktops.

[0064] Example 2

[0065] Figure 2 This is a flowchart of a dual-arm folding robot control method according to Embodiment 2 of this application. Based on the technical solutions of the above embodiments, this embodiment refines the process of "determining the target folding action sequence of the clothing to be processed by the action planning component according to the clothing description vector, the low-risk action set, and the protective area mask" into "determining the target action template library from the candidate action template library by the action planning component according to the clothing description vector, and determining the target flattening and folding action sequence of the clothing to be processed from the target action template library according to the clothing description vector; identifying risk actions in the target flattening and folding action sequence by the action planning component according to the low-risk action set and the protective area mask, obtaining a low-risk flattening and folding action set and a high-risk flattening and folding action set; and determining the target fixed action sequence of the clothing to be processed by the action planning component according to the high-risk flattening and folding action set and the protective area mask." It should be noted that for parts not detailed in this embodiment, please refer to the relevant descriptions in other embodiments. Figure 2 As shown, the method includes:

[0066] S210. Using a visual perception component based on a preset acquisition frequency, images of the target workbench where the target folding robot is currently located are acquired, obtaining the current color image and current depth image of the target workbench, and determining the three-dimensional point cloud data of the target workbench based on the current color image and current depth image.

[0067] S220. The 3D point cloud data is fitted to a plane using the scene segmentation component to obtain the desktop plane point set of the target workbench. Based on the 3D point cloud data and the desktop plane point set, the scene label map and desktop edge line set of the target workbench are determined.

[0068] S230. Based on the scene label map, the clothing classification component determines the clothing mask of the clothing to be processed in the target workbench, and determines the clothing description vector of the clothing to be processed based on the clothing mask.

[0069] S240. Using the safety protection component, determine the low-risk action set from the candidate clothing folding action set based on the scene label map, the desktop edge line set, and the clothing mask, and determine the protection area mask of the target workbench based on the scene label map.

[0070] S250. Based on the clothing description vector, the motion planning component determines the target motion template library from the candidate motion template library, and based on the clothing description vector, determines the target flattening and folding motion sequence of the clothing to be processed from the target motion template library.

[0071] The candidate action template library is a database containing various action templates, each representing a general action sequence applicable to different tasks or clothing types. When performing clothing folding tasks, the candidate action template library provides a range of possible action templates to choose from. The target action template library is a filtered and optimized collection of action templates, a set of precise action templates required based on the current task objective (e.g., folding, flattening). The target flattening and folding action sequence refers to the specific action steps generated based on the clothing description vector and the selected action templates, used to flatten and fold the clothing to be processed.

[0072] S260. Using the motion planning component, risk actions are identified in the target flattening and folding motion sequence based on the low-risk motion set and the protected area mask, resulting in a low-risk flattening and folding motion set and a high-risk flattening and folding motion set.

[0073] The low-risk flattening and folding motion set refers to those low-risk motions selected from the target flattening and folding motion sequence through the risk motion identification process. These motions are considered to pose minimal risk to the robot, clothing, and the surrounding environment, and are typically precise and controllable. The high-risk flattening and folding motion set refers to those high-risk motions identified from the target flattening and folding motion sequence during the risk motion identification process. Performing these motions may result in collisions, damage to clothing, damage to the robot itself, or mission failure.

[0074] S270. Based on the set of high-risk flattening and folding actions and the protective area mask, the motion planning component determines the target fixed action sequence of the clothing to be processed.

[0075] Optionally, the motion planning component determines the area of ​​clothing that needs to be fixed for each high-risk action in the high-risk flattening and folding action set and the protective area mask; the motion planning component determines the target fixing action sequence for the clothing to be processed based on the timestamps corresponding to each high-risk action in the high-risk flattening and folding action set and the area of ​​clothing that needs to be fixed.

[0076] The "clothing-secured area" refers to the crucial step in securing certain parts of the clothing within a specific area during high-risk flattening and folding actions to prevent pulling, movement, or damage. A timestamp is a time label assigned to each high-risk action during motion planning; each action has a specific execution time, and timestamps help the system precisely control the execution sequence, ensuring actions occur on time and allowing for monitoring and optimization. The target securing action sequence refers to the series of actions the robot must perform to ensure the clothing remains stable during high-risk actions; this sequence includes steps to secure the clothing before, during, or after the high-risk actions, preventing displacement or damage.

[0077] For example, after receiving the clothing description vector, the set of low-risk actions, and the protective area mask, the action planning component first selects the corresponding action template library based on the clothing type and size in the clothing description vector. Then, based on the set of low-risk actions, it searches the candidate action template library for flattening and folding action sequences that best meet the set of low-risk actions and uses them as the target flattening and folding action sequences. For candidate actions that cannot be avoided and are marked as high-risk, they are not discarded directly. Instead, the action planning component configures the target fixed action sequence for the clothing to be processed based on the protective area mask and the set of high-risk flattening and folding actions.

[0078] In one alternative implementation, after determining the target fixed action sequence through the action planning component, the action planning component can also optimize the set of high-risk flattening and folding actions in the corresponding target flattening and folding action sequence based on the target fixed action sequence, to obtain the optimized target flattening and folding action sequence.

[0079] Understandably, to avoid conflicts between local fixation and the original flattening and folding actions, which could lead to excessive pulling of clothing, the motion planning component optimizes the flattening and folding actions by utilizing local fixation actions. For example, the edge of the clothing that is about to cross the boundary is fixed to the table, and then a local stretching action is regenerated in the remaining free area to replace the original overall dragging action. This allows for "active correction" of the flattening process at the motion level, preventing the clothing from sliding as a whole during execution and avoiding excessive pulling during the folding process.

[0080] S280: The robot control component controls the arms of the target folding robot and the suction tool on the target workbench according to the target folding action sequence to fold the clothes to be processed.

[0081] Optionally, the robot control component includes a dual-arm motion control sub-component and a local fixation sub-component; correspondingly, the dual-arm motion control sub-component controls the dual arms of the target folding robot to perform flattening and folding operations on the clothes to be processed according to the target flattening and folding action sequence; the local fixation sub-component controls the suction tool on the target worktable to perform local fixation operations on the clothes to be processed during the flattening and folding process according to the target fixation action sequence.

[0082] The dual-arm motion control sub-component refers to the sub-component in the robot control assembly used to coordinate and control the movements of the two arms to complete the folding task. The local fixation sub-component refers to the sub-component in the robot control assembly used to fix certain areas of the clothing. This component usually uses suction tools, clamping devices, or other methods to ensure that the clothing remains stable during operation and avoids slippage or deformation.

[0083] For example, the target flattening and folding motion sequence is decomposed into the joint motion sequence of the left and right robotic arms by the dual-arm motion control sub-component. Multi-parameter closed-loop control, including joint position, speed, and end torque, is used to control the dual arms of the target folding robot according to the joint motion sequence to flatten and fold the clothes to be processed, ensuring that the motion avoids the finished product area in space and does not excessively pull the clothes in terms of force control. The local fixing sub-component controls the vacuum suction cup or flexible pressure plate on the target worktable to adsorb, hold, or release the fabric in the designated area according to the target fixing motion sequence, so that the clothes form multiple "anchor points" at the key edges, so that even if the dual arms perform a large-amplitude local stretching motion, the clothes will not be dragged to the edge of the table.

[0084] This application embodiment uses a visual perception component to acquire images of the target workbench where the target folding robot is currently located, based on a preset acquisition frequency, to obtain the current color image and current depth image of the target workbench. Based on the current color image and current depth image, the 3D point cloud data of the target workbench is determined. A scene segmentation component performs planar fitting on the 3D point cloud data to obtain the desktop plane point set of the target workbench. Based on the 3D point cloud data and the desktop plane point set, the scene label map and desktop edge line set of the target workbench are determined. A clothing classification component determines the clothing mask of the clothing to be processed in the target workbench based on the scene label map, and determines the clothing description vector of the clothing to be processed based on the clothing mask. Finally, a safety protection component determines a low-risk action set from the candidate folding action set based on the scene label map, desktop edge line set, and clothing mask. The system combines the following steps: First, it determines the protective area mask of the target workbench based on the scene label map. Second, it uses the motion planning component to determine the target motion template library from the candidate motion template library based on the clothing description vector, and then determines the target flattening and folding motion sequence of the clothing to be processed from the target motion template library based on the clothing description vector. Third, it uses the motion planning component to identify risky actions in the target flattening and folding motion sequence based on the low-risk motion set and the protective area mask, obtaining low-risk flattening and folding motion sets and high-risk flattening and folding motion sets. Fourth, it uses the motion planning component to determine the target fixed motion sequence of the clothing to be processed based on the high-risk flattening and folding motion set and the protective area mask. Finally, it uses the robot control component to control the arms of the target folding robot and the suction tool on the target workbench according to the target folding motion sequence to fold the clothing to be processed. The above technical solution, while maintaining the basic framework of the dual-arm robot folding system of "scene recognition + arbitrary shape clothing classification + dual-arm flattening and folding", introduces a safety boundary prediction and local fixed cooperative control mechanism to suppress or even completely avoid unrestrained sliding and out-of-bounds drift of clothing during the flattening stage, and avoids interference with folded clothing and table clutter. This enables the folding robot to stably and continuously complete the automated operation from pile of clothes to folded finished product in a real home environment with limited space and clutter on the table.

[0085] Example 3

[0086] Figure 3This is a structural schematic diagram of a dual-arm folding robot control device according to Embodiment 3 of this application. It is applicable to complex scenarios such as multiple items stacked, limited table space, and surrounding objects, enabling the dual-arm robot to efficiently fold different types of clothing. This dual-arm folding robot control device can be implemented in hardware and / or software and can be configured in a computer device, such as the intelligent folding control system of a target folding robot. The intelligent folding control system includes a visual perception component, a scene segmentation component, a clothing classification component, a safety protection component, a motion planning component, and a robot control component. The components in the intelligent folding control system are interconnected. Figure 3 As shown, the device includes:

[0087] The image acquisition module 310 is used to acquire images of the target workbench where the target folding robot is currently located through the visual perception component based on a preset acquisition frequency, to obtain the current color image and current depth image of the target workbench, and to determine the three-dimensional point cloud data of the target workbench based on the current color image and current depth image.

[0088] The point cloud analysis module 320 is used to perform planar fitting on the 3D point cloud data through the scene segmentation component to obtain the desktop plane point set of the target workbench. Based on the 3D point cloud data and the desktop plane point set, it determines the scene label map and desktop edge line set of the target workbench.

[0089] The clothing classification module 330 is used to determine the clothing mask of the clothing to be processed in the target workbench based on the scene label map through the clothing classification component, and to determine the clothing description vector of the clothing to be processed based on the clothing mask.

[0090] The action evaluation module 340 is used to determine a set of low-risk actions from a set of candidate folding actions based on a scene label map, a set of desktop edge lines, and a clothing mask, and to determine a protective area mask for the target workbench based on a scene label map.

[0091] The motion planning module 350 is used to determine the target folding action sequence of the clothing to be processed based on the clothing description vector, the low-risk action set and the protective area mask through the motion planning component;

[0092] The clothing folding module 360 ​​is used to control the arms of the target folding robot and the suction tool on the target worktable according to the target folding action sequence through the robot control component, so as to fold the clothes to be processed.

[0093] This application embodiment uses a visual perception component to acquire images of the target workbench where the target folding robot is currently located, based on a preset acquisition frequency. This yields the current color image and current depth image of the target workbench, and based on these images, determines the 3D point cloud data of the target workbench. A scene segmentation component performs planar fitting on the 3D point cloud data to obtain the desktop plane point set of the target workbench. Based on the 3D point cloud data and the desktop plane point set, a scene label map and a set of desktop edge lines are determined for the target workbench. Finally, a clothing classification component uses the scene label map to determine the clothing categories of the garments to be processed on the target workbench. The system employs a masking mechanism to determine the clothing description vector of the garments to be processed. A safety protection component identifies a set of low-risk actions from a candidate folding action set based on a scene label map, a set of desktop edge lines, and the clothing mask. It also determines the protective area mask for the target workbench based on the scene label map. An action planning component determines the target folding action sequence for the garments to be processed based on the clothing description vector, the low-risk action set, and the protective area mask. Finally, a robot control component controls the arms of the target folding robot and the suction tools on the target workbench according to the target folding action sequence to fold the garments. This technical solution, while maintaining the basic framework of the dual-arm robot folding system—"scene recognition + arbitrary shape clothing classification + dual-arm flattening and folding"—introduces a safety boundary prediction and local fixation collaborative control mechanism. This suppresses or even completely avoids unrestrained sliding and out-of-bounds drifting of clothing during the flattening stage, preventing interference with already folded clothing and desktop clutter. This allows the folding robot to stably and continuously complete the automated operation from piled-up clothes to folded finished products in a real home environment with limited space and cluttered desktops.

[0094] Optionally, the target folding action sequence includes a target flattening and folding action sequence and a target fixing action sequence; correspondingly, the motion planning module 350 includes:

[0095] The first action sequence determination unit is used to determine the target action template library from the candidate action template library based on the clothing description vector through the action planning component, and to determine the target flattening and folding action sequence of the clothing to be processed from the target action template library based on the clothing description vector.

[0096] The risk action identification unit is used to identify risk actions in the target flattening and folding action sequence based on the low-risk action set and the protected area mask through the action planning component, so as to obtain the low-risk flattening and folding action set and the high-risk flattening and folding action set.

[0097] The second action sequence determination unit is used to determine the target fixed action sequence of the clothing to be processed based on the set of high-risk flattening and folding actions and the protective area mask through the action planning component.

[0098] Optionally, the second action sequence determination unit is specifically used for:

[0099] The motion planning component determines the areas of clothing that need to be fixed for each high-risk action in the high-risk flattening and folding action set and the protective area mask.

[0100] The motion planning component determines the target fixed motion sequence for the clothing to be processed based on the timestamps corresponding to each high-risk action in the high-risk flattening and folding action set and the area of ​​clothing that needs to be fixed.

[0101] Optionally, the robot control components include a dual-arm motion control sub-component and a local fixation sub-component; correspondingly, the clothing folding module 360 ​​is specifically used for:

[0102] The dual-arm motion control sub-component controls the dual arms of the target folding robot to perform flattening and folding operations on the clothes to be processed according to the target flattening and folding action sequence.

[0103] The local fixing sub-component controls the adsorption tool on the target worktable to perform local fixing operations on the clothes to be treated during the process of flattening and folding the clothes, according to the target fixing action sequence.

[0104] Optionally, the safety protection component includes a risk prediction sub-component; correspondingly, the action evaluation module 340 is specifically used for:

[0105] The risk prediction subcomponent predicts the clothing pose of the clothing to be processed after each folding action in the candidate folding action set based on the outline of the finished clothing area and the clothing mask in the scene label image, thus obtaining the clothing pose set corresponding to the candidate folding action set.

[0106] The risk prediction subcomponent determines a set of low-risk actions from the candidate clothes-folding action set based on the clothing pose set and the desktop edge line set.

[0107] Optionally, the safety protection component also includes a zone protection sub-component; correspondingly, the action evaluation module 340 is also specifically used for:

[0108] The region protection sub-component determines the outlines of the finished clothing area and the desktop clutter area based on the scene label map;

[0109] The area protection sub-component determines the protective area mask of the target workbench based on the outline of the finished clothing area and the outline of the desktop clutter area.

[0110] The dual-arm folding robot control device provided in this application embodiment can execute the dual-arm folding robot control method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing each dual-arm folding robot control method.

[0111] According to embodiments of this application, this application also provides an electronic device, a readable storage medium, and a computer program product.

[0112] Example 4

[0113] Figure 4 This is a schematic diagram of the electronic device 410 implementing the dual-arm folding robot control method of the embodiments of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0114] like Figure 4 As shown, the electronic device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0115] Multiple components in electronic device 410 are connected to I / O interface 415, including: input unit 416, such as keyboard, mouse, etc.; output unit 417, such as various types of displays, speakers, etc.; storage unit 418, such as disk, optical disk, etc.; and communication unit 419, such as network card, modem, wireless transceiver, etc. Communication unit 419 allows electronic device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0116] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as the control method for a dual-arm folding clothes robot.

[0117] In some embodiments, the dual-arm folding robot control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the dual-arm folding robot control method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured for the dual-arm folding robot control method by any other suitable means (e.g., by means of firmware).

[0118] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to the processor of a general-purpose computer, special-purpose computer, or other programmable dual-arm folding robot control device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on the machine, partially on the machine, or as a standalone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0120] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0121] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0122] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0123] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0124] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0125] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A double-arm laundry folding robot control method, characterized by, The application relates to an intelligent folding control system applied to a target folding robot; the intelligent folding control system comprises a visual perception component, a scene division component, a clothes classification component, a safety protection component, an action planning component and a robot control component; The intelligent folding control system is connected in communication among the components; the method comprises the following steps: Through the visual perception component, image acquisition is performed on a target workbench currently located by the target folding robot based on a preset acquisition frequency, current color images and current depth images of the target workbench are obtained, and three-dimensional point cloud data of the target workbench is determined according to the current color images and the current depth images; Through the scene division component, plane fitting is performed on the three-dimensional point cloud data, a desktop plane point set of the target workbench is obtained, and a scene label graph and a desktop edge line set of the target workbench are determined according to the three-dimensional point cloud data and the desktop plane point set; Through the clothes classification component, a clothes mask of clothes to be processed in the target workbench is determined according to the scene label graph, and a clothes description vector of the clothes to be processed is determined according to the clothes mask; Through the safety protection component, a low-risk action set is determined from a candidate folding action set according to the scene label graph, the desktop edge line set and the clothes mask, and a protection area mask of the target workbench is determined according to the scene label graph; Through the action planning component, a target folding action sequence of the clothes to be processed is determined according to the clothes description vector, the low-risk action set and the protection area mask; Through the robot control component, the target folding action sequence is used to control the dual arms of the target folding robot and a suction tool on the target workbench, and folding processing is performed on the clothes to be processed.

2. The method of claim 1, wherein, The target folding action sequence comprises a target flattening and folding action sequence and a target fixing action sequence; correspondingly, the target folding action sequence of the clothes to be processed is determined by the action planning component according to the clothes description vector, the low-risk action set and the protection area mask, and the target folding action sequence comprises the following steps: The action planning component is used to determine a target action template library from a candidate action template library according to the clothes description vector, and to determine the target flattening and folding action sequence of the clothes to be processed from the target action template library according to the clothes description vector; The action planning component is used to perform risk action identification on the target flattening and folding action sequence according to the low-risk action set and the protection area mask, to obtain a low-risk flattening and folding action set and a high-risk flattening and folding action set; The action planning component is used to determine the target fixing action sequence of the clothes to be processed according to the high-risk flattening and folding action set and the protection area mask.

3. The method of claim 2, wherein, The target fixing action sequence of the clothes to be processed is determined by the action planning component according to the high-risk flattening and folding action set and the protection area mask, and the target fixing action sequence comprises the following steps: determine, by the action planning component, a garment fixation region corresponding to each high-risk action in the high-risk flattening and folding action set according to the high-risk flattening and folding action set and the protection region mask; determine, by the action planning component, a target garment fixation action sequence of the garment to be processed according to the garment fixation region and the time stamp corresponding to each high-risk action in the high-risk flattening and folding action set.

4. The method of claim 2, wherein, The robot control component includes a dual-arm motion control subcomponent and a local fixation subcomponent; accordingly, the process of controlling the target folding robot, the dual arms of the target folding robot, and the adsorption tool on the target workbench to perform folding processing on the garment to be processed according to the target folding action sequence includes: controlling, by the dual-arm motion control subcomponent, the dual arms of the target folding robot to perform garment flattening and folding operations on the garment to be processed according to the target flattening and folding action sequence; controlling, by the local fixation subcomponent, the adsorption tool on the target workbench to perform garment local fixation operations on the garment to be processed during the garment flattening and folding process according to the target fixation action sequence.

5. The method of claim 1, wherein, The safety protection component includes a risk prediction subcomponent; accordingly, the process of determining a low-risk action set from a candidate folding action set according to the scene label map, the desktop edge line set, and the garment mask includes: predicting, by the risk prediction subcomponent, the garment pose of the garment to be processed after the execution of each folding action in the candidate folding action set according to the finished garment region contour in the scene label map and the garment mask to obtain a garment pose set corresponding to the candidate folding action set; determining, by the risk prediction subcomponent, the low-risk action set from the candidate folding action set according to the garment pose set and the desktop edge line set.

6. The method of claim 5, wherein, The safety protection component further includes a region protection subcomponent; accordingly, the process of determining a protection region mask of the target workbench according to the scene label map includes: determining, by the region protection subcomponent, a finished garment region contour and a desktop clutter region contour according to the scene label map; determining, by the region protection subcomponent, the protection region mask of the target workbench according to the finished garment region contour and the desktop clutter region contour.

7. A dual-arm laundry folding robot control device characterized by comprising: An intelligent folding control system configured on a target folding robot; the intelligent folding control system includes a visual perception component, a scene division component, a garment classification component, a safety protection component, an action planning component, and a robot control component; The intelligent folding control system includes: an image acquisition module configured to acquire images of a target workbench currently located by the target folding robot based on a preset acquisition frequency to obtain a current color image and a current depth image of the target workbench, and determine three-dimensional point cloud data of the target workbench according to the current color image and the current depth image; a point cloud analysis module, configured to perform plane fitting on the three-dimensional point cloud data by the scene division component to obtain a desktop plane point set of the target workbench, and determine a scene label map and a desktop edge line set of the target workbench according to the three-dimensional point cloud data and the desktop plane point set, a clothes classification module, configured to determine a clothes mask of clothes to be processed in the target workbench according to the scene label map by the clothes classification component, and determine a clothes description vector of the clothes to be processed according to the clothes mask; an action evaluation module, configured to determine a low-risk action set from a candidate folding action set according to the scene label map, the desktop edge line set and the clothes mask by the safety protection component, and determine a protection area mask of the target workbench according to the scene label map; an action planning module, configured to determine a target folding action sequence of the clothes to be processed according to the clothes description vector, the low-risk action set and the protection area mask by the action planning component; a clothes folding module, configured to control a dual-arm of the target folding robot and a suction tool on the target workbench to perform folding processing on the clothes to be processed according to the target folding action sequence by the robot control component.

8. An electronic device, comprising: comprise: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the dual-arm folding robot control method according to any one of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the dual-arm folding robot control method according to any one of claims 1-6.

10. A computer program product comprising a computer program which, when executed by a processor, implements the dual-arm folding robot control method according to any one of claims 1-6.

10. A computer program product comprising a computer program which, when executed by a processor, implements the dual-arm folding robot control method according to any one of claims 1-6.

Images