Fixtureless dual-arm robot adaptive sewing method and system based on real-time depth perception of rgbd camera

Abstract

This application discloses an adaptive sewing method and system for a gripperless dual-arm robot based on real-time depth perception using an RGBD camera. The method involves extracting three geometric features—maximum height deviation, standard deviation of the normal vector angle, and wrinkle area ratio—from a 3D point cloud of the fabric surface constructed from color and depth images, and weighted and fused them into a comprehensive wrinkle metric Φ. Based on the comprehensive wrinkle metric Φ, the wrinkle state of the fabric is defined as a smooth state, a slightly wrinkled state, and a severely wrinkled state. Adaptive gain scheduling is performed on the dual-arm impedance controller of the dual-arm robot according to the comprehensive wrinkle metric Φ, allowing the dual-arm impedance controller to adjust the interaction force between the end effector of the dual-arm robot and the fabric using an impedance control law. This invention is the first to introduce 3D depth information into sewing wrinkle detection. The extracted Φ index simultaneously includes three dimensions: height, normal vector, and area, providing a quantitative description of wrinkles that is far superior to methods based on 2D segmentation.

Description

Technical Field

[0001] This invention belongs to the field of industrial robot and intelligent manufacturing technology, specifically relating to a method for real-time sensing and adaptive control of fabric wrinkles in a gripperless dual-arm robot sewing system. Background Technology

[0002] Sewing is the most crucial process in garment production, handling approximately 80% of fabric joining and accounting for about 40% of the overall production cost. Industrial sewing machines can achieve high-speed sewing of up to 5,000 stitches per minute (approximately 80 stitches per second), while skilled workers must repeatedly perform operations such as grasping, aligning, feeding, and inspecting during the sewing process. At the same time, they must control the tension of the fabric with their hands to prevent wrinkling. Production efficiency and quality heavily depend on the worker's experience.

[0003] The main challenges in achieving sewing automation are: fabrics are flexible materials with complex deformations that are difficult to model; sewing requires extremely high real-time performance; in fixtureless sewing, the fabric relies entirely on the friction of the end effector to maintain its shape, and stress concentration can easily cause wrinkles during the gripping and switching of multiple sewing segments. Without active intervention, the presser foot will compact the wrinkles into permanent creases, severely affecting the appearance and structural strength of the garment.

[0004] Existing solutions can be categorized into three types: Sewing systems based on fixed-parameter impedance control employ dual-arm impedance control with fixed stiffness and damping parameters, using the same control parameters throughout the sewing process. This approach cannot adaptively adjust to changes in fabric shape and cannot proactively respond when local wrinkles occur, leading to continuous wrinkle accumulation and eventually permanent creases. Sewing systems based on 2D image fabric edge detection utilize ordinary color or grayscale cameras to detect fabric edges and control the sewing trajectory through visual servoing. This approach can only acquire 2D contour information of the fabric and cannot perceive the 3D undulations of the fabric surface; wrinkle detection relies entirely on manual intervention. Wrinkle detection methods based on fabric segmentation model the fabric edge detection problem as a semantic segmentation problem, with a detection frame rate of approximately 30 FPS and an average error of approximately 0.61 mm. While this approach can detect edges, it still relies on 2D information and does not utilize depth data, making it impossible to quantitatively assess the 3D geometric features of wrinkles. Furthermore, the detection speed is insufficient to meet the real-time control requirements of high-speed sewing. Summary of the Invention

[0005] The purpose of this invention is to address the core problem of fabric wrinkling caused by free deformation in gripperless dual-arm robot sewing. It introduces RGBD depth sensing into the sewing control closed loop and achieves at least one of the following specific objectives: real-time acquisition of the three-dimensional morphology of the fabric surface during sewing to quantitatively assess the severity of wrinkles; adaptive adjustment of the impedance control parameters of the dual-arm robot's impedance controller based on the wrinkle state, actively stretching the fabric during minor wrinkles; automatic pause, ironing, and resuming of stitches when severe wrinkles occur, without manual intervention throughout the process; and an overall sensing-control closed-loop frequency of no less than 60Hz to meet the real-time control requirements of high-speed industrial sewing.

[0006] To this end, some embodiments of this application provide an adaptive sewing method for a fixtureless dual-arm robot based on real-time depth perception using an RGBD camera, for achieving high-quality automatic sewing of two pre-aligned fabric pieces along their edges. This invention introduces an RGBD camera into the fixtureless sewing system of a dual-arm robot to perceive the three-dimensional morphology of the fabric surface in the sewing area in real time, extract the comprehensive wrinkle metric Φ, and drive the dual-arm robot's dual-arm impedance controller to perform adaptive gain scheduling through finite state machine scheduling. When severe wrinkles occur, the fabric ironing operation R6 is automatically triggered to resume sewing from the breakpoint, thereby actively suppressing wrinkle defects caused by free deformation of the fabric during fixtureless sewing.

[0007] In some embodiments, the main steps of the method provided by the present invention include: continuously acquiring color and depth images of the sewing area using an RGBD camera, and constructing a three-dimensional point cloud of the fabric surface after preprocessing; extracting three geometric features from the point cloud: maximum height deviation, standard deviation of normal vector angle, and wrinkle area ratio, and weighted fusing them into a comprehensive wrinkle metric Φ; evaluating Φ at a frequency of not less than 60Hz using a finite state machine, classifying the wrinkle state of the fabric into three types: flat state, slightly wrinkled state, and severely wrinkled state, and driving the dual-arm impedance controller to perform adaptive gain scheduling; when the severely wrinkled state continues to occur, pausing sewing and recording the breakpoint coordinates, performing a dual-arm fabric ironing operation, and resuming sewing from the breakpoint after the wrinkles are eliminated.

[0008] The beneficial effects of this invention include: all schemes are based on two-dimensional images for fabric wrinkle perception; this invention, for the first time, introduces three-dimensional depth information into sewing wrinkle detection, and the extracted Φ index simultaneously includes three dimensions: height, normal vector, and area, providing a quantitative description of wrinkles that is far superior to methods based on two-dimensional segmentation. Existing impedance control schemes use fixed gain parameters and cannot adaptively respond to wrinkle states. This invention, through a gain scheduling mechanism, actively increases lateral tension and reduces normal stiffness in the slight wrinkle stage, achieving early intervention in wrinkles rather than waiting for them to worsen. Existing schemes lack an automatic recovery mechanism after severe wrinkles occur, requiring manual intervention. The R6 ironing operation defined in this invention, combined with a breakpoint sewing continuation mechanism, achieves automatic wrinkle repair without manual intervention throughout the entire process. The perception-control closed-loop frequency of this invention reaches 60Hz, meeting the real-time control requirements of high-speed sewing in industrial sewing machines, and is superior to existing segmentation-based methods (approximately 30 FPS). Attached Figure Description

[0009] Figure 1 This is a hardware topology diagram of the fixtureless dual-arm robot adaptive sewing system based on real-time depth perception using an RGBD camera, as described in this application.

[0010] Figure 2 This is a flowchart of the adaptive sewing method for a gripperless dual-arm robot based on real-time depth perception using an RGBD camera, as described in this application.

[0011] Figure 3 The state transition diagram of the finite state machine for wrinkle treatment. Detailed Implementation

[0012] Terminology Explanation: Fixtureless sewing refers to a sewing method in which no fixed fixtures are used to constrain the fabric during the sewing process. The fabric shape is maintained entirely by the friction of the end effector of the robotic arm, allowing for free sewing along the fabric edge.

[0013] Impedance control is a robot force control method that sets the dynamic response characteristics of the end effector, including stiffness, damping, and inertia, when subjected to external forces, so that the robot exhibits elastic behavior when interacting with the environment, taking into account both force and position control.

[0014] A finite state machine is a control model used to describe discrete event systems. It consists of a finite number of state nodes and state transition arcs with triggering conditions. The system is in a unique state at any given time, and switches to the next state when the transition condition is met.

[0015] The wrinkle comprehensive measurement index Φ is a composite index proposed in this invention for quantifying the severity of wrinkles on the surface of fabrics. It is calculated by weighted fusion of three geometric features: maximum height deviation, standard deviation of normal vector angle, and wrinkle area ratio.

[0016] In terms of overall system architecture, such as Figure 1 As shown, this invention provides an adaptive sewing system for a fixtureless dual-arm robot based on real-time depth perception using an RGBD camera. This system enables high-quality automatic sewing of two pre-aligned fabric pieces along their edges, effectively solving the wrinkling problem caused by free deformation of the fabric during fixtureless sewing. Specifically, the system includes a dual-arm robot 100 as the actuator that directly interacts with the fabric, an industrial sewing machine unit 200 for high-speed sewing, an RGBD camera 300 (a depth camera) for real-time perception of the three-dimensional morphology of the fabric surface, a wide-angle color camera 400 for global fabric pose estimation, and an industrial control computer for deploying adaptive sewing control software to implement the fixtureless dual-arm robot adaptive sewing method based on real-time depth perception using an RGBD camera proposed in this application.

[0017] The dual-arm robot 100 has six degrees of freedom in both its left arm 100A and right arm 100B, and its wrists are equipped with torque sensors. The industrial sewing machine 200 includes a servo spindle, an electrically controlled fabric pressing foot, and an automatic thread cutting device. The RGBD camera 300 and the wide-angle color camera 400 are used to acquire image information of the sewing area. The industrial control computer serves as the core control unit of the system, deploying adaptive sewing control software. For hardware communication, the industrial control computer communicates in real-time with the servo drivers of the dual-arm robot and the industrial sewing machine via an EtherCAT bus. The hardware for these devices is already commercially available and will not be described in detail here.

[0018] Furthermore, the adaptive sewing control software in the industrial control computer of the present invention adopts a five-layer hierarchical control architecture, which includes a perception layer, a discrete event control layer, a basic operation layer, a control primitive layer, and a hardware layer. The perception layer is used to acquire the pose information and wrinkle state of the fabric in real time; the discrete event control layer schedules the sewing task flow based on Petri nets; the basic operation layer defines nine atomic operations, R1 to R6 and S1 to S3; the control primitive layer provides low-level control algorithms such as trajectory generation, impedance control, and visual servoing; and the hardware layer is responsible for executing the final control commands.

[0019] In terms of hardware communication, the industrial control computer communicates in real time with the servo drivers of the dual-arm robot 100 and the industrial sewing machine 200 via the EtherCAT bus, with a control cycle of 1ms; the RGBD camera 300 synchronously acquires color and depth images at a frame rate of 60fps via the USB 3.0 interface.

[0020] Coordinate system definition: This includes defining a world coordinate system, with the origin fixed at the projection of the sewing needle point onto the table, the X-axis pointing in the opposite direction of the feed direction, and the Z-axis perpendicular to the table and pointing upwards. It also defines a sewing motion coordinate system, with the X-axis parallel to the tangent of the current sewing thread at the needle point. Finally, it defines an RGBD camera coordinate system. The transformation relationship between the coordinate system and the world coordinate system is established through hand-eye calibration. .

[0021] Basic operation definition: The basic operations of the dual-arm robot include: R1: moving the end effector to the target pose; R2: grasping / releasing the fabric by pressing or lifting it off the table; R3: collaboratively moving the fabric to the target pose; R4: aligning the fabric edges with the sewing direction using edge detection visual servoing; R5: the two arms follow the feed direction and sew synchronously; and R6: ironing the fabric—when the sensing layer detects severe wrinkles, sewing is paused, and the two arms apply horizontal smoothing tension outward to flatten the fabric. Sewing resumes from the break point after the wrinkles are eliminated. R6 is the core operation of this invention addressing the problem of free fabric deformation in jigless sewing: in jigless sewing, the fabric relies entirely on the friction of the end effector to maintain its shape. During the grasping and switching process of multi-segment sewing, wrinkles are easily generated due to stress concentration. If not actively intervened, the presser foot will press the wrinkles into permanent creases, affecting the sewing quality.

[0022] Step S1: Initialization and calibration.

[0023] After the system is powered on, the initialization and calibration process is completed first. Specifically, this step includes obtaining the intrinsic parameters of the RGBD camera using Zhang Zhengyou's checkerboard method. A depth correction model is then established to compensate for sensor errors, as shown in equation (1): (1); where, It is a linear proportionality coefficient. The bias is obtained through multiple acquisitions and fitting of known distance planes. In this embodiment, five known distance planes within a range of 200-600mm from the camera are used for calibration, and typical values ​​are obtained through fitting. After correction, the absolute depth error is less than 0.5mm within the working distance range of 300~500mm. Simultaneously, hand-eye calibration is performed to establish the transformation matrix between the camera coordinate system and the world coordinate system, and the average depth of the ROI on the platform is collected under non-woven conditions as a reference height for flatness.

[0024] Step S2, fabric pose estimation.

[0025] The process includes: after the operator places and pre-aligns the fabric, a wide-angle camera acquires a color image, extracts the fabric contour point cloud, and aligns the real-time point cloud with the CAD model using RANSAC coarse registration and ICP fine registration algorithms. The maximum number of iterations for RANSAC coarse registration is 1000, with an in-point distance threshold of 3.0 mm; the maximum number of iterations for ICP fine registration is 50, with a convergence root mean square error threshold of 0.1 mm. The total registration time is approximately 15~30 ms, meeting real-time requirements. The global pose of the fabric is then estimated, and the initial gripping points for the dual arms are planned accordingly.

[0026] Step S3: RGBD depth map preprocessing.

[0027] An RGBD camera continuously monitors a Region of Interest (ROI) with a radius of 80mm centered on the needle point. Upon receiving the image, the industrial control computer first performs depth correction and segmentation on each frame. A lightweight semantic segmentation network, such as a ResNet-18 backbone, is used to generate a fabric mask. This lightweight semantic segmentation network employs a ResNet-18 encoder and a lightweight decoder structure. The input image resolution is 256×256 pixels, and the network is trained on a self-built fabric dataset containing approximately 2000 labeled images. This dataset includes samples of cotton, polyester, and blended fabrics under different folding conditions. The inference speed on the NVIDIA Jetson AGXOrin platform is no less than 120 FPS, and the mIoU of the fabric region segmentation is no less than 0.93. Denoising processing is then performed, including filling holes within the fabric mask and applying statistical outlier removal (SOR). For each pixel... Nearest neighbors, if they meet the statistical outlier removal criteria. Then, it is discarded. Next, Gaussian smoothing is applied to filter out high-frequency noise. The Gaussian smoothing kernel size is 5×5 pixels, and the standard deviation is... =1.5 pixels. After the above preprocessing, the standard deviation of the depth map noise was reduced from approximately 2.5 mm to below 0.8 mm.

[0028] Step S4, Point Cloud Construction and Coordinate Transformation Furthermore, the effective pixels within the fabric mask are back-projected onto the camera coordinate system, as shown in equation (2): (2); in, This is the camera's intrinsic parameter. It is then transformed to the world coordinate system, as shown in equation (3): (3); Obtain the fabric point cloud of the current frame .

[0029] Step S5, surface geometry analysis.

[0030] This step includes: obtaining the point cloud In China, Take each point The nearest neighbor constructs the covariance matrix, as shown in equation (4): (4); Through the Eigenvalue decomposition is performed, and the eigenvector corresponding to the smallest eigenvalue is the normal vector. And uniformly adjust the orientation to make Simultaneously calculate the height of each point relative to the reference plane, as shown in equation (5): (5); Step S6, then derive the comprehensive wrinkle measurement index. This includes extracting three geometric features and performing weighted fusion to obtain a comprehensive wrinkle measurement index. As shown in equation (6): (6); Among them, the maximum height deviation ; Standard deviation of normal vector The wrinkle area ratio is the ratio of the area of ​​the wrinkled region to the total area. ,in, .

[0031] The wrinkled region is determined by a combined threshold of height deviation and normal vector deviation.

[0032] Weighting coefficient Offline calibration was performed using least squares regression on 100 manually rated samples. In this embodiment, the three geometric features were first normalized by dividing by the reference height. , Divided by reference angle , It is itself Dimensionless quantity. After least squares regression of 100 standardized samples, the typical weighting coefficient is... , , The sum of the three is Regression fit determination coefficient .

[0033] In this embodiment, , .Right now Determined to be flat, Determined to be a slight wrinkle. The folds are classified as severe wrinkles. This threshold combination achieves a wrinkle classification accuracy of over 95% on the validation set. In practical applications, the classification can be further refined based on the fabric material (e.g., cotton, polyester, silk). Fine-tuning within ±15%.

[0034] Step S7, using dual thresholds , The wrinkle state is defined as shown in Equation (8), which defines three wrinkle states and judgment criteria, the transition conditions between any two states, and the corresponding operations after the transition conditions are met: (8); Further execute step S8 to perform adaptive dual-arm impedance control.

[0035] The interaction force between the end effector of any arm of the dual-arm robot and the fabric is adjusted using an impedance control law, as shown in equation (9): (9); in, Represents the i-th arm of the left or right arm of a dual-arm robot; Let be the expected inertia matrix of the end effector of the i-th arm; and These are the comprehensive measurement indicators of wrinkles. Dynamically scheduled damping and stiffness matrices; , , These represent the position deviation, velocity deviation, and acceleration deviation of the i-th arm end effector relative to the desired trajectory in the world coordinate system; This is the selection matrix for the force spinor to the end effector coordinate system; Let be the six-dimensional force spinor of the end effector of the i-th arm measured in the world coordinate system.

[0036] Among them, the stiffness matrix With damping matrix It switches dynamically according to the status.

[0037] like Figure 3 As shown, the system uses a finite state machine to monitor the movement of the two arms in the feeding direction and the synchronous stitching process (i.e., During execution, the wrinkle state response is adaptively scheduled, and the finite state machine continuously evaluates the current wrinkle comprehensive metric at a frequency of 60Hz or higher. And after the transition conditions are met, a state transition is triggered: State 0 to State 1: Transition condition is After triggering, a gain scheduling command is sent to the control primitive layer, switching to the slight wrinkle parameter group, and the sewing speed is reduced to the rated value. ; State 1 to State 0: Transition conditions are After being triggered, normal sewing parameters are restored; State 1 to State 2: Transition condition is Execute after triggering Ironing fabric; Status 2 Completed To state 0: The transition condition is And duration After being triggered, from the breakpoint coordinates Resume sewing.

[0038] All state transition condition judgments are completed within a single control cycle, such as 1 ms.

[0039] Under the above transfer conditions Execute after being triggered The fabric ironing operation specifically includes: first, recording the current sewing stroke coordinates. As a breakpoint, let the spindle speed be... Then execute Lift the presser foot, Release the normal constraint; under impedance control, both arms perform a smoothing motion from the inside to the outside along the fabric surface, applying lateral tension outwards. The reference value for the lateral tension is... =3~5N, normal contact force maintained at 0.3~0.8N to prevent fabric from detaching from the table and to avoid indentation, smoothing motion speed at 30~50mm / s, single smoothing stroke not exceeding 60mm, normal force maintained at light contact and continuously monitored. .

[0040] when Persistently below The ironing is considered complete after 0.3 seconds. If the smoothing action is performed three times consecutively... It has not yet dropped to If the system then issues an alarm and pauses, it will wait for manual intervention. The typical total time for a single R6 operation is 1.5~3.0 seconds.

[0041] After ironing, the dual-arm robot returns to the sewing position to perform... Press down the fabric foot, from the breakpoint coordinates For stitch resumption at the last stitch position, the start-up time of the sewing machine spindle from stop to return to rated speed is approximately 0.2 seconds. The stitch recovery positioning accuracy is no greater than ±0.5mm, guaranteed by a closed-loop encoder. The first to third stitches of the resumed sewing use a deceleration start, for example, 50% of the rated speed, and then linearly accelerate to the rated speed.

[0042] Compared with the prior art, the present invention has the following significant advantages: (1) In view of the shortcomings of existing solutions that are based on two-dimensional images for fabric state perception and cannot quantitatively evaluate three-dimensional wrinkles, the present invention introduces the three-dimensional depth information of the RGBD camera into sewing wrinkle detection in steps S4 to S6 of the specific implementation method. The wrinkle comprehensive measurement index Φ shown in Equations (2) and (3) is formed by weighted fusion of three dimensions: maximum height deviation, standard deviation of normal vector angle and wrinkle area ratio. This realizes the real-time acquisition of the three-dimensional morphology of the fabric surface and the accurate quantitative evaluation of the severity of wrinkles. Its quantitative description ability is far superior to the existing two-dimensional segmentation-based methods. (2) In view of the shortcomings of the existing impedance control scheme which uses fixed gain parameters and cannot adapt to the wrinkle state, in step S8 of this invention, the impedance control law shown in equation (9) is used and Φ is evaluated by a finite state machine at a frequency of 60Hz or higher to achieve adaptive gain scheduling: in the slight wrinkle stage (state 1), the transverse tension is actively increased and the normal stiffness is reduced, so as to achieve early active intervention on the wrinkles, rather than passively waiting for the wrinkles to deteriorate. (3) In view of the shortcomings of existing solutions that lack an automatic recovery mechanism after severe wrinkles occur and require manual intervention, this invention defines an R6 fabric ironing operation and, together with a breakpoint sewing mechanism, automatically pauses sewing, performs ironing, and resumes sewing from the breakpoint when severe wrinkles (state 2) occur, thus achieving automatic wrinkle repair without manual intervention throughout the process. (4) The sensing-control closed-loop frequency of the present invention is not less than 60Hz, which meets the real-time control requirements of high-speed sewing in industrial sewing machines and is significantly better than the existing segmentation-based method (about 30FPS).

Claims

1. A fixtureless dual-arm robot adaptive sewing method based on real-time depth perception of an RGBD camera, characterized in that, The method includes the following steps: After preprocessing the color and depth images of the acquired sewing area, a three-dimensional point cloud of the fabric surface is constructed. The maximum height deviation, the standard deviation of the normal vector angle, and the proportion of wrinkle area are extracted from the three-dimensional point cloud and then weighted and fused into a comprehensive wrinkle metric Φ. Based on the comprehensive wrinkle measurement index Φ, the wrinkle state of the fabric is defined as a smooth state, a slightly wrinkled state, and a severely wrinkled state. The transition conditions between the three states and the operation in each state are also defined. A finite state machine is used to evaluate the wrinkle comprehensive metric Φ at a preset frequency, and adaptive gain scheduling is performed on the dual-arm impedance controller of the dual-arm robot based on the wrinkle comprehensive metric Φ, so that the dual-arm impedance controller uses the impedance control law to adjust the interaction force between the end effector of the dual-arm robot and the fabric.

2. The method of claim 1, wherein, The wrinkle state is divided using the dual thresholds Φ1 and Φ2 of the wrinkle comprehensive measurement index Φ: Φ≤Φ1 is a smooth state, Φ1<Φ≤Φ2 is a slightly wrinkled state, and Φ>Φ2 is a severely wrinkled state.

3. The method of claim 2, wherein, When severe wrinkles are detected to be continuously occurring, suspend sewing and record the coordinates of the break point. Perform an ironing operation on the fabric and resume sewing from the break point after the wrinkles are eliminated.

4. The method according to claim 1, characterized in that, The construction of the 3D point cloud includes: back-projecting the effective pixels within the fabric mask of the fabric to the camera coordinate system, and then transforming them to the world coordinate system.

5. The method according to claim 1, characterized in that, The comprehensive wrinkle measurement index Φ is obtained through Calculation, where the maximum height deviation is... ; The standard deviation of the normal vector; The wrinkle area ratio is the ratio of the wrinkled area to the total area; weighting coefficient. The samples were calibrated offline using least squares regression with multiple sets of manually rated samples.

6. The method according to claim 1, characterized in that, The interaction force between the end effector and the fabric is adjusted using the following control law formula: ; in, The i-th arm represents the left or right arm of the dual-arm robot. Let be the desired inertia matrix of the i-th arm end effector; and These are the comprehensive measurement indicators of wrinkles. Dynamically scheduled damping and stiffness matrices; , , These represent the position deviation, velocity deviation, and acceleration deviation of the i-th arm end effector relative to the desired trajectory in the world coordinate system; This is the selection matrix for the force spinor to the end effector coordinate system; The six-dimensional force spinor of the i-th arm end effector measured in the world coordinate system; For the corresponding expected six-dimensional force spinor; stiffness matrix With damping matrix The system dynamically switches according to the wrinkle state, increasing the lateral tension and reducing the normal stiffness during the slight wrinkle stage, thus achieving early active intervention in wrinkles.

7. The method according to claim 3, characterized in that, The fabric ironing operation includes: recording the current sewing stroke coordinates as the breakpoint, lifting the presser foot, applying lateral tension from the inside to the outside along the fabric surface under impedance control by the dual-arm robot to smooth the fabric, continuously monitoring the comprehensive wrinkle measurement index Φ, and pressing down the presser foot and continuing the sewing from the breakpoint position when Φ meets the set conditions.

8. The method according to claim 1, characterized in that, The wrinkle composite metric Φ is evaluated using the finite state machine at a frequency of not less than 60 Hz.

9. An adaptive sewing system for a gripperless dual-arm robot based on real-time depth perception using an RGBD camera, characterized in that, include Dual-arm robots serve as actuators that directly interact physically with fabrics; Industrial sewing machine units are used to achieve high-speed sewing of fabrics; RGBD camera, used to realize real-time perception of the three-dimensional shape of fabric surface; A wide-angle color camera is used to achieve global pose estimation of the fabric, and The control device includes a memory for storing code for adaptive sewing control software; The communication unit is used to communicate with the dual-arm robot, the industrial sewing machine unit, the RGBD camera, and the wide-angle color camera; And a processor for executing the code to implement the adaptive sewing method as described in any one of claims 1 to 8.

Images