Method and apparatus for treating flexible textile

Abstract

This invention discloses a method and apparatus for processing flexible textiles. The method includes: acquiring an image of the disordered state of the flexible textile using a multi-view vision system; identifying key feature points of the flexible textile in the disordered image using a target detection model to obtain the coordinates of these key feature points; performing coordinate transformation on the key feature point coordinates to obtain their three-dimensional spatial coordinates in the robot's base coordinate system; and controlling a robot to perform a flattening operation on the flexible textile based on these three-dimensional spatial coordinates. This invention solves the technical problems of high failure rates and low automation levels in traditional towel processing equipment due to the irregular shape of soft fabrics, difficulty in accurately identifying feature points, and the risk of collisions during robot movement.

Description

Technical Field

[0001] This invention relates to the field of robot automation technology, and more specifically, to a method and apparatus for processing flexible textiles. Background Technology

[0002] While smart home and automated cleaning technologies have gradually introduced garment sorting equipment, the sorting process for soft, wrinkle-prone, and highly irregularly shaped textiles such as towels still heavily relies on manual operation. Traditional mechanical sorting devices generally use conveyor belts with fixed trajectories or rigid clamping structures, which are difficult to adapt to the complex shapes of towels when naturally stacked, such as disorder, twisting, and obstruction. This leads to frequent problems such as corner recognition failure, fold misalignment, residual wrinkles, and even gripper slippage. Existing technologies mostly rely on traditional image processing methods such as edge detection or template matching, which are inaccurate in low-contrast, partially obscured, or lighting-changing scenarios, and lack the ability to perceive and respond to the dynamic deformation process of towels. In terms of robot motion control, there are generally problems with static trajectory planning and no predictive mechanism. When two arms work together, they are prone to collisions due to inconsistent postures, or gripping failure due to failure to predict the flexible deformation of towels during the gripping process. The overall system has a low level of intelligence and poor adaptability, making it difficult to achieve stable, efficient, and unattended automated sorting in real complex home or commercial environments.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a method and apparatus for processing flexible textiles, which at least solves the technical problems of high failure rate and low automation in traditional towel processing equipment due to the irregular shape of soft fabrics, difficulty in accurately identifying feature points, and easy collisions caused by robot movement.

[0005] According to one aspect of the present invention, a method for processing flexible textiles is provided, comprising: acquiring an image of the disordered state of the flexible textile in a disordered state through a multi-view vision system, wherein the multi-view vision system includes at least one fixedly mounted main camera and at least two auxiliary cameras mounted at the end effector of a robotic arm; identifying key feature points of the flexible textile in the disordered state image using a target detection model to obtain the coordinates of key feature points of the flexible textile; wherein the target detection model is a model trained by machine learning using multiple sets of training data, the input of the target detection model is a sample image of the flexible textile, and the output of the target detection model is the coordinates of the sample feature points of the flexible textile, wherein the key feature points include at least one of the following of the flexible textile: corner points, geometric highest points; performing coordinate transformation on the coordinates of the key feature points to obtain the three-dimensional spatial coordinates of the key feature points in the robot's base coordinate system; and controlling the robot to perform a flattening operation on the flexible textile based on the three-dimensional spatial coordinates.

[0006] Optionally, before identifying key feature points of the flexible textile in the disordered state image using a target detection model to obtain the coordinates of the key feature points of the flexible textile, the processing method further includes: acquiring the sample image and labeling the sample image to obtain the coordinates of the sample feature points; preprocessing the sample image to obtain a preprocessed sample image, wherein the preprocessing includes at least one of the following: size normalization processing and illumination equalization processing; mapping the coordinates of the sample feature points to an image pixel coordinate system to obtain multiple sets of training data including the preprocessed sample image and the coordinates of the sample feature points; and processing the multiple sets of training data using machine learning to obtain the target detection model.

[0007] Optionally, the target detection model is obtained by processing the multiple sets of training data through machine learning, including: inputting the multiple sets of training data into a convolutional neural network to extract multi-scale feature images; calculating the spatial gradient magnitude of the multi-scale feature images to obtain spatial gradient magnitude calculation results; generating a normalized spatial attention map based on the spatial gradient magnitude calculation results, and simultaneously performing global average pooling processing on the channel dimension to obtain channel activation vectors; calculating channel attention weights based on the channel activation vectors; weighting and fusing the spatial attention map and the channel attention weights according to learning parameters to obtain enhancement weights; obtaining enhancement features based on the enhancement weights, and inputting the features into the detection head for bounding box regression and corner point category scoring to obtain the target detection model.

[0008] Optionally, based on the enhancement weights, enhanced features are obtained and input into the detection head for bounding box regression and corner category scoring to obtain the target detection model. This includes: nonlinearly modulating the enhancement weights through an activation function and multiplying them element-wise with the original feature map to obtain the enhanced features; feeding the enhanced features into the detection head for bounding box regression and corner category scoring; and removing redundant prediction boxes through non-maximum suppression to optimize the loss function until convergence, thereby obtaining the target detection model.

[0009] Optionally, the key feature points of the flexible textile in the disordered state image are identified by a target detection model to obtain the coordinates of the key feature points of the flexible textile. This includes: inputting the disordered state image into the target detection model to process the disordered state image using the target detection model; when no corner points of the flexible textile are detected, controlling the robot to grasp the geometric highest point of the flexible textile to deploy the flexible textile; performing corner point identification again until an intersection point that meets the confidence level is identified; and outputting the coordinates of the key feature points of the flexible textile.

[0010] Optionally, coordinate transformation is performed on the coordinates of the key feature points to obtain the three-dimensional spatial coordinates of the key feature points in the robot base coordinate system. This includes: determining the camera number from which the image originates based on the image corresponding to the key feature point coordinates; selecting a hand-eye calibration matrix based on the camera number, and restoring the image coordinates corresponding to the key feature points to normalized coordinates in the camera coordinate system through inverse camera intrinsic parameter transformation; and transforming the normalized coordinates to the robot base coordinate system through the hand-eye calibration matrix to obtain the three-dimensional spatial coordinates of the flexible textile.

[0011] Optionally, controlling the robot to perform a flattening operation on the flexible textile based on the three-dimensional spatial coordinates includes: generating a candidate path for the robot's end effector from its initial pose to the target gripping pose, using the three-dimensional spatial coordinates as the gripping target; selecting the target gripping path from the candidate paths; and controlling the robot to perform a flattening operation on the flexible textile according to the target gripping path.

[0012] Optionally, selecting a target gripping path from the candidate paths includes: determining the motion path points for robot teaching; and optimizing the target gripping path using the motion path points to obtain an optimized target gripping path.

[0013] According to another aspect of the present invention, a processing apparatus for flexible textiles is also provided, comprising: a data acquisition unit for acquiring images of the disordered state of the flexible textile in a disordered state through a multi-view vision system, wherein the multi-view vision system includes at least one fixedly mounted main camera and at least two auxiliary cameras mounted at the end effector of a robotic arm; a recognition unit for identifying key feature points of the flexible textile in the disordered state images using a target detection model to obtain the coordinates of key feature points of the flexible textile; wherein the target detection model is a model trained using multiple sets of training data through machine learning, the input of the target detection model is a sample image of the flexible textile, and the output of the target detection model is the coordinates of the sample feature points of the flexible textile, wherein the key feature points include at least one of the following of the flexible textile: corner points, geometric highest points; a coordinate transformation unit for performing coordinate transformation on the coordinates of the key feature points to obtain the three-dimensional spatial coordinates of the key feature points in the robot's base coordinate system; and a control unit for controlling the robot to perform a flattening operation on the flexible textile based on the three-dimensional spatial coordinates.

[0014] Optionally, the processing apparatus further includes: an acquisition unit, configured to acquire the sample image and annotate the sample image to obtain the coordinates of the sample feature points before performing key feature point identification on the flexible textile in the disordered state image using a target detection model to obtain the coordinates of the key feature points of the flexible textile; a preprocessing unit, configured to preprocess the sample image to obtain a preprocessed sample image, wherein the preprocessing includes at least one of the following: size normalization processing and illumination equalization processing; a mapping unit, configured to map the coordinates of the sample feature points to an image pixel coordinate system to obtain multiple sets of training data including the preprocessed sample image and the coordinates of the sample feature points; and a processing unit, configured to process the multiple sets of training data using machine learning to obtain the target detection model.

[0015] Optionally, the processing unit includes: a first input module for inputting the multiple sets of training data into a convolutional neural network to extract multi-scale feature images; a first calculation module for calculating the spatial gradient magnitude of the multi-scale feature images to obtain spatial gradient magnitude calculation results; a first generation module for generating a normalized spatial attention map based on the spatial gradient magnitude calculation results, and simultaneously performing global average pooling processing on the channel dimension to obtain channel activation vectors; a second calculation module for calculating channel attention weights based on the channel activation vectors; a weighted fusion module for performing weighted fusion of the spatial attention map and the channel attention weights according to learning parameters to obtain enhancement weights; and a second input module for obtaining enhancement features based on the enhancement weights and inputting the features into a detection head for performing bounding box regression and corner point category scoring to obtain the target detection model.

[0016] Optionally, the second input module includes: a calculation submodule, used to multiply the enhancement weights element-wise with the original feature map after nonlinear modulation by an activation function to obtain the enhancement features; an input submodule, used to input the enhancement features into the detection head for bounding box regression and corner point category scoring; and a first optimization submodule, used to remove redundant prediction boxes through nonmaximum suppression to optimize the loss function until convergence, thereby obtaining the target detection model.

[0017] Optionally, the recognition unit includes: a third input module, used to input the disordered state image into the target detection model, so as to process the disordered state image using the target detection model, and when no corner of the flexible textile is detected, control the robot to grab the geometric highest point of the flexible textile to deploy the flexible textile, and perform corner recognition again until an intersection point that meets the confidence level is identified, and output the coordinates of the key feature points of the flexible textile.

[0018] Optionally, the coordinate transformation unit includes: a determination module, used to determine the camera number from which the image originates based on the image corresponding to the coordinates of the key feature points; a first transformation module, used to select a hand-eye calibration matrix based on the camera number and restore the image coordinates corresponding to the key feature points to normalized coordinates in the camera coordinate system through inverse transformation of camera intrinsic parameters; and a second transformation module, used to transform the normalized coordinates to the robot base coordinate system through the hand-eye calibration matrix to obtain the three-dimensional spatial coordinates of the flexible textile.

[0019] Optionally, the control unit includes: a second generation module, used to generate a candidate path for the robot's end effector from its initial pose to the target gripping pose, using the three-dimensional spatial coordinates as the gripping target; a selection module, used to select the target gripping path from the candidate paths; and a control module, used to control the robot to perform a flattening operation on the flexible textile according to the target gripping path.

[0020] Optionally, the selection module includes: a determination submodule for determining the motion path points for robot teaching; and a second optimization submodule for optimizing the target gripping path using the motion path points to obtain an optimized target gripping path.

[0021] According to one aspect of the present invention, a flexible textile processing system is provided, the flexible textile processing system using the flexible textile processing method described in any one of the above embodiments.

[0022] According to one aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program performs the processing method for flexible textiles described in any one of the foregoing embodiments.

[0023] According to one aspect of the present invention, a processor is provided for running a program, wherein the program, when running, performs the processing method for flexible textiles as described in any one of the above embodiments.

[0024] According to one aspect of the present invention, a computer program product is provided, including computer instructions that, when executed by a processor, perform the processing method for flexible textiles described in any one of the above embodiments.

[0025] Applying the technical solution of this application, the above-mentioned flexible textile processing method involves acquiring disordered images of the flexible textile in a disordered state through a multi-view vision system; identifying key feature points of the flexible textile in the disordered state image using a target detection model to obtain the coordinates of the key feature points of the flexible textile; performing coordinate transformation on the coordinates of the key feature points to obtain the three-dimensional spatial coordinates of the key feature points in the robot's base coordinate system; and controlling the robot to perform a flattening operation on the flexible textile based on the three-dimensional spatial coordinates. This achieves high-precision identification of corner points of the flexible textile in a small sample size through the dynamic feature enhancement module in the improved detection model. Combined with a time-series trajectory prediction algorithm based on a neural network architecture, it achieves zero-collision gripping and flattening folding of the robot for the first time in a dual-arm flexible textile processing system. This solves the technical problems of high failure rate and low automation in traditional towel processing equipment due to the irregular shape of soft fabrics, difficulty in accurately identifying feature points, and easy collisions during robot movement. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0027] Figure 1 This is a hardware structure block diagram of a mobile terminal for a flexible textile processing method according to an embodiment of the present invention.

[0028] Figure 2 This is a flowchart of a method for processing flexible textiles according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the robot operation hardware distribution according to an embodiment of the present invention;

[0030] Figure 4 This is a flowchart of the improved YOLO-v11 algorithm according to an embodiment of the present invention;

[0031] Figure 5 This is a framework diagram of a YOLO-v11-based feature point detection and robot trajectory prediction algorithm according to an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of a flexible textile processing apparatus according to an embodiment of the present invention.

[0033] The above figures include the following reference numerals:

[0034] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device; 301. Robot vision main camera; 302. Right camera on the robotic arm; 303. Right robotic arm gripper; 304. Left robotic arm gripper; 305. Left camera on the robotic arm; 306. Right robotic arm; 307. Left robotic arm; 308. Robot body hardware support; 309. Industrial control computer. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] As described in the background section, traditional towel processing equipment suffers from high failure rates and low automation due to the irregular shape of soft fabrics, difficulty in accurately identifying feature points, and the risk of collisions during robot movement. This invention provides a method and apparatus for processing flexible textiles, a flexible textile processing system, a computer-readable storage medium, a processor, and a computer program product.

[0038] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0039] The methods and embodiments provided in this invention can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a flexible textile processing method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0040] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the mobile robot control method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] Example 1

[0042] According to an embodiment of the present invention, a method embodiment for processing flexible textiles is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] Figure 2 This is a flowchart of a method for processing flexible textiles according to an embodiment of the present invention, such as... Figure 2 As shown, the method includes the following steps:

[0044] Step S202: Acquire images of the disordered state of the flexible textile fabric through a multi-view vision system, wherein the multi-view vision system includes at least one fixedly mounted main camera and at least two auxiliary cameras mounted at the end of the robotic arm.

[0045] Optionally, in this embodiment of the invention, a towel is used as an example of a flexible textile. Of course, the flexible textile can also be other types of objects, which will not be elaborated here.

[0046] In this embodiment, the multi-view vision system refers to an image acquisition system consisting of a main camera fixed in the environment and two auxiliary cameras respectively mounted at the ends of the left and right robotic arms. Its function is to overcome the problem of missed features caused by occlusion or limited field of view when identifying soft, wrinkled, and irregularly shaped towels from a single perspective. The main camera is used to acquire a top-down or frontal panoramic image of the towel as a whole, while the auxiliary cameras move with the robotic arms, capturing local details from a side or oblique perspective near the towel gripping point. This is especially useful when the towel is partially occluded or folded and stacked, providing crucial corner or edge information.

[0047] This method utilizes the main camera to provide a global perspective to observe the overall distribution of the towels, while the auxiliary camera moves with the robotic arm to get closer and take pictures, thus making up for the problem of missed corner detection caused by the main camera's limited field of view or occlusion, thereby comprehensively obtaining clear image information of the towel's edges, wrinkles and key geometric features.

[0048] By implementing this control method and using multi-view collaborative acquisition, the system can fully acquire the surface morphology of towels even when they are in an arbitrary, disordered stacking state. This provides reliable and redundant visual input for subsequent accurate identification of key feature points, significantly improving the system's accuracy in complex scenarios.

[0049] Step S204: Use a target detection model to identify key feature points of the flexible textile in the disordered state image to obtain the coordinates of the key feature points of the flexible textile. The target detection model is a model trained by machine learning using multiple sets of training data. The input of the target detection model is a sample image of the flexible textile, and the output of the target detection model is the coordinates of the sample feature points of the flexible textile. The key feature points include at least one of the following of the flexible textile: corner points, geometric highest points.

[0050] In this embodiment, the object detection model refers to a deep learning model based on an improved YOLO-v11 architecture, trained with a small number of samples, specifically designed to identify the corner points and geometric highest points of a towel. Its input is towel images captured from multi-view cameras, and its output is the two-dimensional pixel coordinates of each key feature point in the image. Unlike traditional edge detection or template matching methods, this model introduces a "dynamic feature enhancement module" to adaptively improve the feature response of low-contrast, weak-texture, or partially occluded areas. Therefore, even with only a few dozen labeled images as a small sample, it can still stably identify the four corner points or highest points of the towel. These points are the core geometric markers that determine the towel's grasping posture and folding path.

[0051] This method uses a convolutional neural network to extract texture and contour features from images, and through a newly added dynamic feature enhancement module, it adaptively strengthens the response intensity of low-contrast, occluded areas or tiny corners, enabling the model to accurately identify key structural features such as the four corners or geometric highest points of a towel using only a few dozen labeled images.

[0052] By implementing this control method, the shortcomings of traditional edge detection or template matching algorithms in being susceptible to wrinkles, material reflection and background interference on flexible fabrics, resulting in missed detections and false detections, are effectively overcome. This method enables high-precision detection of key feature points of towels in complex environments, providing reliable and stable two-dimensional coordinate output for subsequent spatial positioning and precise grasping by robotic arms. It also significantly improves the practicality and adaptability of the system in disordered and unstructured scenarios.

[0053] Step S206: Perform coordinate transformation on the key feature point coordinates to obtain the three-dimensional spatial coordinates of the key feature point coordinates in the robot base coordinate system.

[0054] In this embodiment, "coordinate transformation" refers to mapping the two-dimensional image pixel coordinates output by the target detection model to the three-dimensional base coordinate system of the robot body through a pre-calibrated "hand-eye calibration matrix", thereby converting the image information perceived by the vision system into position information that can be directly used by the robot control system.

[0055] This method utilizes the spatial geometric relationship between the vision system and the robot body to map the position of feature points in the camera imaging plane to the absolute spatial coordinate system on which the robot's motion depends. This process relies on the precise calibration results of multiple cameras and the end effector of the robotic arm to ensure a stable mathematical correspondence between the image coordinates and the robot's joint poses. This maps the feature point information in the image coordinate system to the global base coordinate system on which the robot's motion control depends, achieving precise alignment between the visual perception space and the robot's execution space.

[0056] By implementing this control method, the robot no longer relies on a preset fixed position, but can dynamically adjust its grasping action according to the real-time identified position of the towel. This effectively avoids grasping failures or collisions caused by random placement, significantly improving the system's autonomy and positioning accuracy in unstructured environments, and providing an accurate and unified spatial reference benchmark for subsequent trajectory planning and collaborative operations.

[0057] Step S208: Control the robot to perform a flattening operation on the flexible textile based on three-dimensional spatial coordinates.

[0058] In this embodiment, the "flattening operation" refers to the robot planning and executing a series of actions such as gripping, lifting, moving, and folding based on the converted three-dimensional keypoint coordinates and a time-series-based trajectory prediction algorithm to automatically arrange the towel. The robot's two arms move synchronously according to the predicted trajectory to avoid collisions with each other, while ensuring that the gripper accurately aligns with the corner or highest point when approaching the towel, picking it up with minimal deformation, and laying the towel flat on the folding platform along the optimal path.

[0059] This method utilizes historical motion data and a dynamic obstacle avoidance penalty function to optimize the spatiotemporal path of the robotic arm's end-effector pose, ensuring that the gripper accurately aligns with the feature point and picks up the towel with minimal deformation. At the same time, it predicts and avoids collision risks between the two arms and between the gripper and the placement plane, achieving zero-collision collaborative operation during the motion process.

[0060] By implementing this control method, the visual recognition results are transformed into a controllable physical operation closed loop, which significantly improves the accuracy, consistency and efficiency of towel sorting. The system can stably complete the unstructured sorting task of complex flexible objects without human intervention, achieving a deep integration of automated equipment in terms of intelligent perception and precise execution.

[0061] Figure 3 This is a schematic diagram of the robot operation hardware distribution according to an embodiment of the present invention, such as... Figure 3 As shown, the dual-arm robot hardware mainly consists of a vision system and a robot control system. The vision system mainly includes: an industrial control computer 309, a robot vision main camera 301, a left camera on the robotic arm 305, and a right camera on the robotic arm 302. The industrial control computer 309 is responsible for implementing the vision algorithm and communicating with the three cameras to acquire images from different perspectives. The robot control system includes: a robot body hardware support 308, a left robotic arm 307, a right robotic arm 306, a left robotic arm gripper 304, and a right robotic arm gripper 303. The robotic arms and the grippers on the robotic arms wait for instructions from the industrial control computer. After the robotic arms reach the designated location, the grippers on the robotic arms perform their actions.

[0062] As described above, in this embodiment, a multi-view vision system acquires images of the disordered state of the flexible textile; a target detection model identifies key feature points of the flexible textile in the disordered state image to obtain the coordinates of these key feature points; coordinate transformation is performed on the key feature point coordinates to obtain their three-dimensional spatial coordinates in the robot's base coordinate system; the robot is then controlled to perform a flattening operation on the flexible textile based on these three-dimensional spatial coordinates. This achieves high-precision identification of towel corner points in small samples using the improved YOLO-v11 dynamic feature enhancement module, and combined with a Transformer-based time-series trajectory prediction algorithm, enables zero-collision gripping and flattening folding of the robot in the dual-arm towel sorting system. This achieves the technical effect of improving the accuracy, safety, and automation level of intelligent sorting.

[0063] Therefore, the technical solutions provided by the above embodiments of the present invention solve the technical problems of high failure rate and low degree of automation in traditional towel sorting equipment due to the irregular shape of soft fabrics, difficulty in accurately identifying feature points, and easy collisions caused by robot movement.

[0064] According to the above embodiments of the present invention, before identifying key feature points of flexible textiles in disordered images using a target detection model to obtain the coordinates of key feature points of the flexible textiles, the processing method further includes: acquiring sample images and labeling the sample images to obtain sample feature point coordinates; preprocessing the sample images to obtain preprocessed sample images, wherein the preprocessing includes at least one of the following: size normalization processing and illumination equalization processing; mapping the sample feature point coordinates to the image pixel coordinate system to obtain multiple sets of training data including the preprocessed sample images and sample feature point coordinates; and processing the multiple sets of training data using machine learning to obtain a target detection model.

[0065] In this embodiment, before collecting disordered towel images, it is necessary to first collect towel image data before the robot picks up the towel in the current scene, and mark the corner positions of the towel. Then, the marked data is used to train the model, and the model file generated after training is saved.

[0066] Figure 4 This is a flowchart of the improved YOLO-v11 algorithm according to an embodiment of the present invention, as shown below. Figure 4As shown, the CNN backbone network extracts image features and generates feature maps through multiple convolutional layers. To address the issue of limited feature extraction capabilities in complex environments such as low contrast, occlusion, and small targets, leading to increased false negative rates, a dynamic feature enhancement module is introduced after feature map acquisition. This module adaptively enhances the feature response of key regions, improving the model's ability to perceive weak targets. In the diagram, the input image is first processed, then the CNN backbone network extracts feature maps. Subsequently, the detection head performs bounding box regression and class probability prediction. A post-processing step using non-maximum suppression removes redundant detection boxes, ultimately outputting the class and bounding box parameters of multiple targets. By adaptively enhancing the response of key regions, the detection capability for weak targets is improved.

[0067] This method first acquires sample images of flexible textiles covering various scenarios and manually annotates the coordinates of their key feature points. Then, it performs preprocessing operations such as size normalization and illumination equalization on the sample images to eliminate interference introduced by factors such as shooting distance and light source changes during image acquisition and improve data consistency. The annotated feature point coordinates are synchronously mapped to the pixel coordinate system of the corresponding images to form multiple sets of training samples containing preprocessed images and accurate labels. On this basis, machine learning methods are used to optimize the training data end-to-end to train a high-precision target detection model that can stably output feature point coordinates.

[0068] By implementing this control method, the convergence efficiency and accuracy of the model under small sample conditions are effectively improved, and the recognition deviation caused by image quality fluctuations is significantly reduced, laying a reliable data-driven foundation for subsequent high-precision spatial positioning and robot control.

[0069] According to the above embodiments of the present invention, a target detection model is obtained by processing multiple sets of training data through machine learning, including: inputting multiple sets of training data into a convolutional neural network to extract multi-scale feature images; calculating the spatial gradient magnitude of the multi-scale feature images to obtain the spatial gradient magnitude calculation result; generating a normalized spatial attention map based on the spatial gradient magnitude calculation result, and simultaneously performing global average pooling processing on the channel dimension to obtain channel activation vectors; calculating channel attention weights based on the channel activation vectors; performing weighted fusion of the spatial attention map and channel attention weights according to the learning parameters to obtain enhancement weights; obtaining enhancement features based on the enhancement weights, and inputting the features into the detection head for bounding box regression and corner point category scoring to obtain the target detection model.

[0070] In this embodiment, the input feature map Calculate spatial gradient magnitude Then, normalization processing is performed to generate a spatial attention map. Simultaneously, global average pooling is performed on the channel dimension to obtain the channel activation vector. Channel attention weights are generated through a two-layer MLP; finally, spatial and channel attention weights are fused to generate enhanced weights. ,in Learnable parameters; nonlinear feature enhancer Original features Multiply the components and then perform nonlinear modulation using the Sigmoid activation function to output the enhanced features. .

[0071] This method first inputs multiple sets of training data into a convolutional neural network to extract multi-level spatial feature maps to capture texture and structural information at different scales. Then, it calculates the spatial gradient magnitude for each feature map to generate a normalized spatial attention map, which represents the salient regions of edges and textures in the image. Simultaneously, global average pooling is performed on the channel dimension to obtain channel activation vectors, and channel attention weights are derived using a multilayer perceptron (MLP) to reflect the contribution of each feature channel to target discrimination. Subsequently, the spatial attention map and channel attention weights are weighted and fused according to a learnable parameter α to generate adaptive enhancement weights. These weights are applied to the original feature map using a sigmoid nonlinear modulation to output the enhanced feature representation. Finally, the enhanced features are input into a detection head to complete bounding box regression and corner point category probability prediction, achieving high-precision feature point localization.

[0072] By implementing this control method, the model's feature discrimination ability in low-contrast, occluded, and small target scenes is significantly improved, effectively suppressing missed detections and false detections. While maintaining the model's lightweight nature, it enhances accuracy and provides stable and reliable feature localization output for subsequent precise robot grasping.

[0073] According to the above embodiments of the present invention, enhanced features are obtained based on enhanced weights, and the features are input into a detection head for bounding box regression and corner category scoring to obtain an object detection model. The process includes: multiplying the enhanced weights element-wise with the original feature map after nonlinear modulation by an activation function to obtain enhanced features; feeding the enhanced features into a detection head for bounding box regression and corner category scoring; and removing redundant prediction boxes by nonmaximum suppression to optimize the loss function until convergence to obtain the object detection model.

[0074] In this embodiment, the detection head maps the feature map to the prediction space, including bounding box position regression and class score. Post-processing commonly uses non-maximum suppression (NMS) to remove redundant overlapping boxes and outputs the final detection result.

[0075] In this method, the enhancement weights are nonlinearly modulated using the Sigmoid activation function and then multiplied element-wise with the original feature map to achieve adaptive enhancement of the key region feature response and suppression of background noise, thereby generating more discriminative enhanced features. Subsequently, the enhanced features are input to the detection head, which performs coordinate regression of the target bounding box and confidence scoring of the corner point category, and outputs the initial detection results. Furthermore, the non-maximum suppression (NMS) algorithm is used to eliminate overlapping redundant prediction boxes, ensuring that each corner point retains only the unique response with the highest confidence, and end-to-end optimization is performed by combining a multi-task loss function (including localization error and classification entropy) until the model converges.

[0076] By implementing this control method, a close collaboration between feature enhancement and detection output is achieved. While ensuring the model's lightweight nature and real-time performance, the recall rate and localization accuracy of corner detection are significantly improved, providing a highly reliable feature localization basis with a low false detection rate for subsequent robot vision guidance.

[0077] According to the above embodiments of the present invention, key feature point identification of flexible textiles in a disordered state image is performed by a target detection model to obtain the coordinates of key feature points of the flexible textiles. The method includes: inputting the disordered state image into the target detection model to process the disordered state image using the target detection model; when no corner points of the flexible textiles are detected, controlling the robot to grab the geometric highest point of the flexible textiles to place the flexible textiles; performing corner point identification again until an intersection point that meets the confidence level is identified; and outputting the coordinates of the key feature points of the flexible textiles.

[0078] In this embodiment, the model file after data annotation is input, images of towels placed in random order are collected, and the corner points of the towels are identified. If no corner point is detected, the highest point of the towel is grabbed, the towel is dropped from a high place, and the corner point is identified again until a corner point that meets the confidence requirement is identified.

[0079] This method inputs disordered images into a target detection model and uses the feature representation capabilities trained by the model to identify corners of flexible textiles in real time. When the detection results output by the model do not include corners with sufficient confidence, the system triggers a fault-tolerant mechanism, controlling the robot to grasp the geometrically highest point of the textile in the image space and perform a directional delivery action to change its posture and occlusion relationship, thereby breaking the detection failure state caused by the original visual occlusion or deformation. After delivery, the image is re-acquired and corner recognition is performed again, iterating until the detection results meet the preset confidence threshold, and finally outputting stable and reliable corner coordinates.

[0080] By implementing this control method, the problem of missed detection caused by wrinkles, stacking or low contrast in flexible textiles under unconstrained conditions is effectively overcome. This improves the robustness and adaptability of the system under complex working conditions, ensures the reliable acquisition of key feature points, and provides a stable spatial reference for subsequent accurate trajectory planning and collision-free grasping.

[0081] According to the above embodiments of the present invention, coordinate transformation is performed on the coordinates of key feature points to obtain the three-dimensional spatial coordinates of the key feature points in the robot base coordinate system, including: determining the camera number from which the image comes based on the image corresponding to the key feature point coordinates; selecting a hand-eye calibration matrix based on the camera number, and restoring the image coordinates corresponding to the key feature points to normalized coordinates in the camera coordinate system through inverse transformation of camera intrinsic parameters; transforming the normalized coordinates to the robot base coordinate system through the hand-eye calibration matrix to obtain the three-dimensional spatial coordinates of the flexible textile.

[0082] In this embodiment, multiple camera eye calibrations are performed, either with the camera on the hand or with the camera outside the hand, and multiple sets of hand-eye calibration matrices are saved. The robot's movement path points are taught, and the hand-eye calibration matrices are input to meet the towel images required when taking pictures with different cameras. The feature point image coordinates in the towel images are then transformed to other robot coordinate systems.

[0083] The method first determines the visual sensor number corresponding to the image source, and then selects the pre-calibrated hand-eye transformation matrix corresponding to the camera. Subsequently, the image pixel coordinates are inversely projected using the camera's intrinsic parameter matrix to restore the two-dimensional image coordinates to normalized spatial coordinates in the camera coordinate system. Based on this, the coordinates in the camera coordinate system are transformed to the robot base coordinate system using the known hand-eye calibration matrix, thereby obtaining the accurate three-dimensional spatial coordinates of the key feature points of the flexible textile in the robot's workspace.

[0084] By implementing this control method, high-precision spatial alignment between visual perception and mechanical motion systems is achieved, ensuring that visual recognition results can directly drive the robotic arm to perform grasping and positioning actions. This eliminates positioning errors caused by multi-camera collaboration or inconsistencies in coordinate systems, and significantly improves the system's perception consistency and execution accuracy of the spatial pose of flexible objects in complex unstructured environments.

[0085] According to the above embodiments of the present invention, controlling a robot to perform a flattening operation on a flexible textile based on three-dimensional spatial coordinates includes: generating candidate paths from the robot's end effector from its initial pose to the target gripping pose, using the three-dimensional spatial coordinates as the gripping target; selecting the target gripping path from the candidate paths; and controlling the robot to perform a flattening operation on the flexible textile according to the target gripping path.

[0086] In this embodiment, the "candidate path" is a set of several feasible trajectories from the initial pose to the target gripping pose, generated based on robot kinematic constraints, obstacle avoidance requirements, and trajectory smoothness; the "target gripping path" is the optimal execution trajectory selected from the candidate paths based on optimality criteria (such as shortest movement time, lowest energy consumption, smallest posture change, and largest obstacle avoidance margin).

[0087] This method first uses the three-dimensional spatial coordinates obtained in the robot's base coordinate system as the gripping target point, and generates multiple candidate paths from the initial pose to the target gripping pose based on the robot's kinematic model and environmental constraints. Then, taking into account path length, motion smoothness, end effector posture stability, and obstacle avoidance safety with surrounding structures, an optimization algorithm is used to select the optimal target gripping path, ensuring that the gripping action minimizes mechanical vibration and trajectory changes while avoiding collisions. Finally, the robot controller drives the end effector to move precisely to the target pose according to the path instruction, completing the stable gripping and initial flattening operation of the flexible textile.

[0088] By implementing this control method, non-rigid operation control of soft and easily deformable objects is achieved, significantly improving the success rate of gripping and the flatness of the finished product. It effectively overcomes the inaccuracy and deformation problems caused by the uncertainty of position and posture in the operation of flexible objects by traditional fixed trajectory control.

[0089] According to the above embodiments of the present invention, selecting a target gripping path from candidate paths includes: determining the motion path points for robot teaching; and optimizing the target gripping path using the motion path points to obtain an optimized target gripping path.

[0090] In this embodiment, the optimal trajectory of the robot is optimized based on the path points taught by the dual-arm robot, and a time-series-based optimal trajectory prediction algorithm is used for trajectory optimization prediction. The end-effector poses of the two robotic arms are assumed to be as follows: and The distance between the ends of the two robotic arms is: ,when If a collision risk is detected at that moment, operation will be immediately stopped. This is the minimum safe distance threshold allowed between the end effectors of a dual-arm robot.

[0091] Assume the starting and ending poses of the trajectory input by the current single robotic arm are respectively... and Historical trajectory data also exists. The Transformer model is used to train several sets of historical trajectory data, and then, based on the input... The objective function is defined as follows: Among them, penalty ( ) is the obstacle avoidance penalty function. , , They are respectively , as well as The corresponding weighted penalty coefficient, when a satisfying condition is found... When the minimum trajectory is reached, output the predicted trajectory data. .

[0092] When both robotic arms are running simultaneously, the joint optimization objective function is defined as follows: Among them, penalty ( ) is the obstacle avoidance penalty function. , , They are respectively , as well as and The corresponding weighted penalty coefficient, The predicted trajectories for the left and right arms are respectively. When a satisfactory condition is found... When the minimum trajectory is reached, output the predicted trajectory data for the left and right arms. .

[0093] Optionally, the robotic arm performs actions based on the teaching points, executing different gripper opening and closing commands and determining whether the gripper has picked up an object when reaching different teaching points; and completes tasks of sorting and folding towels in random order based on the teaching points.

[0094] To achieve optimal planning of the robot's gripping path, this method introduces preset robot teaching motion path points as constraints on the candidate paths. The initial candidate trajectories are reconstructed through a path optimization algorithm, so that the optimized path strictly conforms to the operational experience and spatial taboo areas reflected in human teaching while satisfying kinematic accessibility and dynamic smoothness. This process transforms the implicit knowledge in human operation (such as gripping angle, approach direction, and avoidance posture) into explicit path constraints.

[0095] By implementing this control method, the optimized target gripping path not only ensures that the end effector can safely and stably approach the target point in space, but also significantly reduces the slippage or deformation of the textile caused by sudden changes in the path. This lays a reliable motion foundation for subsequent high-precision gripping and flattening operations, and achieves a deep integration of experience-driven and algorithm optimization.

[0096] Figure 5 This is a framework diagram of a YOLO-v11-based feature point detection and robot trajectory prediction algorithm according to an embodiment of the present invention, as shown below. Figure 5As shown, firstly, randomized towel images are acquired and combined with data acquisition and annotation information. YOLO-v11 is used to detect towel feature points. At the same time, multi-camera calibration is used to set the path points for teaching the dual-arm robot. Then, the robot's motion path is generated by optimal trajectory prediction based on time series. The robot is controlled to reach the teaching point and perform actions, sequentially completing the towel sorting and folding operations, and finally ending the task. A closed-loop control system from visual perception to intelligent execution is systematically constructed, realizing the deep integration and collaborative optimization of "perception-decision-execution" in the intelligent sorting task of flexible textiles.

[0097] As can be seen from the above, the technical solution provided by the above embodiments of the present invention introduces a dynamic feature enhancement module into the YOLO-v11 network. Through the adaptive fusion of spatial gradient attention and channel activation weights, the detection accuracy of weak targets such as towel corners under small sample training is significantly improved, breaking through the limitation of traditional methods relying on a large amount of labeled data. A time series trajectory prediction model based on Transformer is constructed, and the pose, velocity and acceleration historical data of the dual robotic arms are jointly optimized. An obstacle avoidance penalty function is introduced to realize real-time prediction of collision risk and optimal trajectory generation, ensuring zero collision between the gripper and the towel and the dual arms, and realizing high-precision and high-safety autonomous collaborative sorting.

[0098] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0100] Example 2

[0101] According to embodiments of the present invention, a flexible textile processing apparatus for implementing the above-described flexible textile processing method is also provided. Figure 6This is a schematic diagram of a flexible textile processing apparatus according to an embodiment of the present invention, such as... Figure 6 As shown, the device includes: a data acquisition unit 601, an identification unit 603, a coordinate transformation unit 605, and a control unit 607. The device will now be described in detail.

[0102] The acquisition unit 601 is used to acquire images of the disordered state of a flexible textile fabric through a multi-view vision system, wherein the multi-view vision system includes at least one fixedly mounted main camera and at least two auxiliary cameras mounted at the end of a robotic arm.

[0103] The recognition unit 603 is used to identify key feature points of flexible textiles in disordered state images through a target detection model, so as to obtain the coordinates of key feature points of the flexible textiles. The target detection model is a model trained by machine learning using multiple sets of training data. The input of the target detection model is a sample image of the flexible textiles, and the output of the target detection model is the coordinates of the sample feature points of the flexible textiles. The key feature points include at least one of the following of the flexible textiles: corner points, geometric highest points.

[0104] The coordinate transformation unit 605 is used to perform coordinate transformation on the coordinates of key feature points to obtain the three-dimensional spatial coordinates of the key feature points in the robot base coordinate system.

[0105] Control unit 607 is used to control the robot to perform a flattening operation on the flexible textile based on three-dimensional spatial coordinates.

[0106] It should be noted that the acquisition unit 601, the identification unit 603, the coordinate transformation unit 605 and the control unit 607 mentioned above correspond to steps S202 to S208 in the above embodiments. The four units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments.

[0107] As can be seen from the above, in the scheme described in the above embodiments of the present invention, an acquisition unit is used to acquire images of a disordered flexible textile through a multi-view vision system. The multi-view vision system includes at least one fixedly mounted main camera and at least two auxiliary cameras mounted at the end of a robotic arm. A recognition unit is used to identify key feature points of the flexible textile in the disordered image using a target detection model to obtain the coordinates of the key feature points of the flexible textile. The target detection model is a model trained using multiple sets of training data through machine learning. The input of the target detection model is a sample image of the flexible textile, and the output of the target detection model is the coordinates of the sample feature points of the flexible textile. The key feature points include at least one of the following of the flexible textile: corner points and geometric highest points. A coordinate transformation unit is used to perform coordinate transformation on the coordinates of the key feature points to obtain the three-dimensional spatial coordinates of the key feature points in the robot's base coordinate system. A control unit is used to control the robot to perform a flattening operation on the flexible textile based on the three-dimensional spatial coordinates. The above technical solution achieves high-precision identification of towel corner points in small samples by using the improved YOLO-v11 dynamic feature enhancement module. Combined with the Transformer-based time series trajectory prediction algorithm, it enables zero-collision grasping and flat folding of the robot in the dual-arm towel sorting system, thereby improving the accuracy, safety and automation level of intelligent sorting.

[0108] Therefore, the technical solutions provided by the above embodiments of the present invention solve the technical problems of high failure rate and low degree of automation in traditional towel sorting equipment due to the irregular shape of soft fabrics, difficulty in accurately identifying feature points, and easy collisions caused by robot movement.

[0109] Optionally, the processing apparatus further includes: an acquisition unit, configured to acquire a sample image and annotate the sample image to obtain sample feature point coordinates before identifying key feature points of the flexible textile in the disordered state image using a target detection model; a preprocessing unit, configured to preprocess the sample image to obtain a preprocessed sample image, wherein the preprocessing includes at least one of the following: size normalization processing and illumination equalization processing; a mapping unit, configured to map the sample feature point coordinates to an image pixel coordinate system to obtain multiple sets of training data including the preprocessed sample image and the sample feature point coordinates; and a processing unit, configured to process the multiple sets of training data using machine learning to obtain a target detection model.

[0110] Optionally, the processing unit includes: a first input module for inputting multiple sets of training data into a convolutional neural network to extract multi-scale feature images; a first calculation module for calculating the spatial gradient magnitude of the multi-scale feature images to obtain the spatial gradient magnitude calculation result; a first generation module for generating a normalized spatial attention map based on the spatial gradient magnitude calculation result, and simultaneously performing global average pooling processing on the channel dimension to obtain channel activation vectors; a second calculation module for calculating channel attention weights based on the channel activation vectors; a weighted fusion module for performing weighted fusion of the spatial attention map and channel attention weights according to the learning parameters to obtain enhancement weights; and a second input module for obtaining enhancement features based on the enhancement weights and inputting the features into the detection head for bounding box regression and corner point category scoring to obtain an object detection model.

[0111] Optionally, the second input module includes: a calculation submodule, used to multiply the enhancement weights element-wise with the original feature map after nonlinear modulation by the activation function to obtain the enhancement features; an input submodule, used to input the enhancement features into the detection head for bounding box regression and corner point category scoring; and a first optimization submodule, used to remove redundant prediction boxes through nonmaximum suppression to optimize the loss function until convergence, thereby obtaining the object detection model.

[0112] Optionally, the recognition unit includes: a third input module, used to input the disordered state image into the target detection model, so as to process the disordered state image using the target detection model, and when no corner of the flexible textile is detected, control the robot to grab the geometric highest point of the flexible textile to place the flexible textile, and perform corner recognition again until an intersection point that meets the confidence level is identified, and output the coordinates of the key feature points of the flexible textile.

[0113] Optionally, the coordinate transformation unit includes: a determination module, used to determine the camera number from which the image comes based on the image corresponding to the coordinates of the key feature points; a first transformation module, used to select a hand-eye calibration matrix based on the camera number and restore the image coordinates corresponding to the key feature points to normalized coordinates in the camera coordinate system through inverse transformation of camera intrinsic parameters; and a second transformation module, used to transform the normalized coordinates to the robot base coordinate system through the hand-eye calibration matrix to obtain the three-dimensional spatial coordinates of the flexible textile.

[0114] Optionally, the control unit includes: a second generation module for generating candidate paths from the initial pose to the target gripping pose of the robot's end effector, using three-dimensional spatial coordinates as the gripping target; a selection module for selecting the target gripping path from the candidate paths; and a control module for controlling the robot to perform a flattening operation on the flexible textile according to the target gripping path.

[0115] Optionally, the selection module includes: a determination submodule for determining the motion path points for robot teaching; and a second optimization submodule for optimizing the target gripping path using the motion path points to obtain the optimized target gripping path.

[0116] According to one aspect of the present invention, a flexible textile processing system is provided, which uses any of the above-described flexible textile processing methods.

[0117] According to one aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program performs the processing method of flexible textiles described in any of the above.

[0118] According to one aspect of the present invention, a processor is provided for running a program, wherein the program executes the processing method for flexible textiles described above.

[0119] According to one aspect of the present invention, a computer program product is provided, including computer instructions, which, when executed by a processor, perform a method for processing flexible textiles according to any of the above-described methods.

[0120] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any communication device in a group of communication devices.

[0121] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: acquiring disordered images of a flexible textile in a disordered state through a multi-view vision system, wherein the multi-view vision system includes at least one fixedly mounted main camera and at least two auxiliary cameras mounted at the end of a robotic arm; identifying key feature points of the flexible textile in the disordered state image using a target detection model to obtain the coordinates of key feature points of the flexible textile; wherein the target detection model is a model trained using multiple sets of training data through machine learning, the input of the target detection model is a sample image of the flexible textile, and the output of the target detection model is the coordinates of the sample feature points of the flexible textile, the key feature points including at least one of the following of the flexible textile: corner points, geometric highest points; performing coordinate transformation on the coordinates of the key feature points to obtain the three-dimensional spatial coordinates of the key feature points in the robot's base coordinate system; controlling the robot to perform a flattening operation on the flexible textile based on the three-dimensional spatial coordinates.

[0122] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: acquiring a sample image and annotating the sample image to obtain the coordinates of sample feature points; preprocessing the sample image to obtain a preprocessed sample image, wherein the preprocessing includes at least one of the following: size normalization processing and illumination equalization processing; mapping the coordinates of sample feature points to the image pixel coordinate system to obtain multiple sets of training data including the preprocessed sample image and the coordinates of sample feature points; and processing the multiple sets of training data through machine learning to obtain an object detection model.

[0123] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: inputting multiple sets of training data into a convolutional neural network to extract multi-scale feature images; calculating the spatial gradient magnitude of the multi-scale feature images to obtain the spatial gradient magnitude calculation result; generating a normalized spatial attention map based on the spatial gradient magnitude calculation result, and simultaneously performing global average pooling processing on the channel dimension to obtain channel activation vectors; calculating channel attention weights based on the channel activation vectors; performing weighted fusion of the spatial attention map and channel attention weights according to the learning parameters to obtain enhancement weights; obtaining enhancement features based on the enhancement weights, and inputting the features into the detection head for bounding box regression and corner point category scoring to obtain an object detection model.

[0124] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: multiplying the enhancement weights element-wise with the original feature map after nonlinear modulation by an activation function to obtain enhanced features; feeding the enhanced features into a detection head for bounding box regression and corner category scoring; removing redundant prediction boxes by nonmaximum suppression to optimize the loss function until convergence, thereby obtaining the target detection model.

[0125] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: inputting a disordered state image into a target detection model to process the disordered state image using the target detection model, and when no corner of the flexible textile is detected, controlling the robot to grab the geometric highest point of the flexible textile to deploy the flexible textile, performing corner recognition again until an intersection point that meets the confidence level is identified, and outputting the coordinates of the key feature points of the flexible textile.

[0126] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: determining the camera number from which the image comes based on the image corresponding to the coordinates of the key feature points; selecting a hand-eye calibration matrix based on the camera number, and restoring the image coordinates corresponding to the key feature points to normalized coordinates in the camera coordinate system through inverse transformation of camera intrinsic parameters; transforming the normalized coordinates to the robot base coordinate system through the hand-eye calibration matrix to obtain the three-dimensional spatial coordinates of the flexible textile.

[0127] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: generating candidate paths from the initial pose to the target gripping pose of the robot's end effector, using three-dimensional spatial coordinates as the gripping target; selecting the target gripping path from the candidate paths; and controlling the robot to perform a flattening operation on the flexible textile according to the target gripping path.

[0128] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: determining the motion path points for robot teaching; optimizing the target gripping path using the motion path points to obtain an optimized target gripping path.

[0129] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0131] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0133] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0134] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0135] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for treating flexible textiles, characterized in that, include: Images of a disordered flexible textile are acquired using a multi-view vision system, wherein the multi-view vision system includes at least one fixed main camera and at least two auxiliary cameras mounted at the end of a robotic arm. The object detection model is used to identify key feature points of the flexible textile in the disordered state image to obtain the coordinates of the key feature points of the flexible textile. The object detection model is a model trained by machine learning using multiple sets of training data. The input of the object detection model is a sample image of the flexible textile, and the output of the object detection model is the coordinates of the sample feature points of the flexible textile. The key feature points include at least one of the following of the flexible textile: corner points, geometric highest points. The coordinates of the key feature points are transformed to obtain the three-dimensional spatial coordinates of the key feature points in the robot base coordinate system. The robot is controlled to perform a flattening operation on the flexible textile based on the three-dimensional spatial coordinates.

2. The method for treating flexible textiles according to claim 1, characterized in that, Before identifying key feature points of the flexible textile in the disordered state image using a target detection model to obtain the coordinates of the key feature points of the flexible textile, the processing method further includes: The sample image is acquired and annotated to obtain the coordinates of the sample feature points; The sample image is preprocessed to obtain a preprocessed sample image, wherein the preprocessing includes at least one of the following: size normalization processing and illumination equalization processing; The coordinates of the sample feature points are mapped to the image pixel coordinate system to obtain multiple sets of training data including the preprocessed sample image and the coordinates of the sample feature points; The target detection model is obtained by processing the multiple sets of training data through machine learning.

3. The method for treating flexible textiles according to claim 2, characterized in that, The target detection model is obtained by processing the multiple sets of training data using machine learning, including: The multiple sets of training data are input into a convolutional neural network to extract multi-scale feature images; Spatial gradient magnitude is calculated on the multi-scale feature image to obtain the spatial gradient magnitude calculation result; Based on the spatial gradient magnitude calculation results, a normalized spatial attention map is generated, and global average pooling is performed on the channel dimension to obtain the channel activation vector. Calculate channel attention weights based on channel activation vectors; The spatial attention map and the channel attention weights are weighted and fused according to the learning parameters to obtain the enhancement weights; Enhanced features are obtained based on the enhanced weights and input into the detection head for bounding box regression and corner point category scoring, thus obtaining the target detection model.

4. The method for treating flexible textiles according to claim 3, characterized in that, Enhanced features are obtained based on the enhanced weights, and these features are input into the detection head for bounding box regression and corner point category scoring, resulting in the target detection model, including: The enhanced weights are nonlinearly modulated by an activation function and then multiplied element-wise with the original feature map to obtain the enhanced features; The enhanced features are fed into the detection head for bounding box regression and corner point category scoring; Redundant prediction boxes are removed by nonmaximum suppression to optimize the loss function until convergence, thus obtaining the target detection model.

5. The method for treating flexible textiles according to claim 1, characterized in that, The key feature points of the flexible textile in the disordered state image are identified using a target detection model to obtain the coordinates of the key feature points of the flexible textile, including: The disordered state image is input into the target detection model to process the disordered state image. When no corner of the flexible textile is detected, the robot is controlled to grab the geometric highest point of the flexible textile and place it on the flexible textile. Corner recognition is performed again until an intersection point that meets the confidence level is identified, and the coordinates of the key feature points of the flexible textile are output.

6. The method for treating flexible textiles according to claim 1, characterized in that, The coordinates of the key feature points are transformed to obtain their three-dimensional spatial coordinates in the robot's base coordinate system, including: Based on the image corresponding to the coordinates of the key feature points, determine the camera number from which the image originates; Select the hand-eye calibration matrix according to the camera number, and restore the image coordinates corresponding to the key feature points to normalized coordinates in the camera coordinate system through inverse transformation of camera intrinsic parameters. The normalized coordinates are transformed to the robot base coordinate system using the hand-eye calibration matrix to obtain the three-dimensional spatial coordinates of the flexible textile.

7. The method for treating flexible textiles according to any one of claims 1 to 6, characterized in that, The robot is controlled to perform a flattening operation on the flexible textile based on the three-dimensional spatial coordinates, including: Using the three-dimensional spatial coordinates as the gripping target, a candidate path is generated for the robot's end effector from its initial pose to the target gripping pose; Select the target clipping path from the candidate paths; The robot is controlled to perform a flattening operation on the flexible textile according to the target gripping path.

8. The method for treating flexible textiles according to claim 7, characterized in that, Selecting the target clipping path from the candidate paths includes: Determine the movement path points for the robot during teaching; The target clamping path is optimized using the motion path points to obtain the optimized target clamping path.

9. A processing apparatus for flexible textiles, characterized in that, include: The acquisition unit is used to acquire images of the disordered state of a flexible textile fabric through a multi-view vision system, wherein the multi-view vision system includes at least one fixedly mounted main camera and at least two auxiliary cameras mounted at the end of a robotic arm. The identification unit is used to identify key feature points of the flexible textile in the disordered state image through a target detection model, so as to obtain the coordinates of the key feature points of the flexible textile; wherein, the target detection model is a model trained by machine learning using multiple sets of training data, the input of the target detection model is a sample image of the flexible textile, and the output of the target detection model is the coordinates of the sample feature points of the flexible textile, and the key feature points include at least one of the following of the flexible textile: corner points, geometric highest points; The coordinate transformation unit is used to perform coordinate transformation on the coordinates of the key feature points to obtain the three-dimensional spatial coordinates of the key feature points in the robot base coordinate system. The control unit is used to control the robot to perform a flattening operation on the flexible textile based on the three-dimensional spatial coordinates.

10. A flexible textile processing system, characterized in that, The flexible textile processing system uses the flexible textile processing method described in any one of claims 1 to 8.

Images