A method for trajectory optimization of a robot arm

Abstract

The application relates to a mechanical arm trajectory optimization method, comprising the following steps: acquiring an actual outer contour of a target flexible material, and determining a transformation relationship between the actual outer contour and a predefined template contour; respectively determining target information associated with a predefined sewing trajectory in the actual outer contour and the template contour as first target information and second target information; mapping the second target information and the sewing trajectory to an actual outer contour coordinate system based on the transformation relationship; in the coordinate system, taking the difference between the mapped sewing trajectory and the mapped second target information as a reference, optimizing the difference between the sewing trajectory and the first target information, and finally outputting optimized sewing trajectory data adapted to the actual material form. The mechanical arm trajectory optimization method provided by the application can effectively compensate for the deformation and placement error of the flexible material, and improve the sewing trajectory fitting degree and sewing precision.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a method for optimizing the trajectory of a robotic arm. Background Technology

[0002] In the field of automated sewing, especially for processing non-rigid materials such as fabric, accurately obtaining the sewing start point and trajectory is crucial. Current solutions mainly fall into two categories. The first is absolute coordinate positioning: fabric is a flexible material, and its placement involves not only rigid body transformations but also stretching, shearing, and minor deformations at local edges. The second is feature point visual positioning: this involves identifying corner points or marker points on the fabric using a camera and calculating translation and rotation matrices to correct the trajectory.

[0003] However, existing solutions for automated sewing suffer from poor adaptability and low precision. When sewing flexible materials, existing solutions cannot adapt to the rigid transformation, stretching, and localized minor deformation of flexible materials, resulting in deviation of the sewing trajectory and reduced sewing precision when using existing solutions to sew flexible materials. Summary of the Invention

[0004] Therefore, it is necessary to provide a robotic arm trajectory optimization method to address the problem that traditional methods cannot adapt to the rigid body transformation, stretching, and localized minor deformation of flexible materials, which leads to deviation of the sewing trajectory and reduced sewing accuracy when using existing solutions to sew flexible materials.

[0005] This application provides a method for optimizing the trajectory of a robotic arm, including: Obtain the actual outer contour of the target flexible material and determine the transformation relationship between the actual outer contour and the predefined template contour; Determine the target information in the actual outer contour that is associated with the predefined sewing trajectory, and use this target information as the first target information; and determine the target information in the predefined template contour that is associated with the predefined sewing trajectory, and use this target information as the second target information; Based on the transformation relationship between the actual outer contour and the predefined template contour, the second target information is mapped to the coordinate system of the actual outer contour to obtain the mapped second target information, and the predefined sewing trajectory is mapped to the coordinate system of the actual outer contour to obtain the mapped predefined sewing trajectory. In the coordinate system of the actual outer contour, the difference between the mapped predefined sewing trajectory and the mapped second target information is used as a benchmark to continuously optimize the difference between the mapped predefined sewing trajectory and the first target information, so as to obtain the optimized sewing trajectory data. Output optimized sewing trajectory data.

[0006] Further, the step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour includes: The acquired image of the target flexible material is converted to HSV space to obtain an HSV image; The HSV image is binarized and segmented based on a preset pixel threshold to obtain a mask image of the target flexible material. The mask image of the target flexible material is processed by median filtering and opening operation to obtain the processed mask image of the target flexible material. The processed mask image of the target flexible material is defined as the target image. Extract the largest closed contour in the target image as the actual outer contour.

[0007] Furthermore, the step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour also includes: Based on the original distance, calculate the area and centroid position information of the actual outer contour and the predefined template contour respectively; The main axis directions of the actual outer contour and the predefined template contour are calculated based on the center distance.

[0008] Furthermore, the step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour also includes: The scaling ratio between the actual outer contour and the predefined template contour is calculated by taking the square root of the ratio of the area of ​​the actual outer contour to the area of ​​the predefined template contour. The predefined template contour is then scaled based on the scaling ratio to obtain the scaled image of the predefined template contour. Based on the positional relationship between the centroid of the actual outer contour and the centroid of the predefined template contour, the translation amount between the actual outer contour and the predefined template contour is calculated. Based on the translation amount, the scaled predefined template contour image is translated to obtain the translated predefined template contour image. Based on the principal axis direction of the actual outer contour and the principal axis direction of the predefined template contour, the rotation angle between the actual outer contour and the predefined template contour is calculated by the covariance matrix. Based on the rotation amount, the translated predefined template contour image is rotated to obtain the rotated predefined template contour image.

[0009] Furthermore, the step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour also includes: The preset angle search range and search step size are used to search around the angle value of the rotation angle between the actual outer contour and the predefined template contour in order to obtain multiple angle search results. Calculate the IOU score for each angle search result, select the angle search result with the highest IOU score, and adjust the rotation angle between the actual outer contour and the predefined template contour to obtain the final rotation angle.

[0010] Further, the step of determining the target information associated with the predefined sewing trajectory in the actual outer contour, and using this target information as the first target information, and determining the target information associated with the predefined sewing trajectory in the predefined template contour, and using this target information as the second target information, includes: The actual outer contour and the predefined template contour are segmented separately to obtain multiple contour segments; Map the predefined sewing trajectory to the coordinate system of the actual outer contour and the coordinate system of the predefined template contour, respectively; In the coordinate system of the actual outer contour, the target segment associated with the sewing trajectory is determined from multiple contour segments of the actual outer contour and is taken as the first target segment; In the coordinate system of the predefined template outline, a target segment is determined from multiple outline segments of the predefined template outline and associated with the sewing trajectory, serving as the second target segment.

[0011] Further, the step of mapping the second target information to the coordinate system of the actual outer contour based on the transformation relationship between the actual outer contour and the predefined template contour to obtain the mapped second target information, and mapping the predefined sewing trajectory to the coordinate system of the actual outer contour to obtain the mapped predefined sewing trajectory, includes: The second target segment is mapped to the coordinate system of the actual outer contour to obtain the third target segment.

[0012] Furthermore, in the coordinate system of the actual outer contour, based on the difference between the mapped predefined sewing trajectory and the mapped second target information, the difference between the mapped predefined sewing trajectory and the first target information is continuously optimized to obtain optimized sewing trajectory data, including: The distance between the sewing trajectory in the coordinate system of the actual outer contour and the first target segment is calculated as the first distance, and the distance between the sewing trajectory in the coordinate system of the actual outer contour and the third target segment is calculated as the second distance. Adjusting the position of the trajectory points in the sewing trajectory in the coordinate system of the actual outer contour to generate optimized sewing trajectory data; the adjustment of the position of the trajectory points in the sewing trajectory in the coordinate system of the actual outer contour to generate optimized sewing trajectory data includes: continuously adjusting the first distance based on the second distance.

[0013] Furthermore, before mapping the second target segment to the coordinate system of the actual outer contour to obtain the third target segment, the robotic arm trajectory optimization method further includes: The first target segment is fitted to obtain the fitted first target segment.

[0014] Furthermore, the robotic arm trajectory optimization method also includes: The optimized sewing trajectory data is converted to the robot's execution coordinate system to generate an executable sewing trajectory.

[0015] This application relates to a robotic arm trajectory optimization method. First, the actual outer contour of the flexible material to be sewn is acquired and extracted, and the transformation relationship between the actual outer contour and a predefined template contour is calculated. Then, target segments associated with the predefined sewing trajectory are located on both the actual contour and the predefined template contour, and the target segments on the predefined template contour and the sewing trajectory are mapped to the coordinate system of the actual contour. Based on this, using the distance between the mapped sewing trajectory and the target segment from the template as a reference, the position of the sewing trajectory on the corresponding segment of the actual contour is adjusted to generate an optimized trajectory that matches the actual shape of the material. The robotic arm trajectory optimization method provided in this application can effectively overcome the problem of inaccurate trajectory fitting caused by placement deviations and tensile deformation of flexible materials. It achieves high-precision adaptive adjustment of the sewing trajectory without strict alignment, significantly improving sewing quality and automation adaptability. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a robotic arm trajectory optimization method provided in an embodiment of this application.

[0017] Figure 2 This is a schematic diagram of the final cusp distribution of a predefined template contour in a robotic arm trajectory optimization method provided in an embodiment of this application.

[0018] Figure 3 This is a schematic diagram of the final cusp distribution of the actual outer contour in a robotic arm trajectory optimization method provided in an embodiment of this application.

[0019] Figure 4 This is a schematic diagram showing the mapping of a predefined sewing trajectory to a predefined template contour coordinate system in a robotic arm trajectory optimization method provided in an embodiment of this application.

[0020] Figure 5 This is a schematic diagram showing the mapping of a predefined sewing trajectory to the actual outer contour coordinate system in a robotic arm trajectory optimization method provided in an embodiment of this application.

[0021] Figure 6 This is a schematic diagram illustrating the mapping of the second target segment to the actual outer contour coordinate system in a robotic arm trajectory optimization method provided in an embodiment of this application.

[0022] Figure 7 for Figure 6Enlarged view of point a in the middle.

[0023] Figure 8 This is a schematic diagram of the structure of a robotic arm trajectory optimization system provided in an embodiment of this application.

[0024] Figure label: 10-Robotic arm sewing module; 20-Image acquisition module; 30-Processing module; 40-Transportation module. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] This application provides a method for optimizing the trajectory of a robotic arm.

[0027] like Figure 1 As shown, in one embodiment of this application, the robotic arm trajectory optimization method includes the following steps S100 to S500.

[0028] S100: Obtain the actual outer contour of the target flexible material and determine the transformation relationship between the actual outer contour and the predefined template contour.

[0029] Specifically, the image acquisition module 20 can acquire images of the target flexible material, and the actual outer contour can be obtained after image preprocessing and contour extraction. The image acquisition module 20 can be a camera.

[0030] The predefined template outline is: read the CAD file of the clothing design and obtain the outer outline lines of the CAD file of the clothing design.

[0031] The obtained outer contour lines are mapped onto a high-resolution image.

[0032] Next, the pixels inside the outer contour line are assigned a value of 255, and the pixels outside the outer contour line are assigned a value of 0, thus obtaining the predefined template contour.

[0033] S200: Determine the target information in the actual outer contour that is associated with the predefined sewing trajectory, and use this target information as the first target information. Also, determine the target information in the predefined template contour that is associated with the predefined sewing trajectory, and use this target information as the second target information.

[0034] S300, based on the transformation relationship between the actual outer contour and the predefined template contour, maps the second target information to the coordinate system of the actual outer contour to obtain the mapped second target information. Also, it maps the predefined sewing trajectory to the coordinate system of the actual outer contour to obtain the mapped predefined sewing trajectory.

[0035] S400, in the coordinate system of the actual outer contour, takes the difference between the mapped predefined sewing trajectory and the mapped second target information as the benchmark, and continuously optimizes the difference between the mapped predefined sewing trajectory and the first target information to obtain optimized sewing trajectory data.

[0036] S500 outputs optimized sewing trajectory data.

[0037] In this embodiment, the actual outer contour of the flexible material to be sewn is first acquired and extracted, and the transformation relationship between the actual outer contour and the predefined template contour is calculated. Then, target segments associated with the predefined sewing trajectory are located on both the actual contour and the predefined template contour, and the target segments and sewing trajectory on the predefined template contour are mapped to the actual contour coordinate system. Based on this, using the distance between the mapped sewing trajectory and the target segment from the template as a reference, the position of the sewing trajectory on the corresponding segment of the actual contour is adjusted to generate an optimized trajectory that matches the actual shape of the material. The robotic arm trajectory optimization method provided in this application can effectively overcome the problem of inaccurate trajectory fitting caused by placement deviations and tensile deformation of flexible materials, achieving high-precision adaptive adjustment of the sewing trajectory without strict alignment, significantly improving sewing quality and automation adaptability.

[0038] In one embodiment of this application, obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour includes the following steps S101 to S104.

[0039] S101, the acquired image of the target flexible material is converted to HSV space to obtain an HSV image.

[0040] Specifically, the image of the target flexible material is a picture taken by a camera as a fabric piece moves from the worktable to the inspection area.

[0041] S102, the HSV image is binarized and segmented based on a preset pixel threshold to obtain a mask image of the target flexible material.

[0042] Specifically, a corresponding background HSV range is set for the background color of the workbench area, and each pixel is detected. If a pixel is within the background HSV color range, it is determined to be the background and defined as black; the remaining pixels are determined to be the fabric and defined as white.

[0043] S103, perform median filtering and opening operation on the mask image of the target flexible material to obtain the processed mask image of the target flexible material, and define the processed mask image of the target flexible material as the target image.

[0044] Specifically, median filtering of the mask image of the target flexible material includes: Create a sliding convolution kernel, a 3×3 or 5×5 box.

[0045] Clipping area: This window clips onto a single pixel of the mask image and centers on that pixel.

[0046] Pick up (collect values): Take out all pixel values ​​within the window's coverage area.

[0047] Sort (arrange): Arrange these values ​​from smallest to largest (or from largest to smallest).

[0048] Select a representative (take the median): Find the number that is exactly in the middle.

[0049] Replace: Replace the original center pixel value with this intermediate number.

[0050] Opening operations on the mask image of the target flexible material include: Create a small convolutional kernel and slide it across the mask image. The center point remains white only when the kernel is completely surrounded by a white area; the center point becomes black as soon as the kernel touches any black area around it. Then repeat this process, but the detection condition becomes that the center point becomes white as long as the kernel touches any white area around it.

[0051] S104: Extract the closed contour with the largest area in the target image as the actual outer contour.

[0052] Specifically, when the camera captures real-time images of the flexible material, it also captures objects in the surrounding environment. However, the flexible material occupies the largest area in the image. Therefore, the image with the largest contour area after binarization segmentation is the actual outer contour of the flexible material.

[0053] In this embodiment, median filtering and opening operations are performed on the mask image of the target flexible material to remove noise and edge burrs from the image.

[0054] In one embodiment of this application, the step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour further includes the following steps S105 to S106.

[0055] S105, calculate the area and centroid position information of the actual outer contour and the predefined template contour respectively based on the original distance.

[0056] Specifically, the primitive moments are responsible for positioning and weighing, calculated based on the image origin (0,0). They primarily utilize low-order moments, 0th and 1st order, to calculate the object's area and centroid position.

[0057] S106, calculate the main axis directions of the actual outer contour and the predefined template contour based on the center distance.

[0058] Specifically, the central moment is responsible for determining the orientation, and it is calculated based on the center of mass. Because the coordinates of the center of mass are subtracted, it has translation invariance. The principal axis directions of the object are mainly calculated using the second-order moments.

[0059] In this embodiment, the calculation process is performed in the coordinate system of the actual outer contour and the coordinate system of the predefined template contour, and then the values ​​are compared under a unified logic.

[0060] The centroid (CxT, CyT) and angle θT are calculated using the top left corner of the predefined template contour as (0,0).

[0061] In the actual outer contour, with the upper left corner of its image as (0,0), its centroid (CxI, CyI) and angle θI are calculated.

[0062] In practical applications, the predefined template contour map and the actual outer contour map have the same resolution. If they are inconsistent, resolution alignment can be performed to facilitate conversion. The translation amount (dx, dy) is obtained by direct subtraction. The rotation angle difference (θinit) can also be obtained by direct subtraction of θI and θT: θinit = θI − θT.

[0063] In one embodiment of this application, the step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour further includes the following steps S107 to S109.

[0064] S107. Calculate the scaling ratio between the actual outer contour and the predefined template contour based on the square root of the ratio of the area of ​​the actual outer contour to the area of ​​the predefined template contour. Scale the predefined template contour based on the scaling ratio to obtain the scaled image of the predefined template contour.

[0065] Specifically, in practical applications, the resolution of the predefined template outline and the actual outer outline is the same. Therefore, by calculating the ratio between the area of ​​the actual outer outline and the area of ​​the predefined template outline and then taking the square root, the scaling ratio between the two can be obtained.

[0066] S108. Based on the positional relationship between the centroid of the actual outer contour and the centroid of the predefined template contour, the translation amount between the actual outer contour and the predefined template contour is calculated. Based on the translation amount, the scaled predefined template contour image is translated to obtain the translated predefined template contour image.

[0067] Specifically, the translation amount is the difference between the centroid position of the actual outer contour and the centroid position of the predefined template contour in the same coordinate system. Since the resolution of the predefined template contour and the actual outer contour is the same in practical applications, they can be directly mapped to the same coordinate system, which facilitates the calculation of the translation amount.

[0068] S109, based on the principal axis direction of the actual outer contour and the principal axis direction of the predefined template contour, the rotation angle between the actual outer contour and the predefined template contour is calculated through the covariance matrix. Based on the rotation amount, the translated predefined template contour image is rotated to obtain the rotated predefined template contour image.

[0069] In one embodiment of this application, the step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour further includes the following steps S110 to S111.

[0070] S110, presets the angle search range and search step size, and searches around the angle value of the rotation angle between the actual outer contour and the predefined template contour to obtain multiple angle search results.

[0071] Specifically, in the vicinity of the initially calculated rotation angle, a search is performed within a range of ±4° with a preset step size of 0.2° to generate multiple candidate angles.

[0072] S111, calculate the IOU score of each angle search result, select the angle search result with the highest IOU score to adjust the rotation angle between the actual outer contour and the predefined template contour, and obtain the final rotation angle.

[0073] Specifically, the predefined template contour after each fine-tuning is mapped to the coordinate system of the actual outer contour, and then the ratio of the intersection of the two contours to the union of the two contours is calculated to obtain the IOU score after each fine-tuning.

[0074] In this embodiment, the accuracy of contour alignment is improved by finely adjusting the rotation angle.

[0075] In one embodiment of this application, the steps of determining the target information associated with the predefined sewing trajectory in the actual outer contour and using the target information as the first target information, and determining the target information associated with the predefined sewing trajectory in the predefined template contour and using the target information as the second target information, include the following steps S201 to S203.

[0076] S201, the actual outer contour and the predefined template contour are segmented respectively to obtain multiple contour segments.

[0077] like Figure 2 and Figure 3As shown, A is the image coordinate system of the predefined template contour; A1 is the final cusp of the predefined template contour; A2 is the contour segment of the predefined template contour; B is the image coordinate system of the actual outer contour; B1 is the final cusp of the actual outer contour; B2 is the contour segment of the actual outer contour. In one embodiment of this application, the step of segmenting the actual outer contour and the predefined template contour to obtain multiple contour segments includes the following steps S201a to S201c.

[0078] S201a, detect the cusps on the actual outer contour and the predefined template contour respectively, and obtain the cusp data of the actual outer contour and the cusp data of the predefined template contour.

[0079] Specifically, S201a includes: S201a1, analyze the actual outer contour to obtain the contour point sequence of the actual outer contour.

[0080] S201a2, traverse each contour point in the sequence of contour points of the actual outer contour along the trajectory, and determine whether each contour point is a candidate cusp based on the local geometric features of each contour point, so as to obtain the candidate cusp.

[0081] S201a3, based on the preset cusp determination conditions, selects the final cusp from the candidate cusps obtained in S201a2 to obtain the actual outer contour cusp data.

[0082] S201a4, parse the predefined template contour to obtain the contour point sequence of the predefined template contour.

[0083] S201a5, traverse the contour point sequence of the predefined template contour in sequence along the trajectory, and determine whether each contour point is a candidate cusp based on the local geometric features of each contour point in order to obtain the candidate cusp.

[0084] S201a6: Based on the preset cusp determination criteria, the final cusp is selected from the candidate cusps obtained in S201a5 to obtain the predefined template contour cusp data.

[0085] The determination of candidate cusps in S201a2 and S201a5 both include the following steps: W110 starts from any point on the contour line, slides along the arrangement of contour points to traverse each contour point, and selects a contour point as the current point to be calculated.

[0086] W120 calculates the vertex angle corresponding to the current calculated point as the vertex, based on the Nth contour point before and the Nth contour point after the current calculated point, according to the triangle formed by the current calculated point, the Nth contour point before the current calculated point, and the Nth contour point after the current calculated point; N is a preset positive integer. N is a positive integer and N is greater than 2.

[0087] W130 determines whether the apex angle is less than a preset angle threshold.

[0088] W140, if the vertex angle is less than a preset angle threshold, then the currently calculated point is marked as a candidate cusp, and the vertex angle corresponding to the current calculated point is taken as the angle of the candidate cusp.

[0089] W150, return to W110, that is, return to any point based on the contour line, slide along the arrangement order of the contour points to traverse each contour point, select a contour point as the current calculation point, until all contour points have been selected.

[0090] In S201a3 and S201a6, the preset cusp determination condition is: to merge multiple consecutive candidate cusps and select the candidate cusp with the smallest corresponding angle from the multiple consecutive candidate cusps as the final cusp output.

[0091] S201b, based on the actual outer contour cusp data, breaks the actual outer contour to obtain multiple contour segments of the actual outer contour.

[0092] Specifically, the actual outer contour cusp data includes the number of final cusps in the final actual outer contour and the position of the final cusps.

[0093] S201b includes: S201b1, based on the position of each final cusp, breaks the contour lines of the actual outer contour to obtain multiple broken contour segments.

[0094] S201c, based on the cusp data of the predefined template contour, the predefined template contour is broken to obtain contour segments of multiple predefined template contours.

[0095] Specifically, the predefined template profile cusp data includes the number of final cusps and the position of the final cusps in the final predefined template profile.

[0096] S201c includes: S201c1, based on the position of each final cusp, breaks the contour lines of the predefined template contour to obtain multiple contour segments after breaking.

[0097] S202, map the predefined sewing trajectory to the coordinate system of the actual outer contour and the coordinate system of the predefined template contour respectively.

[0098] Specifically, based on the coordinate system transformation relationship, the predefined sewing trajectory is mapped to the coordinate system of the predefined template outline to obtain the mapped sewing trajectory.

[0099] Based on the transformation relationship between the actual outer contour and the predefined template contour, the sewing trajectory in the coordinate system of the predefined template contour is mapped to the coordinate system of the actual outer contour to obtain the mapped sewing trajectory.

[0100] S203, in the coordinate system of the actual outer contour, determine the target segment associated with the sewing trajectory from multiple contour segments of the actual outer contour, and use it as the first target segment.

[0101] Specifically, in the coordinate system of the actual outer contour, the shortest distance between the mapped sewing trajectory and each contour segment of the actual contour is calculated, and the contour segment with the smallest distance is selected as the first target segment. The sewing trajectory in S203 is the sewing trajectory obtained by mapping the predefined sewing trajectory to the coordinate system of the actual outer contour.

[0102] like Figure 5 As shown, B3 is the first target segment; B4 is the sewing trajectory mapped to B.

[0103] In one embodiment of this application, determining the target segment associated with the sewing trajectory from multiple contour segments of the actual outer contour in the coordinate system of the actual outer contour as the first target segment includes the following step S203a.

[0104] S203a, calculate the distance from the sewing trajectory to each contour segment of the actual outer contour in the coordinate system of the actual outer contour, and take the contour segment of the actual outer contour that is closest to the sewing trajectory as the first target segment.

[0105] Specifically, S203a includes: S203a1, determine the midpoint position of the sewing trajectory in the coordinate system of the actual outer contour, and the midpoint position of each contour segment of the actual outer contour, calculate the distance from the midpoint position of the sewing trajectory to the midpoint position of each contour segment of the actual outer contour, and select the contour segment with the shortest distance as the first target segment.

[0106] S204, in the coordinate system of the predefined template contour, determine the target segment associated with the sewing trajectory from multiple contour segments of the predefined template contour, and use it as the second target segment.

[0107] Specifically, in the coordinate system of the predefined template contour, the shortest distance between the mapped sewing trajectory and each contour segment of the predefined template contour is calculated, and the contour segment with the smallest distance is selected as the second target segment. The sewing trajectory in S204 is the sewing trajectory obtained by mapping the predefined sewing trajectory to the coordinate system of the predefined template contour; therefore, the sewing trajectory in S204 is not the sewing trajectory in S203.

[0108] like Figure 4 As shown, A3 is the second target segment; A4 is the sewing trajectory mapped to A. In one embodiment of this application, determining the target segment associated with the sewing trajectory from multiple contour segments of the predefined template contour in the coordinate system of the predefined template contour, as the second target segment, includes the following step S204a.

[0109] S204a, calculate the distance from the sewing trajectory to the contour segment of each predefined template contour in the coordinate system of the predefined template contour, and take the contour segment of the predefined template contour that is closest to the sewing trajectory as the second target segment.

[0110] Specifically, S204a includes: S204a1, determine the midpoint position of the sewing trajectory in the coordinate system of the predefined template contour, as well as the midpoint position of the contour segments of each predefined template contour, calculate the distance from the midpoint position of the sewing trajectory to the midpoint position of the contour segments of each predefined template contour, and select the contour segment with the shortest distance as the second target segment.

[0111] In one embodiment of this application, the step of mapping the second target information to the coordinate system of the actual outer contour based on the transformation relationship between the actual outer contour and the predefined template contour to obtain the mapped second target information, and mapping the predefined sewing trajectory to the coordinate system of the actual outer contour to obtain the mapped predefined sewing trajectory, includes the following step S301.

[0112] S301, map the second target segment to the coordinate system of the actual outer contour to obtain the third target segment.

[0113] Specifically, such as Figure 6 As shown, B is the image coordinate system of the actual outer contour; B3 is the first target segment; A5 is the third target segment; and B4 is the sewing trajectory mapped to the image coordinate system of the actual outer contour.

[0114] Based on the transformation relationship in step S100, the second target segment is mapped from the template coordinate system to the actual contour coordinate system to obtain the third target segment.

[0115] In one embodiment of this application, the step of continuously optimizing the difference between the mapped predefined sewing trajectory and the first target information in the coordinate system of the actual outer contour, based on the difference between the mapped predefined sewing trajectory and the mapped second target information, to obtain optimized sewing trajectory data, includes the following steps S401 to S402.

[0116] S401, calculate the distance between the sewing trajectory in the coordinate system of the actual outer contour and the first target segment as the first distance, and calculate the distance between the sewing trajectory in the coordinate system of the actual outer contour and the third target segment as the second distance.

[0117] Specifically, the sewing trajectory in the coordinate system of the actual outer contour in S401 and the sewing trajectory in S203 are the same sewing trajectory, both of which are obtained by mapping the predefined sewing trajectory to the coordinate system of the actual outer contour.

[0118] S402, Adjust the position of the trajectory point in the sewing trajectory in the coordinate system of the actual outer contour to generate optimized sewing trajectory data; The step of adjusting the position of the trajectory point in the sewing trajectory in the coordinate system of the actual outer contour to generate optimized sewing trajectory data includes: continuously adjusting the first distance based on the second distance.

[0119] Optionally, in S402, the first distance is adjusted as the optimization goal. By continuously adjusting the coordinates of the sewing trajectory points, the first distance gradually approaches the second distance, so that the adjusted sewing trajectory better fits the structural features of the actual contour.

[0120] like Figure 7 As shown, B3 is the first target segment; B5 is the first distance; A6 is the second distance; A5 is the third target segment; and B6 is the i-th sewing trajectory point. A loss function is constructed with the difference between the first and second distances in the initial state as the optimization objective, and the direction of the normal vector of the sewing trajectory is used as the optimization variable for optimization. The loss function is solved using the gradient descent method to obtain the optimized sewing trajectory data.

[0121] First, calculate the second distance between the sewing trajectory points using Formula 1: D_ideal_i=min(||pi-q||) Formula 1; Where D_ideal_i is the second distance of the i-th sewing trajectory point; pi is the position of the i-th sewing trajectory point; and q is the position of the contour point corresponding to pi on the third target segment.

[0122] Next, the optimized first distance is calculated using Formulas 2 and 3: p'i=R*pi+t Formula 2; Where p'i is the position of the i-th sewing trajectory point after the position is changed; R is the rotation amount of the i-th sewing trajectory point; pi is the position of the i-th sewing trajectory point; and t is the translation amount of the i-th sewing trajectory point.

[0123] D_real_i(θ)=min(||p'i-q||) Formula 3; Where D_real_i(θ) is the first distance of the i-th sewing trajectory point after optimization; θ is {R,t}, which is the combination of sewing trajectory point variables; q is the position of the contour point corresponding to pi on the third target segment; and p'i is the position of the i-th sewing trajectory point after transformation.

[0124] Next, we will calculate the loss function used in optimizing the sewing trajectory data using Formula 4.

[0125] The loss function is: L(θ)=(1 / N)*∑|D_real_i(θ)-D_ideal_i| (Equation 4); Where L(θ) is the average absolute error between the first distance and the second distance of the i-th sewing trajectory point after optimization; D_real_i(θ) is the first distance of the i-th sewing trajectory point after optimization; and D_ideal_i is the second distance of the i-th sewing trajectory point.

[0126] The optimized sewing trajectory data, obtained by solving the loss function using the gradient descent method, includes:

[0127] initialization: Set the initial position θ_0={R_0,t_0}.

[0128] Set the maximum number of iterations, K_max.

[0129] Set a convergence threshold ε, which is the convergence value after a single update.

[0130] Counter k=0.

[0131] Iteration loop: 1. Forward computation: Calculate all D_real_i using the current θ_k, and substitute them into the formula to obtain L(θ_k).

[0132] 2. Gradient calculation: Calculate the derivative of the loss function with respect to R and t: ∇L=[∂L / ∂R,∂L / ∂t] Formula 5; 3. Parameter update: θ_(k+1)=θ_k-α*∇L Formula 6; Where θ_(k+1) is the updated position; θ_k is the unupdated position; and α is the learning rate.

[0133] 4. Check the termination condition of the iteration. If either the maximum number of iterations K_max or the convergence value after a single update is less than the set convergence threshold ε is met, exit. Otherwise, k=k+1 and repeat the step "forward calculation".

[0134] In one embodiment of this application, before mapping the second target segment to the coordinate system of the actual outer contour to obtain the third target segment, the robotic arm trajectory optimization method further includes the following step A100.

[0135] A100 is used to fit the first target segment to obtain the fitted first target segment.

[0136] Specifically, A100 can be fitted to the first target segment three times using B-spline fitting to obtain the fitted first target segment. The distance between the sewing trajectory in the coordinate system of the actual outer contour and the fitted first target segment is used as the first distance.

[0137] The fitted first target segment makes the distance function differentiable everywhere with respect to the position variable when calculating the "distance from the sewing trajectory point to the actual outer contour segment". Thus, the gradient ∇L of the loss function with respect to the rotation R and translation t can be accurately calculated using the chain rule.

[0138] Cubic B-spline fitting ensures the continuity of the second derivative of the first target segment at the connection point, which means that the generated optimized sewing trajectory has a continuous change in curvature. This can avoid jitter or abrupt changes in the robotic arm when performing sewing, and ensure the smoothness of the sewing process.

[0139] In one embodiment of this application, the robotic arm trajectory optimization method further includes the following step S600.

[0140] The S600 converts the optimized sewing trajectory data to the robot's execution coordinate system, generating executable sewing trajectory data.

[0141] Specifically, the optimized sewing trajectory data includes the coordinates of each point on the sewing trajectory in the coordinate system of the actual outer contour.

[0142] In this embodiment, the optimized sewing trajectory data is mapped to the coordinate system of the robot using a calibration matrix to generate three-dimensional coordinate data.

[0143] This application also provides a robotic arm trajectory optimization system.

[0144] like Figure 8As shown. In one embodiment of this application, the robotic arm trajectory optimization system includes a robotic arm sewing module 10, an image acquisition module 20, a processing module 30, and a transportation module 40.

[0145] The robotic arm sewing module 10 is disposed on the transport module 40, and the robotic arm sewing module 10 is used to sew the flexible material transported on the transport module 40.

[0146] The image acquisition module 20 is disposed on the top of the transport module 40, and the image acquisition module 20 is used to acquire a top view image of the flexible material transported on the transport module 40.

[0147] The processing module 30 is communicatively connected to the image acquisition module 20 and the robotic arm sewing module 10, respectively. The processing module 30 is used to execute the robotic arm trajectory optimization method as described in any of the above embodiments.

[0148] The transport module 40 is used to transport the cut flexible material.

[0149] In this embodiment, the transport module 40 transports the cut flexible material to the bottom of the image acquisition module 20. The image acquisition module 20 acquires a top-view image of the flexible material and transmits it to the processing module 30. The processing module 30 processes the top-view image to generate sewing trajectory data for the cut flexible material and transmits the generated sewing trajectory data to the robotic arm sewing module 10. The robotic arm sewing module 10 performs a sewing operation on the cut flexible material based on the sewing trajectory data.

[0150] The technical features of the above embodiments can be combined arbitrarily, and the execution order of the method steps is not restricted. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for optimizing the trajectory of a robotic arm, characterized in that, The robotic arm trajectory optimization method includes: Obtain the actual outer contour of the target flexible material and determine the coordinate transformation relationship between the actual outer contour and the predefined template contour; Determine the target information in the actual outer contour that is associated with the predefined sewing trajectory, and use this target information as the first target information; and determine the target information in the predefined template contour that is associated with the predefined sewing trajectory, and use this target information as the second target information; Based on the transformation relationship between the actual outer contour and the predefined template contour, the second target information is mapped to the coordinate system of the actual outer contour to obtain the mapped second target information, and the predefined sewing trajectory is mapped to the coordinate system of the actual outer contour to obtain the mapped predefined sewing trajectory. In the coordinate system of the actual outer contour, the difference between the mapped predefined sewing trajectory and the mapped second target information is used as a benchmark to continuously optimize the difference between the mapped predefined sewing trajectory and the first target information, so as to obtain the optimized sewing trajectory data. Output the optimized sewing trajectory data; The process of determining the target information associated with the predefined sewing trajectory in the actual outer contour, and using this target information as the first target information, and determining the target information associated with the predefined sewing trajectory in the predefined template contour, and using this target information as the second target information, includes: The actual outer contour and the predefined template contour are segmented separately to obtain multiple contour segments; Map the predefined sewing trajectory to the coordinate system of the actual outer contour and the coordinate system of the predefined template contour, respectively; In the coordinate system of the actual outer contour, the target segment associated with the sewing trajectory is determined from multiple contour segments of the actual outer contour and is taken as the first target segment; In the coordinate system of the predefined template outline, the target segment associated with the sewing trajectory is determined from multiple outline segments of the predefined template outline and used as the second target segment; The process of mapping the second target information to the coordinate system of the actual outer contour based on the transformation relationship between the actual outer contour and the predefined template contour, to obtain the mapped second target information, and mapping the predefined sewing trajectory to the coordinate system of the actual outer contour, to obtain the mapped predefined sewing trajectory, includes: The second target segment is mapped to the coordinate system of the actual outer contour to obtain the third target segment.

2. The robotic arm trajectory optimization method according to claim 1, characterized in that, The process of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour includes: The acquired image of the target flexible material is converted to HSV space to obtain an HSV image; The HSV image is binarized and segmented based on a preset pixel threshold to obtain a mask image of the target flexible material. The mask image of the target flexible material is processed by median filtering and opening operation to obtain the processed mask image of the target flexible material. The processed mask image of the target flexible material is defined as the target image. Extract the closed contour with the largest area in the target image as the actual outer contour.

3. The robotic arm trajectory optimization method according to claim 1, characterized in that, The step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour further includes: Based on the original moments, calculate the area and centroid position information of the actual outer contour and the predefined template contour respectively; The principal axis directions of the actual outer contour and the predefined template contour are calculated based on the center moment.

4. The robotic arm trajectory optimization method according to claim 3, characterized in that, The step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour further includes: The scaling ratio between the actual outer contour and the predefined template contour is calculated by taking the square root of the ratio of the area of ​​the actual outer contour to the area of ​​the predefined template contour. The predefined template contour is then scaled based on the scaling ratio to obtain the scaled predefined template contour. Based on the positional relationship between the centroid of the actual outer contour and the centroid of the predefined template contour, the translation amount between the actual outer contour and the predefined template contour is calculated. Based on the translation amount, the scaled predefined template contour is translated to obtain the translated predefined template contour. Based on the principal axis direction of the actual outer contour and the principal axis direction of the predefined template contour, the rotation angle between the actual outer contour and the predefined template contour is calculated by the covariance matrix. Based on the rotation angle, the translated predefined template contour is rotated to obtain the rotated predefined template contour.

5. The robotic arm trajectory optimization method according to claim 4, characterized in that, The step of obtaining the actual outer contour of the target flexible material and determining the coordinate transformation relationship between the actual outer contour and the predefined template contour further includes: The preset angle search range and search step size are used to search around the angle value of the rotation angle between the actual outer contour and the predefined template contour in order to obtain multiple angle search results. Calculate the IOU score for each angle search result, select the angle search result with the highest IOU score, and adjust the rotation angle between the actual outer contour and the predefined template contour to obtain the final rotation angle.

6. The robotic arm trajectory optimization method according to claim 1, characterized in that, In the coordinate system of the actual outer contour, based on the difference between the mapped predefined sewing trajectory and the mapped second target information, the difference between the mapped predefined sewing trajectory and the first target information is continuously optimized to obtain optimized sewing trajectory data, including: The distance between the sewing trajectory in the coordinate system of the actual outer contour and the first target segment is calculated as the first distance, and the distance between the sewing trajectory in the coordinate system of the actual outer contour and the third target segment is calculated as the second distance. Adjusting the position of the trajectory points in the sewing trajectory in the coordinate system of the actual outer contour to generate optimized sewing trajectory data; the adjustment of the position of the trajectory points in the sewing trajectory in the coordinate system of the actual outer contour to generate optimized sewing trajectory data includes: continuously adjusting the first distance based on the second distance.

7. The robotic arm trajectory optimization method according to claim 1, characterized in that, Before mapping the second target segment to the coordinate system of the actual outer contour to obtain the third target segment, the robotic arm trajectory optimization method further includes: The first target segment is fitted to obtain the fitted first target segment.

8. The robotic arm trajectory optimization method according to claim 1, characterized in that, The robotic arm trajectory optimization method also includes: The optimized sewing trajectory data is converted to the robot's execution coordinate system to generate an executable sewing trajectory.

Images