A trajectory deviation correction control method

Abstract

This application relates to a trajectory correction control method, comprising acquiring and preprocessing real-time images during the sewing process; performing feature recognition on the preprocessed real-time images to obtain the needle position and fabric outline; calculating a vertical distance deviation value based on the needle position and fabric outline; comparing the vertical distance deviation value with a safety threshold, and correcting subsequent sewing processes based on the comparison result. The trajectory correction control method provided in this application can effectively improve the efficiency and accuracy of sewing trajectory correction, solving the problems of simple correction judgment logic and short-sighted adjustment in traditional automated sewing.

Description

Technical Field

[0001] This application relates to the field of intelligent sewing technology, and in particular to a trajectory correction control method. Background Technology

[0002] In the field of automated sewing, especially for sewing processes requiring high precision and consistency, achieving precise control and real-time correction of the sewing trajectory has always been a key challenge in technological development. Traditional sewing equipment typically relies on mechanical guiding devices or manual operation to maintain the stability of the sewing trajectory. However, these methods are prone to problems such as trajectory deviation, uneven stitching, and fabric shifting when dealing with flexible fabrics, complex contours, or high-speed sewing conditions, severely impacting sewing quality and production efficiency.

[0003] Currently, some automated sewing systems have incorporated visual inspection methods. By acquiring fabric images and performing edge recognition, they can monitor the sewing trajectory and make corrections based on the monitoring results. However, this method only adjusts for the error at the current moment, ignoring the geometric characteristics of the remaining unsewn path, and also fails to fully consider the impact of kinematic hysteresis after rotational correction.

[0004] Existing technologies typically use fixed monitoring windows or waiting times to delay judgment. However, this static mechanism has limitations; it cannot effectively distinguish between two different states: the error is rapidly converging and the error is continuously diverging. This can lead to unnecessary secondary shutdowns during the error self-correction process because the monitoring window ends and the residual error slightly exceeds the threshold; or, when the error is diverging maliciously, the existence of the window period may cause the system to miss the optimal emergency stop opportunity. Summary of the Invention

[0005] Therefore, it is necessary to provide a trajectory correction control method to address the problem of low correction efficiency and accuracy caused by the short-sightedness of the simple judgment logic and correction adjustment in traditional automated sewing when correcting the sewing trajectory.

[0006] This application provides a trajectory correction control method, including: Acquire real-time images during the sewing process and preprocess the acquired real-time images to obtain preprocessed real-time images; Feature recognition is performed on the preprocessed real-time image to obtain the needle position and fabric outline in the preprocessed real-time image; The vertical distance deviation value in the preprocessed real-time image is calculated based on the needle position and the fabric outline. The vertical distance deviation value in the preprocessed real-time image is compared with the safety threshold, and the subsequent sewing process is corrected based on the comparison result.

[0007] This application relates to a trajectory correction control method. It acquires and preprocesses real-time images during the sewing process, performs feature recognition on the preprocessed images to obtain the needle position and fabric outline, and calculates the vertical distance deviation value based on the needle position and fabric outline. This converts the trajectory deviation into a quantifiable numerical value. The vertical distance deviation value is then compared with a safety threshold, and subsequent sewing processes are corrected in a tiered manner based on the comparison result. This method not only achieves real-time monitoring and precise correction of the sewing trajectory but also avoids over-correction or under-correction caused by a single threshold through a tiered judgment mechanism. It effectively solves the problems of simple correction judgment logic and short-sighted correction adjustment in traditional automated sewing, significantly improving the efficiency and accuracy of sewing trajectory correction and ensuring consistent sewing quality. Attached Figure Description

[0008] Figure 1 This is a schematic flowchart of a trajectory correction control method provided in an embodiment of this application.

[0009] Figure 2 This is a schematic diagram showing the relationship between the actual sewing trajectory and the fabric outline position in a trajectory correction control method provided in an embodiment of this application.

[0010] Figure 3 This is a schematic diagram of the vertical distance deviation value in a trajectory correction control method provided in an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0012] This application provides a trajectory correction control method.

[0013] like Figures 1 to 3 As shown, where a is the needle position in the real-time image; b is the position of the fabric contour point corresponding to the needle position; c is the fabric contour; d is the theoretical sewing trajectory; g is the tangent direction of the fabric contour corresponding to the needle position in the real-time image; f is the sewing direction in the real-time image; L1 is the actual distance; L2 is the vertical distance deviation value; and L3 is the first preset distance. In one embodiment of this application, the trajectory correction control method includes the following steps S100 to S400.

[0014] S100 acquires real-time images during the sewing process and preprocesses the acquired real-time images to obtain preprocessed real-time images.

[0015] Specifically, real-time images can be acquired by continuously shooting images above the sewing area using an industrial camera, with an image acquisition frequency of 60 FPS or higher.

[0016] Optionally, preprocessing the real-time image includes the following steps S101 to S105.

[0017] S101, Gaussian filtering is performed on the acquired real-time image to remove high-frequency noise in the real-time image, resulting in a filtered image.

[0018] Specifically, the Gaussian filter uses a 5×5 kernel with a standard deviation σ=1.5 to suppress high-frequency noise from the sensor.

[0019] S102, perform HSV color segmentation on the filtered image to obtain the segmented image.

[0020] Specifically, HSV color segmentation extracts the binarized fabric region based on the calibrated fabric color threshold.

[0021] S103, based on morphological closing operation, smooth the contours in the segmented image to obtain a smoothed image.

[0022] Specifically, the morphological closing operation uses a 3×3 rectangular structuring element to first dilate and then erode the image, filling in the small holes within the fabric area and smoothing the outer contour.

[0023] S104, Obtain the set of fabric edge points in the image after contour smoothing.

[0024] S105, the fabric edge point set in the smoothed image is fitted using the least squares method to obtain the fabric contour fitted image, which is denoted as the preprocessed real-time image.

[0025] By sequentially filtering, HSV color segmentation, fabric contour smoothing, and fabric contour fitting on the real-time acquired images, noise interference is reduced and feature recognition is clearer during subsequent feature recognition, which helps improve the accuracy of the recognition results.

[0026] S200 performs feature recognition on the preprocessed real-time image to obtain the needle position and fabric outline in the preprocessed real-time image.

[0027] Specifically, the fabric outline is the outline line of the fabric closest to the sewing trajectory. The needle position is the position where the sewing machine needle goes down. Theoretically, the needle position is fixed because the sewing machine head is fixed at the end of the robotic arm and drives the fabric being sewn to move during the sewing process. However, the position of the sewing machine needle relative to the position of the fabric changes.

[0028] S300 calculates the vertical distance deviation value in the preprocessed real-time image based on the needle position and fabric outline.

[0029] Draw a perpendicular line from the needle position to the fabric outline. The resulting perpendicular line distance is the actual distance. The distance between the needle position and the fabric outline is a pre-set preset distance. The difference between the actual distance and the preset distance is the vertical distance deviation value.

[0030] S400 compares the vertical distance deviation value in the preprocessed real-time image with the safety threshold, and corrects the subsequent sewing process based on the comparison result.

[0031] In this embodiment, real-time images of the sewing process are acquired and preprocessed. Feature recognition is performed on the preprocessed real-time images to obtain the needle position and fabric outline. The vertical distance deviation value is calculated based on the needle position and fabric outline, thereby converting the trajectory deviation into a quantifiable specific value. Then, the vertical distance deviation value is compared with a safety threshold, and the subsequent sewing process is corrected in a graded manner based on the comparison result. This not only realizes real-time monitoring and accurate correction of the sewing trajectory, but also avoids over-correction or under-correction caused by a single threshold through a graded judgment mechanism. It effectively solves the problems of simple correction judgment logic and short-sighted correction adjustment in traditional automated sewing, significantly improves the efficiency and accuracy of sewing trajectory correction, and ensures the consistency of sewing quality.

[0032] In one embodiment of this application, the step of comparing the vertical distance deviation value in the preprocessed real-time image with a safety threshold and correcting the subsequent sewing process based on the comparison result includes the following steps S401 to S403.

[0033] S401, determine whether the vertical distance deviation value in the preprocessed real-time image is greater than or equal to K times the second safety threshold; K is greater than 1.

[0034] S402, if the vertical distance deviation value in the preprocessed real-time image is less than K times the second safety threshold, then it is further determined whether the vertical distance deviation value in the preprocessed real-time image is greater than or equal to the first safety threshold; the second safety threshold is greater than the first safety threshold.

[0035] S403, if the vertical distance deviation value in the preprocessed real-time image is greater than or equal to the first safety threshold, an emergency stop is triggered and a sewing correction operation is performed.

[0036] Specifically, after step S403, the method further includes: if the vertical distance deviation value in the preprocessed real-time image is less than the first safety threshold, then return to the process of acquiring the real-time image during the sewing process, and preprocess the obtained real-time image to obtain the preprocessed real-time image.

[0037] The tiered judgment of vertical distance deviation mainly involves adopting differentiated correction strategies based on different degrees of deviation. First, it checks if the vertical distance deviation is greater than or equal to K times the second safety threshold. If it hasn't reached but has exceeded the first safety threshold, the deviation is within a controllable range, and correction can be achieved by triggering an emergency stop and performing a sewing correction operation. This tiered judgment method avoids using the same correction strategy for all deviation cases, improving correction efficiency.

[0038] In this embodiment, K is set to 2, the second safety threshold is 5mm, and the first safety threshold is 4mm. Therefore, K times the second safety threshold is 10mm. When the vertical distance deviation is between 4mm and 10mm, an emergency stop correction operation is triggered. When the vertical distance deviation is less than 4mm, the sewing trajectory is considered to be within the normal range and no correction is required. When the vertical distance deviation is greater than or equal to 10mm, a severe deviation correction operation is initiated.

[0039] In one embodiment of this application, the triggering of an emergency stop and the execution of a sewing correction operation includes the following steps S403a to S403e.

[0040] S403a, Stop sewing.

[0041] S403b, receive a first correction operation instruction; the first correction operation instruction includes a start correction time node, an end correction time node, and sewing strategy correction information; the start correction time node and the end correction time node constitute a first correction monitoring time window.

[0042] Specifically, the sewing strategy correction information includes the adjustment angle of the sewing direction. The start correction time node and the end correction time node together constitute the two time endpoints of the first correction monitoring time window.

[0043] S403c, within the first correction monitoring time window, a sewing action is executed according to the first correction operation instruction; while executing the sewing action according to the first correction operation instruction within the first correction monitoring time window, a real-time image corresponding to the sewing action is captured and stored, and the real-time image corresponding to the sewing action is used as the first monitoring image.

[0044] S403d: Retrieve the first monitoring image, determine whether the correction is effective based on the first monitoring image, and generate the first judgment result.

[0045] S403e, when the first judgment result is valid, control the sewing action to continue, return to the acquisition of real-time images during the sewing process, and preprocess the obtained real-time images to obtain preprocessed real-time images.

[0046] Specifically, upon triggering an emergency stop, the sewing action immediately ceases to prevent further deviation. Subsequently, a first correction operation command containing a correction time window and sewing strategy correction information is received. Within the first correction monitoring time window, the sewing action is executed according to the correction command, and real-time images within that period are simultaneously acquired as the first monitoring image.

[0047] By analyzing the first monitoring image, it is determined whether the correction operation causes the vertical distance deviation value to converge, thus determining whether the correction is effective. If the correction is effective, normal sewing is resumed and monitoring continues.

[0048] In this embodiment, the duration of the first correction monitoring time window is set to 0.6 seconds. Images acquired within this monitoring time window are used to determine the correction trend. If the image acquisition frame rate is 60 FPS, then 36 frames can be acquired within this monitoring time window. The calculated vertical distance deviation value sequence is [2.1, 2.0, 1.9, 1.8, ...], and the first derivative of the linear regression equation is −0.25 mm / s < 0. Therefore, the correction action is deemed effective, and sewing continues.

[0049] If the first judgment result is valid, it means that the correction operation has successfully introduced the deviation into a convergence trend, and then return to S100 to continue regular monitoring.

[0050] In one embodiment of this application, the triggering of an emergency stop and the execution of a sewing correction operation further includes the following steps S403f to S403i.

[0051] S403f, when the first judgment result is invalid, the sewing action is controlled to continue within the second correction monitoring time window; while the sewing action is controlled to continue within the second correction monitoring time window, a real-time image corresponding to the sewing action is captured and stored, and the real-time image corresponding to the sewing action is used as the second monitoring image; the duration of the second correction monitoring time window is less than the duration of the first correction monitoring time window.

[0052] S403g: Retrieve the second monitoring image, determine whether the correction is effective based on the second monitoring image, and generate a second judgment result.

[0053] S403h, if the second judgment result is valid, then control the sewing action to continue, return to the step of acquiring the real-time image during the sewing process, and preprocess the obtained real-time image to obtain the preprocessed real-time image.

[0054] S403i, if the second judgment result is invalid, terminate the sewing action.

[0055] Specifically, if the correction effect within the first correction monitoring time window is unsatisfactory, that is, if the first judgment result is invalid, the sewing operation is not immediately terminated, but a second monitoring and verification opportunity is given. The duration of the second correction monitoring time window is shorter than that of the first correction monitoring time window. Its main purpose is to quickly verify whether the correction can take effect in a short period of time, avoiding accidental machine stoppages caused by brief interference.

[0056] If the second judgment result is valid, it means that although the correction operation did not take effect in the first window, it can converge in the subsequent shorter window, and normal sewing is resumed; if the second judgment result is still invalid, it means that the current correction strategy cannot effectively correct the deviation, and the sewing action is terminated to avoid the generation of sewing waste.

[0057] In this embodiment, the duration of the second correction monitoring time window is set to 0.1 seconds. Six frames of images are acquired within the second correction monitoring time window. If the first derivative of the linear regression equation for the vertical distance deviation value within the first correction monitoring time window is +0.05, and the first derivative of the linear regression equation for the vertical distance deviation value within the second correction monitoring time window is +0.03, then the correction is deemed invalid, and sewing is stopped.

[0058] By setting a second correction monitoring time window, erroneous shutdowns caused by unreasonable monitoring window settings are avoided, and emergency shutdown opportunities are also prevented from being delayed due to an excessively long window period when corrections are ineffective.

[0059] In one embodiment of this application, the step of retrieving the first monitoring image, determining whether the correction is effective based on the first monitoring image, and generating a first determination result includes the following A100 to A600.

[0060] A100 retrieves the first monitoring image, performs feature extraction on each first monitoring image, and obtains the needle position and fabric outline in each first monitoring image.

[0061] A200 calculates the vertical distance deviation value for each first monitoring image based on the needle position and fabric outline in each first monitoring image.

[0062] A300 performs regression analysis based on the vertical distance deviation values ​​of multiple first monitoring images, establishes a regression equation for the vertical distance deviation values ​​as a function of time, and calculates the first derivative of the regression equation for the vertical distance deviation values ​​as a function of time.

[0063] A400, determine whether the first derivative of the regression equation for the vertical distance deviation value over time is less than zero.

[0064] A500, if the first derivative of the regression equation for the vertical distance deviation value over time is less than zero, then the correction is determined to be effective, and the generated content is the first judgment result of the correction being effective.

[0065] Preferably, to avoid the influence of measurement noise, the result is considered valid only when the first derivative is less than a first preset derivative threshold, for example, the first preset derivative threshold is -0.1.

[0066] A600, if the first derivative of the regression equation for the vertical distance deviation value over time is greater than or equal to zero, then the correction is determined to be invalid, and the generated content is the first judgment result of the correction being invalid.

[0067] Specifically, the judgment criterion for the first monitoring image is the trend of the vertical distance deviation value over time. By performing regression analysis on the vertical distance deviation values ​​of multiple first monitoring images, a regression equation is established to show the change of the vertical distance deviation value over time, and the first derivative of this equation is calculated. The first derivative reflects the rate and direction of change of the vertical distance deviation value. If the first derivative is less than zero, it indicates that the vertical distance deviation value is decreasing over time, and the correction operation is effective; if the first derivative is greater than or equal to zero, it indicates that the vertical distance deviation value is not decreasing, and the correction operation is ineffective. This trend-based judgment method is more reliable than simply comparing whether the vertical distance deviation value is below a threshold.

[0068] In this embodiment, the regression analysis uses a linear regression model, which assumes that the vertical distance deviation value changes linearly with time. The first derivative of the regression equation is the slope of the regression line. When the slope is less than zero, the correction is considered valid; when the slope is greater than or equal to zero, the correction is considered invalid. To eliminate the influence of individual outlier data points on the regression results, outlier removal processing can be performed on the vertical distance deviation value sequence before regression analysis.

[0069] In one embodiment of this application, the step of retrieving the second monitoring image, determining whether the correction is effective based on the second monitoring image, and generating a second determination result includes the following B100 to B600.

[0070] B100 retrieves the second monitoring image, performs feature extraction on each second monitoring image, and obtains the needle position and fabric outline in each second monitoring image.

[0071] B200 calculates the vertical distance deviation value of each second monitoring image based on the needle position and fabric outline in each second monitoring image.

[0072] B300 performs regression analysis based on the vertical distance deviation values ​​of multiple second monitoring images, establishes a regression equation for the vertical distance deviation value as a function of time, and calculates the first derivative of the regression equation for the vertical distance deviation value as a function of time.

[0073] B400, determine whether the first derivative of the regression equation for the vertical distance deviation value over time is less than zero.

[0074] B500, if the first derivative of the regression equation for the vertical distance deviation value over time is less than zero, then the correction is determined to be effective, and the generated content is the second judgment result of the correction being effective.

[0075] B600, if the first derivative of the regression equation for the vertical distance deviation value over time is greater than or equal to zero, then the correction is determined to be invalid, and the generated content is the second judgment result of the correction being invalid.

[0076] In this embodiment, the analysis principle for the second monitoring image is the same as that for the first monitoring image, both based on the trend of the vertical distance deviation value over time to determine whether the correction is effective. The difference lies in the smaller number of second monitoring images and the shorter monitoring time window. If the second judgment result is effective, it indicates that the correction operation shows a convergence trend in a short time, and normal sewing can be resumed; if the second judgment result is invalid, sewing is terminated.

[0077] like Figure 2 and Figure 3 As shown, where a is the needle position in the real-time image; b is the position of the fabric outline point corresponding to the needle position; c is the fabric outline; d is the theoretical sewing trajectory; g is the tangent direction of the fabric outline corresponding to the needle position in the real-time image; f is the sewing direction in the real-time image; L1 is the actual distance; L2 is the vertical distance deviation value; L3 is the first preset distance. In one embodiment of this application, the calculation of the vertical distance deviation value in the preprocessed real-time image based on the needle position and the fabric outline includes the following S301 to S303.

[0078] S301, Obtain the first preset distance; the first preset distance is the vertical distance between the theoretical sewing trajectory and the fabric outline.

[0079] Specifically, the first preset distance is the vertical distance between the theoretical sewing trajectory and the fabric outline. This distance is determined by the sewing process requirements. For example, in garment sewing, the seam allowance is usually a standard value such as 5mm, 8mm or 10mm.

[0080] S302, calculate the shortest vertical distance between the needle position and the fabric outline, and record it as the actual distance.

[0081] Specifically, the actual distance is the shortest vertical distance from the current needle position to the fabric outline, reflecting the gap between the actual sewing path and the fabric outline. Subtracting the actual distance from the first preset distance gives the vertical distance deviation. A positive difference indicates that the actual sewing path is far from the fabric outline; a negative difference indicates that the actual sewing path is close to the fabric outline or even presses against the fabric.

[0082] S303, calculate the difference between the actual distance and the first preset distance, and define the difference as the vertical distance deviation value.

[0083] In this embodiment, the first preset distance is pre-set according to the sewing process parameters. The actual distance is calculated in pixels, converted to millimeters using camera calibration parameters, and then compared with the first preset distance. The calculation frequency of the vertical distance deviation value is consistent with the image acquisition frequency; that is, the vertical distance deviation value is calculated once for each frame of image acquired, so as to achieve real-time monitoring of the sewing trajectory.

[0084] In one embodiment of this application, the step of comparing the vertical distance deviation value in the preprocessed real-time image with a safety threshold and correcting the subsequent sewing process based on the comparison result further includes the following steps S404 to S407.

[0085] S404, if the vertical distance deviation value in the preprocessed real-time image is greater than or equal to K times the second safety threshold, then control the sewing direction to correct it at the maximum correction angle until the vertical distance deviation value is less than half of the second safety threshold.

[0086] Specifically, when the vertical distance deviation value in the real-time image is greater than or equal to K times the second safety threshold, it indicates that the sewing has deviated significantly. The vertical distance deviation value needs to be pulled back within a short time to prevent the sewing machine needle from hitting the edge of the fabric or slipping out of the fabric, effectively avoiding accidents such as broken needles and skipped stitches.

[0087] It should be noted that correcting with the maximum correction angle means that the end effector of the robotic arm rotates at the maximum permissible angular velocity to approach the maximum correction angle value.

[0088] Optionally, after performing the correction of the sewing direction with the maximum correction angle, the following steps are also included: D101 to D104.

[0089] D101 acquires real-time images during the sewing process and preprocesses the acquired real-time images to obtain preprocessed real-time images.

[0090] D102 performs feature recognition on the preprocessed real-time image to obtain the needle position and fabric outline in the preprocessed real-time image.

[0091] D103 calculates the vertical distance deviation value in the preprocessed real-time image based on the needle position and fabric outline.

[0092] D104 compares the vertical distance deviation value with half of the second safety threshold.

[0093] After each execution of controlling the sewing direction to correct at the maximum correction angle and performing sewing for a duration of T1, the real-time image of the sewing process is acquired again, and the vertical distance deviation value of the real-time image is calculated. The vertical distance deviation value of the real-time image is compared with half of the second safety threshold. When the vertical distance deviation value is less than half of the second safety threshold, the fast correction is completed.

[0094] S405, when the vertical distance deviation value is less than half of the second safety threshold, predict the position of the sewing end point based on the current sewing direction.

[0095] The predicted position of the sewing endpoint based on the current sewing direction includes F101 to F102.

[0096] F101 retrieves the remaining planned trajectory parameters of the robotic arm controlling the movement of the fabric.

[0097] F102, starting from the current needle position and taking the current sewing direction as the subsequent sewing direction, controls the robotic arm to execute the remaining planned trajectory parameters to obtain the predicted sewing endpoint position.

[0098] Specifically, the remaining planned trajectory parameters of the robotic arm are read from the controller, including the remaining displacement vector of the end effector in the base coordinate system and the angular velocity curves of each joint. Starting from the current needle position, the sewing direction remains unchanged, and the robotic arm moves forward according to the remaining planned parameters. The final position reached is the predicted sewing endpoint.

[0099] By establishing the discrete-time motion equations of the robotic arm end effector and the fabric, as shown in Equation 1: Formula 1.

[0100] in, This is the state vector for the next time step; A is the current state vector; B is the state transition matrix; C is the control input matrix. These are the control commands for the robotic arm's end effector.

[0101] ,in, This represents the vertical distance deviation value of the current needle position; This represents the sewing angle deviation value in the current sewing direction; This shows the trend of the vertical distance deviation value. This represents the changing trend of the sewing angle deviation value.

[0102] This represents the integral effect of angular deviation on distance deviation. Even if there is no current control input, the existing sewing angle deviation value... The vertical distance deviation will also be caused by the sewing feed. It will automatically increase in the next moment. This refers to the feed step length for sewing.

[0103] This indicates the rotation angle control command of the robot's end effector.

[0104] This refers to the direct object of a rotational motion. The rotational motion of a robot's end effector. What is directly changed is the sewing angle deviation value at the next moment. Regarding the vertical distance deviation value The impact is indirect and needs to be reflected through the integral of the state transition matrix A in the next cycle.

[0105] S406, determine whether the distance between the predicted sewing end point position and the preset sewing end point position is less than the second preset distance.

[0106] Specifically, the second preset distance is a parameter value set in advance. It is used to limit the distance between the predicted sewing end point position and the preset sewing end point position so that the actual sewing trajectory can closely match the pre-designed sewing trajectory.

[0107] S407, if the distance between the predicted sewing endpoint position and the preset sewing endpoint position is greater than or equal to the second preset distance, then correct the sewing direction until the distance between the predicted sewing endpoint position and the preset sewing endpoint position is less than the second preset distance.

[0108] Specifically, the sewing direction is first corrected at the maximum correction angle until the vertical distance deviation drops to less than half of the second safety threshold. Then, the position of the sewing end point is predicted based on the current sewing direction, and it is determined whether the distance between the predicted end point and the preset end point is less than the second preset distance. If the distance between the predicted end point and the preset end point is greater than or equal to the second preset distance, the sewing direction is further corrected until the distance between the predicted end point and the preset end point is less than the second preset distance. This strategy of rapid correction followed by end-point calibration balances the efficiency of correction and the final sewing accuracy.

[0109] In this embodiment, the second preset distance is set to 2mm, meaning the distance between the predicted sewing end point and the preset sewing end point should be less than 2mm. When the vertical distance deviation drops to less than half of the second safety threshold, the end point prediction and calibration process is triggered. By dividing the correction process into a rapid correction stage and an end point calibration stage, both rapid response in cases of severe deviation and accuracy of the final sewing position are ensured.

[0110] In one embodiment of this application, if the vertical distance deviation value in the preprocessed real-time image is greater than or equal to K times the second safety threshold, the sewing direction is controlled to be corrected at the maximum correction angle until the vertical distance deviation value is less than half of the second safety threshold, including the following S404a to S404g.

[0111] S404a performs feature recognition on the real-time image to obtain the fabric outline tangent direction corresponding to the sewing direction and the needle position in the real-time image.

[0112] Specifically, the sewing direction in the real-time image is the direction of movement of the current needle relative to the previous stitch. The tangent direction information of the fabric contour corresponding to the stitch position is the tangent direction of the fabric contour point with the shortest perpendicular distance corresponding to the stitch position.

[0113] S404b, calculate the angle between the sewing direction in the real-time image and the tangent direction of the fabric contour corresponding to the needle position, and define the obtained angle as the sewing angle deviation value.

[0114] S404c, obtain the rotation angle value of the robotic arm that controls the movement of the fabric.

[0115] Specifically, the rotation of the robotic arm's end effector is limited by joint limits, speed restrictions, and surrounding objects; for example, the presence of a sewing machine head restricts the rotation angle of the robotic arm's end effector.

[0116] S404d: Select the smaller of the sewing angle deviation value and the rotatable angle value of the robotic arm that controls the movement of the fabric as the maximum correction angle value, and perform correction monitoring on the current sewing for a duration of T1 to obtain the corrected vertical distance deviation value.

[0117] Specifically, the current sewing is monitored for a duration of T1 to obtain the corrected vertical distance deviation value, including the following values: E101 to E104.

[0118] E101 receives the operation command for the maximum correction angle value.

[0119] E102 executes sewing actions according to the operation command of the maximum correction angle value within a time window of length T1; while executing sewing actions according to the operation command of the maximum correction angle value within a time window of length T1, it captures and stores the real-time image of the last moment, and uses the real-time image of the last moment as the target monitoring image.

[0120] E103 performs feature recognition on the target monitoring image to obtain the needle position and fabric outline in the target monitoring image.

[0121] E104 calculates the vertical distance deviation value of the target monitoring image based on the needle position and fabric outline in the target monitoring image.

[0122] Specifically, the vertical distance deviation value of the target monitoring image is the corrected vertical distance deviation value obtained after each correction using the maximum correction angle value.

[0123] S404e, determine whether the corrected vertical distance deviation value is less than half of the second safety threshold.

[0124] S404f, if the corrected vertical distance deviation value is greater than or equal to half of the second safety threshold, then return to the step of performing feature recognition on the real-time image to obtain the fabric outline tangent direction corresponding to the sewing direction and the needle position in the real-time image.

[0125] S404g, if the corrected vertical distance deviation is less than half of the second safety threshold, then the correction ends.

[0126] Specifically, determining the maximum correction angle requires considering the angle between the current sewing direction and the tangent direction of the fabric contour, as well as the range of rotation angles of the robotic arm. The smaller value between the forward sewing direction and the tangent direction of the fabric contour is selected as the maximum correction angle value to ensure that the correction operation is feasible within the robotic arm's range of motion. Subsequently, the current sewing is monitored for a duration of T1 using this maximum correction angle, and the vertical distance deviation value is recalculated after T1. If the vertical distance deviation value still does not drop below half of the second safety threshold, the above process is repeated until the vertical distance deviation value meets the requirements.

[0127] In this embodiment, T1 is set to 0.1 seconds. After each correction monitoring, real-time images are reacquired and feature recognition is performed to update the sewing angle deviation value and vertical distance deviation value. The iterative correction method can adapt to changes in sewing direction caused by changes in the curvature of the fabric contour, ensuring that the correction angle is always the maximum feasible angle under the current conditions throughout the correction process, thereby achieving the fastest deviation convergence.

[0128] Meanwhile, by making multiple corrections for a duration of T1, the vertical distance deviation value is gradually brought closer to less than half of the second safety threshold. This avoids excessive single corrections that would require repeated corrections and prevent the vertical distance deviation value from being smoothly corrected to less than half of the second safety threshold.

[0129] In one embodiment of this application, the step of correcting the sewing direction if the distance between the predicted sewing endpoint position and the preset sewing endpoint position is greater than or equal to a second preset distance, until the distance between the predicted sewing endpoint position and the preset sewing endpoint position is less than the second preset distance, includes the following steps S407a to S407h.

[0130] S407a, acquire multiple consecutive preset robotic arm end-effector movement trajectory points and multiple consecutive predicted robotic arm end-effector movement trajectory points.

[0131] S407b, based on the positional differences between multiple consecutive preset robotic arm end-effector trajectory points and multiple consecutive predicted robotic arm end-effector trajectory points, a correction cost function is constructed.

[0132] S407c, solve the correction cost function to obtain the optimal correction scheme.

[0133] S407d controls the robotic arm end effector to apply the optimal correction scheme to perform the sewing action for a duration of T2, and obtains the needle position after T2.

[0134] Specifically, the actual meaning of the stitch position is the current sewing point's location on the fabric being sewn.

[0135] S407e: Obtain the remaining planned trajectory parameters of the end effector of the robotic arm that controls the movement of the fabric. Taking the needle position after T2 as the starting point and the current sewing direction as the subsequent sewing direction, control the end effector of the robotic arm to execute the remaining planned trajectory parameters and obtain the updated predicted sewing endpoint position.

[0136] S407f, determine whether the distance between the updated predicted sewing endpoint position and the preset sewing endpoint position is less than the second preset distance.

[0137] S407g, if the distance between the updated predicted sewing endpoint position and the preset sewing endpoint position is greater than or equal to the second preset distance, then return to execute the process of obtaining multiple consecutive preset robotic arm end-effector movement trajectory points and multiple consecutive predicted robotic arm end-effector movement trajectory points.

[0138] S407h, if the distance between the updated predicted sewing endpoint position and the preset sewing endpoint position is less than the second preset distance, then control the end of the robotic arm to apply the optimal correction scheme to perform the sewing action until the sewing is completed.

[0139] Specifically, obtain N consecutive preset end-effector trajectory points and their corresponding predicted end-effector trajectory points. Construct a corrected cost function and solve the optimization problem to obtain the optimal corrected control sequence. Execute the first control variable in the optimal corrected control sequence and continue for time T2. Then update the predicted sewing endpoint position and determine whether the distance between the predicted sewing endpoint position and the preset sewing endpoint position is less than a second preset distance. If not, repeat the solution and correction process until the updated position is reached. and The distance between them is less than the second preset distance.

[0140] In this embodiment, a modified cost function is constructed, as shown in Formula 2: Formula 2.

[0141] Where i is the index of the trajectory point at the end of the robotic arm. The next robotic arm end-effector trajectory point is predicted based on the current robotic arm end-effector trajectory point; is the preset theoretical next robotic arm end-effector trajectory point; u is the smoothness of the current robotic arm end-effector trajectory point; This is the predicted end-effector position of the robotic arm; This is the preset end position of the robotic arm. , , These are the corresponding weight items.

[0142] The optimal control sequence is obtained by solving the modified cost function using quadratic programming. Select the optimal control sequence After applying the correction angle as the current time, the time is maintained at T2. Then, the latest real-time image is reacquired, and the updated angle is calculated. and The distance between them.

[0143] Determine the updated and Is the distance between them less than the second preset distance? If the updated distance is less than the second preset distance, then... and If the distance between them is greater than or equal to the second preset distance, then the optimal control sequence is resolved using the new current actual state vector, and the angle correction of the first term in the new optimal control sequence is performed; the above solution and correction process is repeated until the updated optimal control sequence is reached. and The distance between them is less than the second preset distance.

[0144] If the updated and If the distance between them is less than the second preset distance, the robotic arm will execute the remaining planning parameters to drive the fabric to sew.

[0145] The technical features of the above embodiments can be combined arbitrarily, and the execution order of the method steps is not restricted. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A trajectory correction control method, characterized in that, The trajectory correction control method includes: Acquire real-time images during the sewing process and preprocess the acquired real-time images to obtain preprocessed real-time images; Feature recognition is performed on the preprocessed real-time image to obtain the needle position and fabric outline in the preprocessed real-time image; The vertical distance deviation value in the preprocessed real-time image is calculated based on the needle position and the fabric outline. The vertical distance deviation value in the preprocessed real-time image is compared with the safety threshold, and the subsequent sewing process is corrected based on the comparison result.

2. The trajectory correction control method according to claim 1, characterized in that, The step of comparing the vertical distance deviation value in the preprocessed real-time image with a safety threshold, and correcting subsequent sewing processes based on the comparison result, includes: Determine whether the vertical distance deviation value in the preprocessed real-time image is greater than or equal to K times the second safety threshold; K is greater than 1; If the vertical distance deviation value in the preprocessed real-time image is less than K times the second safety threshold, then it is further determined whether the vertical distance deviation value in the preprocessed real-time image is greater than or equal to the first safety threshold; the second safety threshold is greater than the first safety threshold. If the vertical distance deviation value in the preprocessed real-time image is greater than or equal to the first safety threshold, an emergency stop is triggered and a sewing correction operation is performed.

3. The trajectory correction control method according to claim 2, characterized in that, The triggering of an emergency stop and execution of sewing correction operations includes: Stop sewing; Receive a first correction operation instruction; the first correction operation instruction includes a start correction time node, an end correction time node, and sewing strategy correction information; the start correction time node and the end correction time node constitute a first correction monitoring time window; Within the first correction monitoring time window, a sewing action is performed according to the first correction operation instruction; while performing the sewing action according to the first correction operation instruction within the first correction monitoring time window, a real-time image corresponding to the sewing action is captured and stored, and the real-time image corresponding to the sewing action is used as the first monitoring image; Retrieve the first monitoring image, determine whether the correction is effective based on the first monitoring image, and generate the first judgment result; When the first judgment result is valid, the sewing action is controlled to continue, and the process of acquiring real-time images during the sewing process is returned. The obtained real-time images are then preprocessed to obtain preprocessed real-time images.

4. The trajectory correction control method according to claim 3, characterized in that, The method of triggering an emergency stop and performing sewing correction operations also includes: When the first judgment result is invalid, the sewing action is controlled to continue within the second correction monitoring time window; while the sewing action is controlled to continue within the second correction monitoring time window, a real-time image corresponding to the sewing action is captured and stored, and the real-time image corresponding to the sewing action is used as the second monitoring image; the duration of the second correction monitoring time window is less than the duration of the first correction monitoring time window; Retrieve the second monitoring image, determine whether the correction is effective based on the second monitoring image, and generate a second judgment result; If the second judgment result is valid, the sewing action is controlled to continue, and the process of obtaining real-time images during the sewing process is returned. The obtained real-time images are then preprocessed to obtain preprocessed real-time images. If the second judgment result is invalid, the sewing action is terminated.

5. The trajectory correction control method according to claim 3, characterized in that, The process of retrieving the first monitoring image, determining whether the correction is effective based on the first monitoring image, and generating a first determination result includes: Retrieve the first monitoring image, extract features from each first monitoring image, and obtain the needle position and fabric outline in each first monitoring image; Based on the needle position and fabric outline in each first monitoring image, calculate the vertical distance deviation value for each first monitoring image; Regression analysis was performed based on the vertical distance deviation values ​​of the multiple first monitoring images obtained. A regression equation for the vertical distance deviation values ​​changing with time was established, and the first derivative of the regression equation for the vertical distance deviation values ​​changing with time was calculated. Determine whether the first derivative of the regression equation for the vertical distance deviation over time is less than zero; If the first derivative of the regression equation for the vertical distance deviation value over time is less than zero, then the correction is determined to be effective, and the generated content is the first judgment result of the correction being effective. If the first derivative of the regression equation for the vertical distance deviation value over time is greater than or equal to zero, the correction is determined to be invalid, and the generated content is the first judgment result of the invalid correction.

6. The trajectory correction control method according to claim 4, characterized in that, The process of retrieving the second monitoring image, determining whether the correction is effective based on the second monitoring image, and generating a second determination result includes: Retrieve the second monitoring image, extract features from each second monitoring image, and obtain the needle position and fabric outline in each second monitoring image; Based on the needle position and fabric outline in each second monitoring image, the vertical distance deviation value of each second monitoring image is calculated. Regression analysis was performed based on the vertical distance deviation values ​​of the obtained second monitoring images to establish a regression equation for the vertical distance deviation values ​​as a function of time, and the first derivative of the regression equation for the vertical distance deviation values ​​as a function of time was calculated. Determine whether the first derivative of the regression equation for the vertical distance deviation over time is less than zero; If the first derivative of the regression equation for the vertical distance deviation value over time is less than zero, then the correction is determined to be effective, and the generated content is the second judgment result of the correction being effective. If the first derivative of the regression equation for the vertical distance deviation value over time is greater than or equal to zero, then the correction is determined to be invalid, and the generated content is the second judgment result of the correction being invalid.

7. The trajectory correction control method according to claim 1, characterized in that, The vertical distance deviation value in the preprocessed real-time image calculated based on the needle position and fabric contour includes: Obtain a first preset distance; the first preset distance is the vertical distance between the theoretical sewing trajectory and the fabric outline; Calculate the shortest vertical distance between the stitch position and the fabric outline, and record it as the actual distance; Calculate the difference between the actual distance and the first preset distance, and define this difference as the vertical distance deviation value.

8. The trajectory correction control method according to claim 2, characterized in that, The step of comparing the vertical distance deviation value in the preprocessed real-time image with a safety threshold, and correcting subsequent sewing processes based on the comparison result, also includes: If the vertical distance deviation value in the preprocessed real-time image is greater than or equal to K times the second safety threshold, the sewing direction is controlled to correct it at the maximum correction angle until the vertical distance deviation value is less than half of the second safety threshold. When the vertical distance deviation is less than half of the second safety threshold, the position of the sewing end point is predicted based on the current sewing direction; Determine whether the distance between the predicted sewing end point position and the preset sewing end point position is less than a second preset distance; If the distance between the predicted sewing endpoint and the preset sewing endpoint is greater than or equal to the second preset distance, the sewing direction is corrected until the distance between the predicted sewing endpoint and the preset sewing endpoint is less than the second preset distance.

9. The trajectory correction control method according to claim 8, characterized in that, If the vertical distance deviation value in the preprocessed real-time image is greater than or equal to K times the second safety threshold, then the sewing direction is controlled to correct at the maximum correction angle until the vertical distance deviation value is less than half of the second safety threshold, including: Feature recognition is performed on the real-time image to obtain the sewing direction and the direction of the fabric outline tangent corresponding to the needle position in the real-time image; Calculate the angle between the sewing direction in the real-time image and the tangent direction of the fabric outline corresponding to the needle position, and define the obtained angle as the sewing angle deviation value. Obtain the rotation angle value of the robotic arm that controls the movement of the fabric; The smaller of the sewing angle deviation value and the rotatable angle value of the robotic arm controlling the fabric movement is selected as the maximum correction angle value, and the current sewing is monitored for correction for a duration of T1 to obtain the corrected vertical distance deviation value. Determine whether the corrected vertical distance deviation value is less than half of the second safety threshold; If the corrected vertical distance deviation value is greater than or equal to half of the second safety threshold, then return to the step of performing feature recognition on the real-time image to obtain the sewing direction and the fabric outline tangent direction corresponding to the needle position in the real-time image; If the corrected vertical distance deviation is less than half of the second safety threshold, the correction ends.

10. The trajectory correction control method according to claim 9, characterized in that, If the distance between the predicted sewing end point and the preset sewing end point is greater than or equal to a second preset distance, the sewing direction is corrected until the distance between the predicted sewing end point and the preset sewing end point is less than the second preset distance, including: Obtain multiple consecutive preset robotic arm end-effector trajectory points and multiple consecutive predicted robotic arm end-effector trajectory points. Based on the positional differences between multiple consecutive preset robotic arm end-effector trajectory points and multiple consecutive predicted robotic arm end-effector trajectory points, a correction cost function is constructed. Solve the modified cost function to obtain the optimal modification scheme; The robotic arm end effector is controlled to perform a sewing action by applying the optimal correction scheme for a duration of T2, and the needle position after T2 is obtained. Obtain the remaining planned trajectory parameters of the robotic arm end effector that controls the movement of the fabric. Taking the stitch position after T2 as the starting point and the current sewing direction as the subsequent sewing direction, control the robotic arm end effector to execute the remaining planned trajectory parameters to obtain the updated predicted sewing endpoint position. Determine whether the distance between the updated predicted sewing end point position and the preset sewing end point position is less than the second preset distance; If the distance between the updated predicted sewing endpoint position and the preset sewing endpoint position is greater than or equal to the second preset distance, then return to the process of obtaining multiple consecutive preset robotic arm end-effector movement trajectory points and multiple consecutive predicted robotic arm end-effector movement trajectory points. If the distance between the updated predicted sewing endpoint position and the preset sewing endpoint position is less than the second preset distance, then the control robot arm end effector applies the optimal correction scheme to perform the sewing action until the sewing is completed.

Images