Mobile control method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a mobile control method and device, electronic equipment and storage medium, obtaining a visual feature of a thread end; determining a translation attenuation factor based on the visual feature of the thread end, a preset centering position threshold and a preset posture alignment threshold, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor; determining a translation increment parameter based on the translation attenuation factor and the visual feature of the thread end, so that the thread end moves according to the translation increment parameter; determining whether the thread end reaches a preset target position based on a preset success determination function, and ending the thread end movement control process if the thread end reaches the preset target position. The problem that the thread end posture is easily pulled to a position that is not conducive to subsequent rotation alignment when performing translation centering in the prior art is solved.

Description

Technical Field

[0001] This disclosure relates to the field of industrial testing technology, specifically to a mobile control method, device, electronic device, and storage medium. Background Technology

[0002] In industrial sewing, medical needle assembly, electronic wire threading, and precision component assembly, threading typically requires precisely aligning a very small diameter flexible thread end with the needle hole or guide hole while maintaining a stable posture during insertion. These tasks involve small target sizes, with hole diameters often on the order of millimeters or even sub-millimeters. Therefore, with the development and maturation of robotics technology, threading is now generally performed by robots.

[0003] When a robot is threading a needle, it typically first obtains the center deviation of the thread end and the direction angle of the line segment as control errors. Then, it directly drives the end of the robotic arm to perform translation and rotation based on the obtained control errors. Once the error is reduced to a threshold, it performs insertion.

[0004] Currently, when performing translation, only the positional error between the line end and the target position is considered, and translation is performed solely based on the positional error. However, the posture of the line end changes significantly during the translation process, which increases the difficulty of subsequent posture adjustment. Summary of the Invention

[0005] To address the shortcomings of existing technologies that only consider positional errors during the centering process of line head translation, resulting in significant changes in the posture of the line head after centering, this disclosure provides a motion control method, device, electronic device, and storage medium.

[0006] In a first aspect, embodiments of this specification provide a motion control method for moving the end of a flexible wire to a preset target position, characterized in that the method includes:

[0007] Obtain visual features of the thread ends;

[0008] Based on the visual features of the line end, the preset centering position threshold, and the preset posture alignment threshold, a translation attenuation factor is determined, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor.

[0009] Based on the translation attenuation factor and the visual characteristics of the thread end, a translation increment parameter is determined so that the thread end moves according to the translation increment parameter; and

[0010] Based on a preset success determination function, it is determined whether the thread end has reached the preset target position, and if the thread end has reached the preset target position, the thread end movement control process ends.

[0011] In some implementations, determining the translation attenuation factor based on the visual features of the line end, a preset centering position threshold, and a preset pose alignment threshold includes:

[0012] Based on the visual characteristics of the thread end, determine the actual positional error between the thread end and the preset target position;

[0013] Based on the preset centering position threshold, the reference position error between the line end and the preset target position is determined; and

[0014] The first translation attenuation factor is determined based on the actual position error and the reference position error.

[0015] In some embodiments, the visual features of the thread end include positional error information of the thread end relative to a preset target position, and the method further includes:

[0016] Based on a preset translation error dead zone threshold, a translation error dead zone function is established; and

[0017] Based on the translation error dead zone function, position error dead zone processing is performed on the position error information of the line end relative to the preset target position.

[0018] In some implementations, the preset attitude alignment threshold includes an attitude error threshold and an attitude alignment success threshold. The step of determining the translation attenuation factor based on the line end visual features, the preset centering position threshold, and the preset attitude alignment threshold includes:

[0019] Based on the visual features of the thread end and the preset target posture, the thread end posture error is determined;

[0020] Based on the line head attitude error, attitude error threshold, and attitude alignment success threshold, the translation limiting parameters are determined; and

[0021] The second translation attenuation factor is determined based on the translation limiting parameters.

[0022] In some embodiments, the method further includes:

[0023] Based on the pose information in the visual features of the thread end, the preset target pose, and the pose alignment success threshold, pose maintenance parameters are determined; and

[0024] Based on the posture maintenance parameters, the posture information in the visual features of the line head is maintained.

[0025] In some implementations, determining whether the thread end has reached the preset target position based on a preset success determination function includes:

[0026] Calculate the success determination parameters based on the preset success determination function; and

[0027] If the success determination parameter is valid, the thread end is determined to have reached the preset target position; if the success determination parameter is invalid, the thread end is determined to have not reached the preset target position.

[0028] In some embodiments, the method further includes:

[0029] Obtain the auxiliary visual features of the thread end; and

[0030] The validity of the visual features of the thread end is determined based on the auxiliary visual features of the thread end.

[0031] Secondly, embodiments of this specification provide a mobile control device, the device comprising:

[0032] The thread end feature acquisition module is used to acquire the visual features of the thread end;

[0033] The translation attenuation factor determination module is used to determine the translation attenuation factor based on the visual features of the line end, a preset centering position threshold, and a preset posture alignment threshold, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor.

[0034] A translation increment parameter determination module is used to determine translation increment parameters based on the translation attenuation factor and the visual characteristics of the line end, so that the line end moves according to the translation increment parameters; and

[0035] The success determination module is used to determine whether the wire end has reached the preset target position based on a preset success determination function. If the wire end has reached the preset target position, the wire end movement control process ends.

[0036] In some implementations, the translation attenuation factor determination module is configured to:

[0037] Based on the visual characteristics of the thread end, determine the actual positional error between the thread end and the preset target position;

[0038] Based on the preset centering position threshold, the reference position error between the line end and the preset target position is determined; and

[0039] The first translation attenuation factor is determined based on the actual position error and the reference position error.

[0040] In some embodiments, the visual features of the thread end include positional error information of the thread end relative to a preset target position, and the translation attenuation factor determination module is configured to:

[0041] Based on a preset translation error dead zone threshold, a translation error dead zone function is established; and

[0042] Based on the translation error dead zone function, position error dead zone processing is performed on the position error information of the line end relative to the preset target position.

[0043] In some implementations, the preset attitude alignment threshold includes an attitude error threshold and an attitude alignment success threshold, and the translation attenuation factor determination module is configured to:

[0044] Based on the visual features of the thread end and the preset target posture, the thread end posture error is determined;

[0045] Based on the line head attitude error, attitude error threshold, and attitude alignment success threshold, the translation limiting parameters are determined; and

[0046] The second translation attenuation factor is determined based on the translation limiting parameters.

[0047] In some embodiments, the motion control device further includes an attitude maintenance module, which is configured to:

[0048] Based on the pose information in the visual features of the thread end, the preset target pose, and the pose alignment success threshold, pose maintenance parameters are determined; and

[0049] Based on the posture maintenance parameters, the posture information in the visual features of the line head is maintained.

[0050] In some implementations, the success determination module is configured to:

[0051] Calculate the success determination parameters based on the preset success determination function; and

[0052] If the success determination parameter is in a valid state, it is determined that the thread end has reached the preset target position.

[0053] In some embodiments, the motion control device further includes a feature validity determination module, which is configured to:

[0054] Obtain the auxiliary visual features of the thread end; and

[0055] The validity of the visual features of the thread end is determined based on the auxiliary visual features of the thread end.

[0056] Thirdly, embodiments of this specification provide an electronic device, including:

[0057] processor;

[0058] The memory stores computer instructions that cause the processor to perform the method described in any of the above embodiments.

[0059] Fourthly, embodiments of this specification provide a computer-readable storage medium storing computer instructions for implementing the methods described in any of the above embodiments.

[0060] The movement control method described in this specification acquires the visual features of the thread end; based on the visual features of the thread end, a preset centering position threshold, and a preset attitude alignment threshold, a translation attenuation factor is determined, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor; based on the translation attenuation factor and the visual features of the thread end, a translation increment parameter is determined so that the thread end moves according to the translation increment parameter; based on a preset success determination function, it is determined whether the thread end has reached a preset target position, and if the thread end has reached the preset target position, the thread end movement control process ends. By introducing two translation attenuation factors during the centering process of the thread end, the two translation attenuation factors are the translation attenuation factor when the thread end is closer to the target and the translation attenuation factor when the attitude deviation of the thread end is large. The two translation attenuation factors ensure that the thread end can gradually approach the pinhole position, and also avoid the situation where attitude error coupling amplification may be caused by simply forcibly centering, which provides convenience for the subsequent attitude alignment process. Attached Figure Description

[0061] To more clearly illustrate the specific embodiments or technical solutions in the prior art of this specification, the accompanying drawings used in the description of the specific embodiments or prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this specification. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0062] Figure 1 A flowchart illustrating a motion control method according to an embodiment of the present disclosure is shown schematically.

[0063] Figure 2 A flowchart illustrating a pose calibration method according to an embodiment of the present disclosure is shown schematically.

[0064] Figure 3 A flowchart illustrating an alignment processing method according to an embodiment of the present disclosure is shown schematically.

[0065] Figure 4 A schematic diagram of a motion control device according to an embodiment of the present disclosure is shown.

[0066] Figure 5 A schematic diagram of a pose calibration apparatus according to an embodiment of the present disclosure is shown.

[0067] Figure 6 A schematic diagram of an alignment processing apparatus according to an embodiment of the present disclosure is shown.

[0068] Figure 7 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0069] The technical solutions of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification. Furthermore, the technical features involved in the different embodiments of this specification described below can be combined with each other as long as they do not conflict with each other.

[0070] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0071] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0072] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0073] A sewing robot is an automated device that can autonomously thread a needle and automatically perform sewing tasks in scenarios such as industrial sewing, medical needle assembly, electronic wire drilling, and precision component assembly.

[0074] In existing technologies, when a sewing robot performs the task of threading a needle, it generally first obtains the deviation of the thread end center and the direction angle of the line segment as control errors. Then, it directly drives the end of the robotic arm to perform translation and rotation based on the obtained control errors. After the error is reduced to a threshold, it performs insertion. For example, the deviation of the thread end relative to the image center or the needle hole center is first used as the translation control quantity. Then, the slope angle of the line segment or the angle of the connection between the endpoints is used as the attitude control quantity. Afterward, single-stage proportional control or simple pseudo-inverse control is used to adjust the horizontal and vertical translational degrees of freedom as well as the horizontal and vertical rotational degrees of freedom. When the center of the thread end is close to the center of the needle hole, the end straight insertion action is directly performed.

[0075] However, translation alters the projection of the thread tip onto the image, while rotation affects its center position. Therefore, directly controlling all four degrees of freedom simultaneously can easily lead to thread tip position oscillations. Furthermore, due to the flexibility of the thread tip, variations in camera mounting position, and tool offset, translation and rotation collectively affect image plane features. If controlled simultaneously without phased adjustments, the control direction and magnitude are prone to instability. Moreover, simply centering the thread tip relative to the image center does not guarantee that the thread tip's center truly coincides with the pinhole's center, especially when the pinhole deviates from the image center or the camera's viewing angle changes; the error becomes more pronounced then.

[0076] Based on the deficiencies of the aforementioned related technologies, this disclosure provides a method for achieving needle and thread alignment, applied to a sewing robot, aiming to improve the insertion accuracy of the needle and thread in the sewing robot.

[0077] In this embodiment, the needle-thread alignment process may include a thread end movement control process, a thread end pose calibration process, and a secondary needle hole alignment process. The thread end movement control process can be understood as an initial centering translation control, used to ensure the thread end falls into the center of the camera's field of view or near a preset fine-tuning target, providing stable initial conditions for subsequent pose alignment. The thread end pose calibration process controls the robotic arm's end effector to rotate in small steps around the Z-axis, gradually bringing the projection direction of the thread end in the image closer to the desired pose. The secondary needle hole alignment process further aligns the center of the thread end with the center of the needle hole after the initial centering translation and pose alignment of the thread end have been completed.

[0078] In one possible embodiment, this disclosure provides a motion control method that can be executed by a processor integrated into a sewing robot to achieve initial alignment of the thread end with the needle hole, and after execution, move the thread end to a first preset position to obtain a first position parameter of the thread end at the first preset position.

[0079] Taking the needle alignment scenario of a sewing robot as an example, the processor can acquire the visual features of the thread end; based on the visual features of the thread end, a preset centering position threshold, and a preset posture alignment threshold, determine the translation attenuation factor, which includes a first translation attenuation factor and a second translation attenuation factor; based on the translation attenuation factor and the visual features of the thread end, determine the translation increment parameter so that the thread end moves according to the translation increment parameter; based on a preset success determination function, determine whether the thread end has reached the preset target position, and if the thread end has reached the preset target position, end the thread end movement control process.

[0080] As can be seen from the above, in the embodiments of this disclosure, two translation attenuation factors are introduced during the process of centering the line head. The two translation attenuation factors are the translation attenuation factor when the line head is closer to the target and the translation attenuation factor when the attitude deviation of the line head is large. The two translation attenuation factors can ensure that the line head can gradually approach the pinhole position, and also avoid the situation of attitude error coupling amplification that may be caused by simply forcibly centering, thus providing convenience for the subsequent attitude alignment process.

[0081] Figure 1 A flowchart illustrating a motion control method according to an embodiment of the present disclosure is shown schematically. Figure 1 As shown, the motion control method 100 of this embodiment may include steps S101 to S104.

[0082] Step S101: Obtain the visual features of the thread ends.

[0083] Step S102: Based on the visual features of the line end, the preset centering position threshold, and the preset posture alignment threshold, determine the translation attenuation factor, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor.

[0084] Step S103: Based on the translation attenuation factor and the visual characteristics of the wire end, determine the translation increment parameter so that the wire end moves according to the translation increment parameter.

[0085] Step S104: Based on the preset success determination function, determine whether the wire end has reached the preset target position. If the wire end has reached the preset target position, end the wire end movement control process.

[0086] The implementation process of each step is illustrated below as an example.

[0087] Step S101: Obtain the visual features of the thread ends.

[0088] In this embodiment of the disclosure, the thread end is the portion of the wire end used for inserting a pinhole. The processor can identify the target area where the thread end is located in the original image based on preset thread end characteristics. For example, the thread end portion in the original image can be identified based on preset thread end shape, color, and other characteristics.

[0089] Sewing robots can directly acquire images of thread ends using cameras integrated on the robot. These images are then processed to obtain visual features of the thread ends. The processing includes, but is not limited to, region-of-interest (ROI) cropping, preprocessing to extract the target region, and geometric moment calculation to extract position and shape error information. The extracted visual features can include position error information of the thread end relative to a preset target position, differences in the horizontal and vertical expansion of the target region along the image, and the degree of tilt coupling of the target region in the image plane.

[0090] Optionally, in order to ensure the validity of the visual features of the thread end, this disclosure also obtains auxiliary visual features of the thread end; based on the auxiliary visual features of the thread end, the validity of the visual features of the thread end is judged.

[0091] For example, the auxiliary visual features of the thread end include at least the anisotropy intensity η of the thread end, the visual detection confidence, and the area of ​​the segmented region of the thread end. The validity judgment of the visual features of the thread end can include confidence judgment, thread end region area judgment, and visual temporal information judgment. Specifically, the validity judgment can be performed using the following formula:

[0092] ,

[0093] Among them, confidence min As the minimum confidence threshold, A min and A max These represent the upper and lower limits of the area of ​​the line head region, respectively, t feature The timestamp for the visual features of the thread starter, t timeout This is the timeout period for visual characteristics of the thread ends.

[0094] The above-described validity judgment method can be used to determine the validity of the visual features of the thread end. When the visual features of the thread end are invalid, the subsequent steps will not be executed, thus preventing the robotic arm holding the thread end from being driven by incorrect visual information.

[0095] Step S102: Based on the visual features of the line end, the preset centering position threshold, and the preset posture alignment threshold, determine the translation attenuation factor, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor.

[0096] In this embodiment, the subsequent translation increment parameters are determined by two translation attenuation factors. The first translation attenuation factor is a translation attenuation factor related to the position error, and the second translation attenuation factor is a translation attenuation factor related to the attitude error. By using the two translation attenuation factors, it can be ensured that the wire end can gradually approach the pinhole position, and the attitude error coupling amplification may be caused by simply forcibly centering.

[0097] Optionally, when determining the first translation attenuation factor, the actual position error between the line end and the preset target position can be determined based on the visual characteristics of the line end; the reference position error between the line end and the preset target position can be determined based on the preset centering position threshold; and the first translation attenuation factor can be determined based on the actual position error and the reference position error.

[0098] In this embodiment, the first translation attenuation factor is used to enable the sewing robot to translate with a larger translation step size when the thread end is far from the preset target position, and to translate with a smaller translation step size when the thread end is close to the preset target position, so as to avoid the translation amplitude being too large and causing the translation to exceed the preset target position.

[0099] For example, when determining the first translation attenuation factor, the actual position error between the line end and the preset target position is first determined based on the position error information of the line end relative to the preset target position in the visual features of the line end. The specific calculation formula is as follows:

[0100] ,

[0101] in,

[0102] ,

[0103] in, This indicates the dead zone threshold for lateral translation error. This represents the dead zone threshold for longitudinal translation error.

[0104] Specifically, when the thread end is close to the target point, if the sewing robot's robotic arm continues to perform translational movements, the thread end may exceed the preset target position, resulting in a small-scale shaking at the end of the robotic arm. Therefore, to avoid the robotic arm shaking back and forth near the target, this disclosure establishes a translation error dead zone function based on a preset translation error dead zone threshold; based on the translation error dead zone function, position error dead zone processing is performed on the position error information of the thread end relative to the preset target position.

[0105] By performing position error dead zone processing, when the absolute value of the position deviation between the actual position of the line head and the preset target position is less than a preset threshold (i.e., the "dead zone"), the absolute value of the position deviation is regarded as zero, and the actual position of the line head is no longer adjusted, thus avoiding the situation of small-range shaking at the end of the robotic arm.

[0106] Specifically, the dead-time function can be:

[0107] .

[0108] By handling the dead zone of the position error as described above, the system will no longer generate commands for the robotic arm when the end of the line is close to the preset target position, thereby reducing the end effector jitter of the robotic arm.

[0109] For example, the reference position error between the line end and the preset target position can be obtained through a preset centering position threshold, wherein the preset centering position threshold includes a horizontal centering threshold T. u And vertical centering threshold T v The specific formula for calculating the error between the line end and the reference position of the preset target position is as follows:

[0110] .

[0111] After obtaining the actual position error and the reference position error, the first translation attenuation factor can be calculated. The specific calculation formula is as follows:

[0112] ,

[0113] Here, clip(x, a, b) means restricting x to the interval [a, b].

[0114] Optionally, when determining the second translation attenuation factor, the line head posture error can be determined based on the line head visual characteristics and the preset target posture; the translation limiting parameter can be determined based on the line head posture error, the posture error threshold, and the posture alignment success threshold; and the second translation attenuation factor can be determined based on the translation limiting parameter.

[0115] In this embodiment of the disclosure, since the attitude alignment may be amplified by translation centering when the line head is not aligned, which increases the difficulty of the subsequent attitude alignment stage, this disclosure introduces a second translation attenuation factor in the translation centering control stage. The second translation attenuation factor is used to reduce the translation intensity when the attitude feature deviates from the target by a large margin, so as to avoid the translation adjusting the attitude of the line head to a position that is not conducive to attitude rotation alignment.

[0116] For example, when obtaining the second translation attenuation factor, the pose information q2 in the visual features of the line end and the preset target pose features are first used as the basis. The specific formula for determining the line end posture error is as follows:

[0117] .

[0118] When eq2 is greater than the attitude alignment success threshold, the translation limiting parameter can be calculated. The specific calculation formula is as follows:

[0119] ,

[0120] Among them, T entry T represents the attitude error threshold before entering the attitude alignment phase. success This indicates the threshold for successful attitude alignment.

[0121] After obtaining the translation limiting parameter, the second translation attenuation factor can be calculated based on this parameter. The specific calculation formula is as follows:

[0122] .

[0123] In one possible embodiment, the attitude change of the line head should be zero during the line head movement control phase. However, in actual execution, the centering of the line head during translation can cause changes in its attitude information. Therefore, to prevent the line head attitude from deviating from the target attitude during translation, attitude maintenance can be performed on the line head. The purpose of attitude maintenance is to ensure that the line head attitude does not change significantly during translation, so as to facilitate subsequent attitude alignment.

[0124] For example, posture maintenance parameters can be determined based on the posture information in the visual features of the line end, the preset target posture, and the posture alignment success threshold; and posture maintenance can be performed on the posture information in the visual features of the line end based on the posture maintenance parameters.

[0125] For example, the formula for calculating the attitude maintenance parameter ΔRz is:

[0126] ,

[0127] Among them, R z,max To limit the maximum command amplitude, g maintain To maintain the attitude gain, β q The maintenance ratio is adaptively adjusted based on the magnitude of the attitude information deviation.

[0128] Step S103: Based on the translation attenuation factor and the visual characteristics of the wire end, determine the translation increment parameter so that the wire end moves according to the translation increment parameter.

[0129] In this embodiment of the disclosure, the translation increment parameter is used to control the translation of the line end so that the line end reaches a preset target position, which is the first preset position. The translation increment parameter includes a horizontal translation increment and a vertical translation increment.

[0130] Optionally, the formula for calculating the lateral translation increment is:

[0131] .

[0132] The formula for calculating the longitudinal translation increment is:

[0133] .

[0134] Where, k xu k is the proportional gain of the lateral pixel error to the x-direction (lateral) translation. zv γ is the proportional gain from the vertical pixel error to the translation in the z-direction (vertical direction), c is the cross-coupling coefficient used to compensate for the coupling caused by the non-perfect orthogonality between the horizontal and vertical axes of the image and the x and z axes of the sewing robot. c This is the overall translation scaling factor.

[0135] Step S104: Based on the preset success determination function, determine whether the wire end has reached the preset target position. If the wire end has reached the preset target position, end the wire end movement control process.

[0136] In this embodiment of the disclosure, in order to determine whether the line end has reached the preset target position after translation, it is necessary to judge the position of the line end by a success judgment parameter. Specifically, the success judgment parameter can be calculated based on a preset success judgment function. If the success judgment parameter is valid, it is determined that the line end has reached the preset target position. If the success judgment parameter is invalid, it is determined that the line end has not reached the preset target position.

[0137] In one possible embodiment, the success determination function can be:

[0138] ,

[0139] C center This indicates that the parameter was successfully determined when C center =false, meaning that when the parameter is successfully determined to be invalid, the system continues to execute the line head translation and centering process. When C center =true, meaning that the parameter is successfully determined to be valid, it indicates that the line head has reached the preset target position. At this time, the line head movement control process can be ended, and the first position parameter of the line head at the preset target position is recorded. The first position parameter may include the lateral error and longitudinal error of the line head relative to the final target point after the line head translation and centering process is completed.

[0140] During the centering process of the line head translation, two translation attenuation factors are introduced. These two factors are the translation attenuation factor when the line head is closer to the target and the translation attenuation factor when the attitude deviation of the line head is large. By using these two translation attenuation factors, we can ensure that the line head can gradually approach the pinhole position. On the other hand, we can also avoid the situation where attitude error coupling amplification may be caused by simply forcibly centering, which provides convenience for the subsequent attitude alignment process.

[0141] In one possible embodiment, this disclosure also provides a pose calibration method, which can be executed by a processor integrated into a sewing robot to complete the pose calibration process, thereby achieving the alignment of the thread end and the needle hole, and after execution, the pose characteristics of the thread end can conform to the target pose characteristics.

[0142] Taking the needle alignment scenario of a sewing robot as an example, the processor can acquire the current pose features and orientation features of the thread end. The current pose features include position features and orientation features. Based on the orientation features and orientation features, the rotation direction of the thread end is determined. Based on the position features of the thread end, translation compensation control parameters are determined. Based on the rotation direction, orientation features, and preset basic rotation step size of the thread end, rotation control parameters are determined. Based on the rotation control parameters and translation compensation control parameters, the end effector of the robotic arm used to grip the thread end is controlled to move to update the current pose features of the thread end. It is determined whether the updated current pose features of the thread end match the preset target pose features. If not, the rotation control parameters and translation compensation control parameters are regenerated based on the updated current pose features of the thread end, and the end effector of the robotic arm is controlled to continue moving until the updated current pose features of the thread end match the target pose features, thereby achieving thread end pose alignment.

[0143] As can be seen from the above, in the embodiments of this disclosure, by introducing translation compensation control parameters during the alignment of the wire head posture, the wire head can be translated and compensated while being rotated using rotation control parameters, thus avoiding the displacement of the center position of the wire head due to the rotation of the wire head.

[0144] Figure 2 A flowchart illustrating a pose calibration control method according to an embodiment of the present disclosure is shown schematically. Figure 2 As shown, the pose calibration control method 200 of this disclosure embodiment may include steps S201 to S207.

[0145] Step S201: Obtain the current pose features and direction determination features of the line end. The current pose features include position features and attitude features.

[0146] Step S202: Determine the rotation direction of the line end based on the posture features and direction judgment features.

[0147] Step S203: Determine the translation compensation control parameters based on the positional characteristics of the line end.

[0148] Step S204: Determine the rotation control parameters based on the rotation direction and posture characteristics of the line end and the preset basic rotation step size.

[0149] Step S205: Based on rotation control parameters and translation compensation control parameters, control the end effector of the robotic arm used to grip the wire end to move, so as to update the current pose characteristics of the wire end.

[0150] Step S206: Determine whether the current pose features after the line end is updated match the preset target pose features.

[0151] Step S207: If not, based on the updated current pose features of the line end, regenerate the rotation control parameters and translation compensation control parameters, and control the end effector of the robotic arm to continue moving until the updated current pose features of the line end match the target pose features, so as to achieve line end pose alignment.

[0152] The implementation process of each step is illustrated below as an example.

[0153] Step S201: Obtain the current pose features and direction determination features of the line end. The current pose features include position features and attitude features.

[0154] In this embodiment of the disclosure, after entering the attitude alignment control stage, the position feature in the current pose feature is the first position parameter obtained last in the above-mentioned line head movement control method. Both the attitude feature and the direction judgment feature are obtained through the line head visual features provided by the vision system. The direction judgment feature can be the line segment slope or direction auxiliary quantity output by the vision system. This disclosure does not limit this.

[0155] Step S202: Determine the rotation direction of the line end based on the posture features and direction judgment features.

[0156] In this embodiment of the disclosure, to avoid errors in rotation direction, the disclosure first determines the rotation direction of the thread end and verifies the determined rotation direction to avoid invalid posture alignment actions. Optionally, when determining the rotation direction, alternative rotation directions of the thread end can be determined based on the relative relationship between the posture characteristics of the thread end and the posture characteristics of a preset target. Based on the direction judgment characteristics and the trend of posture error change after rotating along the alternative rotation direction, it is determined whether the alternative rotation direction is correct. If so, the alternative rotation direction is taken as the rotation direction of the thread end; if not, the opposite direction of the alternative rotation direction is taken as the rotation direction of the thread end.

[0157] For example, in this embodiment of the disclosure, the current posture feature q2 and the preset target posture feature can be used as a basis for the initial determination. The relative relationship determines the alternative rotation direction s r Specifically, it can be expressed as:

[0158] .

[0159] Simultaneously record the attitude error and rotation direction characteristics, where the attitude error can be expressed as:

[0160] .

[0161] Taking the slope of a line segment as an example, the rotation direction characteristic can be recorded as follows: Here, line_slope represents the slope of the line segment output by the visual end.

[0162] If the attitude error continues to increase after rotating along the alternative rotation direction and the slope of the line segment does not change sufficiently, it indicates that the rotation direction may be incorrect and the alternative rotation direction needs to be reversed. Otherwise, it indicates that the alternative rotation direction is correct.

[0163] For example, when determining the rotation direction, the change in attitude error and the change in direction determination features after rotating along the candidate rotation direction can be obtained; based on the change in attitude error, the change in direction determination features, and the preset flipping conditions, it can be determined whether the candidate rotation direction is correct.

[0164] Specifically, since the wire ends are generally flexible, relying solely on the change in attitude error may lead to misjudgment. Therefore, in this embodiment, after several preheating frames, the attitude error and line segment slope after the preheating frames are recorded and used as a benchmark for judging the error trend. and slope judgment criteria Then, the real-time error change is obtained, as follows:

[0165] .

[0166] when Greater than the preset error change benchmark value If the trend of q2 is incorrect, then the change in the slope of the line segment should be calculated simultaneously, as follows:

[0167] .

[0168] when Greater than the preset slope change benchmark value If the trend of q2 is incorrect, but the slope of the line segment is still changing significantly, it means that the current rotation direction is still valid. Therefore, the alternative rotation direction will still be taken as the correct rotation direction of the line head.

[0169] Therefore, this embodiment can determine whether it is necessary to change the alternative rotation direction based on preset turning conditions. The specific turning conditions can be:

[0170] .

[0171] When the flipping conditions are met, the alternative rotation direction is reversed. This embodiment utilizes the changing trends of attitude error and line segment slope to jointly determine whether the rotation is valid, effectively avoiding misjudgment of the rotation direction in flexible wires.

[0172] Step S203: Determine the translation compensation control parameters based on the positional characteristics of the line end.

[0173] In this embodiment of the disclosure, since the center position of the wire end may change during the attitude alignment control stage, this disclosure performs translation compensation while adjusting the attitude to avoid deviation in the center position of the wire end.

[0174] For example, when determining the translation compensation control parameters, the total translation error can be determined based on the positional characteristics of the line end; the reference position error can be determined based on the preset horizontal centering threshold and vertical centering threshold; the compensation amplification factor can be determined based on the total translation error and the reference position error; and the translation compensation control parameters can be determined based on the compensation amplification factor and the positional characteristics of the line end.

[0175] Specifically, the formula for determining the total translation error is:

[0176] .

[0177] The formula for determining the reference position error is:

[0178] .

[0179] The formula for determining the compensation amplification factor is:

[0180] ,

[0181] Among them, b max This represents the maximum compensation magnification factor.

[0182] Therefore, the translation compensation control parameters can be obtained, specifically:

[0183] ,

[0184] Where, γ o γ is the translation compensation ratio during the attitude alignment control phase. c The translation reference ratio is denoted by c, which is the cross-coupling coefficient, and k is the translation centering control phase.xu k is the proportional gain of the lateral pixel error to the x-direction (lateral) translation. zv This is the proportional gain of the vertical pixel error to the translation in the z-direction (vertical).

[0185] Step S204: Determine the rotation control parameters based on the rotation direction and posture characteristics of the line end and the preset basic rotation step size.

[0186] In this embodiment of the disclosure, the rotation control parameters are used to control the robotic arm holding the wire end to rotate in a stepping manner. Optionally, when determining the rotation control parameters, the posture error can be determined based on the posture characteristics of the wire end and the preset target posture characteristics; the rotation step ratio can be determined based on the relationship between the posture error and the size of the preset step attenuation region; and the rotation control parameters can be determined based on the rotation direction of the wire end, the preset basic rotation step, and the rotation step ratio.

[0187] For example, the attitude error is the difference between the current attitude features and the target attitude features. When the attitude error e q2 Not less than the preset attitude deviation threshold T orient At this time, it indicates that the posture of the line end still needs to be adjusted. Therefore, this disclosure first presets a step size decay region Z. s Its expression is:

[0188] .

[0189] When the attitude error e q2 The decay region Z is smaller than the step size. s When this happens, it indicates that the step size of the robotic arm needs to be reduced to avoid excessive amplitude of the robotic arm causing deviation in rotational position. The rotational step size ratio can be:

[0190] .

[0191] When the attitude error e q2 Not less than the step size decay region Z s At this time, the step size of the robotic arm can be increased to quickly rotate the robotic arm into position, and the rotation step size ratio can be 1.

[0192] Therefore, based on the obtained rotation step size ratio, the final generated rotation control parameters can be:

[0193] ,

[0194] Among them, s r ∈{−1, +1}, representing the current rotation direction, r step This indicates the preset base rotation step size.

[0195] Step S205: Based on rotation control parameters and translation compensation control parameters, control the end effector of the robotic arm used to grip the wire end to move, so as to update the current pose characteristics of the wire end.

[0196] In this embodiment, after determining the rotation control parameters and translation compensation control parameters, the robotic arm holding the wire end can be controlled according to these parameters. Translation compensation is performed simultaneously with rotation to ensure that the center position of the wire end does not change while adjusting its posture. Furthermore, after the robotic arm's movement is completed, the current pose features are updated to facilitate the generation of translation compensation control parameters and rotation control parameters for the next movement of the robotic arm.

[0197] Step S206: Determine whether the current pose features after the line end is updated match the preset target pose features.

[0198] Step S207: If not, based on the updated current pose features of the line end, regenerate the rotation control parameters and translation compensation control parameters, and control the end effector of the robotic arm to continue moving until the updated current pose features of the line end match the target pose features, so as to achieve line end pose alignment.

[0199] In this embodiment, when the updated pose features of the thread end match the preset target pose features, the pose adjustment is complete, and the pose alignment control process can end. Otherwise, it means that the robotic arm needs to continue to be controlled to adjust the pose of the thread end. Therefore, at this time, steps S201 to S206 will be re-executed based on the updated current pose features, and so on, until the updated pose features of the thread end match the preset target pose features.

[0200] For example, the pose error between the target pose feature and the updated current pose feature of the line head can be determined; and based on the pose error, a preset pose deviation threshold, a preset lateral centering threshold, and a longitudinal centering threshold, it can be determined whether the updated current pose feature of the line head conforms to the preset target pose feature. The position of the line head corresponding to the preset target pose feature is the second target position.

[0201] For example, after each step of the robotic arm, it checks whether the current posture error has reached the preset posture deviation threshold T. orient If the target is reached, the position of the line end is further checked to determine whether the lateral and longitudinal errors of the current line end relative to the final target point are less than the preset lateral centering threshold and longitudinal centering threshold. If both the attitude check and the position check pass, it means that the line end attitude alignment is complete. If either one fails, the attitude adjustment or position adjustment needs to be readjusted.

[0202] When it is determined that the attitude of the line end has been rotated to the preset attitude position, the parameters of the line end at the preset attitude position are recorded, namely the second position parameters. The second position parameters may include the coordinates of the centroid of the line end after the line end attitude alignment control process is completed, as well as the lateral and longitudinal errors of the line end relative to the final target point.

[0203] In one possible implementation, to avoid excessive rotation steps of the robotic arm due to visual feature errors, this disclosure also provides a maximum rotation step protection mechanism. Optionally, it determines whether the number of movement steps currently executed by the end of the robotic arm has reached a preset step threshold. If so, the attitude alignment control process is terminated. This can prevent the sewing robot from staying in the attitude alignment control stage for a long time due to unreliable attitude features or invalid rotation direction.

[0204] By introducing translation compensation control parameters during the alignment of the wire head, translation compensation can be performed on the wire head while controlling its rotation using rotation control parameters, thus preventing the center position of the wire head from shifting due to rotation.

[0205] Taking the needle hole alignment scenario of a sewing robot as an example, the processor can acquire the visual features of the thread end; based on the visual features of the thread end, generate primary alignment parameters, and control the thread end to move to a first preset position according to the primary alignment parameters to achieve translational centering of the thread end; based on the visual features of the thread end and the current pose features of the thread end, generate posture control parameters, and control the thread end to rotate at the first preset position according to the posture control parameters until the current pose features of the thread end match the preset target pose features to achieve pose alignment of the thread end; based on the target pose features of the thread end and the needle hole coordinates, generate secondary alignment parameters, and control the alignment of the thread end and the needle hole according to the secondary alignment parameters.

[0206] As described above, in this embodiment, the pinhole alignment process is decomposed into a translation centering control stage, an attitude alignment control stage, and a secondary alignment stage, effectively reducing the control instability caused by the current direct four-degree-of-freedom strong coupling. Moreover, compared to the traditional method of directly inserting the pin after completing the initial position and attitude alignment, this disclosure improves the pinhole alignment accuracy by performing secondary alignment between the thread end and the center of the pinhole, avoiding the situation where the thread end cannot be inserted into the pinhole after position and attitude adjustment.

[0207] Figure 3 A flowchart illustrating an alignment processing method according to an embodiment of the present disclosure is shown schematically. Figure 3 As shown, the alignment processing method 300 of this embodiment may include steps S301 to S304.

[0208] Step S301: Obtain the visual features of the thread ends.

[0209] Step S302: Based on the visual features of the thread end, generate a primary alignment parameter, and control the thread end to move to the first preset position according to the primary alignment parameter.

[0210] Step S303: Based on the visual features of the thread end and the current pose features of the thread end, generate attitude control parameters to control the thread end to rotate at the first preset position according to the attitude control parameters until the current pose features of the thread end match the preset target pose features, so as to achieve the pose alignment of the thread end.

[0211] Step S304: Based on the target pose features of the thread end and the coordinates of the pinhole, generate secondary alignment parameters to control the alignment of the thread end and the pinhole according to the secondary alignment parameters.

[0212] The implementation process of each step is illustrated below as an example.

[0213] Step S301: Obtain the visual features of the thread ends.

[0214] In this embodiment, the method for obtaining the visual features of the thread end is the same as that in the above-described thread end movement control method, and will not be repeated here.

[0215] Step S302: Based on the visual features of the thread end, generate a primary alignment parameter, and control the thread end to move to the first preset position according to the primary alignment parameter, so as to realize the translation and centering of the thread end.

[0216] In this embodiment, a translation attenuation factor can be determined based on the visual features of the thread end, a preset centering position threshold, and a preset posture alignment threshold. The translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor. Based on the translation attenuation factor and the visual features of the thread end, a primary alignment parameter is determined to control the thread end to move based on the primary alignment parameter. The primary alignment parameter includes a translation increment parameter. Based on a preset success determination function, it is determined whether the thread end has reached a first preset position. If the thread end reaches the first preset position, the first position parameter of the thread end is recorded.

[0217] The specific implementation method has been described in detail in the above-mentioned line head movement control method, and will not be repeated here.

[0218] Step S303: Based on the visual features of the thread end and the current pose features of the thread end, generate attitude control parameters to control the thread end to rotate at the first preset position according to the attitude control parameters until the current pose features of the thread end match the preset target pose features, so as to achieve the pose alignment of the thread end.

[0219] In this embodiment, the current pose features and orientation features of the thread end can be obtained. The current pose features include position features and orientation features. Based on the orientation features and orientation features, the rotation direction of the thread end is determined. Based on the position features of the thread end, translation compensation control parameters are determined. Based on the rotation direction, orientation features, and preset basic rotation step size of the thread end, rotation control parameters are determined. Based on the rotation control parameters and translation compensation control parameters, the end effector of the robotic arm used to grip the thread end is controlled to move to update the current pose features of the thread end. It is determined whether the updated current pose features of the thread end conform to the preset target pose features. If not, the rotation control parameters and translation compensation control parameters are regenerated according to the updated current pose features of the thread end, and the end effector of the robotic arm is controlled to continue moving until the updated current pose features of the thread end conform to the target pose features, so as to achieve thread end pose alignment.

[0220] The specific implementation method has been described in detail in the above-mentioned line head attitude alignment control method, and will not be repeated here.

[0221] Step S304: Based on the target pose features of the thread end and the coordinates of the pinhole, generate secondary alignment parameters to control the alignment of the thread end and the pinhole according to the secondary alignment parameters.

[0222] In this embodiment, the first preset position is typically a fixed target point or a preset servo center point in the camera image, its function being to bring the thread end into a suitable viewing state for orientation alignment. However, in actual needle-threading tasks, the final target point is not to center the thread end in the image, but rather to align the thread end with the center of the needle hole in the image. Therefore, this embodiment performs a secondary alignment after orientation alignment is completed to zero out the relative positional error between the thread end and the needle hole.

[0223] Optionally, the pinhole coordinates can be obtained; based on the pinhole coordinates and the target pose features of the thread end, the positional error between the thread end and the pinhole can be determined; based on the target pose features of the thread end and the positional error between the thread end and the pinhole, secondary alignment parameters can be generated to control the alignment of the centroid of the thread end with the center of the pinhole according to the secondary alignment parameters.

[0224] For example, pinhole coordinates This information can be obtained during needle grasping. Afterwards, based on the coordinates of the thread head's centroid and the needle hole after the alignment control process, the positional error between the thread head and the needle hole can be obtained. This positional error is mainly the lateral error between the thread head's centroid and the needle hole's center, which can be specifically expressed as:

[0225] ,

[0226] Where, when d uh When d > 0, it means the thread end is located to the right of the center of the needle hole. uhWhen d < 0, it means the thread end is located to the left of the center of the needle hole. uh When = 0, it indicates that the centroid of the thread end is aligned with the center of the needle hole. Furthermore, to prevent vibration at the end of the robotic arm, this embodiment of the disclosure also performs dead-zone processing on this positional error, specifically:

[0227] ,

[0228] Where, δ u This is the threshold for the lateral error dead zone.

[0229] Once the positional error between the thread end and the pinhole is obtained, secondary alignment parameters can be generated. Specifically, based on the positional error between the thread end and the pinhole, the secondary alignment lateral position adjustment parameters can be determined; based on the target pose characteristics of the thread end, the secondary alignment longitudinal position adjustment parameters can be determined; and based on the secondary alignment lateral and longitudinal position adjustment parameters, the secondary alignment parameters can be generated.

[0230] For example, the calculation method for the secondary alignment lateral position adjustment parameter is as follows:

[0231] ,

[0232] Where, γ s,u k is the lateral translation scaling factor during secondary alignment. xh This is the proportional gain corresponding to the pinhole lateral error.

[0233] The calculation method for the longitudinal position adjustment parameters of the secondary alignment is as follows:

[0234] ,

[0235] Where, γ s,v k is the longitudinal translation scaling factor during secondary alignment. zv This represents the proportional gain corresponding to the longitudinal error. This is the result of processing the longitudinal error dead zone.

[0236] In one possible implementation, when the robotic arm performs the action of gripping the wire end using secondary alignment parameters, it can be determined whether the secondary alignment is successful. Specifically, the secondary alignment success determination condition can be obtained. Based on the secondary alignment success determination condition, the position of the wire end after being controlled by the secondary alignment parameters is verified to determine whether the secondary alignment is successful.

[0237] In this embodiment of the disclosure, the secondary alignment success determination condition consists of a horizontal alignment condition and a vertical alignment condition. The horizontal alignment condition can specifically be:

[0238] ,

[0239] Among them, Tuh This is the threshold for lateral alignment between the thread end and the pinhole.

[0240] Vertical alignment conditions can be:

[0241] ,

[0242] Among them, T v2 This is the longitudinal threshold for secondary alignment.

[0243] Therefore, the final condition for determining successful secondary alignment can be:

[0244] ,

[0245] .

[0246] This disclosure decomposes the pinhole alignment process into a translation centering control stage, an attitude alignment control stage, and a secondary alignment stage, effectively reducing the control instability caused by the current direct four-degree-of-freedom strong coupling. Moreover, compared with the traditional method of directly inserting the pin after completing the initial position and attitude alignment, this disclosure improves the pinhole alignment accuracy by performing secondary alignment of the pinhole center, avoiding the situation where the wire end cannot be inserted into the pinhole after position and attitude adjustment.

[0247] Figure 4 A schematic diagram of a motion control device according to an embodiment of the present disclosure is shown. Figure 4 As shown, the movement control device 400 includes:

[0248] The thread end feature acquisition module 401 is used to acquire the visual features of the thread end;

[0249] The translation attenuation factor determination module 402 is used to determine the translation attenuation factor based on the visual features of the line end, a preset centering position threshold and a preset posture alignment threshold, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor.

[0250] The translation increment parameter determination module 403 is used to determine the translation increment parameter based on the translation attenuation factor and the visual characteristics of the line end, so that the line end moves according to the translation increment parameter; and

[0251] The success determination module 404 is used to determine whether the wire end has reached the preset target position based on the preset success determination function. If the wire end reaches the preset target position, the wire end movement control process ends.

[0252] In some implementations, the translation attenuation factor determination module 402 is configured to:

[0253] Based on the visual characteristics of the thread end, determine the actual positional error between the thread end and the preset target position;

[0254] Based on a preset centering threshold, the reference position error between the line end and the preset target position is determined; and

[0255] The first translation attenuation factor is determined based on the actual position error and the reference position error.

[0256] In some implementations, the visual features of the thread end include positional error information of the thread end relative to a preset target position, and the translation attenuation factor determination module 402 is configured to:

[0257] Based on a preset translation error dead zone threshold, a translation error dead zone function is established; and

[0258] Based on the translation error dead zone function, position error dead zone processing is performed on the position error information of the line end relative to the preset target position.

[0259] In some implementations, the preset attitude alignment threshold includes an attitude error threshold and an attitude alignment success threshold, and the translation attenuation factor determination module 402 is configured as follows:

[0260] Based on the visual characteristics of the thread end and the preset target posture, the thread end posture error is determined.

[0261] Based on the line head attitude error, attitude error threshold, and attitude alignment success threshold, the translation limiting parameters are determined; and

[0262] The second translation attenuation factor is determined based on the translation limiting parameter.

[0263] In some embodiments, the motion control device 400 further includes an attitude maintenance module, which is configured to:

[0264] Based on the pose information from the visual features of the line end, the preset target pose, and the pose alignment success threshold, the pose maintenance parameters are determined; and

[0265] Based on the posture maintenance parameters, the posture information in the visual features of the line ends is maintained.

[0266] In some implementations, the success determination module 404 is configured to:

[0267] Calculate the success determination parameters based on the preset success determination function; and

[0268] If the parameter is successfully determined to be valid, the wire end is determined to have reached the preset target position; if the parameter is successfully determined to be invalid, the wire end is determined to have not reached the preset target position.

[0269] In some embodiments, the motion control device 400 further includes a feature validity determination module, which is configured to:

[0270] Obtain auxiliary visual features of the thread ends; and

[0271] Based on the auxiliary visual features of the thread ends, the validity of the visual features of the thread ends is judged.

[0272] Figure 5 A schematic diagram of a pose calibration apparatus according to an embodiment of the present disclosure is shown. Figure 5 As shown, the pose calibration device 500 includes:

[0273] The feature acquisition module 501 is used to acquire the current pose features and orientation judgment features of the line end. The current pose features include position features and attitude features.

[0274] The orientation determination module 502 is used to determine the rotation direction of the line end based on the posture features and orientation judgment features.

[0275] The translation compensation control parameter determination module 503 is used to determine the translation compensation control parameters based on the positional characteristics of the line end.

[0276] The rotation control parameter determination module 504 is used to determine the rotation control parameters based on the rotation direction, posture characteristics and preset basic rotation step size of the line end.

[0277] The control module 505 is used to control the end effector of the robot arm used to grip the wire end to move based on rotation control parameters and translation compensation control parameters, so as to update the current pose characteristics of the wire end.

[0278] The attitude determination module 506 is used to determine whether the current pose features after the line end is updated match the preset target pose features.

[0279] The update module 507 is used to, if not, regenerate rotation control parameters and translation compensation control parameters based on the current pose characteristics after the line end is updated, and control the end of the robotic arm to continue moving until the current pose characteristics after the line end is updated match the target pose characteristics.

[0280] In some implementations, the direction determination module 502 is configured to:

[0281] Based on the relative relationship between the line end posture characteristics and the preset target posture characteristics, the alternative rotation directions of the line end are determined.

[0282] Based on the orientation judgment characteristics and the attitude error change trend after rotating along the candidate rotation direction, determine whether the candidate rotation direction is correct; and

[0283] If so, use the alternative rotation direction as the rotation direction of the thread end.

[0284] In some implementations, the direction determination module 502 is configured to:

[0285] Obtain the change in attitude error and the change in orientation judgment features after rotating along the candidate rotation direction; and

[0286] Based on the change in attitude error, the change in direction judgment features, and the preset flipping conditions, it is determined whether the candidate rotation direction is correct.

[0287] In some implementations, the translation compensation control parameter determination module 503 is configured as follows:

[0288] Based on the positional characteristics of the line ends, determine the total translation error;

[0289] The reference position error is determined based on the preset horizontal centering threshold and vertical centering threshold.

[0290] The compensation amplification factor is determined based on the total translation error and the reference position error; and

[0291] Based on the compensation amplification factor and the positional characteristics of the wire end, the translation compensation control parameters are determined.

[0292] In some implementations, the rotation control parameter determination module 504 is configured to:

[0293] The attitude error is determined based on the attitude characteristics of the line end and the preset target attitude characteristics;

[0294] Based on the relationship between attitude error and the preset step size decay region, the rotation step size ratio is determined; and

[0295] The rotation control parameters are determined based on the rotation direction of the wire end, the preset basic rotation step size, and the rotation step size ratio.

[0296] In some implementations, the attitude determination module 506 is configured as follows:

[0297] Determine the pose error of the line end based on the target pose features and the updated current pose features; and

[0298] Based on the pose error, the preset pose deviation threshold, the preset horizontal centering threshold, and the vertical centering threshold, it is determined whether the current pose features after the line head is updated conform to the preset target pose features.

[0299] In some embodiments, the pose calibration device 500 further includes a rotation step determination module, used for:

[0300] Determine whether the number of steps currently executed by the robotic arm end effector has reached the preset step threshold. If so, terminate the attitude alignment control process.

[0301] Figure 6A schematic diagram of an alignment processing apparatus according to an embodiment of the present disclosure is shown. Figure 6 As shown, the alignment processing device 600 includes:

[0302] Visual feature acquisition module 601 is used to acquire the visual features of the thread end;

[0303] The primary alignment module 602 is used to generate primary alignment parameters based on the visual features of the thread end, so as to control the thread end to move to a first preset position according to the primary alignment parameters;

[0304] The attitude adjustment module 603 is used to generate attitude control parameters based on the visual features of the line end and the current pose features of the line end, so as to control the line end to rotate at the first preset position according to the attitude control parameters until the current pose features of the line end match the target pose features.

[0305] The secondary alignment module 604 is used to generate secondary alignment parameters based on the target pose features of the thread end and the coordinates of the pinhole, so as to control the alignment of the centroid of the thread end with the center of the pinhole according to the secondary alignment parameters.

[0306] In some implementations, the primary alignment module 602 is configured to:

[0307] Based on the visual features of the line end, the preset centering position threshold, and the preset posture alignment threshold, the translation attenuation factor is determined, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor.

[0308] Based on the translation attenuation factor and the visual characteristics of the thread end, primary alignment parameters are determined to move the thread end according to these parameters. These primary alignment parameters include translation increment parameters; and

[0309] Based on the preset success determination function, it is determined whether the wire end has reached the first preset position. If the wire end has reached the first preset position, the first position parameter of the wire end is recorded.

[0310] In some implementations, the visual features of the thread end include orientation determination features, and the posture adjustment module 603 is configured as follows:

[0311] Obtain the current pose features and orientation features of the line end. The current pose features include position features and attitude features.

[0312] Based on posture features and direction judgment features, the rotation direction of the line end is determined;

[0313] Based on the positional characteristics of the line end, determine the translation compensation control parameters;

[0314] Based on the rotation direction and posture characteristics of the line end, as well as the preset basic rotation step size, the rotation control parameters are determined.

[0315] Based on rotation control parameters and translation compensation control parameters, the robot controls the end effector of the robotic arm used to grip the wire end to move, so as to update the current pose characteristics of the wire end.

[0316] Determine whether the current pose features after the line end is updated match the preset target pose features; and

[0317] If not, based on the updated pose characteristics of the line end, regenerate the rotation control parameters and translation compensation control parameters, and control the end effector of the robotic arm to continue moving until the updated pose characteristics of the line end match the target pose characteristics, so as to achieve line end pose alignment.

[0318] In some implementations, the attitude adjustment module 603 is configured to:

[0319] Based on the rotation direction and posture characteristics of the line end, as well as the preset basic rotation step size, the rotation control parameters are determined, including:

[0320] The attitude error is determined based on the attitude characteristics of the line end and the preset target attitude characteristics;

[0321] Based on the relationship between attitude error and the preset step size decay region, the rotation step size ratio is determined; and

[0322] The rotation control parameters are determined based on the rotation direction of the wire end, the preset basic rotation step size, and the rotation step size ratio.

[0323] In some implementations, the secondary alignment module 604 is configured as follows:

[0324] Obtain the coordinates of the pinhole;

[0325] Based on the pinhole coordinates and the target pose features of the thread end, determine the positional error between the thread end and the pinhole; and

[0326] Based on the target pose features of the thread end and the positional error between the thread end and the pinhole, secondary alignment parameters are generated to control the alignment of the centroid of the thread end with the center of the pinhole.

[0327] In some implementations, the secondary alignment module 604 is configured as follows:

[0328] Based on the positional error between the thread end and the pinhole, determine the horizontal position adjustment parameters for secondary alignment;

[0329] Based on the target pose characteristics of the line end, determine the longitudinal position adjustment parameters for secondary alignment; and

[0330] Secondary alignment parameters are generated based on the secondary alignment lateral position adjustment parameters and the secondary alignment longitudinal position adjustment parameters.

[0331] In some implementations, the secondary alignment module 604 is configured as follows:

[0332] Obtain the conditions for successful secondary alignment; and

[0333] The position of the wire end after secondary alignment is controlled by the secondary alignment parameters, and the secondary alignment is verified by the secondary alignment success criteria to determine whether the secondary alignment is successful.

[0334] Figure 7 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown below, in conjunction with... Figure 7 This specification describes some embodiments of electronic devices.

[0335] Reference Figure 7 The electronic device 700 may include one or more of the following components: processing component 702, memory 704, power supply component 706, multimedia component 708, audio component 710, input / output (I / O) interface 712, sensor component 716, and communication component 718.

[0336] Processing component 702 typically controls the overall operation of electronic device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 702 may include one or more processors 720 to execute instructions. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702. As another example, processing component 702 may read executable instructions from memory to implement relevant functions of the electronic device.

[0337] Memory 704 is configured to store various types of data to support the operation of electronic device 700. Examples of this data include instructions for any application or method operating on electronic device 700, contact data, phonebook data, messages, pictures, videos, etc. Memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0338] Power supply component 706 provides power to various components of electronic device 700. Power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 700.

[0339] Multimedia component 708 includes a display screen that provides an output interface between electronic device 700 and user. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When electronic device 700 is in an operating mode, such as shooting mode or video mode, the front-facing camera and / or rear-facing camera can receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0340] Audio component 710 is configured to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) configured to receive external audio signals when electronic device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 704 or transmitted via communication component 718. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.

[0341] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0342] Sensor assembly 716 includes one or more sensors for providing state assessments of various aspects of electronic device 700. For example, sensor assembly 716 may detect the on / off state of electronic device 700, the relative positioning of components such as the display and keypad of electronic device 700, changes in position of electronic device 700 or a component of electronic device 700, the presence or absence of user contact with electronic device 700, orientation or acceleration / deceleration of electronic device 700, and temperature changes of electronic device 700. Sensor assembly 716 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 716 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 716 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0343] Communication component 718 is configured to facilitate wired or wireless communication between electronic device 700 and other devices. Electronic device 700 can access wireless networks based on communication standards, such as Wi-Fi, 2G, 3G, 4G, 5G, or 6G, or combinations thereof. In one exemplary embodiment, communication component 718 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 718 also includes a short-range communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0344] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0345] In an exemplary embodiment, a non-transitory computer-readable storage medium, such as a memory for instructions, is also provided. When the instructions in the storage medium are executed by the processor of the electronic device 700, the electronic device 700 is able to execute the thread end translation and centering control method, the thread end posture alignment control method, or the needle and thread alignment method of any embodiment of the present disclosure.

[0346] Non-transitory computer-readable storage media can be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices, etc.

[0347] Obviously, the above embodiments are merely examples for clear illustration and are not intended to limit the embodiments. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all embodiments here. However, obvious variations or modifications derived therefrom remain within the scope of protection created by this specification.

Claims

1. A motion control method for moving the end of a flexible wire to a preset target position, characterized in that, The method includes: Obtain visual features of the thread ends; Based on the visual features of the line end, the preset centering position threshold, and the preset posture alignment threshold, a translation attenuation factor is determined, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor. Based on the translation attenuation factor and the visual characteristics of the thread end, a translation increment parameter is determined so that the thread end moves according to the translation increment parameter; and Based on a preset success determination function, it is determined whether the thread end has reached the preset target position, and if the thread end has reached the preset target position, the thread end movement control process ends.

2. The method according to claim 1, characterized in that, The determination of the translation attenuation factor based on the visual features of the line end, a preset centering position threshold, and a preset pose alignment threshold includes: Based on the visual characteristics of the thread end, determine the actual positional error between the thread end and the preset target position; Based on the preset centering position threshold, the reference position error between the line end and the preset target position is determined; and The first translation attenuation factor is determined based on the actual position error and the reference position error.

3. The method according to claim 2, characterized in that, The visual features of the thread end include positional error information of the thread end relative to a preset target position, and the method further includes: Based on a preset translation error dead zone threshold, a translation error dead zone function is established; and Based on the translation error dead zone function, position error dead zone processing is performed on the position error information of the line end relative to the preset target position.

4. The method according to claim 1, characterized in that, The preset attitude alignment threshold includes an attitude error threshold and an attitude alignment success threshold. The step of determining the translation attenuation factor based on the line end visual features, the preset centering position threshold, and the preset attitude alignment threshold includes: Based on the visual features of the thread end and the preset target posture, the thread end posture error is determined; Based on the line head attitude error, attitude error threshold, and attitude alignment success threshold, the translation limiting parameters are determined; and The second translation attenuation factor is determined based on the translation limiting parameters.

5. The method according to claim 4, characterized in that, The method further includes: Based on the pose information in the visual features of the thread end, the preset target pose, and the pose alignment success threshold, pose maintenance parameters are determined; and Based on the posture maintenance parameters, the posture information in the visual features of the line head is maintained.

6. The method according to claim 1, characterized in that, The determination of whether the thread end has reached the preset target position based on the preset success determination function includes: Calculate the success determination parameters based on the preset success determination function; and If the success determination parameter is in a valid state, it is determined that the thread end has reached the preset target position.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obtain the auxiliary visual features of the thread end; and The validity of the visual features of the thread end is determined based on the auxiliary visual features of the thread end.

8. A mobile control device, characterized in that, The device includes: The thread end feature acquisition module is used to acquire the visual features of the thread end; The translation attenuation factor determination module is used to determine the translation attenuation factor based on the visual features of the line end, a preset centering position threshold, and a preset posture alignment threshold, wherein the translation attenuation factor includes a first translation attenuation factor and a second translation attenuation factor. A translation increment parameter determination module is used to determine translation increment parameters based on the translation attenuation factor and the visual characteristics of the line end, so that the line end moves according to the translation increment parameters; and The success determination module is used to determine whether the wire end has reached the preset target position based on a preset success determination function, and to end the wire end movement control process if the wire end has reached the preset target position.

9. An electronic device, characterized in that, include: processor; A memory storing computer instructions for causing a processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The device stores computer instructions for implementing the method according to any one of claims 1 to 7.

Images