Visual servo method, servo device, equipment and system based on template pushing operation

Abstract

The invention provides a visual servo method, servo device, equipment and system based on template pushing operation, and the method comprises the steps: obtaining a first code block used for positioning a template machine, and determining the position coordinates of the template machine in a world space according to the first code block; determining position coordinates of a virtual target code block when the to-be-operated template is pushed into a template machine clamping groove by utilizing a preset transformation matrix; a second code block used for positioning the to-be-operated template is obtained, and a plate pushing mechanism is indicated to press and push the to-be-operated template according to the position coordinates of the second code block; according to the position error between the real-time position coordinate of the second code block and the position coordinate of the virtual target code block, the pushing direction and the pushing distance of the template to be operated are adjusted; and in response to the fact that the position error is within the preset position error range, it is judged that the to-be-operated template is pushed to the target position of the template machine. According to the template pushing method provided by the invention, rapid positioning, high-precision alignment and stable template pushing can be realized.

Description

Technical Field

[0001] The present application belongs to the technical field of automated sewing, and relates to a visual servoing method, servo device, equipment, and system based on template push operation. Background Art

[0002] In the garment industry, template machines achieve standardized and efficient sewing operations by accurately positioning the cutting pieces on the template, and are the core equipment for large-scale production.

[0003] However, existing template machines rely heavily on manual labor, requiring steps such as placing the cut pieces, inserting the template into the machine's slots, and preparing for sewing. These operations are monotonous, time-consuming, and labor-intensive. The current labor shortage is becoming increasingly severe, making it difficult for factories to recruit enough workers for these repetitive tasks, limiting production efficiency and consistency. Summary of the Invention

[0004] The present application provides a visual servoing method, servo device, equipment and system based on template pushing operation, which are used to solve the problem that template machine operation is highly dependent on manual labor.

[0005] In the first aspect, the present application provides a visual servoing method based on a template push operation, the method comprising: obtaining a first code group for positioning a template machine, and determining the position coordinates of the template machine in the world space according to the first code group; using a preset transformation matrix to determine the position coordinates of a virtual target code group when the template to be operated is pushed into the card slot of the template machine; obtaining a second code group for positioning the template to be operated, and instructing a push plate mechanism to press and push the template to be operated according to the position coordinates of the second code group; adjusting the pushing direction and pushing distance of the template to be operated according to the position error between the real-time position coordinates of the second code group and the position coordinates of the virtual target code group; and in response to the position error being within a preset position error range, determining that the template to be operated has been pushed to the target position of the template machine.

[0006] In an implementation of the first aspect, the preset process of the transformation matrix includes: using a template to push into the template machine slot to obtain the first position coordinates of the first code group in the world space and the second position coordinates of the second code group in the world space; generating the transformation matrix according to the first position coordinates and the second position coordinates; generating a virtual target code group with the same position coordinates as the second code group, and the virtual target code group is set on the template machine and maintains a relative relationship with the first code group in the transformation matrix.

[0007] In an implementation of the first aspect, a second code group for locating the template to be operated is obtained, and the step of instructing a push plate mechanism to press and push the template to be operated according to the position coordinates of the second code group includes: obtaining force data monitored in real time by the push plate mechanism; based on the force data, using admittance control to command the push plate mechanism to press the specified position of the template to be operated with a preset constant force and push the template to be operated.

[0008] In an implementation of the first aspect, the position error includes a position deviation and an angular deviation; the step of adjusting the pushing direction and pushing distance of the template to be operated according to the position error between the real-time position coordinates of the second code group and the position coordinates of the virtual target code group includes: ensuring that the second code group and the virtual target code group are on the same horizontal table, ignoring the error in the z direction, and simplifying it to the translation in the xy direction and the rotation around the z axis; subtracting the position coordinate of the virtual target code group in the x direction from the position coordinate of the second code group in the x direction to determine the x-coordinate position deviation, subtracting the position coordinate of the virtual target code group in the y direction from the position coordinate of the second code group in the y direction to determine the y-coordinate position deviation, subtracting the position coordinate of the virtual target code group rotated around the z axis from the position coordinate of the second code group rotated around the z axis to determine the angular deviation value; determining an error vector according to the x-coordinate position deviation, the y-coordinate position deviation and the angular deviation value; and adjusting the pushing direction and pushing distance of the template to be operated based on the error vector.

[0009] In an implementation of the first aspect, the step of adjusting the pushing direction and pushing distance of the template to be operated based on the error vector includes: determining the distance according to the error vector; dynamically adjusting the pushing speed of the template to be operated with the distance; adopting a preset speed in response to the position deviation being higher than or equal to a deviation threshold; and reducing the speed on the basis of the preset speed to improve accuracy in response to the position deviation being lower than the deviation threshold.

[0010] In an implementation of the first aspect, the step of adjusting the pushing direction and pushing distance of the template to be operated based on the error vector further includes:

[0011] The proximity between the second code group and the virtual target code group is determined based on the error vector; and in response to the proximity satisfying a preset position condition, the pushing direction of the template to be operated is adjusted.

[0012] In an implementation of the first aspect, in response to the position error being within a preset position error range, the step of determining that the template to be operated has been pushed to the target position of the template machine includes: in response to the template to be operated being pushed into the card slot, determining the sum of square errors and the angle errors based on the position coordinates of the second code group and the position coordinates of the virtual target code group; in response to the sum of square errors being less than a first error threshold and the angle error being less than a second error threshold, determining that the template pushing operation is completed.

[0013] In a second aspect, the present application provides a servo device, which includes: a template machine positioning module, configured to obtain a first code group for positioning the template machine, and determine the position coordinates of the template machine in the world space based on the first code group; a target position positioning module, configured to use a preset transformation matrix to determine the position coordinates of a virtual target code group when the template to be operated is pushed into the template machine slot; the template positioning module, configured to obtain a second code group for positioning the template to be operated, and instruct a push plate mechanism to press and push the template to be operated based on the position coordinates of the second code group; a template pushing module, configured to adjust the pushing direction and pushing distance of the template to be operated based on the position error between the real-time position coordinates of the second code group and the position coordinates of the virtual target code group; a pushing plate confirmation module, configured to determine that the template to be operated has been pushed to the target position of the template machine in response to the position error being within a preset position error range.

[0014] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method as described in any one of claims 1 to 7.

[0015] In the fourth aspect, the present application provides a template pushing system, which includes: an identification and positioning module, a pushing plate mechanism and the electronic device; the electronic device instructs the pushing plate mechanism to push the template to the target position based on the position information detected by the identification and positioning module, completes the template pushing operation to start the subsequent sewing operation.

[0016] As described above, the visual servoing method, servo device, equipment, and system based on push-plate operation described in this application have the following beneficial effects:

[0017] The application provides a template pushing method based on visual servoing and force-position hybrid control, which is an independent module of a humanoid robot system, receives the output of a cutting piece placing module, replaces manual work to accurately push a template into a template machine card slot, connects a template machine sewing, promotes full-process automation, and provides reliable support for cutting piece sewing. The method combines real-time visual positioning, intelligent motion planning and force control technology, realizes fast, high-precision and stable template pushing operation, effectively fills the gap of full-process unmanned technology in the garment industry, provides seamless connection for the cutting piece placing module, and promotes the upgrading of intelligent manufacturing.

[0018] The application adopts a transformation relationship generation of a fast template generation and template machine standard relationship, and only needs to put the template into the template slot once to establish a reusable transformation relationship. In addition, the application adopts a PBVS (Position-Based Visual Servoing, position-based visual servoing) visual servoing algorithm of a virtual target code to overcome the problem that the desktop code is easily blocked in pushing the template, and ensures the continuous positioning of the template pushing module. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 An application schematic diagram of the visual servoing method based on the template pushing operation described in the embodiments of the application is shown.

[0020] Figure 2 A principle flowchart of the visual servoing method based on the template pushing operation described in the embodiments of the application is shown.

[0021] Figure 3 A parameter preset flowchart of the visual servoing method based on the template pushing operation described in the embodiments of the application is shown.

[0022] Figure 4 An example process diagram of the parameter preset of the visual servoing method based on the template pushing operation described in the embodiments of the application is shown.

[0023] Figure 5 A template pushing flowchart of the visual servoing method based on the template pushing operation described in the embodiments of the application is shown.

[0024] Figure 6 An example process diagram of the template pushing of the visual servoing method based on the template pushing operation described in the embodiments of the application is shown.

[0025] Figure 7 A structure principle diagram of the servo device described in the embodiments of the application is shown.

[0026] Figure 8 A structure connection schematic diagram of the electronic device described in the embodiments of the application is shown.

[0027] Figure 9Shown is a schematic diagram of the principle of the push-type template system described in an embodiment of the present application.

[0028] Component number description

[0029] 1 Template Machine

[0030] 2 Templates to be operated

[0031] 3 First code group

[0032] 4 Second code group

[0033] 5 Virtual target code group

[0034] 7 Press the designated location

[0035] 8 card slots

[0036] 71 Template machine positioning module

[0037] 72 Target Positioning Module

[0038] 73 Template Positioning Module

[0039] 74 Template Push Module

[0040] 75 Template Confirmation Module

[0041] 81 processors

[0042] 82 Memory

[0043] 83 Communication Interface

[0044] 84 System Bus

[0045] Steps S21 to S25

[0046] Steps S241 to S245

[0047] Steps S31 to S33

[0048] Steps S41 to S46

[0049] Steps S61 to S70 DETAILED DESCRIPTION

[0050] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0051] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0052] The following embodiments of the present application provide visual servoing methods, servo devices, equipment and systems based on template pushing operations, including but not limited to scenarios where a template to be operated needs to be pushed into a card slot of a template machine. This application scenario will be described below as an example.

[0053] See also Figure 1 , which is a schematic diagram showing the application of the visual servoing method based on push template operation described in the embodiment of the present application. Figure 1 As shown, this embodiment provides an application scenario of a visual servoing method based on a template push operation, specifically including: a first code group 3 is provided on the template machine 1, a second code group 4 is provided on the template to be operated 2, the push plate mechanism presses the designated position 7 on the template to be operated, and pushes the template to be operated to slide to the card slot 8 of the template machine 1. When the template to be operated reaches the card slot 8, the second code group 4 on the template to be operated coincides with the virtual target code group 5 on the template machine 1, indicating that the pushing plate action is completed.

[0054] In actual application, a camera installed on the chest of the humanoid robot is used for identification and positioning. The template machine 1 is positioned by four rectangular arrangement code groups 3 attached to the desktop of the template machine 1, and the template 2 is positioned by four rectangular arrangement code groups (i.e., the second code group) 4 attached to the surface of the template 2.

[0055] It should be noted that the code group of the positioning template machine and positioning template in this application is an Apriltag code group with 4 rectangular rows. In addition, the code group can also be replaced with ArUco code, etc., and the number is adjusted to 1-4. It also supports non-rectangular arrangements (such as triangles or circles) and any other code group arrangements of different shapes that can be visually detected for positioning.

[0056] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings in the embodiments of the present application.

[0057] See also Figure 2 , which is a flow chart showing the principle of the visual servoing method based on push template operation described in the embodiment of the present application. Figure 2 As shown, this embodiment provides a visual servoing method based on template push operation, which specifically includes the following steps S21 to S25:

[0058] S21 : Acquire a first code group for locating a template machine, and determine the position coordinates of the template machine in the world space according to the first code group.

[0059] S22, using a preset transformation matrix to determine the position coordinates of the virtual target code group when the template to be operated is pushed into the card slot of the template machine.

[0060] See also Figure 3 , which shows a parameter preset flow chart of the visual servoing method based on push template operation according to an embodiment of the present application. Figure 3 As shown, the preset process of the transformation matrix includes steps S31 to S33:

[0061] S31, using a process of pushing a template into the template machine slot to respectively obtain the first position coordinates of the first code group in the world space and the second position coordinates of the second code group in the world space.

[0062] S32: Generate the transformation matrix according to the first position coordinates and the second position coordinates.

[0063] S33, generating a virtual target code group having the same position coordinates as the second code group, wherein the virtual target code group is set on the template machine and maintains a relative relationship of the transformation matrix with the first code group.

[0064] In other embodiments, during the pushing process, if the first code group on the desktop of the template machine is not blocked by the robot arm, the first code group on the desktop can be used to generate a virtual target code group in real time.

[0065] See also Figure 4 , which shows a process diagram of parameter preset examples of the visual servoing method based on push template operation according to an embodiment of the present application, as shown in FIG. Figure 4 As shown, the entire parameter preset process includes steps S41 to S46:

[0066] S41, hand-eye calibration, in the stage of preset operating parameters, manually push the template 2 into the template machine slot 8.

[0067] S42 , checking the clarity or illumination of the first code group and the second code group so that the camera can clearly recognize the code group images.

[0068] S43, the humanoid robot recognizes the first code group 3 of the template machine desktop and the second code group 4 of the template 2 through the chest camera.

[0069] S44, judging whether the first code group 3 and the second code group 4 are recognized successfully.

[0070] S45: If the recognition is successful, the template machine and template pose are calculated, that is, the position coordinates P1 and P2 of the two in the world space are obtained.

[0071] S46, calculate a transformation matrix T (P2=T·P1), and then store the transformation matrix T. A virtual target code group 5 position P3 (P3=P2) is generated at the template 2 code group 4 position, which is fixed on the template machine 1 and maintains the relative relationship of the transformation matrix T with the code group 3. Each time the operation is performed, when the template 2 code group position P2' is pushed to coincide with the virtual target code group 5 position P3, it can be confirmed that the template 2 has been correctly pushed into the designated position of the template machine 1.

[0072] S23, obtain a second code group for positioning the template to be operated, and instruct the push plate mechanism to press and push the template to be operated according to the position coordinate of the second code group.

[0073] In an embodiment, the step of obtaining a second code group for positioning the template to be operated, and instructing the push plate mechanism to press and push the template to be operated according to the position coordinate of the second code group, comprises:

[0074] (1) Obtain the force data monitored by the push plate mechanism in real time.

[0075] (2) Based on the force data, use the admittance control to instruct the push plate mechanism to press a specified position of the template to be operated with a preset constant force and push the template to be operated, and maintain the preset constant force during the pushing process, so as to ensure that the push plate mechanism and the template to be operated maintain sufficient friction force without sliding, while allowing the template to be operated to slide freely on the template machine.

[0076] S24, adjust the pushing direction and pushing distance of the template to be operated according to the position error between the real-time position coordinate of the second code group and the position coordinate of the virtual target code group.

[0077] Please refer to Figure 5 , which shows the template pushing flowchart of the visual servo method based on the template pushing operation described in the embodiments of the present application. As shown in Figure 5 , S24 specifically comprises steps S241 to S245:

[0078] S241, let the position coordinate of the second code group be (x2, y2, z2, rx2, ry2, rz2), and the position coordinate of the virtual target code group be (x3, y3, z3, rx3, ry3, rz3).

[0079] S242, ensure that the second code group and the virtual target code group are on the same horizontal desktop, ignore the error in the z direction, and simplify it to translation in the xy direction and rotation around the z axis.

[0080] S243, make the position coordinates of the virtual target code group in the x direction different from the position coordinates of the second code group in the x direction, and determine the x coordinate position deviation as Δx=x3-x2, make the position coordinates of the virtual target code group in the y direction different from the position coordinates of the second code group in the y direction, and determine the y coordinate position deviation as Δy=y3-y2, make the position coordinates of the virtual target code group rotated around the z axis different from the position coordinates of the second code group rotated around the z axis, and determine the angle deviation value as Δrz=rz3-rz2.

[0081] S244 : Determine an error vector (Δx, Δy, Δrz) according to the x-coordinate position deviation Δx, the y-coordinate position deviation Δy, and the angle deviation value Δrz.

[0082] S245 , adjusting the pushing direction and pushing distance of the template to be operated based on the error vector.

[0083] In one embodiment, the step of adjusting the pushing direction and pushing distance of the template to be operated based on the error vector includes:

[0084] (1) Determine the distance based on the error vector (Δx, Δy, Δrz)

[0085] (2) The pushing speed of the template to be operated changes with the distance Make dynamic adjustments.

[0086] (3) In response to the position deviation being higher than or equal to the deviation threshold, a preset speed v=v is adopted. max , the preset speed is generally a higher speed; in response to the position deviation being lower than the deviation threshold, the speed is reduced based on the preset speed to improve the accuracy Among them, v represents the real-time pushing speed, v max Indicates the maximum pushing speed, v min Indicates the minimum pushing speed, d thresh Indicates a preset constant, d thresh The larger the The smaller it is, the closer the speed v is to v min .

[0087] In actual application, the distance between the template and the card slot is determined by the distance deviation. If the position deviation is higher than or equal to the deviation threshold, it means that the distance to the card slot is far, and a higher speed can be used; if the position deviation is lower than the deviation threshold, it means that the distance to the card slot is close. In addition to using a higher speed v=v max ; and lower speeds In addition, a constant first speed may be used when the distance is far, and a constant second speed may be used when the distance is close, with the first speed being higher than the second speed. Other curve-changing speed regulation methods may also be used.

[0088] Furthermore, the step of adjusting the pushing direction and pushing distance of the template to be operated based on the error vector also includes: judging the proximity between the second code group and the virtual target code group based on the error vector; and adjusting the pushing direction of the template to be operated in response to the proximity satisfying a preset position condition.

[0089] Specifically, the preset position condition refers to a position value set according to a range near the slot, indicating proximity to or proximity to the slot. Adjusting the pushing direction of the template to be operated includes: in response to the template to be operated sliding on the template machine to the vicinity of the slot, to prevent the slot from getting stuck, first pushing the template to be operated to the front of the slot, then adjusting the angle, and then gradually pushing the template into the slot from the front of the slot.

[0090] S25 , in response to the position error being within a preset position error range, determining that the template to be operated has been pushed to the target position of the template machine.

[0091] In one embodiment, in response to the position error being within a preset position error range, the step of determining that the template to be operated has been pushed to the target position of the template machine includes:

[0092] (1) In response to the template to be operated being pushed into the card slot, the square sum error and the angle error are determined according to the position coordinates of the second code group and the position coordinates of the virtual target code group.

[0093] (2) In response to the square sum error being less than the first error threshold and the angle error being less than the second error threshold, it is determined that the template push operation is completed.

[0094] See also Figure 6 , which shows a push plate example process diagram of the visual servoing method based on the push plate operation described in the embodiment of the present application. Figure 6 As shown, combined Figure 1 The symbols shown indicate that the entire plate pushing process includes steps S61 to S70. In practical applications, the plate pushing mechanism can be a humanoid robot, which includes at least a chest camera as a recognition and positioning module, a robotic arm as a hand, and the electronic device of this application as a control center. This application's visual servoing method based on plate pushing operation serves as an independent module of the humanoid robot, taking over the placement of cut pieces and connecting to the template machine sewing process, promoting full process automation.

[0095] S61, positioning the template machine: the humanoid robot locates the template machine 1 by identifying the template machine code group 3, obtains the position coordinates P1 of the template machine 1 in the world space, and uses the transformation matrix T to calculate the position P3 of the virtual target code group 5 when the template 2 is pushed into the card slot 8, where P3 = T·P1.

[0096] S62, positioning and pressing the template: The robot determines its position P2 by identifying the AprilTag code group 4 of template 2, plans to move its arms above template 2, and uses a six-dimensional force sensor at the end of the manipulator to monitor the force in real time. It uses admittance control to press the specified position 7 of template 2 with a constant force of 5N, so as to maintain constant force during the pushing process, ensure sufficient friction with template 2 without sliding, and allow template 2 to slide freely on the template machine 1.

[0097] S63, Visual Servo Push: During the push process, the robot uses its chest camera to detect the current position of template 2's AprilTag code group 4 (P2 = (x2, y2, z2, rx2, ry2, rz2) in real time. This position is compared with the position of the virtual target code group 5 (P3 = (x3, y3, z3, rx3, ry3, rz3)). The robot calculates the position and angular deviation between the two in the world coordinate system, determining the push direction and distance of template 2. Since the robot is on the same horizontal tabletop, the z-direction error can be ignored and simplified to an xy translation and a rotation around the z-axis. The x-coordinate deviation is Δx = x3 - x2, the y-position deviation is Δy = y3 - y2, and the angular deviation is Δrz = rz3 - rz2. The error vector is simplified to ∈ = (Δx, Δy, Δrx). Based on the positional relationship R between template 2's position P2 and the position of the robot's hands P4 (where P2 = R·P4), the robot calculates the push direction and distance, driving the robot to gradually push template 2 toward the target position.

[0098] S64~S66, pushing speed varies with distance Dynamic adjustment: when the distance is far, use a higher speed v=v max ; When approaching the target, reduce speed to improve accuracy

[0099] S67-S68: To prevent the card slot 8 from getting stuck, first push the template 2 to the position P30 just in front of the card slot 8, and then gradually push it into the card slot 8 from the position P3 after adjusting the angle.

[0100] S69, verification and completion: After the template 2 is pushed into the card slot 8, the code group 4 and the virtual target code group 5 are used to verify whether it has reached the specified position, ensuring that the square sum error of the position P2 of the template 2 and the position P3 of the virtual target code group 5 Less than 0.001 and the angle error Δrx is less than 0.001.

[0101] It should be noted that 0.001 is only one implementation of the first error threshold and the second error threshold, and other reasonable thresholds that can be used to evaluate whether the specified position has been reached are also within the scope of protection of this application.

[0102] S70, after confirmation, the robot arm retracts to the chest, sends a completion instruction to the template machine 1, and starts the sewing operation.

[0103] The purpose of this application is to provide an efficient and accurate template pushing method and system. As an independent module of the humanoid robot system, it receives the output of the cutting piece placement module and automatically pushes the template into the template machine slot to support the automated sewing of the cutting pieces. This method integrates position-based visual servoing (PBVS), force-position hybrid control, and dual-arm coordination technology to achieve rapid positioning, high-precision alignment, and stable pushing, meeting the unmanned needs of the entire process of the clothing industry while ensuring the flexibility and independence of the modular design.

[0104] The protection scope of the visual servoing method based on push template operation described in the embodiment of the present application is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the existing technology based on the principles of the present application are included in the protection scope of the present application.

[0105] An embodiment of the present application also provides a servo device, which can implement the visual servo method based on push template operation described in the present application. However, the implementation device of the visual servo method based on push template operation described in the present application includes but is not limited to the structure of the servo device listed in this embodiment. All structural deformations and replacements of the existing technology made according to the principles of the present application are included in the protection scope of the present application.

[0106] See also Figure 7 , which shows the structural principle diagram of the servo device according to the embodiment of the present application. Figure 7 As shown, this embodiment provides a servo device, which specifically includes: a template machine positioning module, a target position positioning module, a template positioning module, a template pushing module and a push plate confirmation module.

[0107] The template machine positioning module is configured to obtain a first code group for positioning the template machine, and determine the position coordinates of the template machine in the world space according to the first code group.

[0108] The target position positioning module is configured to use a preset transformation matrix to determine the position coordinates of the virtual target code group when the template to be operated is pushed into the template machine slot.

[0109] In one embodiment, the preset process of the transformation matrix includes: using a template to push into the template machine slot to obtain the first position coordinates of the first code group in the world space and the second position coordinates of the second code group in the world space; generating the transformation matrix according to the first position coordinates and the second position coordinates; generating a virtual target code group with the same position coordinates as the second code group, and the virtual target code group is set on the template machine and maintains a relative relationship with the first code group in the transformation matrix.

[0110] The template positioning module is configured to obtain a second code group for positioning the template to be operated, and instruct the push plate mechanism to press and push the template to be operated according to the position coordinates of the second code group.

[0111] In one embodiment, the template positioning module is specifically configured to obtain the force data monitored in real time by the push plate mechanism; based on the force data, the admittance control is used to command the push plate mechanism to press the specified position of the template to be operated with a preset constant force and push the template to be operated, and maintain the preset constant force during the pushing process to ensure that the push plate mechanism and the template to be operated maintain sufficient friction without sliding, while allowing the template to be operated to slide freely on the template machine.

[0112] The template pushing module is configured to adjust the pushing direction and pushing distance of the template to be operated according to the position error between the real-time position coordinates of the second code group and the position coordinates of the virtual target code group.

[0113] In one embodiment, the position error includes position deviation and angular deviation; the template pushing module is specifically configured to make the position coordinates of the second code group (x2, y2, z2, rx2, ry2, rz2) and the position coordinates of the virtual target code group (x3, y3, z3, rx3, ry3, rz3); ensure that the second code group and the virtual target code group are on the same horizontal table, ignore the error in the z direction, and simplify it to the translation in the xy direction and the rotation around the z axis; make the position coordinate of the virtual target code group in the x direction subtract from the position coordinate of the second code group in the x direction to determine the x coordinate position The deviation is Δx=x3-x2, and the position coordinates of the virtual target code group in the y direction are subtracted from the position coordinates of the second code group in the y direction to determine the y coordinate position deviation as Δy=y3-y2. The position coordinates of the virtual target code group rotated around the z axis are subtracted from the position coordinates of the second code group rotated around the z axis to determine the angle deviation value as Δrz=rz3-rz2; the error vector is determined to be (Δx, Δy, Δrz) according to the x-coordinate position deviation Δx, the y-coordinate position deviation Δy and the angle deviation value Δrz; and the pushing direction and pushing distance of the template to be operated are adjusted based on the error vector.

[0114] Furthermore, the template pushing module is further configured to determine the distance according to the error vector (Δx, Δy, Δrz) The pushing speed of the template to be operated varies with the distance Dynamic adjustment is performed; in response to the position deviation being higher than or equal to the deviation threshold, a higher speed v=v is adopted max In response to the position deviation being lower than the deviation threshold, at a higher speed v max On the basis of reducing speed to improve accuracy Among them, v represents the real-time pushing speed, v max Indicates the maximum pushing speed, v min Indicates the minimum pushing speed, d thresh In response to the template to be operated sliding on the template machine to the vicinity of the card slot, in order to prevent the card slot from getting stuck, the template to be operated is first pushed to the front of the card slot, and after the angle is corrected, it is gradually pushed into the card slot from the front position of the card slot.

[0115] The push plate confirmation module is configured to determine that the template to be operated has been pushed to the target position of the template machine in response to the position error being within a preset position error range.

[0116] In one embodiment, the push plate confirmation module is specifically configured to determine the proximity between the second code group and the virtual target code group based on the error vector; and in response to the proximity satisfying a preset position condition, adjust the push direction of the template to be operated. Specifically, in response to the template to be operated being pushed into the slot, a sum of squared errors and an angular error are determined based on the position coordinates of the second code group and the position coordinates of the virtual target code group; and in response to the sum of squared errors being less than a first error threshold and the angular error being less than a second error threshold, the template push operation is determined to be complete.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices or methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0118] The modules / units described as separate components may or may not be physically separate, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into a processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into a single module / unit.

[0119] Those skilled in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0120] The present application provides an electronic device, which includes: a processor and a memory; the memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that the electronic device performs the visual servoing method based on push template operation.

[0121] The visual servoing method based on the template push operation includes: obtaining a first code group for positioning the template machine, and determining the position coordinates of the template machine in the world space according to the first code group; using a preset transformation matrix to determine the position coordinates of the virtual target code group when the template to be operated is pushed into the card slot of the template machine; obtaining a second code group for positioning the template to be operated, and instructing the push plate mechanism to press and push the template to be operated according to the position coordinates of the second code group; adjusting the pushing direction and pushing distance of the template to be operated according to the position error between the real-time position coordinates of the second code group and the position coordinates of the virtual target code group; in response to the position error being within the preset position error range, determining that the template to be operated has been pushed to the target position of the template machine.

[0122] See also Figure 8 , which shows a schematic diagram of the structural connection of the electronic device described in the embodiment of the present application. Figure 8As shown, the electronic device of this embodiment includes: a processor 81, a memory 82, a communication interface 83 and / or a system bus 84. The memory 82 and the communication interface 83 are connected to the processor 81 via the system bus 84 and communicate with each other. The memory 82 is used to store computer programs, the communication interface 83 is used to communicate with other devices, and the processor 81 is used to run the computer program to enable the electronic device to perform each step of the visual servoing method based on the push template operation.

[0123] The above-mentioned processor 81 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0124] The memory 82 may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0125] The system bus 84 mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The system bus 84 can be divided into an address bus, a data bus, a control bus, etc. The communication interface is used to implement communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries).

[0126] See also Figure 9 , which is a schematic diagram showing the principle of the push-type template system described in the embodiment of this application. Figure 9 As shown, this embodiment provides a plate pushing plate system, which specifically includes: an identification and positioning module, a plate pushing mechanism and the above-mentioned electronic equipment.

[0127] The electronic device instructs the plate pushing mechanism to push the template to the target position according to the position information detected by the identification and positioning module, and completes the template pushing operation to start the subsequent sewing operation.

[0128] In one embodiment, the electronic device is configured to execute the computer program stored in the memory, so as to cause the electronic device to perform the visual servoing method based on template pushing operation. The visual servoing method based on template pushing operation includes: obtaining a first code group for positioning a template machine, determining the position coordinates of the template machine in world space based on the first code group; determining the position coordinates of a virtual target code group when the template to be operated is pushed into the template machine slot using a preset transformation matrix; obtaining a second code group for positioning the template to be operated, instructing a push plate mechanism to press and push the template to be operated based on the position coordinates of the second code group; adjusting the pushing direction and pushing distance of the template to be operated based on the position error between the real-time position coordinates of the second code group and the position coordinates of the virtual target code group; and determining that the template to be operated has been pushed to the target position of the template machine in response to the position error being within a preset position error range.

[0129] The descriptions of the processes or structures corresponding to the above figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0130] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A visual servoing method based on template push operation, characterized in that: The method comprises: Acquire a first code group for locating the template machine, and determine the position coordinates of the template machine in the world space according to the first code group; Using a preset transformation matrix to determine the position coordinates of the virtual target code group when the template to be operated is pushed into the template machine slot; Obtaining a second code group for locating the template to be operated, and instructing a push plate mechanism to press and push the template to be operated according to the position coordinates of the second code group; Adjusting the pushing direction and pushing distance of the template to be operated according to the position error between the real-time position coordinates of the second code group and the position coordinates of the virtual target code group; In response to the position error being within a preset position error range, it is determined that the template to be operated has been pushed to the target position of the template machine.

2. The method according to claim 1, characterized in that The preset process of the transformation matrix includes: By pushing a template into the card slot of the template machine, first position coordinates of the first code group in the world space and second position coordinates of the second code group in the world space are respectively obtained; generating the transformation matrix according to the first position coordinates and the second position coordinates; A virtual target code group having the same position coordinates as the second code group is generated. The virtual target code group is set on the template machine and maintains a relative relationship of the transformation matrix with the first code group.

3. The method according to claim 1, characterized in that The step of obtaining a second code group for locating the template to be operated, and instructing a push plate mechanism to press and push the template to be operated according to the position coordinates of the second code group includes: Obtaining the force data of the push plate mechanism monitored in real time; Based on the force data, the push plate mechanism is commanded to press the designated position of the template to be operated with a preset constant force and push the template to be operated using admittance control.

4. The method according to claim 1, wherein The position error includes a position deviation and an angle deviation; and the step of adjusting the pushing direction and pushing distance of the template to be operated according to the position error between the real-time position coordinates of the second code group and the position coordinates of the virtual target code group includes: Ensure that the second code group and the virtual target code group are on the same horizontal table, ignore the error in the z direction, and simplify it to the translation in the xy direction and the rotation around the z axis; Subtracting the x-axis position coordinates of the virtual target code group from the x-axis position coordinates of the second code group to determine an x-coordinate position deviation, subtracting the y-axis position coordinates of the virtual target code group from the y-axis position coordinates of the second code group to determine a y-coordinate position deviation, and subtracting the z-axis position coordinates of the virtual target code group from the z-axis position coordinates of the second code group to determine an angle deviation value; Determine an error vector based on the x-coordinate position deviation, the y-coordinate position deviation, and the angular deviation value; The pushing direction and pushing distance of the template to be operated are adjusted based on the error vector.

5. The method according to claim 4, characterized in that The step of adjusting the pushing direction and pushing distance of the template to be operated based on the error vector includes: determining a distance based on the error vector; The pushing speed of the template to be operated is dynamically adjusted according to the distance; In response to the position deviation being higher than or equal to a deviation threshold, a preset speed is adopted; in response to the position deviation being lower than the deviation threshold, a speed is reduced based on the preset speed.

6. The method according to claim 4, characterized in that The step of adjusting the pushing direction and pushing distance of the template to be operated based on the error vector further includes: determining, based on the error vector, how close the second code group is to the virtual target code group; In response to the proximity degree satisfying a preset position condition, the pushing direction of the template to be operated is adjusted.

7. The method according to claim 1, characterized in that In response to the position error being within a preset position error range, the step of determining that the template to be operated has been pushed to the target position of the template machine includes: In response to the template to be operated being pushed into the card slot, determining a square sum error and an angle error according to the position coordinates of the second code group and the position coordinates of the virtual target code group; In response to the square sum error being less than a first error threshold and the angle error being less than a second error threshold, it is determined that the template push operation is completed.

8. A servo device, characterized in that: The device comprises: a template machine positioning module, configured to obtain a first code group for positioning the template machine, and determine the position coordinates of the template machine in the world space according to the first code group; The target position positioning module is configured to determine the position coordinates of the virtual target code group when the template to be operated is pushed into the template machine slot using a preset transformation matrix; a template positioning module configured to obtain a second code group for locating the template to be operated, and instruct the push plate mechanism to press and push the template to be operated according to the position coordinates of the second code group; a template pushing module, configured to adjust the pushing direction and pushing distance of the template to be operated according to a position error between the real-time position coordinates of the second code group and the position coordinates of the virtual target code group; The push plate confirmation module is configured to determine that the template to be operated has been pushed to the target position of the template machine in response to the position error being within a preset position error range.

9. An electronic device, characterized in that: The electronic device includes: a processor and a memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 7.

10. A push formwork system, characterized in that: The system comprises: an identification and positioning module, a push plate mechanism, and the electronic device according to claim 9; The electronic device instructs the plate pushing mechanism to push the template to the target position according to the position information detected by the identification and positioning module, and completes the template pushing operation to start the subsequent sewing operation.

Images