A vision detection-based production robot cooperative control method and system

Abstract

The application discloses a production robot cooperation control method and system based on visual detection, relates to the field of production robot cooperation control based on visual detection, and is used for improving the precision and reliability of fine mold assembly tasks and comprises the following steps: acquiring first pose data of an end effector of a robot and second pose data of a workpiece based on a visual detection system; the end effector is used for adsorbing the workpiece and making the workpiece enter a positioning groove; according to the first pose data and the second pose data, the relative displacement between the end effector of the robot and the workpiece is calculated; based on the relative displacement, the deformation state of a suction cup on the end effector is quantified; in combination with the deformation state of the suction cup, it is judged whether the current assembly state is a false to-position state caused by the deformation of the suction cup; when it is judged that the false to-position state exists, a preset fine adjustment action is performed to make the workpiece truly enter the positioning groove.

Description

Technical Field

[0001] This invention relates to the field of collaborative control of production robots based on vision detection, and in particular to a collaborative control method and system for production robots based on vision detection. Background Technology

[0002] In the precision mold assembly task of modified plastics production lines, industrial robots, under the precise guidance of a vision inspection system, typically use end-effector vacuum suction cups to pick up precision mold inserts and insert them into the corresponding positioning slots of the template with a micron-level clearance. In existing technologies, when performing this high-precision closed-loop operation, the vision system simultaneously observes the reference features on both the insert and the template, calculating the three-dimensional pose deviation between them in real time. The robot then slowly adjusts its posture accordingly until the deviation vector from the visual feedback is completely zero, thus completing the alignment and insertion.

[0003] However, in the final stage of actual assembly alignment, when the edge of the insert has contacted but not yet fully entered the positioning slot, upward resistance and lateral deflection torque are often generated. At this time, the robot continues to perform vision-guided downward insertion and minor posture adjustment movements, causing the rubber lip of the suction cup connecting the insert to undergo minor compression and shear deformation under contact resistance. During this process, the vision inspection system continuously tracks the insert body or the marked points on it. Because the elastic deformation of the suction cup absorbs part of the displacement of the robot's end effector, the change in insert pose observed by the vision system lags behind and is smaller than the actual displacement of the robot flange.

[0004] When the robot flange has moved into position and stopped, the insert is still pre-pressed above the slot due to the elastic force accumulated by the deformation of the suction cup. However, the vision system at this time judges that the insert has reached the commanded pose and made contact with the template based on the position of the marker point, and thus determines that the closed-loop correction is complete, and the deviation is displayed as zero. Existing vision control methods, because they directly measure the pose of the insert, lack the ability to perceive the dynamic deformation state of the actuator-workpiece mating surface, and are easily deceived by this overshoot illusion. This raises a specific technical problem: in the scenario of precision mold assembly, when the suction cup undergoes elastic deformation when the insert is obstructed in the slot, because the vision inspection system directly measures the pose of the insert and does not perceive the deformation state of the actuator-workpiece mating surface, the system misjudges the false positioning under the suction cup deformation as true positioning, causing the insert to rebound slightly or deflect after the robot releases the vacuum, resulting in assembly clearance exceeding tolerance or hidden failure. Summary of the Invention

[0005] This application discloses a collaborative control method and system for production robots based on vision inspection, which aims to solve the problem in precision mold assembly scenarios where, when a suction cup is obstructed during insert insertion, it undergoes elastic deformation. Because the vision inspection system directly measures the insert's pose without sensing the deformation state of the connection surface between the actuator and the workpiece, the system misjudges the false positioning of the suction cup under deformation as true positioning. This causes the insert to experience slight rebound or deflection after the robot releases the vacuum, leading to assembly gap deviations or hidden failures.

[0006] In a first aspect, this application discloses a collaborative control method for production robots based on vision detection, comprising the following steps: The robot's end effector first pose data and the workpiece's second pose data are acquired using a vision inspection system; the end effector is used to attract the workpiece and move it into the positioning slot. Based on the first pose data and the second pose data, calculate the relative displacement between the robot's end effector and the workpiece; Based on relative displacement, the deformation state of the suction cup on the end effector is quantified; Based on the suction cup deformation state, determine whether the current assembly state is a false positioning state caused by suction cup deformation; When the position is determined to be false, a preset fine-tuning action is performed to make the workpiece actually enter the positioning groove.

[0007] Optionally, based on the first pose data and the second pose data, the relative displacement between the robot's end effector and the workpiece is calculated, including: Determine if the end effector has entered a blocked state; After entering the obstructed state, the first displacement increment of the end effector is determined based on the first pose data; Based on the second pose data, determine the second displacement increment on the workpiece; The relative displacement is determined based on the difference between the first displacement increment and the second displacement increment.

[0008] Optionally, determining whether the end effector has entered a blocked state includes: If the deviation between the end effector and the workpiece on the z-axis is within a first preset range, and the deviation between them on the planes containing the X and Y axes is within a second preset range, then the end effector is determined to enter a blocked state.

[0009] Optionally, based on relative displacement, the deformation state of the suction cup on the end effector is quantified, including: Obtain the stiffness coefficient matrix of the suction cup on the end effector in each degree of freedom; Multiplying the relative displacement by the stiffness coefficient matrix yields the current accumulated elastic potential energy of the suction cup and the deformation in different directions, thus quantifying the deformation state of the suction cup on the end effector.

[0010] Optionally, determine whether the current assembly state is a false positioning state caused by suction cup deformation, including: Based on the deformation state of the suction cup, determine whether the elastic potential energy is greater than the preset potential energy threshold. If the elastic potential energy is greater than the preset potential energy threshold and the relative displacement is greater than the preset displacement threshold, then the current assembly state is determined to be a false positioning state caused by the deformation of the suction cup.

[0011] Optionally, perform preset fine-tuning actions to ensure the workpiece actually enters the positioning groove, including: Perform high-frequency jittering in the plane containing the X and Y axes; If the end effector experiences a sudden displacement change within a preset range in the Z-axis direction during high-frequency jittering, it is confirmed that the workpiece has actually entered the positioning groove, and the high-frequency jittering stops.

[0012] Optionally, the method also includes: If the execution duration of the high-frequency jitter exceeds the preset duration, the end effector will be retracted along the Z-axis, and a manual operation request will be sent to the user terminal. Receive manual operation commands from user terminals; Based on manual operation instructions, the workpiece is actually put into the positioning groove.

[0013] Optionally, the method also includes: After confirming that the workpiece has actually entered the positioning slot, a physical stability verification operation is performed; the physical stability verification operation is used to ensure that the assembly state of the workpiece does not depend on the pressure applied by the end effector. Based on the results of the physical stability verification operation, the end effector is controlled to release the workpiece.

[0014] Optionally, perform physical stability verification operations, including: Obtain material property information for the workpiece and the positioning groove; Based on the material properties of the workpiece and the positioning groove, obtain the contact friction coefficient between the suction cup and the workpiece, the contact friction coefficient between the workpiece and the positioning groove, and the equivalent elastic modulus of the rubber lip of the suction cup. Based on the contact friction coefficient between the suction cup and the workpiece, the contact friction coefficient between the workpiece and the positioning groove, and the equivalent elastic modulus of the rubber lip of the suction cup, the verification parameters for the physical stability verification operation are adjusted; the verification parameters for the physical stability verification operation include the displacement of the reverse tensile test and the upper limit of the test force. Perform physical stability verification based on the adjusted verification parameters.

[0015] Secondly, this application also discloses a collaborative control system for production robots based on vision detection, the system comprising: The pose data acquisition module is used to acquire the first pose data of the robot's end effector and the second pose data of the workpiece based on the vision inspection system; the end effector is used to attract the workpiece and make the workpiece enter the positioning groove. The displacement transmission difference recognition module is used to calculate the relative displacement between the robot's end effector and the workpiece based on the first pose data and the second pose data. The quantization module is used to quantify the deformation state of the suction cup on the end effector based on relative displacement. The false positioning detection module is used to determine whether the current assembly state is a false positioning state caused by the deformation of the suction cup, based on the suction cup deformation state. The fine-tuning action execution module is used to execute preset fine-tuning actions to make the workpiece actually enter the positioning groove when the position is determined to be false.

[0016] Beneficial Effects: This application discloses a collaborative control method for production robots based on vision inspection. By introducing the perception and quantification of the suction cup deformation state, it effectively solves the problem in existing technologies where vision systems cannot perceive the dynamic deformation state of the connection surface between the actuator and the workpiece, leading to misjudging false positioning under suction cup deformation as true positioning. Specifically, this application obtains the first pose data of the robot's end effector and the second pose data of the workpiece using a vision inspection system, and calculates the relative displacement between the end effector and the workpiece based on these data. Furthermore, this application quantifies the suction cup deformation state on the end effector based on the relative displacement, and determines whether the current assembly state is a false positioning state caused by suction cup deformation. When a false positioning state is determined, the system executes a preset fine-tuning action to ensure the workpiece truly enters the positioning groove. Through this series of technical solutions, this application can accurately identify and correct assembly errors caused by the elastic deformation of the suction cup, avoiding assembly gap deviations or hidden failures due to misjudgment, and significantly improving the accuracy and reliability of precision mold assembly tasks. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a collaborative control method for production robots based on vision detection provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of another collaborative control method for production robots based on vision detection provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a collaborative control system for production robots based on vision detection, provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0019] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0020] To better understand the technical solution proposed in this application, some key terms involved will be explained first.

[0021] In this application, "robot" generally refers to an industrial robot whose end effector is equipped with a suction cup for adsorbing workpieces. During assembly, the suction cup grasps the workpiece using vacuum adsorption and guides it to the target positioning slot. The vision inspection system is a high-precision optical measurement system capable of capturing and analyzing the spatial position and orientation information of the robot's end effector and the workpiece in real time. The "false positioning state" that this application focuses on refers to a situation during assembly where, due to the elastic deformation of the suction cup, the vision system mistakenly determines that the workpiece has reached the target position, but in reality, the workpiece has not fully entered the positioning slot.

[0022] The following specific embodiments will provide a detailed description and explanation of a visual inspection-based collaborative control method for production robots provided in this application.

[0023] Reference Figure 1 This invention provides a collaborative control method for production robots based on vision detection, comprising the following steps: S1, based on the vision inspection system, acquires the first pose data of the robot's end effector and the second pose data of the workpiece.

[0024] The end effector is used to pick up the workpiece and move it into the positioning groove.

[0025] Pose data typically includes position information (e.g., X, Y, Z coordinates) and orientation information (e.g., rotation angles around the X, Y, Z axes). Visual inspection systems can accurately calculate this pose data by setting specific visual markers on the end effector and workpiece, and by using multiple cameras for image acquisition and processing. For example, high-precision 3D pose information can be obtained through stereo vision systems or structured light scanning systems.

[0026] S2. Based on the first pose data and the second pose data, calculate the relative displacement between the robot's end effector and the workpiece.

[0027] Relative displacement can be calculated in several ways. For example, the pose data of the end effector and the workpiece can be directly compared to obtain their relative position and orientation deviations in each degree of freedom. Another approach is to continuously monitor the pose changes of the end effector and the workpiece during robot movement; when an unexpected displacement difference occurs between them, the relative displacement can be determined.

[0028] S3. Based on relative displacement, quantify the deformation state of the suction cup on the end effector.

[0029] For example, a suction cup can be modeled as an elastic body with a certain stiffness coefficient. By multiplying the relative displacement by the stiffness coefficient matrix of the suction cup, the elastic potential energy currently stored in the suction cup and the deformation in different directions can be estimated. Another approach is to integrate micro-sensors, such as strain gauges or pressure sensors, inside or on the surface of the suction cup to directly measure the deformation or internal pressure changes of the suction cup under stress, thereby quantifying its deformation state.

[0030] S4. Based on the suction cup deformation state, determine whether the current assembly state is a false positioning state caused by suction cup deformation.

[0031] For example, an elastic potential energy threshold and a relative displacement threshold can be set. When the elastic potential energy accumulated by the suction cup is greater than the preset potential energy threshold, and the relative displacement between the end effector and the workpiece is greater than the preset displacement threshold, the current assembly state can be determined to be a false positioning state. These thresholds can be obtained through experimental calibration or simulation analysis to adapt to the requirements of different suction cup materials, workpiece weights, and assembly clearances.

[0032] S5. When the position is determined to be false, perform a preset fine-tuning action to make the workpiece actually enter the positioning groove.

[0033] Specifically, high-frequency jittering can be performed in the plane containing the X and Y axes. During the high-frequency jittering process, if the end effector experiences a sudden displacement change within a preset range in the Z-axis direction, it is determined that the workpiece has actually entered the positioning groove, and the high-frequency jittering stops.

[0034] High-frequency jittering in the plane containing the X and Y axes refers to controlling the robot's end effector to perform reciprocating motion in the horizontal direction (i.e., the plane formed by the X and Y axes) with a smaller than preset amplitude and a higher than preset frequency. This jittering aims to help the workpiece overcome friction or slight jamming with the positioning groove through mechanical vibration, thereby facilitating the smooth sliding of the workpiece into the positioning groove.

[0035] The frequency and amplitude of high-frequency vibration can be preset and adjusted according to the size and material of the workpiece, the geometry of the positioning groove, and the assembly accuracy requirements.

[0036] During high-frequency vibration, if the end effector experiences a sudden displacement change within a preset range in the Z-axis direction, this can be understood as a significant and rapid displacement change in the vertical direction (Z-axis) of the end effector when the workpiece successfully overcomes obstacles and falls into the positioning slot under vibration. This displacement change occurs due to gravity or the sudden release of suction cup pressure. The magnitude of this displacement change should be within a preset range to distinguish between normal mechanical vibration and the actual positioning of the workpiece. For example, the occurrence of a displacement change can be determined by monitoring the rate of change or displacement of the Z-axis position sensor data of the end effector. In practical applications, once such a displacement change in the Z-axis direction is detected, it indicates that the workpiece has actually entered the positioning slot. At this point, the system will immediately determine that the workpiece has actually entered the slot and stop the ongoing high-frequency vibration to avoid unnecessary energy consumption and excessive manipulation of the workpiece.

[0037] In some preferred embodiments, suppose a robot is suctioning a rectangular workpiece and attempting to insert it into a rectangular positioning slot slightly larger than the workpiece. Due to the elastic deformation of the suction cup, the vision inspection system may determine that the workpiece is "in place," but in reality, the edge of the workpiece may only be stuck at the entrance of the positioning slot and not fully inserted. In this case, the system will judge it as a false positioning state. To solve this problem, the robot control system instructs the end effector to perform a high-frequency jitter with a frequency of 50Hz and an amplitude of 0.5mm in the plane containing the X and Y axes. During the jitter, if the workpiece successfully overcomes the jamming and falls into the positioning slot, the end effector will suddenly move downward about 1mm in the Z-axis direction. Once this 1mm displacement abrupt change is detected, the system immediately determines that the workpiece has truly entered the positioning slot and stops jittering. This method ensures that even if the workpiece is misjudged due to suction cup deformation, the workpiece can still achieve accurate and true assembly through physical fine-tuning.

[0038] The collaborative control method for production robots based on vision inspection proposed in this application works by introducing the perception and quantification of the deformation state of the robot's end effector suction cup. This solves the technical problem of false positioning misjudgment caused by the elastic deformation of the suction cup during fine assembly in traditional vision inspection systems. Specifically, when the robot performs workpiece assembly tasks, the vision inspection system continuously acquires real-time pose data of the end effector and the workpiece. This data is not only used to guide the robot for initial alignment, but more importantly, by calculating the relative displacement between the end effector and the workpiece, this method can identify the elastic deformation of the suction cup under obstructed conditions. Once a significant relative displacement is detected, combined with the physical characteristics of the suction cup (such as stiffness coefficient), the system can quantify the deformation state of the suction cup, such as calculating its accumulated elastic potential energy. When both the elastic potential energy and the relative displacement exceed a preset threshold, the system can accurately determine that the current assembly state is false positioning, that is, the workpiece is not actually inserted into the slot, but is being "pushed" above the slot opening by the elastic force of the suction cup. At this point, the system no longer blindly trusts the "zero deviation" of visual feedback, but actively triggers preset fine-tuning actions, such as high-frequency vibration in the XY plane, to help the workpiece overcome resistance and actually slide into the positioning groove. During the fine-tuning process, by monitoring the sudden changes in displacement in the Z-axis direction, it can confirm in real time whether the workpiece has successfully entered the groove, thus stopping the fine-tuning action in time. The entire process forms a closed-loop control, ensuring the accurate and reliable assembly of the workpiece and avoiding assembly failures or quality problems caused by false positioning.

[0039] Compared to traditional robot control methods that rely solely on visual feedback for pose correction, this application offers significant advantages and innovations. Traditional methods, when workpiece entry is obstructed or the suction cup deforms, fail to detect the dynamic deformation of the suction cup-workpiece connection surface because the vision system directly measures the workpiece's pose. This leads to misjudging false positioning caused by suction cup deformation as true positioning. Such misjudgment results in the workpiece potentially rebounding or deflecting due to the suction cup's elasticity after release, causing assembly clearance errors or hidden failures. The core innovation of this application lies in introducing the calculation of the relative displacement between the robot's end effector and the workpiece, and quantifying the suction cup's deformation state accordingly. This allows for accurate identification of false positioning caused by suction cup deformation. For example, when both the elastic potential energy and relative displacement caused by suction cup deformation exceed preset thresholds, the system can clearly determine that the current positioning is not true. Furthermore, this application proposes executing preset fine-tuning actions when a false positioning is detected, prompting the workpiece to actually enter the groove. This method not only improves the accuracy and reliability of assembly, but also avoids decreased production efficiency and product quality problems caused by misjudgment. By sensing and actively intervening in the deformation state of the suction cup, this application effectively solves the "overshooting illusion" problem existing in the prior art, and provides a more robust and intelligent control strategy for high-precision robotic collaborative tasks such as precision mold assembly.

[0040] like Figure 2 As shown, based on the first pose data and the second pose data, the relative displacement between the robot's end effector and the workpiece is calculated, including: S101. Determine whether the end effector has entered a blocked state.

[0041] For example, when the end effector's trajectory or speed changes significantly as it approaches the workpiece, or when the robot's joint torque fluctuates abnormally, it can be considered that the end effector has entered a blocked state.

[0042] Specifically, when it is determined that the deviation between the end effector and the workpiece on the z-axis is within a first preset range, and the deviation in the planes containing the X-axis and Y-axis is within a second preset range, the end effector is determined to enter a blocked state.

[0043] The deviation between the end effector and the workpiece along the z-axis refers to the difference between the actual position of the workpiece perpendicular to the assembly plane (usually the Z-axis direction) and the Z-axis position of the target positioning groove after the end effector has adsorbed the workpiece during assembly. The setting of the first preset range is intended to allow for a certain alignment error in the Z-axis direction, while ensuring that the workpiece has initially contacted or approached the positioning groove, providing a basis for subsequent obstruction judgment. For example, this first preset range can be empirically set or determined experimentally based on the workpiece size, the depth of the positioning groove, and the elastic deformation characteristics of the suction cup.

[0044] Furthermore, the deviation between the end effector and the workpiece in the X and Y planes refers to the difference between the center position of the workpiece and the center position of the positioning groove in the horizontal direction (i.e., in the plane perpendicular to the Z-axis). The second preset range is set to limit the degree of alignment of the workpiece in the horizontal direction. When the workpiece is nearly in place in the Z-axis direction, and the deviation in the XY plane is also within an acceptable small range, it can be considered that the end effector and the workpiece it adsorbs have already contacted or are about to contact the positioning groove, thus potentially entering an obstructed state. This second preset range is typically smaller than the first preset range to ensure alignment accuracy in the horizontal direction.

[0045] The above technical solution effectively avoids subsequent calculation errors caused by inaccurate obstruction state judgment, improving the accuracy of identifying the actual contact state between the workpiece and the positioning groove during robot assembly. This precise obstruction state judgment provides a solid foundation for subsequent relative displacement calculation, suction cup deformation state quantification, and false positioning state identification, thereby enhancing the robustness of the entire collaborative control method and the assembly success rate.

[0046] S102. After entering the obstructed state, determine the first displacement increment of the end effector based on the first posture data.

[0047] The first displacement increment refers to the displacement change of the end effector within a certain time period under obstructed conditions.

[0048] S103. Based on the second pose data, determine the second displacement increment of the workpiece.

[0049] The second displacement increment refers to the change in displacement of the workpiece under obstructed conditions.

[0050] S104. Determine the relative displacement based on the difference between the first displacement increment and the second displacement increment.

[0051] This application's solution introduces a "resistance state" judgment and calculates the difference in displacement increments between the end effector and the workpiece under this state to determine the relative displacement. This effectively solves the problem of traditional methods struggling to accurately calculate relative displacement after the end effector contacts the workpiece. When the end effector contacts the workpiece and enters a restricted state, the suction cup may begin to deform. At this time, relying solely on absolute position difference may fail to capture the minute relative movements caused by the suction cup deformation. By calculating the displacement increment difference, this solution can more sensitively detect the subtle relative displacements generated between the end effector and the workpiece during the suction cup's deformation under force, thus providing more accurate input data for subsequent quantification of the suction cup's deformation state. It is precisely this incremental calculation method that significantly improves the measurement accuracy of relative displacement under complex contact conditions.

[0052] In some preferred embodiments, a specific example is given below. Assume a robot's end effector is adsorbing a workpiece and attempting to insert it into a positioning slot. As the end effector moves downwards, and the suction cup makes slight contact with the edge of the positioning slot, or the workpiece makes slight contact with the edge of the positioning slot, a vision inspection system continuously acquires the first pose data of the end effector and the second pose data of the workpiece. The system first determines whether the end effector has entered an obstructed state, for example, by monitoring whether the robot joint torque exceeds a preset threshold, or by detecting a sudden decrease in the rate of change of distance between the end effector and the workpiece through the vision system. Once an obstructed state is determined, the system begins recording that from that moment, the end effector moves downwards by 0.5 mm (first displacement increment) in the following 0.1 seconds, while the workpiece moves downwards by only 0.2 mm (second displacement increment) in the same time period. At this point, based on the difference between the first and second displacement increments, i.e., 0.5 mm - 0.2 mm = 0.3 mm, the actual relative displacement of the end effector relative to the workpiece is determined to be 0.3 mm. This 0.3 mm relative displacement is then used to quantify the deformation state of the suction cup, thereby more accurately determining whether there is false positioning.

[0053] Specifically, the above-mentioned quantification of the suction cup deformation state on the end effector based on relative displacement includes: S201. Obtain the stiffness coefficient matrix of the suction cup on the end effector in each degree of freedom.

[0054] The stiffness coefficient matrix describes the relationship between the deformation of the suction cup in each degree of freedom (e.g., translation in the X, Y, and Z directions and rotation about the X, Y, and Z axes) and the applied force or torque when subjected to external force or displacement. This matrix is ​​usually predetermined through experimental testing or finite element analysis and stored in the control system.

[0055] S202. Multiply the relative displacement by the stiffness coefficient matrix to obtain the current accumulated elastic potential energy of the suction cup and the deformation in different directions, so as to quantify the deformation state of the suction cup on the end effector.

[0056] Relative displacement refers to the change in relative position and attitude between the end effector and the workpiece. It can be a six-degree-of-freedom vector, containing three translational components and three rotational components. Elastic potential energy refers to the energy stored in the suction cup due to deformation; its magnitude is proportional to the stiffness of the suction cup and the square of the deformation. Deformation refers to the actual displacement or rotation of the suction cup in each degree of freedom. By multiplying the relative displacement by the stiffness coefficient matrix, the forces and torques acting on the suction cup in its current state can be calculated, and thus the stored elastic potential energy and the specific deformation in different directions can be derived.

[0057] The solution proposed in this application obtains the stiffness coefficient matrix of the suction cup and multiplies it with the relative displacement between the end effector and the workpiece. This transforms the abstract relative displacement into concrete physical quantities, namely the elastic potential energy and deformation of the suction cup. This transformation allows for precise quantification of the suction cup's deformation state, providing a solid physical basis for subsequent judgments regarding whether a false positioning state has occurred. In this way, it is possible to more accurately assess whether the suction cup has experienced excessive stress or abnormal deformation during assembly, thereby avoiding misjudgments caused by suction cup deformation.

[0058] This application further proposes a method for determining whether the current assembly state is a false positioning state caused by suction cup deformation, including: S301. Based on the deformation state of the suction cup, determine whether the elastic potential energy is greater than the preset potential energy threshold.

[0059] Specifically, elastic potential energy refers to the energy stored in a suction cup during deformation. Its magnitude reflects the degree of deformation and the suction cup's tendency to return to its original shape. The preset potential energy threshold is an energy value pre-set based on the suction cup's material properties, structural design, and actual assembly experience. It is used to distinguish between normal deformation and excessive deformation that may lead to false positioning.

[0060] S302. If the elastic potential energy is greater than the preset potential energy threshold and the relative displacement is greater than the preset displacement threshold, then the current assembly state is determined to be a false positioning state caused by the deformation of the suction cup.

[0061] When the elastic potential energy of the suction cup exceeds the preset potential energy threshold, it indicates that the suction cup has been subjected to greater pressure or has undergone significant deformation.

[0062] Relative displacement can be understood as the displacement difference between the robot's end effector and the workpiece, directly reflecting the actual fit between the workpiece and the positioning groove. The preset displacement threshold is a displacement value set based on factors such as the fit tolerance between the workpiece and the positioning groove, and assembly accuracy requirements. When the relative displacement exceeds this threshold, it indicates that the workpiece may not have fully entered the positioning groove, but is instead "supported" or "stuck" by the deformation of the suction cup.

[0063] In practical applications, the above judgment logic requires two conditions to be met simultaneously: the elastic potential energy must be greater than a preset potential energy threshold, and the relative displacement must be greater than a preset displacement threshold. These two conditions complement each other and together ensure the accuracy of the judgment.

[0064] In some preferred embodiments, suppose that during an assembly task, the robot's end effector attracts a workpiece and attempts to insert it into a positioning slot. The vision inspection system continuously acquires the pose data of the end effector and the workpiece and calculates the relative displacement. Simultaneously, based on the stiffness coefficient matrix of the suction cup, the elastic potential energy of the suction cup is quantified. Specifically, if the preset potential energy threshold is set to E_th and the preset displacement threshold is set to D_th, during the assembly process, the system detects that the elastic potential energy E_current of the suction cup is 1.5 * E_th, and the relative displacement D_current between the end effector and the workpiece is 1.2 * D_th. Since E_current is greater than E_th and D_current is greater than D_th, both conditions are met, and the system determines that the current assembly state is a false positioning state caused by suction cup deformation. At this time, the system will trigger subsequent fine-tuning actions to ensure that the workpiece truly enters the positioning slot. Conversely, if the elastic potential energy E_current is 0.8 * E_th, even if the relative displacement D_current is 1.2 * D_th, the system will not identify it as a false contact because the elastic potential energy has not exceeded the threshold, and it may only be a slight normal contact. Alternatively, if the elastic potential energy E_current is 1.5 * E_th, but the relative displacement D_current is 0.5 * D_th, the system will not identify it as a false contact because the relative displacement is small, and it may only be the deformation of the suction cup within its normal operating range. This dual judgment mechanism ensures that a false contact is only identified when the suction cup deformation reaches a certain level and the workpiece is indeed not in place, thereby avoiding unnecessary fine-tuning operations and improving assembly efficiency.

[0065] This application's solution, by simultaneously considering the elastic potential energy of the suction cup and the relative displacement between the end effector and the workpiece, can more comprehensively and accurately identify false positioning states. The consideration of elastic potential energy reflects the internal mechanical state generated by the suction cup deformation, i.e., whether the suction cup has undergone significant deformation due to excessive force; while the consideration of relative displacement directly reflects the geometric fit between the workpiece and the positioning groove, i.e., whether the workpiece is actually in the correct position. Since false positioning states caused by suction cup deformation usually manifest as the suction cup bearing excessive pressure (high elastic potential energy) and the workpiece not actually entering the positioning groove (large relative displacement), combining these two indicators for judgment can effectively avoid misjudgments that may arise from judging by a single indicator, thereby improving the robustness of false positioning state identification.

[0066] In some embodiments described above, when a false positioning state is detected, high-frequency jitter is performed in the plane containing the X and Y axes to ensure the workpiece actually enters the positioning slot. However, in actual assembly processes, due to various complex factors such as the fit tolerance between the workpiece and the positioning slot, friction, robot control accuracy, or environmental factors, high-frequency jitter may not effectively guide the workpiece into the positioning slot within a preset time. This results in the robot performing ineffective operations for an extended period, reducing production efficiency and potentially causing assembly failure.

[0067] In this regard, this application further proposes that the above-mentioned method also includes: S401. If the execution duration of high-frequency jitter exceeds the preset duration, the end effector will be retracted along the Z-axis, and a manual operation request will be sent to the user terminal.

[0068] Specifically, the "preset duration" refers to the maximum allowable execution time set for high-frequency jitter operations. This duration can be empirically set or determined through experimental optimization based on factors such as the specific assembly task, workpiece material, positioning slot design, and robot performance. Its purpose is to prevent the robot from getting bogged down in ineffective jitter attempts for extended periods. When the execution time of high-frequency jitter exceeds this preset duration, it indicates that automatic fine-tuning may not be able to effectively solve the false positioning problem. "Retracting the end effector along the Z-axis" means that the robot control system instructs the end effector to move upwards a distance perpendicular to the assembly plane to disengage the workpiece from the positioning slot or reduce contact pressure, thus creating conditions for subsequent manual intervention. In practical applications, the "user terminal" can be understood as any human-machine interface device connected to the robot control system, such as an operation panel, tablet, computer, or smartphone. "Sending a manual operation request" means that the system sends an alarm or prompt to the operator through the user terminal, informing them that the current assembly task has encountered difficulties and requires manual intervention.

[0069] S402, Receive manual operation commands from the user terminal.

[0070] "Receiving manual operation instructions from the user terminal" refers to the operator inputting corresponding instructions through the user terminal after receiving a request, such as adjusting the workpiece posture, cleaning the positioning slot, replacing the workpiece, or directly performing manual assisted assembly.

[0071] S403. Based on manual operation instructions, the workpiece is actually put into the positioning groove.

[0072] This step refers to the robot control system executing corresponding actions based on the received human instructions to ultimately ensure that the workpiece can accurately and stably enter the positioning slot.

[0073] This application's solution effectively addresses the problem that the aforementioned fine-tuning actions might fail to successfully guide the workpiece into its correct position within a limited time by introducing a retraction and manual intervention mechanism after high-frequency jitter failure. When the high-frequency jitter operation lasts longer than the preset duration, the system determines that automatic fine-tuning has stalled. At this point, by retracting the end effector along the Z-axis, the jamming or excessive pressure between the workpiece and the positioning slot can be relieved, preventing damage to the workpiece or equipment. Simultaneously, sending a manual operation request to the user terminal allows for timely notification of operator intervention, leveraging human judgment and flexibility to handle complex situations that robots cannot autonomously manage. By receiving and executing manual operation commands, assembly tasks can be effectively advanced even under limitations of automation, thus avoiding prolonged production line downtime and improving overall robustness and production efficiency.

[0074] Traditional collaborative control methods for production robots typically release the end effector's grip on the workpiece directly after fine-tuning the workpiece into the positioning slot. However, this approach has potential limitations. Specifically, even if the workpiece appears to be properly positioned under the end effector's grip, factors such as fit tolerances between the workpiece and the positioning slot, insufficient friction, or the workpiece's own structural characteristics can cause the workpiece to fail to remain stably within the slot once the end effector releases its gripping pressure. This can lead to assembly failure or problems in subsequent processes. Failure to address these issues can result in hidden defects on the production line, increasing rework costs and reducing overall production efficiency. To address this, this application proposes a scheme that, after confirming the workpiece's proper entry into the positioning slot, further ensures assembly quality through a physical stability verification operation.

[0075] In some embodiments of this application described above, after confirming that the workpiece has actually entered the positioning groove, the method further includes: S501. After confirming that the workpiece has actually entered the positioning groove, perform a physical stability verification operation.

[0076] Specifically, physical stability verification refers to evaluating the workpiece's fixation and stability within the positioning groove through a series of controlled physical tests after the workpiece is believed to have actually entered the groove. Its purpose is to simulate the real-world situation after the end effector releases its suction pressure, verifying whether the workpiece can remain stable solely due to its interaction with the positioning groove. This operation aims to ensure that the workpiece's assembly state is robust, independent of any external pressure or suction force applied by the end effector during assembly. For example, physical stability verification may include applying a test force of a preset direction and magnitude to the workpiece and monitoring its displacement or force feedback to determine whether it remains within the positioning groove.

[0077] S502. Based on the results of the physical stability verification operation, control the end effector to release the workpiece.

[0078] The process of controlling the end effector to release the workpiece based on the results of the physical stability verification operation can be understood as follows: if the physical stability verification results show that the workpiece is stable within the positioning slot, i.e., the verification has passed, then the end effector can safely release its grip on the workpiece to complete the assembly. Conversely, if the verification fails, it indicates that the assembly state of the workpiece may be unstable. In this case, the end effector should not immediately release the workpiece, but should trigger an exception handling mechanism, such as re-executing the fine-tuning action, triggering an alarm, or requesting manual intervention, to avoid the unstable workpiece being mistakenly judged as successfully assembled.

[0079] This application's solution effectively solves the problem of workpiece instability and detachment after the release of the end effector, which may occur in the aforementioned methods, by introducing a physical stability verification operation after the workpiece has actually entered the positioning slot. Specifically, after confirming that the workpiece has truly entered the positioning slot, the end effector is not released immediately; instead, a physical stability verification operation is performed first. This operation assesses the actual fit between the workpiece and the positioning slot by actively applying external disturbances or detecting the inherent stability of the workpiece. If the workpiece can withstand the verification operation, it indicates that sufficient mechanical locking or friction has been formed between it and the positioning slot, enough to remain stable without the support of the end effector. Therefore, the decision to release the workpiece is based on the verification result, ensuring that only truly stable assembled workpieces are considered successfully assembled, thereby avoiding subsequent production problems caused by false stability.

[0080] In some preferred embodiments, it is assumed that a robot is inserting a small electronic component (workpiece) into a slot (positioning slot) on a circuit board. After visual inspection and fine-tuning, the system determines that the electronic component has indeed entered the slot. At this point, to ensure the physical stability of the assembly, the robot does not immediately release the suction cup. Instead, it performs a physical stability verification operation. For example, the end effector can maintain the suction state, but simultaneously apply a slight, predetermined direction (e.g., perpendicular to the slot plane) pulling or pushing force to the electronic component through a slight movement of the robot body, and use a visual inspection system or force sensor to monitor whether the electronic component has experienced displacement beyond the allowable range. If the electronic component remains in the slot after the test force is applied, and the displacement is within acceptable tolerances, the physical stability verification is considered to have passed. At this point, the system controls the end effector to safely release the electronic component. Conversely, if the electronic component undergoes significant displacement or detaches from the slot under the test force, the verification is considered to have failed, and the system will trigger an exception handling procedure, such as retrying the assembly or issuing an alarm.

[0081] Specifically, the above-mentioned physical stability verification operation includes: S601. Obtain the material property information of the workpiece and the positioning groove.

[0082] Specifically, obtaining material property information for the workpiece and positioning groove refers to acquiring physical characteristic data related to the workpiece and positioning groove, such as material type, hardness, surface roughness, and elastic modulus, through methods such as pre-set database queries, sensor detection, or manual input. This information forms the basis for subsequent calculations and adjustments to verification parameters.

[0083] S602. Based on the material properties of the workpiece and the positioning groove, obtain the contact friction coefficient between the suction cup and the workpiece, the contact friction coefficient between the workpiece and the positioning groove, and the equivalent elastic modulus of the rubber lip of the suction cup.

[0084] This step can be understood as using known material properties, through methods such as looking up tables, calculating with empirical formulas, or finite element analysis, to determine the frictional characteristics between the suction cup and the workpiece, and between the workpiece and the positioning groove, as well as the deformation characteristics of the suction cup's rubber lip under stress, under specific contact conditions. These parameters directly affect the stability of the workpiece within the positioning groove and the behavior of the suction cup during the verification process.

[0085] S603. Based on the contact friction coefficient between the suction cup and the workpiece, the contact friction coefficient between the workpiece and the positioning groove, and the equivalent elastic modulus of the rubber lip of the suction cup, adjust the verification parameters for the physical stability verification operation.

[0086] The verification parameters for physical stability verification include the displacement and upper limit of the test force during reverse tensile testing.

[0087] The aim is to make the verification process more targeted and effective. For example, if the coefficient of friction between the workpiece and the positioning groove is low, the displacement or upper limit of the test force for reverse tensile testing may be set to a relatively small value to avoid unnecessary damage or misjudgment to the workpiece. Conversely, if the coefficient of friction is high, a larger test force or displacement may be required to fully verify its stability. The equivalent elastic modulus of the suction cup's rubber lip is used to assess the degree of deformation of the suction cup under the test force, thereby distinguishing between instability of the workpiece itself and elastic deformation of the suction cup.

[0088] S604. Perform physical stability verification operation based on the adjusted verification parameters.

[0089] This application's solution dynamically acquires and utilizes the material property information of the workpiece, positioning groove, and suction cup to intelligently adjust the verification parameters for physical stability verification operations. This adjustment mechanism transforms the verification process from a rigid, fixed pattern into one that can be optimized based on specific assembly scenarios and material characteristics. For example, when handling workpieces of different materials, the system can automatically calculate the most suitable displacement and upper limit of the test force for reverse tensile testing, thereby avoiding misjudgments or omissions due to parameter mismatches. This adaptive verification method ensures that the assembly stability of the workpiece can be evaluated in the most reasonable and effective way, regardless of the material combination.

[0090] In some preferred embodiments, a specific example is given below. Suppose a robot needs to insert a workpiece made of polymer material into a metal positioning slot.

[0091] First, the system will acquire the material properties of the polymer workpiece and the metal positioning groove, such as the elastic modulus and surface roughness of the polymer, and the hardness and surface finish of the metal.

[0092] Secondly, based on this material property information, the system calculates the coefficient of friction between the polymer workpiece and the metal positioning groove, as well as the coefficient of friction between the suction cup (e.g., made of silicone rubber) and the polymer workpiece, and determines the equivalent elastic modulus of the silicone rubber suction cup lip. For example, the coefficient of friction between the polymer and the metal may be relatively low, while the elastic modulus of the silicone rubber suction cup reflects its flexibility.

[0093] Next, based on these calculated coefficients and moduli, the system intelligently adjusts the verification parameters for the physical stability verification operation. Considering that polymer workpieces may be relatively fragile and have low friction with the metal positioning groove, the system may set the displacement and upper limit of the reverse test stretching to relatively conservative values ​​to avoid damaging the workpiece and ensure that it can be determined whether it is truly in place under a small external force.

[0094] Finally, the robot will perform a physical stability verification operation based on these adjusted verification parameters. For example, it might apply a reverse stretch with a preset, relatively small test force limit and monitor whether the workpiece undergoes displacement beyond the allowable range. If the workpiece remains stable under the set test force, it is considered to have truly entered the positioning groove.

[0095] like Figure 3 As shown, this embodiment of the invention also provides a collaborative control system for production robots based on vision inspection. The system includes: The pose data acquisition module is used to acquire the first pose data of the robot's end effector and the second pose data of the workpiece based on the vision inspection system; the end effector is used to attract the workpiece and make the workpiece enter the positioning groove. The displacement transmission difference recognition module is used to calculate the relative displacement between the robot's end effector and the workpiece based on the first pose data and the second pose data. The quantization module is used to quantify the deformation state of the suction cup on the end effector based on relative displacement. The false positioning detection module is used to determine whether the current assembly state is a false positioning state caused by the deformation of the suction cup, based on the suction cup deformation state. The fine-tuning action execution module is used to execute preset fine-tuning actions to make the workpiece actually enter the positioning groove when the position is determined to be false.

[0096] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by a computer program instructing related hardware. This program can be stored in the computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be an internal storage unit of the task execution device (including a data sending end and / or a data receiving end) of any of the foregoing embodiments, such as the hard disk or memory of the task execution device. The computer-readable storage medium can also be an external storage device of the terminal device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device. Further, the computer-readable storage medium can include both the internal storage unit of the task execution device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the task execution device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0097] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0098] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0099] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the scope of protection of this application.

Claims

1. A collaborative control method for production robots based on vision detection, characterized in that, include: The first pose data of the robot's end effector and the second pose data of the workpiece are obtained based on a vision inspection system. The end effector is used to pick up the workpiece and bring the workpiece into the positioning groove; Based on the first pose data and the second pose data, the relative displacement between the robot's end effector and the workpiece is calculated; Based on the relative displacement, the deformation state of the suction cup on the end effector is quantified; Based on the suction cup deformation state, determine whether the current assembly state is a false positioning state caused by suction cup deformation; When the workpiece is determined to be in a false positioning state, a preset fine-tuning action is performed to make the workpiece actually enter the positioning groove.

2. The collaborative control method for production robots based on vision detection according to claim 1, characterized in that, The step of calculating the relative displacement between the robot's end effector and the workpiece based on the first pose data and the second pose data includes: Determine whether the end effector has entered a blocked state; After entering the obstructed state, the first displacement increment of the end effector is determined based on the first pose data; Based on the second pose data, determine the second displacement increment of the workpiece; The relative displacement is determined based on the difference between the first displacement increment and the second displacement increment.

3. The collaborative control method for production robots based on vision detection according to claim 2, characterized in that, Determining whether the end effector has entered a blocked state includes: If the deviation between the end effector and the workpiece on the z-axis is determined to be within a first preset range, and the deviation between them on the planes containing the X-axis and Y-axis is determined to be within a second preset range, then the end effector is determined to be in a blocked state.

4. The collaborative control method for production robots based on vision detection according to claim 2, characterized in that, The quantification of the suction cup deformation state on the end effector based on the relative displacement includes: Obtain the stiffness coefficient matrix of the suction cup on the end effector in each degree of freedom; Multiplying the relative displacement by the stiffness coefficient matrix yields the current accumulated elastic potential energy of the suction cup and its deformation in different directions, thereby quantifying the deformation state of the suction cup on the end effector.

5. The collaborative control method for production robots based on vision detection according to claim 4, characterized in that, The determination of whether the current assembly state is a false positioning state caused by the deformation of the suction cup includes: Based on the deformation state of the suction cup, determine whether the elastic potential energy is greater than a preset potential energy threshold. If the elastic potential energy is greater than the preset potential energy threshold and the relative displacement is greater than the preset displacement threshold, then the current assembly state is determined to be a false positioning state caused by the deformation of the suction cup.

6. The collaborative control method for production robots based on vision detection according to claim 2, characterized in that, The step of performing a preset fine-tuning action to ensure that the workpiece actually enters the positioning groove includes: Perform high-frequency jittering in the plane containing the X and Y axes; If the end effector experiences a sudden displacement change within a preset range in the Z-axis direction during high-frequency jittering, it is determined that the workpiece has actually entered the positioning groove, and the high-frequency jittering is stopped.

7. The collaborative control method for production robots based on vision detection according to claim 6, characterized in that, The method further includes: If the execution duration of the high-frequency jitter exceeds the preset duration, the end effector will be retracted along the Z-axis, and a manual operation request will be sent to the user terminal. Receive manual operation commands from user terminals; Based on the manual operation command, the workpiece is actually put into the positioning groove.

8. The collaborative control method for production robots based on vision detection according to claim 1, characterized in that, The method further includes: After confirming that the workpiece has actually entered the positioning slot, a physical stability verification operation is performed; the physical stability verification operation is used to ensure that the assembly state of the workpiece does not depend on the pressure applied by the end effector. Based on the results of the physical stability verification operation, the end effector is controlled to release the workpiece.

9. The collaborative control method for production robots based on vision detection according to claim 8, characterized in that, The physical stability verification operation includes: Obtain the material property information of the workpiece and the positioning groove; Based on the material properties of the workpiece and the positioning groove, the contact friction coefficient between the suction cup and the workpiece, the contact friction coefficient between the workpiece and the positioning groove, and the equivalent elastic modulus of the rubber lip of the suction cup are obtained. Based on the contact friction coefficient between the suction cup and the workpiece, the contact friction coefficient between the workpiece and the positioning groove, and the equivalent elastic modulus of the rubber lip of the suction cup, the verification parameters of the physical stability verification operation are adjusted; the verification parameters of the physical stability verification operation include the displacement of the reverse test tensile and the upper limit of the test force. Perform physical stability verification based on the adjusted verification parameters.

10. A collaborative control system for production robots based on vision inspection, characterized in that, The system includes: The pose data acquisition module is used to acquire the first pose data of the robot's end effector and the second pose data of the workpiece based on the vision inspection system; the end effector is used to adsorb the workpiece and make the workpiece enter the positioning groove. The displacement transmission difference recognition module is used to calculate the relative displacement between the robot's end effector and the workpiece based on the first pose data and the second pose data. A quantization module is used to quantify the deformation state of the suction cup on the end effector based on the relative displacement; The false positioning detection module is used to determine whether the current assembly state is a false positioning state caused by the deformation of the suction cup, based on the deformation state of the suction cup. The fine-tuning action execution module is used to execute a preset fine-tuning action to make the workpiece actually enter the positioning groove when it is determined to be in a false positioning state.

Images