Automatic plugging method and system based on multi-perception fusion and readable storage medium

CN116544753BActive Publication Date: 2026-10-16北京瓦特曼智能科技有限公司
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202310203057.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-06
Publication Date
2026-10-16
Estimated Expiration
2043-03-06

AI Technical Summary

Technical Problem

[0003]现有技术提出了点云或者图像识别控制机械臂,基本的设计逻辑在于通过图像识别或者点云锁定插座对接的位置,通过机械臂的端部的夹具并采用驱动机构锁紧;但是实际过程中,由于现场环境的影响,现实是充满各种误差的,如插座被污染,导致点云或者图像识别控制所判断的插座位置可能存在误差,此时机器臂会严格按照预先设定的位置轨迹进行运动,导致无法实现自动插拔;位置上很小的误差,经过刚度的放大会产生很大的接触力,甚至导致插头和插座直接损坏

Benefits of technology

本发明实施例所提供的基于多感知融合的自动插拔方法,首先通过对现有的连接件进行改进,提供一种柔性接触式的连接件,再集视觉感知和力觉反馈相融合的算法结合机器人自动控制技术,末端夹具浮动单元(即柔性接触端子)等,实现了连接件的柔性插拔;系统整体作业成功率达到99.5%以上。通过力觉反馈和浮动单元,即使第一连接件的安装位置误差超过一定径向范围,仍然可以保证系统具有较高的成功率。且可精准的判定插拔是否到位,避免过渡插接或者插接过松,实现智能化的自动插拔。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116544753B_ABST
    Figure CN116544753B_ABST
Patent Text Reader

Abstract

Embodiments of the present application provide a kind of automatic plug based on multi-perception fusion method, system and readable storage medium;Automatic plug method is applied to connector;Connector includes first connector and second connector, first connector includes plug, second connector includes socket, socket includes flexible contact terminal and is electrically connected with plug, socket is connected on the end effector of robot;Automatic plug method includes: by image recognition, plug is connected to socket according to moving track;Contact force is read during the connection of plug and socket;System overall operation success rate reaches 99.5% or more.Through force feedback and floating unit, even if the installation position error of first connector exceeds certain radial range, still can guarantee that system has higher success rate.And whether plug is in place can be accurately judged, avoid transition plug or plug is too loose, realize intelligent automatic plug.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of intelligent device design, and in particular relates to an automatic plugging and unplugging method, system and readable storage medium based on multi-sensor fusion. Background Technology

[0002] Torpedo-type molten iron ladle (also known as a torpedo car or molten iron mixing car). Molten iron ladles are traditional and widely used equipment for transporting molten iron. Under certain protective conditions, molten iron mixing cars can transport molten iron to locations tens of kilometers away, with minimal temperature drop, thus saving energy. The torpedo-type molten iron ladle, including the molten iron inside, weighs approximately 400-500 tons. The motor power required for rotation is substantial, and dedicated transport trains cannot provide the corresponding high-power supply, necessitating external power supply. This is typically done manually by manually pulling out the plug on-site. Traditional manual operation is mechanical and tedious, and the working environment is harsh, involving high temperatures and dust, resulting in high labor intensity and health hazards for workers. Steel plants urgently need an automatic plug-in / plug-out power system for torpedo ladles to replace manual operation.

[0003] Existing technologies have proposed point cloud or image recognition-controlled robotic arms. The basic design logic is to lock the socket docking position through image recognition or point cloud, and then lock it with a clamp at the end of the robotic arm and a drive mechanism. However, in actual practice, due to the influence of the on-site environment, reality is full of various errors. For example, if the socket is contaminated, the socket position determined by point cloud or image recognition control may be inaccurate. In this case, the robotic arm will move strictly according to the pre-set position trajectory, resulting in the inability to achieve automatic plugging and unplugging. Even a small error in position can generate a large contact force after being amplified by stiffness, and may even cause direct damage to the plug and socket.

[0004] This application is made on this basis. Summary of the Invention

[0005] To address the technical problems identified in the prior art, this invention provides an automatic plugging and unplugging method, system, and readable storage medium based on multi-sensor fusion.

[0006] The first aspect of this application provides an automatic plugging and unplugging method based on multi-sensor fusion, which is applied to a connector. The connector includes a first connector and a second connector. The first connector includes a plug, and the second connector includes a socket. The socket includes a flexible contact terminal electrically connected to the plug. The socket is connected to an end effector of a robot. The automatic plugging and unplugging method includes: connecting the plug to the socket according to a movement trajectory through image recognition; reading the contact force during the connection process between the plug and the socket; performing multi-point force analysis on the contact force to adjust the movement trajectory and determine whether the plug is installed in place.

[0007] In an optional embodiment of this application, the first connector further includes a positioning pin and a first fixing structure, and the second connector includes a positioning hole and a second fixing structure. The positioning pin and the positioning hole are clearance-fitted, and an elastic seal is provided at the opening of the positioning hole. During the contact process between the first connector and the second connector, the positioning pin contacts the positioning hole in advance, and then the plug and the flexible contact terminal contact to form an electrical connection. Finally, the first fixing structure and the second fixing structure achieve a snap-fit. The multi-point force analysis of the contact force to adjust the movement trajectory includes: when the positioning pin enters the positioning hole, adjusting the position of the first connector according to the radial force of the positioning pin fed back by the elastic seal to pre-position the plug.

[0008] In a further embodiment of this application, adjusting the position of the first connector based on the radial force of the locating pin in response to feedback from the elastic seal to pre-position the plug includes: comparing the radial contact forces; determining that the first connector is offset when the difference in the radial contact forces is greater than a preset first threshold; and controlling the first connector to move radially in the case of the first connector offset, wherein the direction of the radial displacement is opposite to the direction of the larger radial contact force, and the moving distance of the radial displacement is adjusted according to the difference feedback.

[0009] In a further embodiment of this application, determining whether the plug is installed correctly includes: when the plug and the flexible contact terminal are in contact, determining whether the plug is installed correctly based on the magnitude of the contact force in the axial direction.

[0010] In a further embodiment of this application, determining whether the plug is installed in place based on the magnitude of the axial contact force when the plug and the flexible contact terminal are in contact includes: during the contact process between the plug and the flexible contact terminal, feeding back and controlling the driving force of the first connector based on the magnitude of the axial contact force; when the axial contact force meets the preset conditions, determining that the plug is installed in place.

[0011] In a further embodiment of this application, the automatic plugging and unplugging method further includes: receiving a first signal and controlling a robot to move the plug to a preset position; obtaining the three-dimensional position coordinates of the socket; and calculating the movement trajectory of the plug based on the three-dimensional position coordinates and the preset position.

[0012] In an optional embodiment of this application, the end effector further includes a first camera, and acquiring the three-dimensional position coordinates of the socket includes: triggering the first camera to scan the socket to obtain a target image; and performing feature recognition on the target image to determine the three-dimensional position coordinates of the socket in space.

[0013] In a further embodiment of this application, determining the three-dimensional position coordinates of a socket in space through feature recognition of a target image includes: preprocessing the target image; segmenting the preprocessed target image to obtain the pixel coordinates corresponding to the socket; and obtaining the three-dimensional position coordinates of the socket based on the pixel coordinates and depth information.

[0014] In a further embodiment of this application, calculating the movement trajectory of the plug based on the three-dimensional position coordinates includes: pre-establishing the workpiece coordinate system and completing hand-eye calibration; acquiring the current target image in real time; and extracting the feature points of the socket in the target image based on the region growing method to generate the movement trajectory.

[0015] In a further embodiment of this application, the automatic plugging and unplugging method further includes: receiving a second signal and removing the plug from the socket according to a movement trajectory; reading the contact force of a force feedback sensor during the removal of the plug and socket; and performing multi-point force analysis on the contact force to adjust the movement trajectory and determine whether the plug has been successfully removed.

[0016] In a second aspect of this application, an automatic plug-in / plug-out system based on multi-sensor fusion is also provided. The automatic plug-in / plug-out system includes: a connector, including a first connector and a second connector, the first connector including a plug, the second connector including a socket, the socket including a flexible contact terminal electrically connected to the plug; a robot, including a multi-degree-of-freedom robotic arm and an end effector connected to the robotic arm, the end effector including a tooling fixture, a force feedback sensor and a first camera, the tooling fixture fixing the first connector, and the second connector fixing to an external device; and a controller configured to the above-described automatic plug-in / plug-out method.

[0017] In a further embodiment of this application, the first connector further includes a positioning pin and a first fixing structure, and the second connector includes a positioning hole and a second fixing structure; the positioning pin and the positioning hole are clearance-fitted and an elastic seal is provided at the opening of the positioning hole. During the contact process between the first connector and the second connector, the positioning pin contacts the positioning hole in advance, and then the plug and the flexible contact terminal contact to form an electrical connection. Finally, the first fixing structure and the second fixing structure are snapped together; wherein, the first fixing structure is a chuck and the second fixing structure is a pull pin.

[0018] In a further embodiment of this application, the plug is a planar terminal with multiple electrical contact points, and the socket and plug are connected by contact.

[0019] Finally, this application also provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the above-described automatic plugging and unplugging method.

[0020] Beneficial effects: The automatic insertion and removal method based on multi-sensor fusion provided in this invention first improves existing connectors by providing a flexible contact connector. Then, it integrates visual perception and force feedback algorithms with robot automatic control technology and an end-effector floating unit (i.e., a flexible contact terminal), achieving flexible insertion and removal of the connector. The overall system success rate reaches over 99.5%. Through force feedback and the floating unit, even if the installation position error of the first connector exceeds a certain radial range, the system can still maintain a high success rate. Furthermore, it can accurately determine whether the insertion or removal is in place, avoiding over-insertion or excessively loose insertion, thus achieving intelligent automatic insertion and removal.

[0021] Other features and advantages of the embodiments of the present invention will be described in the following detailed description section. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a connector in the prior art; Figure 2 and Figure 3 A schematic diagram of the structure of the connector used in the improved automatic insertion and removal method of the present invention, as provided in Embodiment 1. Figure 2 As the first connector, Figure 3 (For the second connector); Figure 4 A flowchart illustrating the automatic plugging and unplugging method based on multi-sensor fusion provided in an embodiment of the present invention; Figure 5 This is a simplified structural diagram of the plug and socket in contact according to an embodiment of the present invention; Figure 6 This is a flowchart of step S3 in the automatic insertion and removal method provided in the embodiment of the present invention; Figure 7 This is a second flowchart of the automatic insertion and removal method provided in an embodiment of the present invention; Figure 8 This is a flowchart illustrating step S12 of the automatic insertion and removal method provided in this embodiment of the invention. Figure 9 This is a flowchart illustrating step S122 in the automatic insertion / removal method provided in this embodiment of the invention. Figure 10This is a flowchart illustrating step S13 of the automatic insertion and removal method provided in this embodiment of the invention. Figure 11 This is the third flowchart of the automatic insertion and removal method provided in the embodiments of the present invention; Figures 12-13 A schematic diagram of the structure of the connector provided in Embodiment 2, which is applicable to the automatic insertion and removal method of the present invention. Figure 12 As the first connector, Figure 13 (for the second connector); and Figure 14 The flowchart shows step S3 in the improved automatic insertion and removal method applicable to embodiments of the present invention, as provided in Example 2.

[0024] Figure Labels Connector; 200, end effector; First connector; 102, Second connector; First tooling pallet; 1012, First fixing structure; Plug; 1014, positioning pin; Second tooling pallet; 1022, Second fixing structure; Socket; 1024; positioning hole; First camera; 202, End mounting plate; 1024a, Elastic seal. Detailed Implementation

[0025] To make the above and other features and advantages of the present invention clearer, the invention will be further described below with reference to the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explanation to those skilled in the art and are exemplary only, not restrictive.

[0026] [Overall Invention Concept] Continuing from the previous point, to address the technical problems existing in the current technology, namely that current torpedo canisters require manual insertion and removal, or automatic insertion and removal can be achieved through image recognition combined with a robotic arm, both of these methods have certain drawbacks. Manual insertion is time-consuming and labor-intensive; while existing image recognition solutions lack fault tolerance and cannot provide more accurate insertion and removal. Therefore, the inventive concept of Embodiment 1 of this invention provides an automatic insertion and removal method based on multi-sensor fusion.

[0027] In this general inventive concept, the first step is to improve the connector used for plugging and unplugging to provide more data feedback. Then, by using multi-sensor fusion of image and force feedback, the redundancy and contradictions that may exist between multi-source sensor information are eliminated. By using information complementarity, the uncertainty in previous solutions is reduced, so as to obtain the best comprehensive estimate and evaluation of the object under test.

[0028] This system utilizes industrial robotic arm control technology that integrates visual guidance and force feedback to automate the insertion and removal of electrical plugs on torpedo-shaped molten iron ladles. First, based on 3D visual perception technology, the system recognizes the three-dimensional spatial position and posture of the electrical plugs on the torpedo-shaped ladle, guiding the robot to automatically install and remove the plugs via an end effector 200. Second, the end effector 200, combined with force feedback and a floating unit, achieves flexible control of the automatic insertion and removal process. Finally, based on deep learning technology, the system automatically identifies the ladle's serial number.

[0029] It is understood that the connector and automatic insertion / removal method adapted to the connector provided in the embodiments of the present invention can be applied to automatic insertion / removal in other industrial fields, in addition to the field of torpedo-type molten iron ladles.

[0030] For a better understanding of this solution, please refer to Figures 1-3 , Figure 1 This is a schematic diagram of a connector in the prior art. Figures 2-3 This is a schematic diagram of the structure of the connector 100, which is an improved automatic insertion and removal method applicable to embodiments of the present invention as provided in Example 1.

[0031] A connector is a coupling device that connects electrical terminals to form a circuit. In the prior art, high-power industrial connectors typically employ... Figure 1 The connector shown, i.e. the coaxial connector, forms a male and female plug-in joint through its cylindrical structure, and the internal electrical terminals make contact connections.

[0032] Coaxial connectors are rigid connections, with the female and male ends forming a clearance fit between the shaft and hole during connection. However, this method requires high precision in the fit of the connectors, which makes it less tolerant of errors in image recognition.

[0033] To better apply force sensing to the automatic insertion and removal method provided in this embodiment of the invention, this embodiment first provides an improved connector 100, such as... Figure 2 and Figure 3 As shown.

[0034] The connector 100 includes a first connector 101 and a second connector 102; the first connector 101 is the male end mounted on the robotic arm, and the second connector 102 is the female end mounted on an external device. In this embodiment, the robotic arm drives the first connector 101 to connect to the second connector 102 to achieve automatic insertion and removal. Its main advantage lies in multi-sensor fusion to achieve precise automatic insertion and removal of the first connector 101 and the second connector 102. "Multi-sensor fusion" refers to the comprehensive processing of data obtained from different information sources or multiple sensors. The improved connector 100 in this embodiment primarily provides better multi-dimensional data.

[0035] It is understood that one end of the first connector 101 is connected to the robot's end effector 200 and can move automatically; the second connector 102 is fixed in position.

[0036] In this embodiment of the invention, the first connector 101 includes a first tooling tray 1011, a first fixing structure 1012, and a plug 1013; wherein the first fixing structure 1012 and the plug 1013 are both disposed on the side of the first tooling tray 1011 facing the second connector 102.

[0037] Specifically, the first fixed structure 1012 needs to be designed to prevent mistakes, that is, to adopt an asymmetrical design to provide machine vision recognition while providing mistake-proof positioning.

[0038] In this embodiment of the invention, the second connector 102 includes a second tooling tray 1021, a second fixing structure 1022, and a socket 1023. The second tooling tray 1021 and the first tooling tray 1011 are matched with each other to facilitate a fit during insertion.

[0039] Similarly, the second fixed structure 1022 matches the first fixed structure 1012, and the first fixed structure 1012 and the second fixed structure 1022 are connected under the cooperation of the robot.

[0040] Furthermore, the first fixing structure 1012 uses a chuck, and the second fixing structure 1022 uses a pull stud.

[0041] Optionally, four chucks are available, with the four Latin chucks configured to have different lengths: two clamping Latin chucks, one positioning chuck, and one compensation chuck. This creates a quick-change zero-point positioning system.

[0042] It is understood that the first fixed structure 1012 and the second fixed structure 1022 can be arbitrarily combined according to the specific conditions of the workpiece and equipment; the first tooling pallet 1011 and the second tooling pallet 1021 can be circular or square, wherein the number of chucks in the first fixed structure 1012 and the second fixed structure 1022 is determined according to the size of the tooling pallet; in the second fixed structure 1022, in addition to the necessary positioning rivets and compensating rivets to determine the position of the first tooling pallet 1011, clamping rivets are selected at the other chuck positions, and the robot can achieve automatic locking and unlocking by using pneumatic and hydraulic operation.

[0043] The following describes the key modified areas of this application, namely the plug 1013 and the socket 1023. Structurally, the plug 1013 is located in the central area of ​​the first tooling tray 1011, and the first fixing structure 1012 is designed in a ring around the plug 1013. Similarly, the socket 1023 and the plug 1013 are respectively located in the central area of ​​the second tooling tray 1021. The plug 1013 is a planar terminal with multiple electrical contact points. The socket 1023 includes a flexible contact terminal that can form an electrical connection with the electrical contact points. That is, the socket 1023 and the plug 1013 are connected by contact.

[0044] It is understood that both the plug 1013 and the socket 1023 are configured as planar contact electrical connections, which have a certain degree of fault tolerance. At the same time, the first tooling tray 1011, the second tooling tray 1021, and the fixing structure set on them all have obvious structural features, which facilitates image recognition. Most importantly, the flexible contact terminal improved in this embodiment of the invention enables the socket 1023 to have downward elasticity after being squeezed by the plug 1013, that is, it allows the plug 1013 to have a certain displacement in the axial direction. The flexibility of the socket 1023 can provide mechanical data, thereby providing a multi-dimensional data foundation for multi-sensor fusion.

[0045] Of course, the connector can also be improved by using other elastic components as force feedback structures, as long as they achieve the same effect as above.

[0046] Based on the above-described contact structure, this embodiment of the invention provides an automatic plugging and unplugging method based on multi-sensor fusion.

[0047]

Control Methods

[0048] Based on the connector 100 described above, the automatic insertion and removal methods include: Step S1: Connect the plug 1013 to the socket 1023 according to the movement trajectory using image recognition; Step S2: Read the contact force during the connection process of plug 1013 and socket 1023; Step S3: Perform multi-point force analysis on the contact force to adjust the movement trajectory and determine whether the plug 1013 is installed in place.

[0049] The steps provided in the embodiments of the present invention are all executed based on the controller, and the same applies below.

[0050] Step S1 involves the robot connecting the plug 1013 and socket 1023; in step S2, the "contact force" refers to the feedback force experienced by the plug 1013 and socket 1023 as a whole during contact. This contact force is obtained from a force feedback sensor located on the robot's end effector 200. Subsequently, a multi-point force analysis is performed based on the contact force to determine the plug's installation state.

[0051] The aforementioned determination of "proper installation" is to determine whether there is radial deviation in the plug and whether the axial connection of plug 1013 is too tight or insufficient. Understandably, because the socket 1023 and plug 1013 have a flexible contact connection, if the connection is too tight during contact, the flexible contact terminals of the socket 1023 will deform significantly, and repeated plugging and unplugging can easily affect the lifespan of the socket. Conversely, if the connection between the socket 1023 and plug 1013 is too loose, insufficient contact surface can lead to excessive temperature rise at the contact surfaces of the socket 1023 and plug 1013, especially in high-power transmission scenarios in some industrial settings, which can easily generate sparks or even cause fires. Therefore, the determination in step S13 is particularly important.

[0052] Furthermore, determining whether the plug 1013 is properly installed in step S13 includes: When plug 1013 and flexible contact terminal are in contact, the magnitude of the contact force in the axial direction determines whether plug 1013 is installed in place.

[0053] Please see Figures 5-6 , Figure 5 This is a simplified structural diagram of the plug 1013 and socket 1023 when in contact, as provided in an embodiment of the present invention. Figure 6 This is a flowchart of step S3 in the automatic plugging and unplugging method provided in an embodiment of the present invention.

[0054] During the connection process, the plug 1013 and the flexible contact terminal come into contact and are squeezed, the socket 1023 deforms and applies an axial contact force to the plug 1013, and the plug 1013 can be judged as to whether it is in place based on the contact force.

[0055] In one specific embodiment, determining whether the plug 1013 is properly installed based on the magnitude of the axial contact force when the plug 1013 and the flexible contact terminal are in contact includes: Step S31: During the contact process between the plug 1013 and the flexible contact terminal, the driving force of the first connector 101 is controlled by feedback based on the magnitude of the contact force in the axial direction. Step S32: When the axial contact force meets the preset conditions, confirm that the plug 1013 is installed in place.

[0056] It is understood that the axial reaction force generated after the plug 1013 and the flexible contact terminal come into contact is the axial contact force Fd relative to the force feedback sensor. The driving force Fr of the first connector 101 is adjusted according to the feedback of the contact force Fd.

[0057] Specifically, after the plug 1013 contacts the flexible contact terminal, the controller carries the first connector 101 to perform feedback control. This method involves controlling the movement of the first connector 101 by using a driving force Fr. In one example, the formula for calculating the driving force Fr is: Fr=K1(Xr−X)+K2(X'r−X')+K3(X''r−X'') Xr=f(Fd) Where Xr represents the target position change of the first connector 101 (1m, 0m / s, 0m / s²), and Xr is negatively correlated with the contact force Fd, where f(Fd) is a negatively correlated function. When there is no contact force Fd, Xr is a fixed constant. X represents the current actual position change of the first connector 101. The magnitude of the driving force Fr that controls the first connector 101 is adjusted according to the difference between the target position change and the current actual position change. K1, K2, and K3 are stiffness damping coefficients, which can characterize the relationship between force and position. These coefficients are calibrated for user applicability.

[0058] When the plug 1013 contacts the flexible contact terminal, the flexible contact terminal generates an end contact force Fd to counteract Fr, at which point the first connector 101 gradually stops moving forward. When the contact force Fd reaches a preset value, it is determined that the plug 1013 is installed in place, and the first connector 101 stops moving.

[0059] It is understandable that after the plug 1013 gradually comes into contact with the socket 1023, feedback control is formed. As the plug 1013 moves toward the socket 1023, the axial contact force Fd on the plug 1013 increases, and the target position change Xr gradually decreases. At this time, the movement of the plug 1013 slows down until the contact force Fd and the driving force Fr tend to be balanced.

[0060] The advantage of the above control is that the plug 1013 is controlled by the driving force Fr, which is more precise than position control and will not cause irreversible damage to the socket 1023; the continuous feedback of information through feedback adjustment is conducive to achieving a virtuous cycle and improving efficiency.

[0061] Please see Figure 7 and Figure 8 , Figure 7 This is a second flowchart of the automatic insertion and removal method provided in an embodiment of the present invention; Figure 8This is a flowchart illustrating step S12 of the automatic insertion / removal method provided in this embodiment of the invention.

[0062] In this embodiment of the invention, the automatic insertion and removal method further includes: Step S11: Upon receiving the first signal, control the robot to move the plug 1013 to the preset position; Step S12: Obtain the three-dimensional position coordinates of socket 1023; Step S13: Calculate the movement trajectory of plug 1013 based on the three-dimensional position coordinates and the preset position.

[0063] It can be understood that the first signal is the movement command that drives the robot to perform automatic plugging and unplugging. When the controller receives the first signal, the robot moves the plug to the preset position. Then, a movement trajectory is generated based on the three-dimensional position coordinates of the socket 1023 and the current preset position. By performing reasonable trajectory planning for the robot, the smoothness of the trajectory movement during plugging and unplugging is improved, unnecessary displacement is avoided, and the operating efficiency and accuracy are also improved.

[0064] Specifically, the end effector 200 also includes a first camera 201 and an end mount 202, wherein a force feedback sensor (not shown) is disposed on the end mount 202, and the acquisition of the three-dimensional position coordinates of the socket mentioned in step S12 above includes: Step S121: Trigger the first camera 201 to scan the socket and obtain the target image; Step S122: Determine the three-dimensional position coordinates of the socket in space by performing feature recognition on the target image.

[0065] It is understood that steps S121 to S122 provide a way to obtain the socket, namely, to use the first camera 201 to acquire images and perform three-dimensional positioning through image recognition.

[0066] In some embodiments, other positioning methods may also be used, such as three-point lidar positioning or ranging sensor positioning. The purpose is to accurately obtain the three-dimensional position coordinates of the socket.

[0067] Please continue reading. Figure 9 , Figure 9 The flowchart below shows the specific steps of step S122 in the automatic plugging and unplugging method provided in this embodiment of the invention. In one specific solution, step S122, which determines the three-dimensional position coordinates of the socket in space by feature recognition of the target image, includes:

[0068] Step S1221: Perform image preprocessing on the target image; Step S1222: Perform image segmentation on the target image after image preprocessing to obtain the pixel coordinates corresponding to the socket; Step S1223: Obtain the three-dimensional position coordinates of the socket based on pixel coordinates and depth information.

[0069] It is understandable that image preprocessing techniques can effectively remove interference from environmental factors in images, highlight relevant information, and improve image detectability. The image preprocessing techniques used in this system mainly include algorithms for image denoising, image enhancement, and grayscale thresholding.

[0070] Image denoising can improve image quality by employing mean filtering and median filtering algorithms. Smoothing filtering, as a neighborhood averaging method, replaces the gray value of a pixel in the target image with the values ​​of its neighbors, i.e., it's an N×N image of the original image. After processing, the grayscale value of each pixel From image The grayscale values ​​of several neighboring pixels are determined as shown in the following formula: Where, in the formula ; It is a point The set of neighborhoods centered on the center. yes The total number of internal coordinates. Median filtering is a non-linear filter. Its principle is to replace the gray value of a point with the median value of all points in the neighborhood of that point. Median filtering has a more obvious noise reduction effect when the gray value changes little, and it can effectively eliminate impulse interference and achieve a good smoothing effect.

[0071] Image enhancement can effectively highlight the features of the original image, eliminate problems such as feature blurring and key point obscuring caused by external factors, and improve the analytical accuracy of the image.

[0072] In this application, histogram equalization uses a gray-level mapping method to ensure that pixels at each gray level of the original image have similar gray levels, with each pixel evenly distributed and occupying as many gray levels as possible. Then, it uses a histogram to statistically represent the existence of gray levels and their probabilities, describing gray-level statistical information. The gray-level equalization conversion formula is as follows:

[0073] In the above formula, The converted grayscale value. The grayscale value before conversion. Let be the probability density function.

[0074] Histogram equalization is used to process denoised images, resulting in reduced gray-level differences and more uniform changes. However, it can easily lead to the appearance of outlines and blurring of image boundaries, which is not conducive to subsequent analysis and processing.

[0075] Grayscale transformation is essentially the process of increasing image contrast, and there are two types of transformations: linear and nonlinear. Linear transformation linearly stretches the original image. The grayscale value of each pixel, from the grayscale range Expand to The transformed image is As shown in the following formula:

[0076] Nonlinear transformations change the grayscale value of each pixel in the original image using exponential, logarithmic, or other functions. As shown in the following formula:

[0077] Where b is the base, and modulation parameters can be added to increase the dynamic range of the transformation. We obtain the following formula: In the above formula, 'a' can change the position of the curve, and 'c' can change the amplitude of the curve. Exponential transformation is suitable for overly bright high grayscale images.

[0078] A logarithmic transformation is a method where the grayscale values ​​of the original image and the processed image satisfy a logarithmic relationship. Logarithmic transformation is suitable for low-grayscale images that are too dark. The equation is as follows:

[0079] Modulation parameters can also be added. Increase the range of transformation: Gray-level thresholding, also known as binarization, transforms an image with gray levels into a binary image consisting only of black and white. It specifies a gray value as the threshold, dividing the gray values ​​of each pixel in the image. Compared to this dividing line, values ​​smaller than the dividing line are uniformly set to 0, while values ​​larger than the dividing line are set to 255. The grayscale threshold transformation expression is as follows, where T is the specified threshold.

[0080] The target image after grayscale transformation is binarized to eliminate interference factors and clarify boundaries, providing a basis for further analysis.

[0081] Image segmentation not only reduces the amount of data required for subsequent image analysis and recognition processes, but also preserves key image features, enabling computers to better understand the information contained within the image. This system uses edge detection and Hough transform algorithms for image segmentation.

[0082] Edge detection is performed by convolving a differential operator template with the image. The Canny operator, in particular, offers low false positive rates and high localization accuracy. First, a Gaussian filter is selected. To smooth the image, a non-extreme suppression method is used to traverse the gradient magnitude matrix of the image to find edge seed points. Finally, edge points are determined recursively by using dual threshold detection to achieve edge extraction.

[0083] Gauss filter Function-smoothed image: in, It is image data. It's the edge.

[0084] The magnitude and direction of the gradient are obtained by calculating the finite difference of the first-order partial derivative. The amplitude is obtained as follows: The direction obtained is: Non-local maxima suppression (NMR) is used to handle gradient magnitude. Edges are refined by setting non-local maxima to zero, and each neighborhood center pixel is compared to two pixels along the gradient line. If the gradient value at the center point is less than the gradient values ​​of two adjacent pixels along the gradient line, it is set to zero.

[0085] Set two thresholds, high and low. Find the contour lines in the high threshold result image. Since these lines may be discontinuous, find the breaks in the lines in the low threshold result image and connect them in the high threshold image.

[0086] The Hough transform can identify feature information in an image and form closed, complete boundaries from discrete edge pixels. The Hough transform maps each edge point in the original image sequentially to a parameter space, then accumulates the values ​​corresponding to all coordinate elements obtained in the parameter space, and finally infers the location of feature points based on this value. Since the original .bmp format images acquired by industrial cameras are relatively large, performing the Hough transform is time-consuming and inefficient. Therefore, this application optionally employs two Hough transforms. The first transform scales down the image by 30%, and sets half of the smallest side of the image as the processing range for initial extraction. The second transform is performed on a higher-resolution image to obtain more accurate circle parameters. Performing the Hough transform twice on the processed image effectively improves detection efficiency.

[0087] Please see Figure 10 , Figure 10 This is a flowchart illustrating step S13 of the automatic plugging and unplugging method provided in an embodiment of the present invention. In this embodiment, step S13, calculating the movement trajectory of the plug based on three-dimensional position coordinates, includes:

[0088] Step S131: Pre-establish the workpiece coordinate system and complete hand-eye calibration; Step S132: Acquire the current target image in real time; Step S133: Extract feature points of the socket in the target image based on the region growing method to generate the movement trajectory.

[0089] Specifically, in step S131, the hand-eye calibration can be performed using any of the following solutions: one is to fix the first camera in a certain place and fix the calibration plate in the end of the robotic arm; the other is to do the opposite, fix the first camera in the end of the robotic arm and fix the calibration plate in a certain place.

[0090] The two hand-eye calibration modes described above are essentially equivalent in their abstract mathematical principles, ultimately resulting in matrix equations. Noise often exists during measurement; therefore, solving these matrix equations typically employs an optimization iterative method rather than a direct algebraic solution. Thus, during hand-eye calibration, the end-effector pose of the robotic arm is often altered. Under different end-effector poses, the first camera acquires different calibration plate position and pose data. Substituting multiple sets (typically 10 to 20 sets) of relative position and pose data between the first camera and the calibration plate, along with the corresponding end-effector position and pose data of the robotic arm itself, into the aforementioned matrix equation, and after optimization iteration, the optimal value X can be solved, representing the relative position and pose relationship between the first camera and the robotic arm base. Once this relationship is obtained, the object acquired by the first camera can be mapped to its coordinates in the coordinate system of the robotic arm base after coordinate transformation, thereby guiding the robotic arm to perform the corresponding task. In this embodiment of the invention, the second installation method is preferred.

[0091] Furthermore, the robot workpiece coordinate system describes the coordinate system of the workpiece's position, that is, the motion coordinate system for the robot to achieve accurate manipulation. The workpiece coordinate system is usually defined using the three-point method. This involves determining three points to calibrate the workpiece coordinate system, where the line connecting these three points forms the X-axis, a perpendicular line drawn from these three points to the X-axis forms the Y-axis, and the Z-axis is determined using the right-hand rule. The greater the distance between these points, the more precise the definition.

[0092] In step S132, feature points are extracted using the region growing method to generate a RAPID trajectory. The region growing method is based on the similarity of pixels to form a set of points. First, a point is determined as the starting point of growth, called the seed point. Then, pixels with the same or similar properties in the neighborhood of the seed point are collected. These new pixels are used as seed points to continue searching for similar points in the neighborhood, thereby expanding the set of seed points until no more points that meet the conditions can be included.

[0093] Furthermore, the extracted feature points are many discrete point coordinates. To make these coordinates a trajectory that the robot can execute, it is necessary to use instructions to control the robot's movement and write them in RAPID program format. RAPID is a high-level programming language for motion control in ABB robots. The RAPID language allows for the installation of new programs, data objects, and data types for specific robot systems. The torpedo canister insertion / removal trajectory should use MoveL instructions to connect the discrete points with small intervals to form a complete contour. Then, the feature points are defined as working points using a loop. These working points are then written into the trajectory routine according to the RAPID language format, and translational and acceleration speeds are defined and incorporated into the main and initialization programs. The complete code file is stored on the PC in .mod format for synchronization with the robot controller.

[0094] Please continue reading. Figure 11 , Figure 11 This is a third flowchart of the automatic insertion and removal method provided in an embodiment of the present invention. The automatic insertion and removal method further includes:

[0095] Step S4: Receive the second signal and remove the plug from the socket according to the movement trajectory; Step S5: Read the contact force from the force feedback sensor during the removal of the plug and socket; Step S6: Perform multi-point force analysis on the contact force to adjust the movement trajectory and determine whether the plug has been successfully disassembled.

[0096] It can be understood that steps S4 to S6 are the removal steps in the automatic plugging / unplugging method. When the second signal is received, the robot removes the plug from the socket. (See reference...) Figures 2-3 First, the first fixing structure 1012 and the second fixing structure 1022 are separated, and then the contact between the plug and the socket is slowly separated. At this time, the contact force received by the first connector 101 gradually decreases until there is no contact force, at which point the plug is confirmed to be completely disassembled.

[0097] In summary, the automatic insertion and removal method based on multi-sensor fusion provided in this invention first improves existing connectors by providing a flexible contact connector. Then, it integrates visual perception and force feedback algorithms with robot automatic control technology, end-effector floating units (i.e., flexible contact terminals), etc., to achieve flexible insertion and removal of the connector. The overall system success rate reaches over 99.5%. Through force feedback and the floating unit, even if the installation position error of the first connector exceeds a certain range, the system can still maintain a high success rate. Furthermore, it can accurately determine whether the insertion or removal is in place, avoiding over-insertion or excessively loose insertion, thus achieving intelligent automatic insertion and removal.

[0098]

Example 2

[0099] For connectors 100 requiring high precision or ultra-high power, embodiments of the present invention also provide a further improved connector 100.

[0100] In this second embodiment, the connector 100 has the same structure as in the first embodiment, the difference being that... The first connector 101 also includes a positioning pin 1014, and the second connector 102 includes a positioning hole 1024; the positioning pin 1014 and the positioning hole 1024 are clearance-fitted and an elastic seal 1024a is provided at the opening of the positioning hole 1024.

[0101] During the contact process between the first connector 101 and the second connector 102, the positioning pin 1014 pre-contacts the positioning hole 1024 to achieve pre-positioning, then the plug 1013 and the flexible contact terminal contact to form an electrical connection, and finally the first fixing structure 1012 and the second fixing structure 1022 achieve quick snap-fit.

[0102] The elastic seal 1024a can shrink and deform to form an interference fit with the locating pin 1014. When the locating pin 1014 and the locating hole 1024 are connected, the elastic seal 1024a will be subjected to radial compression, thereby generating radial contact force.

[0103] It is understandable that the elastic seal 1024a is annular. When the locating pin 1014 is precisely connected to the center of the elastic seal 1024a, the ring deformation of the elastic seal 1024a is consistent and the radial force is relatively average. When the locating pin 1014 is offset, the deformation of the elastic seal 1024a will be different, and the radial force will be concentrated in the area with larger deformation.

[0104] Based on the above approach, in this embodiment of the invention, step S3, which involves multi-point force analysis of the contact force to adjust the movement trajectory, includes: When the locating pin 1014 enters the locating hole 1024, the position of the first connector 101 is adjusted according to the radial force on the locating pin 1014 fed back by the elastic seal 1024a to pre-position the plug 1013.

[0105] It is understood that when the first connector 101 and the second connector 102 are connected, the radial force originates solely from the compression of the locating pin 1014 and the elastic seal 1024a. During the connection process, precise pre-positioning is achieved by analyzing the radial force and adjusting the position of the locating pin 1014. Please refer to [link to relevant documentation]. Figure 14 , Figure 14 This is a flowchart of step S3 in the improved automatic insertion and removal method applicable to embodiments of the present invention provided in Embodiment 2; in this embodiment of the present invention, adjusting the position of the first connector 101 to pre-position the plug 1013 based on the radial force fed back by the elastic seal 1024a on the positioning pin 1014 includes:

[0106] Step S31': Compare the contact forces located in the radial direction; Step S32': When the difference in radial contact force is greater than a preset first threshold, determine the offset of the first connector; Step S33': When the first connector is offset, control the first connector to move radially, wherein the direction of the radial displacement is opposite to the direction of the larger radial contact force, and the distance of the radial displacement is adjusted according to the difference feedback.

[0107] It is understandable that, based on the radial force collected by the force feedback sensor, as the offset of the positioning pin increases, the radial force in the corresponding offset direction also increases accordingly.

[0108] In a specific scheme, comparing the contact forces located in the radial direction includes: Collect radial forces from multiple preset directions; Compare the radial forces in two opposing directions among multiple radial forces.

[0109] It is understandable that a force feedback sensor can acquire radial forces Fn1, Fn2, Fn3, and Fn4 in four directions (some sensors can acquire eight angles) at 0°, 90°, 180°, and 270°. By comparing Fn2 and Fn4 in the 90° and 270° directions, as well as comparing the radial forces Fn1 and Fn4 in the 0° and 270° directions, a first difference ΔF1 and a second difference ΔF2 can be obtained.

[0110] Furthermore, when the difference in radial contact force is greater than a preset first threshold, it is determined that the compression of the positioning pin 1014 to the elastic seal 1024a is too large, which indicates that the first connector 101 is excessively offset.

[0111] Furthermore, when the first connector 101 is offset, the first connector 101 is controlled to move radially, wherein the direction of the radial displacement is opposite to the direction of the larger radial contact force, and the moving distance of the radial displacement is adjusted according to the difference feedback.

[0112] Specifically, when at least one of the first difference ΔF1 and the second difference ΔF2 is greater than a preset first threshold, the following cases can be identified: 1) If either the first difference △F1 or the second difference △F2 is greater than the first threshold, the first connector 101 will shift positively to one of the four directions. 2) When both the first difference △F1 and the second difference △F2 are greater than the first threshold, the diagonal shift occurs in the included angle direction of the four directions.

[0113] For example, if the radial force Fn1 at 0° is greater than Fn4, and the first difference ΔF1 exceeds the first threshold, then the first connector 101 is offset in the 0° direction. The radial displacement direction to be corrected in step S33' is the opposite direction of 0°, that is, the first connector 101 is controlled to be corrected in the 270° direction.

[0114] If both the first difference △F1 and the second difference △F2 are greater than the first threshold, it proves that the first connector 101 is obliquely offset at the angles of 0°, 90°, 180°, and 270°. The angle of inclination can be estimated based on the magnitude of the larger radial contact force. For example, when the obtained radial forces Fn1 and Fn2 in the 0° and 90° directions are larger contact forces, the first connector 101 has shifted in the direction between 0° and 90°. The first connector 101 can be corrected in the direction of 225°.

[0115] Furthermore, "the radial displacement distance is adjusted based on the difference feedback," meaning the corrected displacement distance is adjusted based on the magnitude of the first difference ΔF1. The larger the first difference ΔF1, the larger the target displacement distance, until the first difference ΔF1 is less than a first threshold. This can be referenced to PID feedback adjustment, setting a control reference difference of 0, the feedback quantity as the magnitude of the first difference ΔF1, and the control quantity as the radial displacement driving force control quantity. When the value of ΔF1 is larger, the radial displacement driving force control quantity is larger, resulting in a larger radial distance movement of the locating pin 1014, until the first difference ΔF1 approaches 0. At this point, the driving force control quantity approaches 0, the compressive force of the elastic seal 1024a causes the movement to stop, and the locating pin 1014 stops moving, thus exiting PID feedback adjustment. Adjusting the control displacement parameters by comparing the difference between the feedback value and the set value improves the accuracy during the adjustment process.

[0116] In summary, Embodiment 2 of the present invention proposes an automatic insertion and removal method for connector 100 in application scenarios with higher precision requirements. Specifically, it adds a pre-positioning step to the automatic insertion and removal method proposed in Embodiment 1. After pre-positioning using a positioning pin and positioning hole, the plug and socket are brought into contact, and then the first and second fixing members are connected. This allows for more precise installation of the first connector and further reduces radial error in automatic insertion and removal.

[0117] Automatic plug-in / plug-out system Embodiment 3 of the present invention provides an automatic insertion and removal system for a workpiece to be inspected (not shown in the figure), the automatic insertion and removal system comprising: A connector, including a first connector and a second connector, the first connector including a plug, the second connector including a socket, the socket including a flexible contact terminal electrically connected to the plug; The robot includes a multi-degree-of-freedom robotic arm and an end effector connected to the robotic arm. The end effector includes a tooling fixture, a force feedback sensor, and a first camera. The tooling fixture fixes a first connector, and a second connector is fixed to an external device. The work area includes controllable automatic safety doors and position sensors, which are used to check whether the equipment to be plugged in has arrived. Light source, used to provide an environment for image recognition; The controller is configured to perform the automatic plugging and unplugging method of Embodiment 1 or Embodiment 2 described above.

[0118] According to the above embodiment 2, based on the above automatic insertion and removal system, the first connector further includes a positioning pin and a first fixing structure, and the second connector includes a positioning hole and a second fixing structure. The positioning pin and positioning hole are fitted with a clearance fit and an elastic seal is provided at the opening of the positioning hole. During the contact process between the first connector and the second connector, the positioning pin contacts the positioning hole in advance, and then the plug and the flexible contact terminal contact to form an electrical connection. Finally, the first fixing structure and the second fixing structure are snapped together.

[0119] The first fixing structure uses a chuck, and the second fixing structure uses a pull stud.

[0120] The plug is a flat terminal with multiple electrical contact points. The socket and plug are connected by contact.

[0121] Taking the automatic insertion and removal application scenario of torpedo tank trucks as an example, the following general control logic is provided: 1) The system is powered on, the equipment is initialized, and the robotic arm is in its initial position and ready to go; 2) Confirm whether the torpedo tanker has arrived; 3) Confirm whether the system status is normal; 4) The central control center sends an automatic plug-in / plug-out command; 5) Automatically open the safety door between the robot and the torpedo tanker, perform dust removal on the first camera and turn on the lights; 6) Trigger the first camera to take a picture and identify the vehicle number of the torpedo tanker; 7) The robotic arm moves the first camera to the preset position; 8) Trigger the first camera to perform 3D imaging of the socket; 9) The three-dimensional spatial position of the system positioning socket; 10) The system automatically calculates the movement trajectory of the plug; 11) Control the robotic arm to drive the plug through the gripper to perform the installation operation; 12) Collision detection during socket installation; and based on the contact force read from the force feedback sensor, fine-tuning of the plug and detection of whether the plug is installed in place. 13) Confirm that the plug is installed correctly; 14) The robotic arm disconnects the plug and returns to its original position, closes the safety door, and waits for the backflow operation to end; 15) After the backflow operation is completed, the central control room confirms whether the system is normal; 16) Open the safety door; 17) The robotic arm moves to the preset position; 18) Trigger the first camera to perform three-dimensional imaging of the socket, and the system locates the three-dimensional spatial position of the socket; 19) Plug unloading trajectory planning; 20) The robotic arm uses a gripper to disassemble the plug; 21) Collision detection during plug disassembly, and simultaneous detection of whether the plug has been successfully disassembled based on force feedback; 22) The robotic arm moves the plug back to its original position; 23) The safety door is closed; 24) Waiting for the next assignment.

[0122] It is understood that this automatic plug-in / plug-out system performs the aforementioned functions by encapsulating the automatic plug-in / plug-out methods in the above method embodiments into various different models through compiled code. This detection system requires the models mentioned in the specific steps of the above method embodiments, which will not be repeated here.

[0123] Furthermore, those skilled in the art should understand that if all or part of the sub-modules involved in the automatic plug-in system 100 provided in the embodiments of the present invention are combined or replaced by means of fusion, simple changes, mutual transformation, etc., such as moving the position of each component; or setting the product they constitute as a whole; or having a detachable design; as long as the combined components can form a device / apparatus / system with a specific function, using such a device / apparatus / system to replace the corresponding components of the present invention also falls within the protection scope of the present invention.

[0124] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0125] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An automatic plugging and unplugging method based on multi-sensor fusion, characterized in that, The automatic plugging / unplugging method is applied to connections. Item; The connector includes a first connector and a second connector. The first connector includes a plug, and the second connector includes a socket. The socket includes a flexible contact terminal that is electrically connected to the plug. The socket is connected to the end effector of the robot. The automatic insertion / removal method includes: The plug is connected to the socket according to the movement trajectory using image recognition; Read the contact force during the connection process of the plug and socket; Multi-point force analysis is performed on the contact force to adjust the movement trajectory and determine whether the plug is installed in place. The first connector further includes a positioning pin and a first fixing structure. The second connector includes a positioning hole and a second fixing structure. The positioning pin and the positioning hole are clearance-fitted and an elastic sealing element is provided at the opening of the positioning hole. During the contact process of the first connector and the second connector, the positioning pin contacts the positioning hole in advance. Then the plug and the flexible contact terminal contact to form an electrical connection. Finally, the first fixing structure and the second fixing structure achieve a snap-fit ​​connection. The step of performing multi-point force analysis on the contact force to adjust the movement trajectory includes: When the locating pin enters the locating hole, the position of the first connector is adjusted according to the radial force of the locating pin fed back by the elastic seal to pre-position the plug.

2. The automatic insertion and removal method according to claim 1, characterized in that, The step of adjusting the position of the first connector based on the radial force of the locating pin fed back by the elastic seal to pre-position the plug includes: The radial contact forces are compared, which includes: collecting radial forces in multiple preset directions; and comparing the radial forces in two opposing directions among the multiple radial forces. When the difference in radial contact force is greater than a preset first threshold, the first connector is determined to be offset. When the first connector is offset, the first connector is controlled to move radially, wherein the direction of the radial displacement is opposite to the direction of the larger radial contact force, and the moving distance of the radial displacement is adjusted according to the difference feedback.

3. The automatic insertion and removal method according to claim 1 or 2, characterized in that, Determining whether the plug is properly installed includes: When the plug and the flexible contact terminal come into contact, the magnitude of the axial contact force determines whether the plug is installed in place.

4. The automatic insertion and removal method according to claim 3, characterized in that, The step of determining whether the plug is properly installed based on the magnitude of the axial contact force when the plug and the flexible contact terminal are in contact includes: During the contact process between the plug and the flexible contact terminal, the driving force of the first connector is controlled by feedback based on the magnitude of the axial contact force. When the axial contact force meets the preset conditions, the plug is determined to be installed in place.

5. The automatic insertion and removal method according to claim 1, characterized in that, The automatic insertion / removal method further includes: Upon receiving the first signal, the robot is controlled to move the plug to a preset position. Obtain the three-dimensional position coordinates of the socket; The movement trajectory of the plug is calculated based on the three-dimensional position coordinates and the preset position.

6. The automatic insertion and removal method according to claim 5, characterized in that, The end effector further includes a first camera, and the acquisition of the three-dimensional position coordinates of the socket includes: The first camera is triggered to scan the socket to obtain a target image; The three-dimensional position coordinates of the socket in space are determined by feature recognition of the target image.

7. The automatic insertion and removal method according to claim 6, characterized in that, The step of determining the three-dimensional position coordinates of the socket in space through feature recognition of the target image includes: Perform image preprocessing on the target image; The target image after image preprocessing is segmented to obtain the pixel coordinates corresponding to the socket; The three-dimensional position coordinates of the socket are obtained based on pixel coordinates and depth information.

8. The automatic insertion and removal method according to claim 7, characterized in that, The calculation of the plug's movement trajectory based on three-dimensional position coordinates includes: Pre-establish the workpiece coordinate system and complete hand-eye calibration; Real-time acquisition of the current target image; The feature points of the socket are extracted from the target image using the region growing method to generate the movement trajectory.

9. The automatic insertion and removal method according to claim 5, characterized in that, The automatic insertion / removal method further includes: Upon receiving the second signal, the plug is removed from the socket according to the movement trajectory; The contact force of the force feedback sensor is read during the removal of the plug and socket; Multi-point force analysis is performed on the contact force of the force feedback sensor to adjust the movement trajectory and determine whether the plug has been successfully disassembled.

10. An automatic plug-in / plug-out system based on multi-sensor fusion, characterized in that, The automatic insertion / removal system includes: A connector includes a first connector and a second connector, the first connector including a plug, the second connector including a socket, the socket including a flexible contact terminal electrically connected to the plug; A robot includes a multi-degree-of-freedom robotic arm and an end effector connected to the robotic arm. The end effector includes a tooling fixture, a force feedback sensor, and a first camera. The tooling fixture fixes a first connector, and a second connector is fixed to an external device. A controller configured to perform the automatic plugging and unplugging method as described in any one of claims 1 to 9.

11. The automatic insertion and removal system according to claim 10, characterized in that, The first connector further includes a positioning pin and a first fixing structure, and the second connector includes a positioning hole and a second fixing structure; The positioning pin and the positioning hole are fitted with a clearance and an elastic seal is provided at the opening of the positioning hole. During the contact process of the first connector and the second connector, the positioning pin contacts the positioning hole in advance, and then the plug and the flexible contact terminal contact to form an electrical connection. Finally, the first fixing structure and the second fixing structure are snapped together.

12. The automatic insertion and removal system according to claim 11, characterized in that, The first fixing structure uses a chuck, and the second fixing structure uses a pull stud.

13. The automatic insertion and removal system according to claim 12, characterized in that, The plug is a planar terminal with multiple electrical contact points, and the socket and the plug are connected by contact.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, one or more of which can be executed by one or more processors to implement the automatic plugging and unplugging method as claimed in any one of claims 1 to 9.

Citation Information

Patent Citations

  • Full-automatic charging robot control method and device

    CN110722561A

  • Buffer type electrode plugging structure and equipment for semiconductor process equipment

    CN215451900U

  • Plug device for industrial equipment

    CN217387743U