Computer control method and system for automatic compensation of pin equipment based on vision

Abstract

The application provides a host computer control method and system for automatic compensation of a pin insertion device based on vision, which comprises the following steps: synchronously triggering a plurality of vision cameras to collect pin and jack images, extracting coordinates and locking the vision triggering time; constructing a weighted global-local rigid registration model, iteratively solving a rotation matrix and a translation vector to obtain an original vision deviation vector, and obtaining a Z-direction height deviation to generate a compression depth compensation amount; within a time window from the vision triggering to the command locking, reading servo speed feedback and temperature change rate, leading to calculate the device autonomous drift amount and superimpose the original deviation to generate the final compensation amount; issuing to a multi-axis servo driver through a real-time Ethernet and synchronously locking the time sequence; calling a cache template to reuse a reference, executing secondary micro compensation according to a following error, triggering reference resampling after the cumulative threshold is reached; storing and visualizing the compensation data, and alarming when the threshold is exceeded. The technical problems of lack of real-time vision compensation and high communication delay of the pin insertion device are solved.

Description

Technical Field

[0001] This application relates to the field of vision-based pin compensation control technology, specifically to a host computer control method and system for automatic compensation of pin insertion devices based on vision. Background Technology

[0002] Automated connector insertion equipment is a key piece of equipment in the field of electronic component assembly, widely used in the automated crimping operations of products such as wire harness connectors, FPCs, PCB connectors, and new energy terminals. Insertion equipment typically relies on servo modules to drive the terminals and connectors to align and complete the crimping operation. The insertion accuracy directly determines the electrical connection reliability and product yield of the connector. However, in long-term continuous production, positioning deviations caused by factors such as mechanical wear, material fluctuations, and environmental changes are unavoidable. Traditional insertion equipment has many shortcomings in areas such as visual perception, compensation control, and communication interaction.

[0003] Currently, some pin insertion devices have attempted to incorporate vision technology to improve positioning accuracy. For example, a camera captures images of the pins and pad holes; binarization and blob analysis are used to locate the pin center coordinates; a pad hole matching template is used to locate the pad hole coordinates; and the offset for visual positioning compensation is calculated to guide the actuator to complete the insertion. Another approach compensates for the connector position by obtaining the overall offset of the PCB, or by combining vision inspection devices to adjust the insertion position in real time during the insertion process. However, these solutions still have significant limitations in practical applications.

[0004] Traditional pin insertion equipment relies primarily on fixtures and stops for material positioning, operating under purely mechanical open-loop control. Over long-term operation, tooling wear, lead screw backlash, equipment temperature rise and deformation, and incoming material dimensional tolerances continuously cause alignment misalignment. Once defects such as bent pins or broken sockets occur, manual shutdown and tooling fine-tuning are required. Calibration accuracy depends heavily on operator experience, and downtime costs are significant. Existing vision solutions are mostly used only for post-insertion photo inspection, a form of defect screening that cannot correct alignment deviations before the pressing action. Defective products are already inserted, resulting in high material losses and rework costs, failing to reduce defect rates at the source. The communication between the vision camera and the equipment controller typically uses polling serial communication, leading to significant data transmission delays. At high-speed pin insertion cycles, compensation commands lag behind the actual movement, preventing synchronous alignment correction. Furthermore, the lack of a dedicated high-speed interaction protocol results in frequent packet loss and timing errors. Switching between different connector models on the production line requires manual recalibration of the reference and adjustment of parameters, taking 10 to 30 minutes per changeover. This cannot meet the needs of flexible production with multiple product varieties and lacks a template buffer reuse mechanism. In multi-needle continuous segmented insertion scenarios, each needle position has independent local deviations. Traditional global unified compensation cannot correct the positioning error of each needle separately, and it also lacks long-term dynamic self-compensation capability for slow drift caused by equipment temperature rise and lead screw creep.

[0005] In summary, existing pin insertion equipment control schemes lack an integrated upper-computer closed-loop control system that combines visual acquisition, deviation calculation, and multi-axis servo real-time compensation, making it difficult to achieve high-speed, high-precision, and automated pin insertion production that eliminates the need for frequent manual calibration. Summary of the Invention

[0006] In view of this, this application provides a host computer control method and system for automatic compensation of pin insertion devices based on vision, so as to solve the technical problems of lack of real-time visual closed-loop compensation, high communication delay and high dependence on manual calibration in the prior art.

[0007] In a first aspect, this application provides a host computer control method for automatic compensation of pin insertion devices based on vision, including: Simultaneously trigger multiple vision cameras to acquire images of terminal pins and connector sockets, extract the actual coordinates of the pins and the theoretical reference coordinates of the sockets through image processing, and lock the vision trigger time; Based on the actual coordinates of the pins and the theoretical reference coordinates of the socket, a weighted global coordinate system is constructed. The objective function of local rigid registration is used to iteratively solve for the global optimal rotation matrix and translation vector to obtain the original visual deviation vector; at the same time, the Z-axis height deviation of the terminal is obtained to generate the Z-axis indentation depth compensation amount. Within the time window from the visual trigger moment to the compensation instruction latching moment, the servo speed feedback value and the equipment temperature change rate are read, the autonomous drift of the equipment is calculated in advance, and the pre-compensation amount is superimposed with the original visual deviation vector to generate the final compensation amount. The final compensation amount is sent to the multi-axis servo driver via a real-time Ethernet channel, and the visual acquisition timestamp and servo motion timing are latched synchronously. The standard product template in the cache is called for benchmark reuse. When changing models, the cached template is loaded and visual fine-tuning is performed. During the pin insertion process, the actual position feedback of the servo is collected, and secondary micro-compensation is added based on the following error. After the cumulative production reaches the threshold, visual benchmark resampling is triggered. Store pin compensation data and visualize compensation trends and calculation time; when the total time, compensation amount, or following error exceeds the threshold, trigger an alarm and retain abnormal information.

[0008] Preferably, the objective function for the weighted global-local rigid registration is: Among them, dynamic weights , The current pin number and , It is the attenuation coefficient and its value ranges from 0.5 to 1.2; In a multi-pin continuous segmented insertion scenario, the current insertion sequence number The process is dynamically updated as the process progresses, and the weight of each pin is recalculated and the objective function is re-solved before each pin insertion. Before iteratively solving, calculate the initial residuals for each pin. and the mean of all pin residuals with standard deviation ,like Then the weights of the outliers will be... Set to zero to eliminate false deviation coordinates caused by material burrs or deformation; The rotation matrix Include Rotation compensation amount, the translation vector Includes X-axis and Y-axis translation compensation.

[0009] Preferably, the final compensation amount is... The calculation formula is: In the formula, , The original visual deviation vector, For servo encoders in time window Internal real-time feedback velocity vector, This refers to the real-time temperature change rate of the equipment. To accumulate temperature difference, It is the characteristic length coefficient of thermal expansion. The dynamic rate compensation coefficient and The value range is 0 to 1.0. This is the static thermal drift compensation coefficient; and Global timestamp unification is achieved based on the IEEE 1588 precision time protocol, mapping the visual exposure time, temperature sampling time, and servo encoder position latching time to the same clock source. The calculation error is ≤1μs.

[0010] Preferably, the multi-channel vision camera is synchronously triggered by an FPGA, and the image acquisition timing deviation is ≤300ns; The image processing includes grayscale equalization, noise filtering and subpixel contour extraction, with a preprocessing time of ≤4ms; Deviation calculation time ≤ 5ms; The real-time Ethernet channel adopts the hard real-time UDP protocol, the data link layer uses a time-aware shaper for traffic scheduling, the compensation instruction is encapsulated as the servo driver native register instruction format, and directly written into the driver register to eliminate protocol parsing delay, and the data transmission delay ≤ 2ms; The total time spent on the entire process ≤15ms, the final pin compensation positioning accuracy stabilizes at ±0.015mm.

[0011] Preferably, the standard product template in the cache stores the theoretical reference coordinates, temperature compensation curves, and coefficients for each connector model. ,coefficient ,coefficient and preset safety thresholds; During initial production, the above parameters are saved to the local cache. When changing models, the corresponding cache template is directly loaded, and the fine-tuning time is ≤0.8ms. The cache template is updated under the following conditions: when the deviation drift exceeds 0.015mm during continuous production, or after a change in material specifications or a major equipment overhaul, the baseline is automatically re-collected and the cache template is updated. When the final compensation amount calculated in a single operation exceeds the preset safety threshold, the host computer determines that the incoming material is seriously misaligned, automatically issues a stop or rejection control signal, and saves the current visual image and compensation log.

[0012] Preferably, the cumulative production threshold is reached by automatically triggering visual reference resampling after every 5,000 products are produced, and recalibrating the template reference coordinates to compensate for the slow drift caused by lead screw wear and thermal deformation; When the servo following error exceeds the threshold, a secondary micro-compensation is automatically applied. During continuous high-volume pin insertion operation, the compensation fluctuation is controlled within ±0.008mm; Production data is automatically archived and visualized, including real-time display of four-axis compensation curves, equipment temperature change curves, and calculation time for each stage. It also supports integration with the MES production management system for quality traceability and equipment wear analysis.

[0013] Preferably, the multi-channel vision camera includes a 2D area array camera and / or a 3D structured light camera, which is compatible with different models of vision hardware. The image data transmission adopts a gigabit Ethernet interface, and the transmission time of a single frame image is ≤1ms. When using a 3D structured light camera, the image processing also includes 3D point cloud reconstruction and Z-axis height extraction. The original visual deviation vector also includes Z-axis indentation depth deviation, and the translation vector... It also includes Z-axis translation compensation, the rotation matrix It is a 3×3 spatial rotation matrix; The synchronous triggering of multiple vision cameras, image processing, deviation calculation, time delay prediction compensation, and servo command issuance adopts a pipelined parallel processing architecture. The vision acquisition and the issuance of the compensation command in the previous cycle are executed overlapping in time, and the throughput is continuously processed in a pipeline manner to ensure that a complete vision-compensation closed loop is completed in each insertion cycle.

[0014] Secondly, this application provides a host computer control system for automatic compensation of pin insertion devices based on vision, comprising: The visual data acquisition module is used to synchronously trigger multiple vision cameras to acquire images of terminal pins and connector sockets, complete image preprocessing and sub-pixel contour extraction, output the actual coordinates of the pins and the theoretical reference coordinates of the sockets, and latch the visual trigger time. The deviation compensation calculation module, connected to the visual data acquisition module, is used to construct a weighted global coordinate system based on the actual coordinates of the pins and the theoretical reference coordinates of the jacks. The objective function of local rigid registration is used to iteratively solve the global optimal rotation matrix and translation vector to obtain the original visual deviation vector, which is then used to obtain the Z-axis height deviation of the terminal and generate the Z-axis indentation depth compensation amount. The time delay prediction and compensation module is connected to the visual data acquisition module and the deviation compensation calculation module respectively. It is used to read the servo speed feedback value and the equipment temperature change rate within the time window from the visual triggering time to the compensation command latching time, calculate the autonomous drift of the equipment in advance, and superimpose the pre-compensation amount with the original visual deviation vector to generate the final issued compensation amount. The servo real-time transmission module is connected to the time delay prediction and compensation module and is used to transmit the final transmission compensation amount to the multi-axis servo driver through a real-time Ethernet channel, and synchronously latch the visual acquisition timestamp and servo motion timing. The template cache reuse module is connected to the deviation compensation calculation module and is used to store and call standard product templates. When changing models, the cached template is loaded and visual fine-tuning is performed. The closed-loop dynamic correction module is connected to the servo real-time transmission module. It is used to collect the actual position feedback of the servo, superimpose secondary micro-compensation based on the following error, and trigger visual reference resampling after the cumulative production reaches the threshold. The data visualization and alarm module is connected to the closed-loop dynamic correction module. It is used to store the pin compensation data and visualize the compensation trend and calculation time. When the total time, compensation amount or following error exceeds the threshold, an alarm is triggered and abnormal information is retained.

[0015] Thirdly, this application provides a computer device, including: a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to realize the host computer control method of the first aspect, which is based on vision for automatic compensation of the pin insertion device.

[0016] Fourthly, this application provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, enable the host computer control method of the first aspect, which is based on vision for automatic compensation of the pin device.

[0017] The host computer control method and system for automatic compensation of pin insertion devices based on vision provided in this application have at least the following beneficial effects: This application integrates visual acquisition, deviation calculation, and servo compensation into a unified closed-loop control link, completing deviation prediction and correction before the pin insertion action is executed, fundamentally different from the post-screening mode of "pin insertion first, quality inspection later" in traditional solutions. Since compensation occurs before the pressing action, it avoids material loss and rework costs caused by defective products already being pressed in, reducing the pin insertion defect rate from the source. Through a global rigid registration model, the relative positional relationship of each pin after compensation remains highly consistent with the theoretical pin array, avoiding the geometric constraint mismatch problem between pins that may be caused by independent pin-by-pin compensation, making it particularly suitable for automatic pin insertion scenarios of multi-pin dense connectors. The dynamic time delay prediction compensation mechanism transforms communication and calculation delays from "passive tolerance" to "active utilization," continuously tracking changes in device status during the waiting period to ensure that compensation commands are still accurate and effective when they reach the servo, eliminating the compensation misalignment problem caused by delays in traditional solutions. The standard product template cache, combined with an automatic resampling mechanism, supports both rapid model changeover and adaptive compensation for long-term drift, eliminating the need for frequent manual intervention in continuous high-volume production and significantly reducing operator workload and equipment downtime. Visualization and alarm functions provide comprehensive data support for production line quality traceability and preventative equipment maintenance. Operators can quickly locate abnormal batches and abnormal needle positions, facilitating continuous improvement of production processes. Attached Figure Description

[0018] Figure 1 A schematic diagram of a host computer control method for automatic compensation of pin insertion devices based on vision is shown. Figure 2 A schematic diagram of a host computer control system for automatic compensation of pin insertion devices based on vision is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] Example 1 See Figure 1 As shown, this application provides a host computer control method for automatic compensation of pin insertion devices based on vision, including the following steps.

[0021] S1. Simultaneously trigger multiple vision cameras to acquire images of terminal pins and connector sockets, extract the actual coordinates of the pins and the theoretical reference coordinates of the sockets through image processing, and lock the vision trigger time. S2. Construct a weighted global coordinate system based on the actual coordinates of the pins and the theoretical reference coordinates of the socket. The objective function of local rigid registration is used to iteratively solve for the global optimal rotation matrix and translation vector to obtain the original visual deviation vector; at the same time, the Z-axis height deviation of the terminal is obtained to generate the Z-axis indentation depth compensation amount. S3. Within the time window from the visual trigger moment to the compensation instruction latching moment, read the servo speed feedback value and the equipment temperature change rate, calculate the equipment autonomous drift amount in advance, and superimpose the pre-compensation amount with the original visual deviation vector to generate the final compensation amount to be sent out. S4. The final compensation amount is sent to the multi-axis servo driver through the real-time Ethernet channel, and the vision acquisition timestamp and servo motion timing are latched synchronously. S5. Call the cached standard product template for benchmark reuse. Load the cached template and make visual fine-tuning when changing models. Collect the actual position feedback of the servo during the pin insertion process and add secondary micro-compensation based on the following error. Trigger visual benchmark resampling after the cumulative production reaches the threshold. S6. Store the pin compensation data and visualize the compensation trend and calculation time; when the total time, compensation amount or following error exceeds the threshold, trigger an alarm and retain the abnormal information.

[0022] This invention provides a host computer control method for automatic compensation of pin insertion devices based on vision. The method first activates multiple vision cameras through a synchronous triggering mechanism to simultaneously acquire images of terminal pins and connector sockets. The multiple vision cameras are synchronously exposed under the control of a unified external trigger signal, ensuring that the spatial positional relationship between pins and sockets is accurately recorded at the same time, avoiding false deviations introduced by time asynchrony.

[0023] After the acquired images are transmitted to the host computer, image processing operations such as grayscale equalization, noise filtering, and subpixel contour extraction are performed in parallel. Grayscale equalization is used to eliminate the interference of uneven illumination on feature extraction, noise filtering is used to suppress random noise introduced by the image sensor, and subpixel contour extraction improves the edge positioning accuracy to the subpixel level through interpolation algorithms, ultimately outputting high-precision actual pin coordinates and theoretical reference coordinates of the jack. The host computer latches the timestamp of the moment when visual acquisition is triggered, which serves as the reference zero point for all subsequent timing calculations.

[0024] After acquiring visual coordinate data, a weighted global-local rigid registration model is constructed. The core idea of ​​this model is to treat the local deviations of all pins relative to the connector socket as a rigid whole. By solving for an optimal rotation matrix and translation vector, the theoretical coordinate set is transformed as a whole to achieve the best fit with the actual coordinate set. This approach differs from the traditional method of calculating compensation values ​​independently for each pin, ensuring that the relative positional relationship between each pin remains strictly consistent after compensation, avoiding the disruption of the inherent geometric constraints of the pin array due to independent compensation. Simultaneously, the height data of the terminal in the Z-axis is acquired using a 3D structured light camera or laser rangefinder. The height difference between the terminal's insertion surface and the connector socket end face is calculated, and the Z-axis insertion depth compensation is generated accordingly to prevent over- or under-pressure.

[0025] During the waiting period after the visual deviation calculation is completed but before the compensation command is actually written to the servo register, a dynamic time-delay prediction compensation mechanism is introduced. Since the servo is already in motion at the moment of visual triggering, there is an inherent delay between visual triggering and the final latching of the compensation command. During this period, the device itself continues to generate displacement, and temperature changes also cause thermal expansion of the mechanical structure. By reading the real-time speed information fed back by the servo encoder, the amount of motion displacement within this time window is calculated by integration; simultaneously, temperature sensor data is read to calculate the rate of temperature change and cumulative temperature difference, and combined with the device's thermal expansion characteristic parameters, the thermally induced deformation is estimated. The aforementioned motion drift and thermal drift are superimposed on the original visual deviation vector to generate a forward-looking compensation amount.

[0026] The final compensation amount is transmitted to the multi-axis servo drive through a real-time Ethernet channel. The synchronous latching mechanism ensures that the visual acquisition time, deviation calculation time, command issuance time, and servo response time strictly correspond on the time axis, eliminating compensation lag or lead caused by timing disorder.

[0027] During the actual pin insertion process, a locally cached standard product template is invoked. This template stores the reference coordinates and various compensation parameters of the connector model under standard operating conditions. It can be directly loaded during model changes without requiring manual recalibration. During the pin insertion process, the actual position feedback from the servo encoder is continuously read and compared with the theoretical command position. If the servo following error exceeds the allowable range, a secondary micro-compensation is immediately applied for dynamic correction. A cumulative production threshold is also set; after every 5000 pieces produced, visual reference resampling is automatically triggered to recalibrate the template coordinates, compensating for systematic drift caused by long-term wear of the lead screw and slow thermal deformation of the frame.

[0028] The entire process records the original visual deviation, servo compensation amount, and final pin insertion coordinates for each pin. The compensation trend curves of each axis and the calculation time of each stage in a single cycle are displayed in real time through a visual interface. When the total time, compensation amount, or following error exceeds the preset threshold, an alarm is triggered and abnormal images and logs are saved for maintenance personnel to trace and analyze afterward.

[0029] This application integrates visual acquisition, deviation calculation, and servo compensation into a unified closed-loop control link, completing deviation prediction and correction before the pin insertion action is executed, fundamentally different from the post-screening mode of "pin insertion first, quality inspection later" in traditional solutions. Since compensation occurs before the pressing action, it avoids material loss and rework costs caused by defective products already being pressed in, reducing the pin insertion defect rate from the source. Through a global rigid registration model, the relative positional relationship of each pin after compensation remains highly consistent with the theoretical pin array, avoiding the geometric constraint mismatch problem between pins that may be caused by independent pin-by-pin compensation, making it particularly suitable for automatic pin insertion scenarios of multi-pin dense connectors. The dynamic time delay prediction compensation mechanism transforms communication and calculation delays from "passive tolerance" to "active utilization," continuously tracking changes in device status during the waiting period to ensure that compensation commands are still accurate and effective when they reach the servo, eliminating the compensation misalignment problem caused by delays in traditional solutions. The standard product template cache, combined with an automatic resampling mechanism, supports both rapid model changeover and adaptive compensation for long-term drift, eliminating the need for frequent manual intervention in continuous high-volume production and significantly reducing operator workload and equipment downtime. Visualization and alarm functions provide comprehensive data support for production line quality traceability and preventative equipment maintenance. Operators can quickly locate abnormal batches and abnormal needle positions, facilitating continuous improvement of production processes.

[0030] Preferably, the weighted global-local rigid registration objective function is: Among them, dynamic weights , The current pin number and , It is the attenuation coefficient and its value ranges from 0.5 to 1.2; In a multi-pin continuous segmented insertion scenario, the current insertion sequence number The process is dynamically updated as the process progresses, and the weight of each pin is recalculated and the objective function is resolved before each pin insertion. Before iteratively solving, calculate the initial residuals for each pin. and the mean of all pin residuals with standard deviation ,like Then the weights of the outliers will be... Set to zero to eliminate false deviation coordinates caused by material burrs or deformation; Rotation matrix Include Rotation compensation amount, translation vector Includes X-axis and Y-axis translation compensation.

[0031] Specifically, in an ideal situation, there is only one rigid body transformation relationship between the theoretical coordinate set and the actual coordinate set of all pins, namely, overall rotation and translation. However, in engineering practice, due to factors such as individual material differences, incoming material positioning deviations, and equipment repeatability accuracy, the actual deviation direction and magnitude of each pin vary. This method uses a weighted least squares framework to assign a weight coefficient to each pin, ensuring that the alignment accuracy of pins with higher weights is prioritized when solving for the optimal rotation matrix and translation vector.

[0032] The weights are not fixed but change in real time as the current pin insertion process progresses. When the k-th pin is about to be inserted, that pin receives the highest weight, the weights of its neighboring pins decrease exponentially with distance, and pins far from the current insertion position receive lower global baseline weights. The essence of this dynamic weight allocation strategy is to introduce a local optimization center within the framework of global rigid transformation, achieving an adaptive balance between "global rigid constraints" and "piecewise local accuracy".

[0033] The attenuation coefficient λ ranges from 0.5 to 1.2. When λ is small, the weight attenuates slowly, and neighboring pins contribute more to the fitting of the current pin, resulting in a smoother transformation matrix. When λ is large, the weight attenuates rapidly, the dominant role of the current pin is more prominent, and the local fit is higher. In practical applications, the appropriate coefficient can be selected flexibly based on the pin spacing and pin insertion accuracy requirements.

[0034] In multi-pin continuous segmented pin insertion scenarios, the pin insertion sequence k is continuously updated as the process progresses. Each time the pin insertion position is advanced, the weight distribution of all pins is recalculated and the objective function is resolved to ensure that the compensation amount for each pin is the optimal solution obtained under the dominance of the weights of that pin and its neighboring pins.

[0035] Before iteratively solving the objective function, an adaptive outlier removal pre-screening procedure is executed. Specifically, the initial residual for each pin is calculated first. Then, the mean μ and standard deviation σ of all pin residuals are calculated. If the residual of a pin exceeds the 3σ threshold, it indicates that the deviation of that point is much larger than the normal statistical distribution range, and it is very likely that it is a false coordinate caused by dust, burrs, abnormal reflection, or local deformation on the material surface. At this time, the weight of the abnormal point is automatically reset to zero, so that it does not participate in the rigid registration solution of the objective function, thereby avoiding the overall module pin collision caused by a single abnormal point contaminating the global optimal solution.

[0036] The rotation matrix R includes a rotation compensation of θ, and the translation vector T includes translation compensations for the X and Y axes. For a 3D structured light camera configuration, the translation vector T is extended to a three-dimensional vector including translation compensations for the X, Y, and Z axes, and the rotation matrix R is extended to a 3×3 spatial rotation matrix.

[0037] The above solution process can be completed quickly using singular value decomposition or orthogonal Procrustes analysis methods. The time taken for a single deviation calculation is no more than 5ms, which meets the cycle time requirements of high-speed production lines.

[0038] Traditional pin insertion compensation schemes calculate XY translation and angular rotation compensation independently for each pin, which inherently contradicts physical principles—the actuator has only one global θ rotation axis, making it impossible to simultaneously meet the different angular compensation requirements of different pins. This scheme uses global rigid registration to unify multi-pin deviations into the same solution framework of rotation matrix and translation vector, ensuring that all pins maintain the geometric consistency of the theoretical array after compensation, fundamentally resolving kinematic conflicts in multi-pin scenarios. The dynamic weighting mechanism achieves an adaptive trade-off between global smoothness and local accuracy. When a pin is advanced to a certain position, that pin and its neighboring pins dominate the optimization objective, ensuring the alignment accuracy of the currently inserted pin; while distant pins retain certain weight constraints to prevent overly aggressive global transformations from causing uncontrolled deviations in distant pins. Compared to fixed-weight schemes, dynamic weighting improves the average accuracy of continuous multi-pin insertion by approximately 48%. The 3σ outlier adaptive elimination logic effectively avoids spurious deviations caused by material surface defects, optical noise, or image processing errors contaminating the global transformation matrix, improving robustness in harsh industrial environments. This logic does not require manual setting of fixed thresholds and can automatically adapt to the tolerance distribution characteristics of different products, thus exhibiting self-adaptability.

[0039] Preferably, the final compensation amount is issued. The calculation formula is: In the formula, , The original visual deviation vector, For servo encoders in time window Internal real-time feedback velocity vector, This refers to the real-time temperature change rate of the equipment. To accumulate temperature difference, It is the characteristic length coefficient of thermal expansion. The dynamic rate compensation coefficient and The value range is 0 to 1.0. This is the static thermal drift compensation coefficient; and Global timestamp unification is achieved based on the IEEE 1588 precision time protocol, mapping the visual exposure time, temperature sampling time, and servo encoder position latching time to the same clock source. The calculation error is ≤1μs.

[0040] Specifically, this application constructs a dynamic time delay prediction compensation model to address the inherent delay between visual acquisition and the actual effectiveness of compensation commands, and determines the calculation formula for the final compensation amount issued by the model.

[0041] First item The original visual deviation vector includes X-axis translation deviation, Y-axis translation deviation, and θ-axis rotation deviation (and, when using a 3D structured light camera, Z-axis indentation depth deviation). This term is defined at the visual trigger moment. The deviation value obtained by solving the aforementioned weighted rigid registration objective function represents "how much should be corrected at the current moment".

[0042] Second item This is a term for compensation of the lead inertia. From arrive Within the time window (total process time ≤ 15ms), the servo does not stop moving, and the device itself continuously generates displacement. By reading the velocity vector fed back in real time by the servo encoder, the displacement of the device within this time window is precisely calculated by performing a definite integral. This displacement is superimposed on the original deviation vector, meaning that what is sent is not the "deviation at the visual trigger moment," but the "deviation predicted to arrive at the compensation command latch moment," transforming communication delay from a negative factor into a calculable predictable window.

[0043] Third item For dynamic thermal rate compensation term, where This represents the real-time rate of temperature change of the equipment, reflecting the rapidity of the current temperature rise. When the equipment heats up rapidly, the rate of thermal expansion is high, and this item provides transient compensation; when the temperature tends to stabilize, As it approaches zero, this term automatically decays. α is the dynamic rate compensation coefficient, determined by the servo response bandwidth, with a value ranging from 0 to 1.0.

[0044] Fourth item This is the static thermal drift compensation term, where This represents the cumulative temperature difference between the current temperature and the reference temperature. β is the characteristic length coefficient of thermal expansion of the lead screw or frame (determined by the mechanical structure dimensions and material properties), and β is the static thermal drift compensation coefficient (obtained through thermal expansion experiments of metallic materials). This term mainly compensates for steady-state thermal drift caused by cumulative temperature rise after long-term operation, complementing the third term—the third term addresses transient temperature rise, and the fourth term addresses steady-state temperature rise.

[0045] To ensure the precise measurability of the start and end times of the aforementioned time windows, this method employs the IEEE 1588 precision time protocol to achieve global timestamp unification. Under this protocol, the visual exposure time, temperature sampling time, and servo encoder position latching time are all mapped to the same clock source, achieving clock synchronization accuracy at the microsecond level (Δt calculation error ≤ 1μs). This high-precision time synchronization makes the time boundaries of the aforementioned integration operations clear and unambiguous, avoiding the accumulation of Δt calculation errors caused by clock asynchrony among subsystems.

[0046] In traditional solutions, the inherent delay between visual triggering and compensation issuance is considered a defect, typically addressed passively by reducing production line cycle time or increasing buffering—essentially "tolerating lag." This solution transforms this delay window into an active prediction interval. It uses motion feedback and temperature change rate data within the window to calculate the device's impending position, ensuring that the issued compensation is a forward-looking value rather than a historical one. Comparative tests show that without predictive compensation, thermal drift resulted in a cumulative deviation of 0.018mm after 2 hours of continuous operation; with this solution, the cumulative deviation remained stable within 0.009mm after 8 hours of continuous operation, demonstrating a 100% improvement in long-term stability. The motion inertia advance compensation precisely offsets the autonomous movement of the servo during communication waiting, ensuring that even with a total latency of ≤15ms, the compensation command arrives at the servo with a precise match to the current actual pose, eliminating the root cause of compensation lag. The dual-path compensation architecture for dynamic thermal rate and static thermal drift addresses thermal effects at two different time scales: unsteady-state temperature rise and steady-state temperature rise. This ensures stable compensation accuracy during cold starts, continuous operation, and under fluctuating ambient temperatures. The application of the IEEE 1588 precise time protocol aligns the data sampling times of the vision, temperature, and servo subsystems at microsecond precision, providing accurate time boundary conditions for the time delay prediction compensation formula and guaranteeing the mathematical rigor of integral and differential calculations.

[0047] Preferably, the multiple vision cameras are synchronously triggered by the FPGA, and the image acquisition timing deviation is ≤300ns; Image processing includes grayscale equalization, noise filtering, and subpixel contour extraction; preprocessing time is ≤4ms. Deviation calculation time ≤5ms; The real-time Ethernet channel adopts the hard real-time UDP protocol, the data link layer uses a time-aware shaper for traffic scheduling, the compensation instruction is encapsulated in the servo driver native register instruction format, and directly written into the driver register to eliminate protocol parsing delay, and the data transmission delay ≤2ms. Total time for the entire process ≤15ms, the final pin compensation positioning accuracy stabilizes at ±0.015mm.

[0048] Specifically, this application employs a multi-channel vision camera synchronous triggering method at the vision acquisition hardware level. Multiple cameras (including 2D area array cameras and / or 3D structured light cameras) are triggered by a unified external pulse signal generated by an FPGA (Field-Programmable Gate Array). All cameras synchronously expose and acquire images at the same time, with a timing deviation not exceeding 300ns. The high-precision timing capability of the FPGA ensures the spatiotemporal consistency of pin and socket images, avoiding positioning errors introduced by the timing deviations of different camera acquisitions.

[0049] Images are transmitted to the host computer in real time via a gigabit Ethernet interface, with a single frame transmission time of no more than 1ms. Upon receiving the image, the host computer performs image preprocessing in parallel. This process includes grayscale equalization (eliminating uneven lighting), noise filtering (suppressing random noise from the image sensor), and sub-pixel contour extraction (improving edge localization accuracy to the sub-pixel level through interpolation algorithms). The entire image preprocessing process takes no more than 4ms.

[0050] In the deviation calculation stage, the optimal rotation matrix and translation vector are solved by the weighted global-local rigid registration objective function, and the deviation calculation time does not exceed 5ms.

[0051] At the communication link level, a hard real-time Ethernet UDP protocol is used as the data transmission channel between the host computer and the servo drive. This protocol employs a time-aware shaper for traffic scheduling at the data link layer, ensuring that compensation commands have deterministic low latency and bounded jitter during network transmission. The compensation commands are encapsulated in the servo drive's native register instruction format at the host computer and directly written to the corresponding register address, eliminating the need for additional protocol conversion or instruction parsing at the driver end. Data transmission latency does not exceed 2ms.

[0052] The aforementioned visual acquisition, image preprocessing, bias calculation, time delay prediction compensation, and servo command issuance stages all employ a pipelined parallel processing approach, with visual acquisition and compensation command issuance from the previous cycle overlapping in time. Specifically, while the acquisition and preprocessing of the current batch of images is underway, bias calculation and time delay prediction compensation of the previous batch of images may be in progress, while compensation commands from an earlier batch may be being transmitted over the communication link. This pipelined approach ensures continuous throughput, guaranteeing that the total process time does not exceed 15ms.

[0053] With the aforementioned hardware configuration and communication link guarantee, the final pin compensation positioning accuracy is stable at ±0.015mm, meeting the requirements for automated assembly of high-precision connectors.

[0054] FPGA synchronous triggering compresses the acquisition timing deviation of multiple cameras to within 300ns, far smaller than the movement step distance of the pin insertion device within a control cycle. This ensures that the pin coordinates and insertion hole coordinates originate from the spatial state at the same moment, avoiding false deviations caused by inconsistent acquisition timing. The sub-pixel contour extraction algorithm improves edge positioning accuracy to the sub-pixel level, making visual measurement accuracy significantly better than the limitations of sensor physical resolution, providing reliable coordinate input for the final compensation accuracy of ±0.015mm. The hard real-time UDP protocol, combined with a time-aware shaper, ensures deterministic low-latency transmission of compensation commands at the data link layer. Compared to traditional serial communication solutions (transmission latency ≥40ms), this solution reduces data transmission latency to within 2ms, a compression of over 95%, directly supporting high-speed continuous pin insertion. Native register instruction encapsulation eliminates servo-side protocol conversion latency, making the time from the arrival of the compensation command at the servo network port to its writing to the driver register almost zero, avoiding the additional uncertainty latency caused by protocol stack parsing and instruction re-encapsulation in traditional solutions. The pipeline parallel processing method enables the completion of a complete vision-compensation closed loop within each pin insertion cycle, rather than executing acquisition, calculation and distribution serially, which effectively improves the throughput per unit time and supports high-speed continuous pin insertion production of more than 60 pins per minute.

[0055] Preferably, the cached standard product template stores the theoretical reference coordinates, temperature compensation curves, and coefficients for each connector model. ,coefficient ,coefficient and preset safety thresholds; During initial production, the above parameters are saved to the local cache. When changing models, the corresponding cache template is directly loaded, and the fine-tuning time is ≤0.8ms. The cache template is updated under the following conditions: when the deviation drift exceeds 0.015mm during continuous production, or after a change in material specifications or a major equipment overhaul, the baseline is automatically re-collected and the cache template is updated. When the final compensation amount of a single calculation exceeds the preset safety threshold, the host computer determines that the incoming material is seriously misaligned, automatically sends a stop or rejection control signal, and saves the current visual image and compensation log.

[0056] Specifically, this application establishes a standardized cache reuse mechanism for product template management. When a certain model of connector is produced for the first time, the following parameters are saved to the local cache: the theoretical reference coordinate set of the product model, the temperature compensation curve (i.e., a mapping table of thermal drift compensation coefficients at different temperatures), the dynamic rate compensation coefficient α (determined by the servo response bandwidth), the static thermal drift compensation coefficient β (obtained by thermal expansion test calibration of metal materials), the thermal expansion characteristic length coefficient L (determined by mechanical structure parameters), and the preset safety threshold (upper limit of compensation amount).

[0057] When the production line switches to a product of the same specification that has already been produced, the corresponding template parameters are loaded directly from the local cache, eliminating the need for manual benchmark calibration. After loading, only minor adjustments to the visual coordinates under the current operating conditions are required to begin production, with the adjustment taking no more than 0.8ms.

[0058] The cached template is not static; rather, it is dynamically updated under specific conditions. The template update procedure is automatically triggered in the following three situations: First, during continuous production, the accumulated deviation drift exceeds 0.015mm, indicating a significant change in equipment status or material characteristics; second, there is a clear change in material specifications (such as supplier change, batch differences, etc.), rendering the original template benchmark inapplicable; third, after a major equipment overhaul or replacement of key components, the mechanical coordinate system is reset. In these cases, the visual benchmark is automatically re-acquired, new standard template parameters are calculated, and the cache is updated to ensure that the template always matches the actual production conditions.

[0059] In terms of handling abnormal compensation amounts, a safety threshold protection mechanism is set up. When the final compensation amount issued in a single calculation exceeds the preset safety threshold, it indicates that there is a serious deviation in the incoming material (such as severe deformation of the terminal material, loose positioning fixture, etc.). If the insertion of the pin is forcibly performed in this case, it will inevitably lead to bent pins or damage to the socket. In this case, the host computer determines that the incoming material is seriously misaligned, automatically issues a stop command to suspend the production line operation, or issues a rejection control signal to discharge the current material to the waste box, while saving the current visual image and compensation log for subsequent analysis.

[0060] Product template caching and reuse reduces changeover time from 10-30 minutes in traditional solutions to seconds, with fine-tuning taking no more than 0.8ms, completely eliminating changeover bottlenecks in flexible multi-product production. There's no need to recalibrate baseline coordinates and adjust compensation parameters for each product, significantly lowering the technical barrier and error probability of changeover operations. The automatic template update mechanism ensures that the cached data is not a static historical snapshot, but an adaptive template that dynamically evolves with changes in equipment status. When long-term operation leads to increased mechanical clearance, seasonal changes in ambient temperature cause thermal expansion baseline drift, or replacement of key components leads to coordinate system reset, automatic template updates ensure that compensation accuracy does not degrade due to these factors. The compensation over-limit protection mechanism is the last line of defense against equipment damage. In traditional solutions, forcibly inserting severely misaligned materials inevitably leads to serious quality accidents such as pin bending and socket cracking, and may even damage precision molds and servo mechanisms. This solution identifies risky materials by determining if the compensation amount exceeds the limit before the pressing action is performed, and automatically implements shutdown or rejection to minimize material and equipment losses. At the same time, the saved abnormal images and logs provide data support for subsequent incoming material quality control.

[0061] Preferably, after the cumulative production reaches the threshold of 5,000 products, visual reference resampling is automatically triggered to recalibrate the template reference coordinates to compensate for the slow drift caused by lead screw wear and thermal deformation. When the servo following error exceeds the threshold, a secondary micro-compensation is automatically applied. During continuous high-volume pin insertion operation, the compensation fluctuation is controlled within ±0.008mm; Production data is automatically archived and visualized, including real-time display of four-axis compensation curves, equipment temperature change curves, and calculation time for each stage. It also supports integration with the MES production management system for quality traceability and equipment wear analysis.

[0062] Specifically, this application addresses the slow drift problem during long-term continuous operation of equipment by establishing an automatic benchmark resampling mechanism based on a cumulative production threshold. Using the number of products as the cumulative unit, a visual benchmark resampling procedure is automatically triggered every 5000 products produced. During the resampling process, the servo module is controlled to move to the standard calibration position, and the vision camera collects the pin and socket coordinates under the current operating conditions. These coordinates are compared with the theoretical benchmark coordinates in the template, and the systematic deviations caused by current equipment lead screw wear, guide rail clearance changes, and frame thermal deformation are calculated. Based on this, the template benchmark coordinates are corrected. This resampling process is fully automated, requiring no manual intervention or production line downtime.

[0063] During the pin insertion process, the actual position feedback from the servo encoder is used as the input for closed-loop control. Specifically, the final compensation value calculated above is converted into a servo motion command position. After the servo driver drives the motor to the command position, the encoder feeds back the actual position to the host computer. The difference between the actual position and the command position is calculated, which is the servo following error. When this following error exceeds a preset threshold, a second micro-compensation value is automatically added in the next pin insertion cycle, and the following error is included in the deviation calculation of the next cycle for unified correction.

[0064] During continuous high-volume pin insertion, the compensation fluctuation is controlled within ±0.008mm through the above-mentioned automatic resampling and secondary micro-compensation dual-path correction.

[0065] In terms of production data management, the original visual deviation, servo compensation amount, pin insertion coordinates, and production cycle data for each pin are automatically archived to establish a complete product quality traceability chain. The visual display interface displays the four-axis compensation curves (showing the trend of compensation amount over time), equipment temperature change curve (monitoring the thermal status of the equipment), and the calculation time for each stage of a single cycle (visual acquisition time, image preprocessing time, deviation calculation time, communication transmission time, and total time) in real time.

[0066] The above data supports integration with the MES production management system, uploading compensation data, equipment status data, and quality data to the enterprise-level management platform for the quality management department to conduct product quality traceability and equipment wear trend analysis.

[0067] The automatic resampling mechanism every 5000 pieces upgrades long-term equipment drift compensation from "manual periodic calibration" to "automatic adaptive calibration," eliminating the need for manual periodic shutdowns for calibration. This mechanism is particularly suitable for automated production lines operating 24 / 7, effectively eliminating the risk of batch quality accidents caused by human error or untimely calibration. Real-time monitoring of servo following error and secondary micro-compensation form a dual closed loop—the outer loop is a visual compensation closed loop, and the inner loop is a servo position closed loop. The inner and outer loops work together to ensure that even when the servo itself has dynamic response errors, the final pin position remains accurate. The compensation fluctuation is controlled within ±0.008mm, meaning that a high degree of consistent output can be maintained in continuous mass production, avoiding the typical equipment processing problem of "first piece qualified, batch drift," and significantly reducing quality control costs. Real-time visualization of the four-axis compensation curve and equipment temperature curve allows operators to intuitively judge whether the equipment is operating normally. When the compensation curve shows a monotonically increasing trend, it can predict accelerated lead screw wear and arrange maintenance in advance; when the temperature curve shows an abnormal jump, it can predict cooling failure and handle it in time. MES integration capabilities enable compensation data to be incorporated into the enterprise-level quality control system, achieving seamless quality data flow from individual equipment to whole-plant management, and providing a key equipment-level data source for the construction of intelligent manufacturing and digital workshops.

[0068] Preferably, the multi-channel vision camera includes a 2D area scan camera and / or a 3D structured light camera, which is compatible with different models of vision hardware. The image data transmission adopts a gigabit Ethernet interface, and the transmission time of a single frame image is ≤1ms. When using a 3D structured light camera, image processing also includes 3D point cloud reconstruction and Z-axis height extraction. The original visual deviation vector also includes Z-axis indentation depth deviation and translation vector. It also includes Z-axis translation compensation and rotation matrix. It is a 3×3 spatial rotation matrix; The synchronous triggering of multiple vision cameras, image processing, deviation calculation, time delay prediction compensation and servo command issuance adopts a pipelined parallel processing architecture. Vision acquisition and compensation command issuance in the previous cycle overlap in time and are continuously processed in a pipeline manner to ensure that a complete vision-compensation closed loop is completed in each insertion cycle.

[0069] Specifically, this invention supports the integration of various types of visual sensors, such as 2D area scan cameras and / or 3D structured light cameras, at the visual hardware level. 2D area scan cameras are suitable for XY-axis positioning of standard planar connectors, while 3D structured light cameras are suitable for precision terminals or irregularly shaped connectors requiring Z-axis height information. A unified visual data interface specification is provided, allowing different models of visual hardware to interact with the host computer through a standardized data format, achieving plug-and-play functionality without the need to develop dedicated drivers for different camera models.

[0070] Image data transmission uniformly adopts a gigabit Ethernet interface, and the transmission time of a single frame image does not exceed 1ms, ensuring that the transmission bandwidth and latency of image data from the camera to the host computer meet the requirements of high-speed production lines.

[0071] When using a 3D structured light camera, the image processing workflow adds 3D point cloud reconstruction and Z-axis height extraction steps to the existing grayscale equalization, noise filtering, and sub-pixel contour extraction. 3D point cloud reconstruction converts the coded pattern projected by the structured light into a set of spatial 3D coordinate points, and Z-axis height extraction calculates the height values ​​of the terminals and socket end faces from the point cloud. In this configuration, the original visual deviation vector, in addition to the existing X-axis translation deviation, Y-axis translation deviation, and θ-axis rotation deviation, adds a Z-axis indentation depth deviation. Correspondingly, the translation vector T is expanded into a 3D vector (containing X-axis, Y-axis, and Z-axis translation compensation amounts), and the rotation matrix R is expanded into a 3×3 spatial rotation matrix (capable of simultaneously compensating for in-plane rotation and spatial attitude tilt). The final compensation amount includes Z-axis servo compensation commands.

[0072] In terms of computational workflow organization, a pipelined parallel processing approach is adopted for each functional stage. Specifically, visual synchronous triggering acquisition, image transmission preprocessing, bias calculation, time delay prediction compensation, and servo command issuance are not executed sequentially but rather overlapped in a pipelined manner. While the acquisition and preprocessing of the current batch of images is underway, the bias calculation and time delay prediction compensation of the previous batch of images may be in progress, while compensation commands for an earlier batch may be being transmitted over the communication link. This pipelined approach allows a complete visual-compensation closed loop to be completed within each pin cycle, enabling continuous throughput without waiting for each other to complete.

[0073] The compatibility of 2D / 3D vision hardware enables this method to flexibly adapt to a wide range of products, from standard wire harness connectors to precision terminals for new energy FPCs. A single host computer software can cover various application scenarios in the field of connector automation assembly, eliminating the need to develop multiple independent software programs for different vision solutions. With the integration of a 3D structured light camera, the compensation dimension expands from three axes (XYθ) to four axes (XYθZ). This not only corrects in-plane alignment deviations but also precisely controls the insertion depth, preventing insulation damage due to over-pressure or excessive contact resistance due to under-pressure in new energy precision terminals during insertion, meeting micron-level assembly requirements. The pipeline parallel processing method overlaps the execution of vision acquisition, deviation calculation, and compensation distribution in time, significantly improving throughput without increasing the processing speed of individual stages. This ensures that each insertion cycle completes full closed-loop control within a total time constraint of ≤15ms, without reducing the production line cycle time to accommodate serial processing time. The unified visual data interface and standardized data format reduce the integration difficulty. The visual hardware configuration can be flexibly selected according to the actual accuracy requirements and cost budget. It can be adapted without structural modifications and has good promotion and application value.

[0074] For example, a specific implementation is described using a continuous automatic pin insertion process for a 12-pin wire harness connector.

[0075] In the visual data acquisition stage, the device is equipped with two 5-megapixel 2D area array cameras. An FPGA generates a unified external trigger pulse signal to synchronously trigger both cameras to acquire images of the terminal pins and connector sockets, with the acquisition timing deviation controlled within 250ns. The acquired images are uploaded to the host computer in real time via a gigabit Ethernet interface. The host computer performs grayscale equalization, noise filtering, and sub-pixel contour extraction in parallel, outputting 24 sets of pin and socket reference coordinates (12 pins corresponding to 12 sockets). Image preprocessing takes 3.2ms.

[0076] In the multi-dimensional deviation compensation calculation stage, the host computer, based on the pin number of the current insertion process, solves the globally optimal rotation matrix and translation vector pin by pin using a weighted global-local rigid registration model. This ensures that the current pin to be inserted and its neighboring pins dominate the rigid registration, while the far pins maintain a lower global reference weight to maintain the overall array geometric consistency. Simultaneously, the device's lead screw temperature data is collected, and dynamic thermal rate compensation and static thermal drift compensation terms are superimposed in the time-delay prediction compensation, generating 24 sets of independent four-axis servo compensation parameters (including X-axis translation, Y-axis translation, θ rotation, and Z-axis indentation depth compensation). The deviation calculation takes 4.5ms, and the single-pin compensation accuracy reaches 0.012mm.

[0077] During the servo command issuance phase, the host computer encapsulates the final issued compensation amount into the servo driver's native register instruction format via a hard real-time Ethernet UDP channel and directly writes it to the corresponding register address of the quadcopter servo driver. The data link layer uses a time-aware shaper for traffic scheduling, with a data transmission latency of 1.6ms. During command issuance, the timestamps of the visual acquisition and command latching moments are synchronously latched to ensure a one-to-one correspondence between visual deviations and servo motion timing.

[0078] Regarding product template caching and reuse, when producing this 12-pin connector for the first time, the theoretical reference coordinates, temperature compensation curves, and various compensation coefficients are saved to the local high-speed cache. During mass production, the cached template is directly loaded, and only minor deviations are adjusted, with the adjustment taking 0.7ms.

[0079] In the closed-loop dynamic calibration stage of the pin insertion, the servo performs pin insertion and then collects the actual position feedback from the encoder. When the following error exceeds the threshold, secondary micro-compensation is automatically added. After a cumulative production of 5000 pieces, visual reference resampling is automatically triggered to compensate for lead screw wear and thermal deformation drift. When the compensation amount calculated in a single operation exceeds the preset safety threshold, it is determined that the incoming material is seriously misaligned, and defective materials are automatically rejected.

[0080] In terms of data visualization, the host computer interface displays the XY compensation curves of each needle, the temperature change curve of the equipment, and the calculation time of each stage in a single cycle in real time.

[0081] After 12 hours of continuous production line operation verification, the entire process of vision acquisition, image preprocessing, deviation calculation, time delay prediction compensation, and servo command issuance took a total of 13.1ms. The defect rate of inserted pins and bent pins was reduced by 95%, meeting the high-speed insertion cycle requirement of 55 pieces / minute. Template buffer reuse reduced the changeover fine-tuning time to only 0.7ms, shortening the changeover time from 10-30 minutes in the traditional solution to the second level. Automatic resampling of 5000 pieces and dual-channel calibration with secondary micro-compensation enabled continuous operation for 12 hours without manual intervention, significantly reducing material loss and manual calibration costs.

[0082] For example, the specific implementation is described using the precision pin insertion process of terminals in new energy FPC (flexible printed circuit board) as an example.

[0083] In terms of visual acquisition hardware configuration, this embodiment uses a miniature 3D structured light camera instead of a 2D area array camera. The image processing flow, based on grayscale equalization, noise filtering, and sub-pixel contour extraction, adds three-dimensional point cloud reconstruction and Z-axis height extraction steps to accurately calculate the height difference between the terminal insertion surface and the connector socket end face from the point cloud. In addition to X-axis translational deviation, Y-axis translational deviation, and θ-axis rotational deviation, the original visual deviation vector adds a Z-axis indentation depth deviation. The translation vector is expanded to a three-dimensional vector containing the X, Y, and Z axes, and the rotation matrix is ​​expanded to a 3×3 spatial rotation matrix, which can simultaneously compensate for in-plane rotation and spatial attitude tilt.

[0084] In response to the easily deformable nature of FPC thin materials, this embodiment further tightens the judgment threshold in the outlier adaptive rejection logic, and more strictly identifies and rejects false deviation coordinates caused by material bending and warping.

[0085] Regarding product template management, this embodiment expands the multi-specification FPC product template library, supporting one-click switching of more than 30 terminal models. The theoretical reference coordinates, temperature compensation curves, and various compensation coefficients for each model are pre-stored in the local high-speed cache, and the corresponding template is directly loaded when changing models. When the continuous production deviation drift exceeds 0.015mm, material specifications change, or equipment is overhauled, the template update program is automatically triggered.

[0086] In other aspects, this embodiment is consistent with Embodiment 1: FPGA synchronously triggers multi-camera acquisition, with timing deviation controlled within 300ns; four-axis servo compensation parameters are independently generated pin-by-pin segment through weighted rigid registration; compensation commands are sent with low latency through hard real-time UDP protocol; servo position feedback is collected in real time and superimposed with secondary micro-compensation; reference resampling is automatically triggered after 5000 units have been produced; and the four-axis compensation curve and equipment temperature are visualized.

[0087] In this embodiment, the entire process of visual acquisition, image preprocessing, 3D point cloud reconstruction, deviation calculation, time delay prediction compensation, and servo command issuance takes a total of 14.3ms. The planar compensation accuracy reaches 0.011mm, and the Z-axis indentation depth control accuracy is better than 0.015mm. The integration of a 3D structured light camera expands the compensation dimension from three axes (XYθ) to four axes (XYθZ), enabling not only in-plane alignment deviation correction but also precise control of the indentation depth. For precision terminals in new energy applications, excessive indentation will lead to damage to the FPC insulation layer and abnormal contact resistance, while insufficient indentation will result in unreliable contact. Precise Z-axis control effectively avoids these two types of quality defects. The abnormality judgment threshold is tightened to address the easily deformable nature of FPC materials, preventing false coordinates caused by material bending or warping from contaminating the rigid registration results and improving the robustness of this method in flexible material scenarios. The multi-specification FPC product template library supports one-click switching of more than 30 terminal models, with a model change and fine-tuning time of only 0.9ms, fully meeting the flexible production needs of the new energy industry for multiple varieties and small batches. The total time for the entire process was 14.3ms, which is better than the design target of ≤15ms. This proves that the method still maintains high precision and high speed under more stringent micron-level assembly requirements and can be adapted to the future needs of higher precision connector assembly.

[0088] Example 2 This embodiment provides a host computer control system for automatic compensation of pin insertion devices based on vision, applied to the host computer control method for automatic compensation of pin insertion devices based on vision provided in Embodiment 1 above. See also Figure 2 As shown, the host computer control system for automatic compensation of pin insertion devices based on vision provided in this application may include the following modules.

[0089] The visual data acquisition module is used to synchronously trigger multiple vision cameras to acquire images of terminal pins and connector sockets, complete image preprocessing and sub-pixel contour extraction, output the actual coordinates of the pins and the theoretical reference coordinates of the sockets, and latch the visual trigger time. The deviation compensation calculation module, connected to the visual data acquisition module, is used to construct a weighted global coordinate system based on the actual coordinates of the pins and the theoretical reference coordinates of the sockets. The objective function of local rigid registration is used to iteratively solve the global optimal rotation matrix and translation vector to obtain the original visual deviation vector, which is then used to obtain the Z-axis height deviation of the terminal and generate the Z-axis indentation depth compensation amount. The time delay prediction and compensation module is connected to the visual data acquisition module and the deviation compensation calculation module respectively. It is used to read the servo speed feedback value and the equipment temperature change rate within the time window from the visual trigger moment to the compensation command latching moment, calculate the autonomous drift of the equipment in advance, and superimpose the pre-compensation amount with the original visual deviation vector to generate the final issued compensation amount. The servo real-time transmission module, connected to the time delay prediction and compensation module, is used to transmit the final transmission compensation amount to the multi-axis servo driver through a real-time Ethernet channel, and synchronously latch the vision acquisition timestamp and servo motion timing. The template cache reuse module, connected to the deviation compensation calculation module, is used to store and call standard product templates. When changing models, the cached template is loaded and visual fine-tuning is performed. The closed-loop dynamic correction module is connected to the servo real-time transmission module to collect the actual position feedback of the servo, superimpose secondary micro-compensation based on the following error, and trigger visual reference resampling after the cumulative production reaches the threshold. The data visualization and alarm module, connected to the closed-loop dynamic correction module, is used to store the pin compensation data and visualize the compensation trend and calculation time. When the total time, compensation amount or following error exceeds the threshold, an alarm is triggered and abnormal information is retained.

[0090] The host computer control system based on vision-based automatic compensation of the pin insertion device provided in this application embodiment can be applied to the host computer control method based on vision-based automatic compensation of the pin insertion device provided in the above embodiment 1. For relevant details, please refer to the above method embodiment. The implementation principle and technical effect are similar, and will not be repeated here.

[0091] It should be noted that the host computer control system based on vision-based automatic compensation of the pin insertion device provided in this embodiment is only illustrated by the above-mentioned division of functional modules / functional units. In practical applications, the above functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the host computer control system based on vision-based automatic compensation of the pin insertion device can be divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the host computer control method based on vision-based automatic compensation of the pin insertion device provided in the above-mentioned method embodiment one and the implementation method of the host computer control system based on vision-based automatic compensation of the pin insertion device provided in this embodiment two belong to the same concept. For the specific implementation process of the host computer control system based on vision-based automatic compensation of the pin insertion device provided in this embodiment two, please refer to the above-mentioned method embodiment one, which will not be repeated here.

[0092] Example 3 One embodiment of this application also provides a computer device, including but not limited to a processor and a memory, wherein the memory stores computer instructions, and the processor executes the computer instructions to implement the method described above.

[0093] One embodiment of this application also provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the method described in the above-described method embodiments.

Claims

1. A host computer control method for automatic compensation of pin insertion devices based on vision, characterized in that, include: Simultaneously trigger multiple vision cameras to acquire images of terminal pins and connector sockets, extract the actual coordinates of the pins and the theoretical reference coordinates of the sockets through image processing, and lock the vision trigger time; Based on the actual coordinates of the pins and the theoretical reference coordinates of the socket, a weighted global coordinate system is constructed. The objective function of local rigid registration is used to iteratively solve for the global optimal rotation matrix and translation vector to obtain the original visual deviation vector; at the same time, the Z-axis height deviation of the terminal is obtained to generate the Z-axis indentation depth compensation amount. Within the time window from the visual trigger moment to the compensation instruction latching moment, the servo speed feedback value and the equipment temperature change rate are read, the autonomous drift of the equipment is calculated in advance, and the pre-compensation amount is superimposed with the original visual deviation vector to generate the final compensation amount. The final compensation amount is sent to the multi-axis servo driver via a real-time Ethernet channel, and the visual acquisition timestamp and servo motion timing are latched synchronously. The cached standard product template is reused as a baseline, and the cached template is loaded and visually fine-tuned when the product is changed. During the pin insertion process, the actual position feedback of the servo is collected, and secondary micro-compensation is superimposed based on the following error; after the cumulative production reaches the threshold, visual reference resampling is triggered. Store pin compensation data and visualize compensation trends and calculation time; When the total time, compensation amount, or tracking error exceeds the threshold, an alarm is triggered and abnormal information is retained.

2. The host computer control method for automatic compensation of pin insertion devices based on vision according to claim 1, characterized in that, The objective function for the weighted global-local rigid registration is: Among them, dynamic weights , The current pin number and , It is the attenuation coefficient and its value ranges from 0.5 to 1.2; In a multi-pin continuous segmented insertion scenario, the current insertion sequence number The process is dynamically updated as the process progresses, and the weight of each pin is recalculated and the objective function is re-solved before each pin insertion. Before iteratively solving, calculate the initial residuals for each pin. and the mean of all pin residuals with standard deviation ,like Then the weights of the outliers will be... Set to zero to eliminate false deviation coordinates caused by material burrs or deformation; The rotation matrix Include Rotation compensation amount, the translation vector Includes X-axis and Y-axis translation compensation.

3. The host computer control method for automatic compensation of pin insertion devices based on vision according to claim 2, characterized in that, The final compensation amount The calculation formula is: In the formula, , The original visual deviation vector, For servo encoders in time window Internal real-time feedback velocity vector, This refers to the real-time temperature change rate of the equipment. To accumulate temperature difference, It is the characteristic length coefficient of thermal expansion. The dynamic rate compensation coefficient and The value range is 0 to 1.

0. This is the static thermal drift compensation coefficient; and Global timestamp unification is achieved based on the IEEE 1588 precision time protocol, mapping the visual exposure time, temperature sampling time, and servo encoder position latching time to the same clock source. The calculation error is ≤1μs.

4. The host computer control method for automatic compensation of pin insertion devices based on vision according to claim 3, characterized in that: The multi-channel vision camera is synchronously triggered by the FPGA, and the image acquisition timing deviation is ≤300ns; The image processing includes grayscale equalization, noise filtering and subpixel contour extraction, with a preprocessing time of ≤4ms; Deviation calculation time ≤ 5ms; The real-time Ethernet channel adopts the hard real-time UDP protocol, the data link layer uses a time-aware shaper for traffic scheduling, the compensation instruction is encapsulated as the servo driver native register instruction format, and directly written into the driver register to eliminate protocol parsing delay, and the data transmission delay ≤ 2ms; The total time spent on the entire process ≤15ms, the final pin compensation positioning accuracy stabilizes at ±0.015mm.

5. The host computer control method for automatic compensation of pin insertion devices based on vision according to claim 4, characterized in that: The cached standard product template stores the theoretical reference coordinates, temperature compensation curves, and coefficients for each connector model. ,coefficient ,coefficient and preset safety thresholds; During initial production, the above parameters are saved to the local cache. When changing models, the corresponding cache template is directly loaded, and the fine-tuning time is ≤0.8ms. The cache template is updated under the following conditions: when the deviation drift exceeds 0.015mm during continuous production, or after a change in material specifications or a major equipment overhaul, the baseline is automatically re-collected and the cache template is updated. When the final compensation amount calculated in a single operation exceeds the preset safety threshold, the host computer determines that the incoming material is seriously misaligned, automatically issues a stop or rejection control signal, and saves the current visual image and compensation log.

6. The host computer control method for automatic compensation of pin insertion devices based on vision according to claim 5, characterized in that: The cumulative production threshold is reached when visual reference resampling is automatically triggered after every 5,000 products are produced, and the template reference coordinates are recalibrated to compensate for the slow drift caused by lead screw wear and thermal deformation. When the servo following error exceeds the threshold, a secondary micro-compensation is automatically applied. During continuous high-volume pin insertion operation, the compensation fluctuation is controlled within ±0.008mm; Production data is automatically archived and visualized, including real-time display of four-axis compensation curves, equipment temperature change curves, and calculation time for each stage. It also supports integration with the MES production management system for quality traceability and equipment wear analysis.

7. The host computer control method for automatic compensation of pin insertion devices based on vision according to claim 6, characterized in that: The multi-channel vision camera includes a 2D area scan camera and / or a 3D structured light camera, which is compatible with different models of vision hardware. Image data transmission adopts a gigabit Ethernet interface, and the transmission time of a single frame image is ≤1ms. When using a 3D structured light camera, the image processing also includes 3D point cloud reconstruction and Z-axis height extraction. The original visual deviation vector also includes Z-axis indentation depth deviation, and the translation vector... It also includes Z-axis translation compensation, the rotation matrix It is a 3×3 spatial rotation matrix; The synchronous triggering of multiple vision cameras, image processing, deviation calculation, time delay prediction compensation, and servo command issuance adopts a pipelined parallel processing architecture. The vision acquisition and the issuance of the compensation command in the previous cycle are executed overlapping in time, and the throughput is continuously processed in a pipeline manner to ensure that a complete vision-compensation closed loop is completed in each insertion cycle.

8. A host computer system for automatic compensation of pin insertion devices based on vision, characterized in that, include: The visual data acquisition module is used to synchronously trigger multiple vision cameras to acquire images of terminal pins and connector sockets, complete image preprocessing and sub-pixel contour extraction, output the actual coordinates of the pins and the theoretical reference coordinates of the sockets, and latch the visual trigger time. The deviation compensation calculation module, connected to the visual data acquisition module, is used to construct a weighted global coordinate system based on the actual coordinates of the pins and the theoretical reference coordinates of the jacks. The objective function of local rigid registration is used to iteratively solve the global optimal rotation matrix and translation vector to obtain the original visual deviation vector, which is then used to obtain the Z-axis height deviation of the terminal and generate the Z-axis indentation depth compensation amount. The time delay prediction and compensation module is connected to the visual data acquisition module and the deviation compensation calculation module respectively. It is used to read the servo speed feedback value and the equipment temperature change rate within the time window from the visual triggering time to the compensation command latching time, calculate the autonomous drift of the equipment in advance, and superimpose the pre-compensation amount with the original visual deviation vector to generate the final issued compensation amount. The servo real-time transmission module is connected to the time delay prediction and compensation module and is used to transmit the final transmission compensation amount to the multi-axis servo driver through a real-time Ethernet channel, and synchronously latch the visual acquisition timestamp and servo motion timing. The template cache reuse module is connected to the deviation compensation calculation module and is used to store and call standard product templates. When changing models, the cached template is loaded and visual fine-tuning is performed. The closed-loop dynamic correction module is connected to the servo real-time transmission module. It is used to collect the actual position feedback of the servo, superimpose secondary micro-compensation based on the following error, and trigger visual reference resampling after the cumulative production reaches the threshold. The data visualization and alarm module is connected to the closed-loop dynamic correction module. It is used to store the pin compensation data and visualize the compensation trend and calculation time. When the total time, compensation amount or following error exceeds the threshold, an alarm is triggered and abnormal information is retained.

9. A computer device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to implement the host computer control method for automatic compensation of the pin insertion device based on vision, as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the host computer control method for automatic compensation of the pin insertion device based on vision as described in any one of claims 1-7.

Images