Connection deviation detection method and device based on machine vision

Abstract

The present application relates to the field of electronic connector, especially to a connection deviation detection method and device based on machine vision, comprising the following steps: collecting initial environment parameters, substituting into a preset environment and deviation compensation model to obtain initial environment compensation deviation, collecting initial state image, combining the initial environment compensation deviation to obtain final initial deviation; synchronously collecting vision data, force data and environment data, respectively obtaining real-time force push deviation and real-time environment compensation deviation, combining the vision data to obtain compensated vision deviation; outputting global fusion deviation through federated Kalman filtering algorithm, outputting deviation prediction value in future set time through pre-trained LSTM deviation trend prediction model; comparing the global fusion deviation and the deviation prediction value with a preset safety threshold, if the global fusion deviation or the deviation prediction value is out of limit, triggering adjustment mechanism. The present application effectively eliminates the influence of unexpected factors on the detection result through scene-based environment sensing and compensation mechanism.

Description

Technical Field

[0001] This invention relates to the field of electronic connectors, and more particularly to a method and apparatus for detecting connection deviations based on machine vision. Background Technology

[0002] In the fields of industrial automation and special equipment, floating connectors, as core components of electrical transmission docking systems, primarily function to achieve stable electrical signal and energy transmission between devices. With the advancement of deep-water exploration to depths of thousands of meters, the increasing demands for miniaturization and reliability in space exploration missions, and the growing need for long-term service in high-temperature radiation environments, the docking scenarios for floating connectors are becoming increasingly complex. These scenarios not only require connectors to possess environmental adaptability such as waterproofing, radiation resistance, and high-temperature resistance, but also impose stringent requirements on the "accuracy of connection deviation detection": machining tolerances during manufacturing and assembly deviations during installation can lead to axial and radial connection deviations during docking. If these deviations exceed the allowable range, additional contact forces will be generated. When the contact force exceeds the connector's rated load-bearing threshold, it can directly cause connector pin bending, shell deformation, or damage to the internal insulation layer, leading to electrical transmission interruption, equipment shutdown, and even major risks such as leakage in deep-water equipment and failure of space probe missions.

[0003] Currently, the mainstream technical solution for detecting connection deviations in floating connectors in the industry is "preset floating reference + force sensing feedback". The specific implementation of this solution is as follows: the socket mounting base of the floating connector is designed as a movable floating mechanical structure, which serves as the deviation simulation reference; during the mating process, axial force sensors and radial force sensors fixed between the floating mounting base and the equipment base are used to collect the contact force signals received by the socket in real time; based on a preset linear empirical formula of "force-deviation", the magnitude of the current connection deviation is calculated, thereby determining whether the mating position needs to be adjusted.

[0004] However, existing solutions suffer from core problems in practical applications, namely insufficient robustness and accuracy under multi-dimensional interference, making it difficult to meet the high-precision detection requirements of special scenarios. Firstly, the solution fails to consider environmental interference in special scenarios. For example, deep-water pressure can cause unexpected deformation of the floating mounting base, which, when superimposed on the connection deviation, affects detection; high temperatures can cause thermal expansion of the connector's metal shell, altering the initial mating gap and amplifying errors; vibration can introduce significant noise into the contact force signal collected by the force sensor, leading to fluctuations in the calculated deviation. Secondly, the linear force-deviation model and single-force sensing relied upon by the solution have limitations. When the contact force exceeds the linear range, the model will produce calculation errors; when the plug obstructs the socket's positioning features during mating or the force signal is weak in the initial contact stage, single-force sensing will fail in deviation calculation. Ultimately, due to inaccurate deviation judgment, the connector damage rate increases, and the system's operational reliability is significantly reduced.

[0005] Therefore, it is necessary to design a connection deviation detection method and device based on machine vision. Summary of the Invention

[0006] To address the technical deficiencies in the background art, this invention proposes a solution that solves the aforementioned technical problems and meets practical needs. The specific technical solution is as follows:

[0007] The machine vision-based connection deviation detection method includes the following steps:

[0008] The preset vision module and force sensor module are calibrated, and the corresponding environmental sensor is selected for calibration based on the target scene;

[0009] Initial environmental parameters are collected and substituted into a preset environmental and deviation compensation model to obtain the initial environmental compensation deviation. Initial state images of the plug and socket are collected, and the initial axial distance and radial offset are calculated. The final initial deviation is obtained by combining the initial environmental compensation deviation.

[0010] Simultaneously collect visual data, force data, and environmental data. Real-time force push deviation and real-time environmental compensation deviation are obtained through pre-trained adaptive nonlinear deviation and force mapping model and environment and deviation compensation model, respectively. Combined with visual data, the compensated visual deviation is obtained.

[0011] Based on real-time force push deviation, real-time environmental compensation deviation, and post-compensation visual deviation, a global fusion deviation is output through a federated Kalman filter algorithm. Based on the global fusion deviation, a pre-trained LSTM deviation trend prediction model is used to output the deviation prediction value within a set time period in the future.

[0012] The global fusion deviation and the predicted deviation value are compared with the preset safety threshold. If the global fusion deviation or the predicted deviation value exceeds the limit, the adjustment mechanism is triggered.

[0013] Furthermore, the target scenarios include deep water scenarios, high temperature scenarios, and vibration scenarios. The environmental sensors include a water pressure sensor, an infrared temperature sensor, and a triaxial accelerometer. When the target scenario is a deep water scenario, a water pressure sensor is selected and calibrated using a standard water pressure device; when the target scenario is a high temperature scenario, an infrared temperature sensor is selected and calibrated using a constant temperature chamber; when the target scenario is a vibration scenario, a triaxial accelerometer is selected and calibrated using a standard vibration table.

[0014] When calibrating the vision module, a standard checkerboard calibration board is used, and the camera intrinsic parameters are calculated using Zhang's calibration algorithm;

[0015] When calibrating the force sensor module, a standard force of 0 to rated range is applied through a standard force loading device to establish a calibration curve of force value versus voltage.

[0016] Furthermore, the calculation formula for the environment and deviation compensation model is as follows:

[0017] When the target scene is a deep-water scene, the following formula is used:

[0018] ;

[0019] When the target scenario is a high-temperature scenario, the following formula is used:

[0020] ;

[0021] When the target scene is a vibration scene, the following formula is used:

[0022] ;

[0023] in, To compensate for axial environmental deviation. To compensate for radial X-axis environmental deviation. To compensate for the radial y-axis environmental deviation, The hydrostatic deformation coefficient of the connector material. For real-time water pressure, The angle of action of water pressure, The coefficient of thermal expansion of the connector material. For real-time temperature, The axial length of the connector. Vibration-offset coefficient, For real-time vibration acceleration, The real-time vibration acceleration along the x-axis. This represents the real-time vibration acceleration along the y-axis.

[0024] Furthermore, the specific steps to obtain the final initial deviation are as follows:

[0025] The initial state image was preprocessed, and an edge detection algorithm was used to extract the center of the socket positioning hole, the center of the plug end face, and the socket edge marker points. The coordinates of the above feature points in the image coordinate system were recorded.

[0026] Convert the coordinates in the image coordinate system to the actual physical coordinate system coordinates, and calculate the initial axial distance and initial radial offset;

[0027] The final initial deviation is obtained by subtracting the initial environmental compensation deviation from the initial axial distance and initial radial offset.

[0028] Furthermore, the adaptive nonlinear deviation and force mapping model includes an input layer, a hidden layer, and an output layer. The input layer contains three neurons for inputting three vectors of force data. The hidden layer contains several neurons for converting the force data from the input layer into deviation data. The output layer contains three neurons for outputting three vectors corresponding to the real-time force deviation.

[0029] Furthermore, the federated Kalman filtering algorithm operates the Kalman filtering independently through three sub-modules, outputting local bias and the corresponding local error covariance;

[0030] The global fusion bias is obtained by weighting the weighted calculation based on the local error covariance.

[0031] The global fusion bias is fed back to the three sub-modules to correct the initial parameters of their local filtering.

[0032] The three sub-modules include a vision sub-filtering module, a force sensing sub-filtering module, and an environmental compensation sub-filtering module, as shown in the following formula:

[0033] The prediction equations for the three sub-modules are as follows:

[0034] ;

[0035] The update equations for the three sub-modules are as follows:

[0036] ;

[0037] in, , and These are the state prediction values ​​for the vision submodule, the force sensing sub-filtering module, and the environmental compensation sub-filtering module, respectively. , and The corresponding prediction error covariance, As a strong tracking factor, , and The process noise covariance of the vision submodule, force sensing sub-filtering module, and environmental compensation sub-filtering module are respectively defined. For Kalman gain, For the observation matrix, To observe the noise covariance matrix, It is the identity matrix. Observation matrix The transpose of the matrix, Let be the observation vector at the current moment. This represents the final deviation value after fusion at time k. This is the predicted value of the connector's true deviation at time k, based on the detection results at time k-1. This is the updated error covariance.

[0038] Furthermore, the formula for calculating the global fusion bias using the federated Kalman filter algorithm is as follows:

[0039] ;

[0040] ;

[0041] in, This is a global fusion bias. To globally integrate covariance, , and Let be the error covariances of the vision sub-module, the force sensing sub-filter module, and the environmental compensation sub-filter module at time k, respectively. , and These are the deviation values ​​of the vision submodule, force sensing sub-filter module, and environmental compensation sub-filter module at time k, respectively.

[0042] Furthermore, the deviation prediction formula of the LSTM deviation trend prediction model is as follows:

[0043] ;

[0044] in, Predicting time t Timing deviation, To predict the time step, The weight matrix consists of parameters obtained by training the model using historical data, used to quantify the influence of historical features on future biases. This is the historical feature vector at the current time t, which typically contains key data from t and several previous times. This is a bias term used to compensate for minor systematic errors during model training or data acquisition, thereby improving prediction accuracy.

[0045] Furthermore, the adjustment mechanism is specifically as follows:

[0046] When the global fusion deviation or deviation prediction value exceeds the preset safety threshold, the electric slide adjustment is triggered. When the deviation is small, fine-tuning is performed with small steps; when the deviation is large, rapid correction is performed with large steps.

[0047] Each time the electric slide completes an adjustment, the global fusion deviation and the predicted deviation value are recalculated to determine whether they still exceed the limits. If they do, the adjustment is repeated until the global fusion deviation and the predicted deviation value are both within the preset safety threshold.

[0048] A machine vision-based connection deviation detection device includes:

[0049] The vision module is used to acquire images of the plug and socket;

[0050] Force sensor module, used to collect contact forces during docking, including axial force sensor, x-axis force sensor and y-axis force sensor;

[0051] The environmental sensor module selects a water pressure sensor, an infrared temperature sensor, or a triaxial accelerometer based on the target scenario to collect real-time environmental parameters.

[0052] The control module is connected to the vision module, force sensor module, environmental sensor module and electric slide, and has a built-in preprocessing calibration unit, initial positioning unit, real-time fusion unit and predictive adjustment unit, which are used to calculate real-time deviation and send adjustment commands.

[0053] Compared with the prior art, the present invention provides the following beneficial effects:

[0054] This invention effectively eliminates the impact of unexpected factors on detection results through a scenario-based environmental sensing and compensation mechanism. It selects an appropriate environmental sensor based on the target scenario and performs precise calibration using a standard device to ensure reliable environmental parameter acquisition. Simultaneously, relying on a pre-set environmental and deviation compensation model, it converts real-time environmental parameters into corresponding environmental compensation deviations, correcting interference such as connector deformation caused by water pressure, thermal expansion gaps caused by high temperatures, and signal noise from vibration. This avoids misjudgments caused by the superposition of environmental factors on connection deviations, significantly improving the stability and reliability of connection deviation detection in complex and extreme scenarios, meeting the stringent robustness requirements of special fields such as deep-sea exploration and space exploration.

[0055] This invention employs a pre-trained adaptive nonlinear deviation and force mapping model, utilizing a network structure with multiple neurons to adapt to the nonlinear relationship between force and deviation, avoiding calculation errors caused by material deformation when the contact force is in the superlinear range. It constructs three independent sub-filtering modules—vision, force sensing, and environmental compensation—using a federated Kalman filter algorithm, dynamically allocating weights based on the local error covariance of each module. For example, the weight of the force sensor module is increased when vision is obstructed, and the weight of the vision module is increased when the force signal is weak, effectively avoiding the risk of single sensor failure. The combination of these two approaches significantly improves deviation detection accuracy, ensuring accurate deviation judgment during docking, reducing the probability of connector damage due to inaccurate detection, and solving the core pain points of insufficient accuracy and low reliability in existing technologies.

[0056] This invention utilizes a pre-trained LSTM deviation trend prediction model, combined with historical fused deviation data, to predict deviation trends within a set future timeframe. This proactively identifies the risk of deviation exceeding limits, avoiding the need for reactive adjustments after deviations exceed acceptable levels. Simultaneously, a tiered adjustment strategy is designed, selecting corresponding step sizes based on the degree of deviation exceeding limits to reduce the number of repeated adjustments. This design not only improves docking efficiency and avoids delays in automated production lines but also reduces mechanical wear on connector pins and housings caused by frequent adjustments, extending connector lifespan and resolving the efficiency and wear issues associated with reactive adjustments in existing technologies. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the connection deviation detection method based on machine vision in this invention.

[0058] Figure 2 This is a schematic diagram of a connection deviation detection device based on machine vision according to the present invention. Detailed Implementation

[0059] In the description of this invention, it should be understood that the terms "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "middle," and "inner," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, it should be noted that unless otherwise explicitly specified and limited, the terms "installed," "connected," and "joined" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention through specific circumstances.

[0060] The embodiments of the present invention will be described below with reference to the accompanying drawings and related examples. The embodiments of the present invention are not limited to the following examples, and the present invention relates to the relevant necessary components in this technical field, which should be regarded as well-known technology in this technical field and can be known and mastered by those skilled in this technical field.

[0061] See Figure 1This invention provides a connection deviation detection method based on machine vision, characterized by comprising the following steps:

[0062] Step S100: Calibrate the preset vision module and force sensor module, and select the corresponding environmental sensor for calibration based on the target scene;

[0063] The vision module comprises a high-resolution camera, an image acquisition card, and image processing algorithms. Its function is to capture the spatial position information during the plug-socket docking process in real time, which can be achieved through the integration of hardware devices and software algorithms. The force sensor module includes axial and radial force sensors. Its function is to convert contact force into electrical signals, which can be achieved through the physical properties of strain gauges or piezoelectric elements. The environmental sensor is a monitoring device selected according to the characteristics of the target scene. Its function is to collect environmental parameters, which is achieved through physical quantity sensing elements.

[0064] Step S200: Collect initial environmental parameters, substitute them into the preset environmental and deviation compensation model to obtain the initial environmental compensation deviation, collect the initial state images of the plug and socket, calculate the initial axial distance and radial offset, and combine the initial environmental compensation deviation to obtain the final initial deviation.

[0065] The environment and deviation compensation model is a pre-defined neural network model whose function is to map environmental parameters to environmental compensation deviation values. It can be obtained through training with historical experimental data or by derivation using physical formulas. The vision module acquires the initial image for 100ms, at which point the environmental sensor starts acquiring data, continuously collecting 10 sets of data and taking the average as the initial environmental parameters.

[0066] Step S300: Simultaneously collect visual data, force data, and environmental data. Obtain real-time force push deviation and real-time environmental compensation deviation through pre-trained adaptive nonlinear deviation and force mapping model and environment and deviation compensation model, respectively. Combine with visual data to obtain the compensated visual deviation.

[0067] The visual data consists of a real-time image sequence of the plug-socket mating process, acquired through continuous acquisition by a camera. The force data comprises electrical signals of axial and radial contact forces, obtained through signal conversion by a force sensor module. The environmental data consists of real-time environmental parameters, acquired through continuous monitoring by environmental sensors. The real-time environmental compensation deviation is the compensation value corresponding to the current environmental parameters; its function is to eliminate the interference of environmental changes on deviation measurement. The adaptive nonlinear deviation and force mapping model is a multi-layer neural network structure; its function is to learn the nonlinear relationship between the force signal and the actual deviation.

[0068] Step S400: Based on the real-time force push deviation, real-time environmental compensation deviation, and compensated visual deviation, the global fusion deviation is output through the federated Kalman filter algorithm. Based on the global fusion deviation, the deviation prediction value within a set time period is output through the pre-trained LSTM deviation trend prediction model.

[0069] Among them, the federated Kalman filter algorithm is a fusion framework containing three sub-filter modules. Its function is to integrate multi-source deviation data and reduce the impact of noise. The LSTM deviation trend prediction model is a time series prediction network. Its function is to predict the future deviation change trend and capture the temporal characteristics of historical deviation sequences through long short-term memory units.

[0070] Step S500: Compare the global fusion deviation and the predicted deviation value with the preset safety threshold. If the global fusion deviation or the predicted deviation value exceeds the limit, trigger the adjustment mechanism.

[0071] The safety threshold is a permissible deviation range set according to the connector's rated load capacity, determined through connector design specifications or experimental verification. The adjustment mechanism is the motion control scheme of the mating mechanism, whose function is to correct deviations and avoid the risk of exceeding limits.

[0072] This invention eliminates inherent errors by calibrating multi-module sensors, combines environmental compensation models and visual processing algorithms to eliminate environmental interference, synchronously collects multi-source data, and achieves high-precision fusion through an adaptive nonlinear model and federated Kalman filtering. It utilizes an LSTM model to predict deviation trends and trigger a hierarchical adjustment mechanism, achieving the technical effects of improving detection accuracy, enhancing system robustness, providing proactive early warning, and optimizing adjustment efficiency. This method solves the detection error problem caused by environmental interference in traditional solutions through multi-dimensional environmental perception and compensation mechanisms. Simultaneously, it addresses nonlinear characteristics and noise interference through multi-sensor data fusion and adaptive algorithms, ultimately meeting the core requirement of secure connector docking in complex scenarios.

[0073] In one embodiment of the present invention, the target scene includes a deep water scene, a high temperature scene, and a vibration scene. The environmental sensors include a water pressure sensor, an infrared temperature sensor, and a triaxial accelerometer. When the target scene is a deep water scene, a water pressure sensor is selected and calibrated using a standard water pressure device; when the target scene is a high temperature scene, an infrared temperature sensor is selected and calibrated using a constant temperature chamber; when the target scene is a vibration scene, a triaxial accelerometer is selected and calibrated using a standard vibration table.

[0074] The deep-water scenario involves the connector mounting base under water pressure in an underwater environment, with the pressure range quantified using standard devices such as high-pressure water chamber simulators. The high-temperature scenario involves the connector's metal shell surface temperature exceeding that of conventional environments, with the temperature range precisely controlled between 200-500℃ using equipment such as constant-temperature chambers. The vibration scenario involves the connector under dynamic loads due to mechanical vibration, with vibration parameters measured by applying sinusoidal vibrations with a frequency range of 10-1000Hz and an acceleration of 0-10g using a standard vibration table. A water pressure sensor is a physical quantity sensing element used to monitor the pressure exerted by water on the mounting base's deformation; its output signal can be represented as a voltage value. An infrared temperature sensor is a non-contact detection device based on the principle of thermal radiation to measure the surface temperature of an object; its temperature measurement value can be compensated for through synchronous comparison with a thermocouple. A triaxial accelerometer is a spatial multidimensional dynamic response element used to acquire vibration frequency and amplitude; its output signal can be analyzed for frequency domain characteristics using Fourier transform. A standard water pressure device is a device such as a high-pressure water chamber simulator capable of applying controllable pressure, with its pressure output range covering the sensor's rated operating range. A constant temperature chamber is an environmental control device that provides a precise temperature field; its temperature regulation accuracy must meet the temperature gradient requirements for sensor calibration. A standard vibration table is an experimental device that generates controllable vibration parameters; its output characteristics must meet the frequency and acceleration requirements of the vibration scenario.

[0075] This can be achieved by selecting environmental sensors that match the target scenario and calibrating them using dedicated calibration equipment, as follows:

[0076] When the target scenario is a deep water scenario, a pressure value within the range of 0 to the rated working water pressure is applied using a high-pressure water chamber simulator. The sensor output voltage and the standard pressure value are recorded simultaneously. A pressure-voltage calibration curve is established through linear regression or piecewise polynomial fitting, thereby eliminating the measurement deviation caused by the nonlinear response of the sensor.

[0077] When the target scenario is a high-temperature scenario, a precise and controllable temperature field is provided by a constant temperature chamber, which enables the infrared temperature sensor and thermocouple to measure the temperature value synchronously. Bayesian estimation is used to optimize the temperature compensation coefficient, thereby solving the measurement drift caused by sensor self-heating or environmental radiation interference.

[0078] When the target scenario is a vibration scenario, sinusoidal vibrations with different frequencies and accelerations are applied through a standard vibration table. The frequency domain response characteristics of the sensor are analyzed using Fourier transform, and a frequency domain compensation model of vibration acceleration and output signal is established, thereby improving the accuracy of vibration noise separation.

[0079] By selecting scene-appropriate sensors and using dedicated calibration equipment, the problem of insufficient environmental adaptability caused by general parameters in traditional calibration methods has been solved. For example, the specialized calibration of water pressure sensors can accurately quantify the elastic deformation of the mounting base in deep water environments, avoiding misjudgment of axial deviation caused by water pressure; the constant temperature chamber calibration of infrared temperature sensors eliminates temperature measurement deviation caused by sensor self-heating in high-temperature scenarios; and the vibration table frequency domain calibration of triaxial accelerometers improves the accuracy of vibration noise separation.

[0080] When calibrating the vision module, a standard checkerboard calibration board is used, and the camera intrinsic parameters are calculated using Zhang's calibration algorithm;

[0081] A standard checkerboard calibration board is a planar calibration tool containing a black and white checkerboard of standard size. Its geometry consists of a two-dimensional projection of a known three-dimensional coordinate matrix. Zhang's calibration algorithm is a mathematical method that calculates the camera intrinsic parameter matrix by detecting the corner pixel coordinates in the checkerboard image and combining this with the actual size of the checkerboard. Its output parameters include focal length, principal point coordinates, and distortion coefficients. For example, multiple images are captured by placing the checkerboard within the field of view at different angles and distances, extracting sub-pixel-level corner coordinates, minimizing reprojection error through nonlinear optimization, and finally outputting the calibrated camera parameters.

[0082] When calibrating the force sensor module, a standard force of 0 to rated range is applied through a standard force loading device to establish a calibration curve of force value versus voltage.

[0083] A standard force loading device is an experimental apparatus capable of applying stepped standard force values, the output of which must cover the sensor's rated measurement range. A calibration curve is a mathematical model describing the relationship between force and sensor output voltage, which can be constructed using least squares fitting or cubic spline interpolation. For example, stepped standard forces such as 0N, 50N, and 100N are applied, and the sensor's output voltage signal is recorded; a calibration curve is then established through linear or piecewise linear fitting. When the sensor exhibits a nonlinear response, cubic spline interpolation can reduce nonlinear errors. Simultaneously, the calibration process must be repeated under the target environment's temperature and vibration conditions. For instance, in a high-temperature environment, force loading and data recording are performed in a constant-temperature chamber to ensure the calibration curve covers the parameter variation range of the actual working environment. This operation, through the construction of an environment-related calibration curve, solves the measurement deviation problem caused by environmental parameter variations in traditional calibration methods.

[0084] The calibrated camera intrinsic parameters, force-voltage curves, and other parameters are automatically stored in the preprocessing calibration unit of the control module. The parameters of this unit are called in subsequent steps S200 (image coordinate transformation) and S300 (force data transformation).

[0085] This invention employs specialized environmental sensors for deep water, high-temperature, and vibration scenarios, along with a scenario-adaptive calibration strategy. By combining Zhang's calibration algorithm from the vision module with the environmental correlation calibration curve from the force sensor module, it achieves high consistency and scenario-specificity in measurement data from multiple sensors under complex conditions. Specialized equipment such as standard water pressure devices, constant temperature chambers, and vibration tables are used to establish a precise mapping relationship between sensor outputs and physical quantities, effectively eliminating the impact of environmental factors such as nonlinear response, self-heating interference, and vibration noise on measurement accuracy. Furthermore, sub-pixel-level corner detection and nonlinear optimization from the vision module, along with stepped force loading and piecewise interpolation modeling from the force sensor module, significantly improve the absolute accuracy of geometric position measurement and force sensing. These combined techniques ultimately enhance the system's detection accuracy and environmental interference resistance in extreme environments, resolving the scenario adaptability deficiencies caused by the universal calibration of sensors in traditional solutions.

[0086] In one embodiment of the present invention, the calculation formula of the environment and deviation compensation model is as follows:

[0087] When the target scene is a deep-water scene, the following formula is used:

[0088] ;

[0089] The derivation is based on Hooke's Law and axial deformation experiments of connector materials. When water pressure P is applied to the socket mounting base, the deformation of the mounting base conforms to... = (P・S)・L / (E・A) (where S is the area under force, E is the elastic modulus of the material, and A is the cross-sectional area), let α = (S・L) / (E・A), simplifying to .

[0090] When the target scenario is a high-temperature scenario, the following formula is used:

[0091] ;

[0092] This directly corresponds to the metal thermal expansion formula ΔL=β・L・ΔT, where T is actually the temperature difference between "real-time temperature and standard temperature (25℃)" and L is the effective axial length of the connector.

[0093] When the target scene is a vibration scene, the following formula is used:

[0094] ;

[0095] Based on the relationship between vibration acceleration and dynamic displacement, the connector offset ΔL under vibration conditions satisfies the relationship between the actual acceleration a and the actual offset ΔL = (a_actual) / (2πf). 2(where f is the vibration frequency), let γ = 1 / (2πf) 2 (f is a common frequency in the scene, such as γ≈5×10 when it is 50Hz) -6 mm / (m / s 2 )).

[0096] in, To compensate for axial environmental deviation and reflect the impact of deep-water pressure on the axial deformation of the connector on the test results, since the water pressure generates a uniform force along the docking axis, the axial deformation is only positively correlated with the magnitude of the water pressure. Therefore, the compensation amount is directly quantified by multiplying the water pressure and the material deformation coefficient.

[0097] To compensate for radial X-axis environmental deviation, reflecting the effect of water pressure on the connector's radial X-axis offset, when water pressure acts on the connector, it generates a component force in the X-axis direction due to the water pressure angle, thus causing radial offset. Therefore, it is necessary to introduce... Quantify the component force effect in the x-axis direction.

[0098] For radial y-axis environmental compensation deviation, the principle is the same as Consistent, passed The component force effect of water pressure in the y-axis direction is quantified to compensate for the radial offset in the y-axis direction.

[0099] This is the water pressure-deformation coefficient of the connector material, determined by the material properties. It is calibrated experimentally and reflects the axial deformation of the material under unit water pressure. The calibration method is as follows:

[0100] Take samples of the same material and structure as the connector to be tested. Using a standard hydraulic pressure device, apply pressure in steps of 0 MPa, 10 MPa, 20 MPa, ..., 50 MPa, holding each pressure level for 5 seconds. Measure the axial deformation of the sample under each pressure level using a laser displacement sensor. ,by x-axis Using the ordinate as the vertical axis, a straight line is fitted using the least squares method. =α・ +b (b is the zero drift error, usually ≤0.002mm), take the slope of the fitted straight line as the value of α.

[0101] The water pressure is real-time and is collected by a water pressure sensor, which needs to cover the water pressure range of the target scene. The water pressure action angle is the angle between the actual water pressure direction and the connector docking axis. It is preset by the scene installation layout and is used to calculate the radial component of the water pressure.

[0102] The coefficient of thermal expansion of the connector material is determined by the properties of the metallic material and reflects the amount of expansion per unit length of material under a unit temperature change. The calibration method is as follows:

[0103] Take a standard sample of the connector body metal material and place it in a constant temperature chamber. Starting from 25℃, increase the temperature at increments of 25℃, 50℃, 75℃, ..., 200℃ (covering the target scene temperature range), holding each temperature increment for 10 minutes. Measure the length of the sample at each temperature increment using a micrometer. According to β=( - ) / ( • (L0=100mm, =Current temperature - 25℃), take the average of multiple sets of data as the β value.

[0104] The temperature is real-time and is collected by an infrared temperature sensor, which needs to cover the temperature range of the target scene. The axial length is the connector's preset structural parameter. Since thermal expansion is positively correlated with the original length of the material, the expansion deviation needs to be quantified based on the axial length.

[0105] The vibration-offset coefficient is determined by the connector mounting structure characteristics and is calibrated experimentally to reflect the offset generated by the connector under unit vibration acceleration. The calibration method is as follows:

[0106] Fix the connector sample on a standard vibration table, ensuring the sample axis is aligned with the vibration direction. Set the vibration frequency to a common value for the target scenario and the acceleration to 0 m / s². 2 2m / s 2 4m / s 2 ... 10 m / s 2 The sample was subjected to loading, with each acceleration level held for 3 seconds. The vibration process was captured by a high-speed camera, and the maximum axial displacement under each acceleration level was calculated through image analysis. ,by x-axis Using the ordinate as the vertical axis, fit a straight line. =γ・ The slope is taken as the value of γ.

[0107] For real-time vibration acceleration, the axial vibration acceleration is collected by a triaxial accelerometer, covering the vibration range of the target scene; The real-time vibration acceleration along the x-axis. This represents the real-time vibration acceleration along the y-axis.

[0108] This invention achieves the technical effect of quantitatively correlating environmental parameters with connector structural characteristics by calculating axial and radial compensation values ​​in deep water scenarios by combining water pressure direction and material deformation characteristics, quantifying axial expansion based on material thermal expansion coefficient and temperature gradient in high-temperature scenarios, and separating interference displacement in each direction by vibration acceleration component and dynamic displacement coefficient in vibration scenarios.

[0109] In one embodiment of the present invention, the specific steps for obtaining the final initial deviation are as follows:

[0110] The initial state image was preprocessed, and an edge detection algorithm was used to extract the center of the socket positioning hole, the center of the plug end face, and the socket edge marker points. The coordinates of the above feature points in the image coordinate system were recorded.

[0111] Edge detection algorithms are computer vision algorithms used to extract object edge information from digital images. Their core principle is to identify the boundary contours between objects and the background by calculating the gradient changes in the grayscale values ​​of image pixels. In this scheme, the edge detection algorithm is used to accurately locate the circular boundary of the socket's positioning hole, the rectangular boundary of the plug end face, and the contours of the socket's edge markers, ensuring the accuracy of feature point coordinate extraction.

[0112] Convert the coordinates in the image coordinate system to the actual physical coordinate system coordinates, and calculate the initial axial distance and initial radial offset;

[0113] The image coordinate system is a three-dimensional rectangular coordinate system established based on the fixed reference point of the detection system.

[0114] The final initial deviation is obtained by subtracting the initial environmental compensation deviation from the initial axial distance and initial radial offset.

[0115] After acquiring the initial state images of the plug and socket, the images need to be preprocessed. Preprocessing includes grayscale conversion, Gaussian denoising, and contrast enhancement to eliminate the impact of image noise on feature point recognition. Then, an edge detection algorithm is used to accurately extract three types of key feature points from the preprocessed image: the center of the socket positioning hole, the center of the plug end face, and the socket edge markers. The coordinates of these feature points in the image coordinate system are recorded. The core purpose of this step is to select reference points that reflect the relative position of the plug and socket, avoiding inaccurate subsequent deviation calculations due to blurred or misidentified feature points. This is a prerequisite for initial deviation quantification.

[0116] Since the image coordinate system is only in pixels, it can only reflect the relative position of feature points on the imaging plane and cannot directly correspond to physical distances in real space. Therefore, it is necessary to combine the camera intrinsic parameters obtained from the vision module calibration and use a coordinate transformation algorithm to convert the image coordinates of feature points into actual physical coordinates. Then, based on the physical coordinates, the initial axial distance and initial radial offset are calculated, completing the key transformation from visual observation to physical quantification of the deviation.

[0117] Considering the environmental interference present in the initial positioning stage, directly using the initial axial distance and radial offset as the initial deviation would lead to the environmental interference being superimposed on subsequent detection processes. Therefore, the initial axial distance and initial radial offset calculated in the first two steps need to be subtracted from the initial environmental compensation deviation obtained through the environmental and deviation compensation model to obtain the final initial deviation. This deviation has eliminated the interference of initial environmental factors and only reflects the inherent relative positional deviation between the plug and socket due to manufacturing and installation, providing a reference starting point free from environmental interference for subsequent real-time deviation detection.

[0118] This invention improves feature extraction accuracy through image preprocessing and subpixel localization, eliminates geometric distortion and installation errors by using coordinate transformation driven by the camera intrinsic parameter matrix, and suppresses external interference by combining an algebraic elimination strategy for environmental compensation deviation, thus achieving high-resolution measurement of axial and radial deviations.

[0119] In one embodiment of the present invention, the adaptive nonlinear deviation and force mapping model includes an input layer, a hidden layer and an output layer. The input layer contains three neurons for inputting three vectors of force data. The hidden layer contains several neurons for converting the force data from the input layer into deviation data. The output layer contains three neurons for outputting three vectors corresponding to the real-time force push deviation.

[0120] The neurons in the input layer directly receive the raw data from the force sensor for the axial force, radial X-axis force, and radial Y-axis force of the stress data, respectively. The hidden layer contains 10 neurons, which are processed by weighted summation and activation function to extract the nonlinear features of the force signal. The three neurons in the output layer correspond to the axial deviation, radial X-axis deviation, and radial Y-axis deviation, respectively. The final force deviation value is obtained by weighted summation of the outputs from the hidden layer.

[0121] The output formula for the hidden layer is: ;

[0122] The output formula of the output layer is: ;

[0123] The activation function is: ;

[0124] The weight update formula for the adaptive nonlinear deviation and force mapping model is as follows:

[0125] ;

[0126] ;

[0127] ;

[0128] in, The output value of the hidden layer is the final output of the j-th neuron in the hidden layer. It is the feature vector of the input signal after nonlinear transformation and is used to pass it to the output layer to calculate the force push deviation. ( ) is the activation function. The elements of the weight matrix represent the connection strength between the "i-th neuron in the input layer" and the "j-th neuron in the hidden layer," and are core parameters of the model. The input layer neuron signal represents the input value of the i-th neuron in the input layer, which is directly derived from the contact force signal collected and calibrated by the force sensor module. This is the hidden layer bias term, representing the bias parameter of the j-th neuron in the hidden layer. It is used to adjust the activation threshold of this neuron and needs to be combined with... Synchronous training optimization.

[0129] in, The output value of the output layer represents the final output of the k-th neuron in the output layer. It directly corresponds to the connection bias derived from the contact force and is one of the core inputs for subsequent data fusion. For the elements of the hidden layer → output layer weight matrix, it represents the connection strength between the "j-th neuron in the hidden layer" and the "k-th neuron in the output layer". It is a core parameter that the model optimizes through "pre-training + online iteration" and directly determines the accuracy of feature to bias conversion. The output layer bias term represents the baseline adjustment parameter of the k-th neuron in the output layer, used to correct the zero drift error of the output layer. Synchronous training optimization.

[0130] in, To control the learning rate, the step size of parameter updates should be controlled: avoid the step size being too large, which will cause parameter oscillation and non-convergence, or the step size being too small, which will cause slow iteration. For error pair The partial derivatives, This represents the actual deviation. For error pair The partial derivatives, For error pair The partial derivatives, For error pair The partial derivative of , where E is the mean square error.

[0131] In one embodiment of the present invention, the federated Kalman filtering algorithm runs Kalman filtering independently through three sub-modules, and outputs local bias and corresponding local error covariance;

[0132] Local bias is the independent predicted value of each submodule for connector bias, calculated through the prediction and update equations of the state-space model. For example, the local bias of the vision submodule includes compensated visual bias based on image feature extraction. Local error covariance is a matrix characterizing the uncertainty of the predicted value, generated through a combination of the state transition matrix and the process noise covariance. For example, the error covariance of the force sensing submodule can reflect the combined impact of its sensor noise level and environmental interference.

[0133] The global fusion bias is obtained by weighting the weighted calculation based on the local error covariance.

[0134] The reciprocal of the local error covariance can be used as the basis for weight allocation, and a confidence index can be constructed through the inverse operation of the covariance matrix.

[0135] The global fusion bias is fed back to the three sub-modules to correct the initial parameters of their local filtering.

[0136] The feedback correction mechanism injects the difference between the global bias and the local estimate of the submodule into the initial value of the prediction for the next period through proportional adjustment.

[0137] The three sub-modules include a vision sub-filtering module, a force sensing sub-filtering module, and an environmental compensation sub-filtering module, as shown in the following formula:

[0138] The prediction equations for the three sub-modules are as follows:

[0139] ;

[0140] The update equations for the three sub-modules are as follows:

[0141] ;

[0142] in, , and These are the state prediction values ​​for the vision submodule, the force sensing sub-filtering module, and the environmental compensation sub-filtering module, respectively. , and The corresponding prediction error covariance, As a strong tracking factor, , and The process noise covariance of the vision submodule, force sensing sub-filtering module, and environmental compensation sub-filtering module are respectively defined. For Kalman gain, For the observation matrix, To observe the noise covariance matrix, It is the identity matrix. Observation matrix The transpose of the matrix, Let be the observation vector at the current moment. This represents the final deviation value after fusion at time k. This is the predicted value of the connector's true deviation at time k, based on the detection results at time k-1. This is the updated error covariance.

[0143] in, The prediction error at time k is the result of the (k-1)th time, which is used to estimate the connection error of the sensor module at the current time. This is the precise deviation at time k-1, which is the final deviation value obtained after data fusion at the previous time, after the error has been corrected. Quantize the "prediction error covariance" at time k. The smaller the value, the more reliable the prediction bias; It is the precise error covariance at time k-1, which is the quantized error value corresponding to the precise deviation at the previous time.

[0144] It should be noted that the formula for calculating the global fusion bias using the federated Kalman filter algorithm is as follows:

[0145] ;

[0146] ;

[0147] in, This is a global fusion bias. To globally integrate covariance, , and Let be the error covariances of the vision sub-module, the force sensing sub-filter module, and the environmental compensation sub-filter module at time k, respectively. , and These are the deviation values ​​of the vision submodule, force sensing sub-filter module, and environmental compensation sub-filter module at time k, respectively.

[0148] This invention achieves adaptive fusion and system bias suppression of multi-source heterogeneous data through the coordinated operation of three sub-modules: distributed prediction and updating, dynamic weight allocation based on the inverse of covariance, and closed-loop feedback correction. The distributed architecture, combined with adaptive adjustment of the strong tracking factor and Kalman gain, enables the algorithm to cope with nonlinear state changes and sensor noise interference. The dynamic weight allocation mechanism adjusts the contribution of each sub-module according to the real-time error covariance, ensuring that high-confidence data dominates the fusion result. Closed-loop feedback eliminates long-term drift of sub-modules through proportional correction. The combined effect of these three mechanisms allows the system to maintain high bias detection accuracy even in extreme environments, significantly enhancing real-time performance and adaptability to complex environments.

[0149] In one embodiment of the present invention, the deviation prediction formula of the LSTM deviation trend prediction model is as follows:

[0150] ;

[0151] in, Predicting time t Timing deviation, To predict the time step, The weight matrix consists of parameters obtained by training the model using historical data, used to quantify the influence of historical features on future biases. This is the historical feature vector at the current time t, which typically contains key data from t and several previous times. This is a bias term used to compensate for minor systematic errors during model training or data acquisition, thereby improving prediction accuracy.

[0152] The LSTM bias trend prediction model employs a temporal network structure consisting of an input layer, hidden layers, and an output layer. The input layer receives global fusion biases from multiple consecutive time points, forming a historical feature vector to ensure sufficient temporal information coverage to capture bias variation patterns. The hidden layer uses two LSTM units and employs a three-gate control mechanism (forget gate, input gate, and output gate) to selectively retain key features and remove forgotten noise interference from historical biases, while simultaneously updating the temporal dependencies of cell state storage. The output layer corresponds to multiple future prediction time steps, outputting multiple predicted bias values. The formulas for the three-gate control mechanism are as follows:

[0153] ;

[0154] in, , and These are the activation values ​​for the forget gate, input gate, and output gate, respectively. This is the output of the hidden layer at time t-1. The input at time t represents the current global fusion bias. , and These represent the cell state, the previous cell state, and the candidate cell state, respectively. Here is the forget gate weight matrix. For the forget gate bias term, It is the Sigmoid activation function. The input gate weight matrix, For the bias term of the input gate, This is the weight matrix for the candidate cell states. This is a bias term for the candidate cell state. This is the weight matrix of the output gate. This is the bias term for the output gate. The output of the hidden layer is the current state feature after being filtered by the output gate.

[0155] In one embodiment of the present invention, the adjustment mechanism is specifically as follows:

[0156] When the global fusion deviation or deviation prediction value exceeds the preset safety threshold, the electric slide adjustment is triggered. When the deviation is small, fine-tuning is performed with small steps; when the deviation is large, rapid correction is performed with large steps.

[0157] The preset safety threshold is a benchmark parameter used to determine whether a deviation needs correction. It can be set through engineering experiments or industry standards. For example, the safety threshold may include axial deviation ±0.1mm and radial deviation ±0.05mm. The deviation exceeding the safety threshold is quantified by the over-limit ratio, calculated as the ratio of the current deviation value to the safety threshold. When the over-limit ratio is greater than the preset value, the deviation is considered large, and a large step size is used for rapid correction. When the over-limit value is greater than 0.05mm, it is considered large, and a large step size deviation is used, calculated using the corresponding empirical formula. When the over-limit value is less than 0.05mm, it is considered small, and a small step size deviation is used, calculated using the corresponding empirical formula. The small step size is 0.01-0.03mm, and the specific value can be calculated based on the actual over-limit value. For example, when the over-limit value is 0.05mm, the step size is calculated using the formula: The empirical formula yields a small step size of 0.03 mm. When the exceedance value is 0.1 mm, it is considered a large exceedance. The step size is determined by... The empirical formula yields the maximum step size value, which is 0.8 mm.

[0158] Each time the electric slide completes an adjustment, the global fusion deviation and the predicted deviation value are recalculated. The control module determines whether the deviation is still exceeded. If it is exceeded, the adjustment is repeated until the global fusion deviation and the predicted deviation value are both within the preset safety threshold.

[0159] This invention dynamically adjusts the displacement amplitude of the electric slide table according to the degree of exceeding the limit through a graded step size selection mechanism. Combined with closed-loop feedback verification, it ensures the continuous effectiveness of deviation correction. At the same time, it avoids infinite looping or over-execution of the adjustment process through a multi-condition termination strategy. This can achieve the technical effects of significantly reducing motor wear, shortening deviation correction time, and improving system robustness under complex working conditions.

[0160] See Figure 2 The present invention also provides a connection deviation detection device based on machine vision, comprising:

[0161] The vision module is used to acquire images of the plug and socket; during the docking process, panoramic images of the initial state and real-time docking state are captured at a set frequency to ensure that the displacement changes of feature points are captured, providing raw visual data for deviation calculation.

[0162] The force sensor module is used to collect contact forces during the docking process, including axial force sensors, x-axis force sensors, and y-axis force sensors; it outputs axial and radial contact forces in real time, covering the full-dimensional force feedback requirements of connector docking. The collected force signals are converted into actual force values ​​through force and voltage calibration curves and input into an adaptive nonlinear deviation and force mapping model to derive real-time force deviation.

[0163] The environmental sensor module selects a water pressure sensor, an infrared temperature sensor, or a triaxial accelerometer based on the target scenario to collect real-time environmental parameters. It then substitutes these real-time environmental parameters into the environmental and parameter compensation model to output the environmental compensation deviation, thereby eliminating unexpected interference from environmental factors on deviation detection.

[0164] The control module is connected to the vision module, force sensor module, environmental sensor module and electric slide, and has a built-in preprocessing calibration unit, initial positioning unit, real-time fusion unit and predictive adjustment unit, which are used to calculate real-time deviation and send adjustment commands.

[0165] The electric slide stage receives adjustment commands from the control module and performs displacement correction according to a hierarchical adjustment strategy. The preprocessing calibration unit performs precise calibration of each sensor. The initial unit processes the initial image from the vision module and calculates the initial deviation to provide a benchmark for subsequent real-time detection. The real-time fusion unit performs data fusion, combining visual deviation, force deviation, and environmental compensation deviation to output a global fused deviation, eliminating errors from individual sensors. The predictive adjustment unit predicts deviation trends using an LSTM model and drives the electric slide stage to adjust according to a hierarchical strategy based on the prediction results.

[0166] This invention replaces the traditional complex floating reference with a multi-sensor collaboration of vision, force, and environment, simplifying the overall structure and reducing hardware costs and maintenance difficulty; it can be equipped with appropriate environmental sensors according to the target scenario, and can work stably in extreme scenarios such as deep water, high temperature, and vibration, greatly expanding the application boundaries.

[0167] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A connection deviation detection method based on machine vision, characterized in that, Includes the following steps: The preset vision module and force sensor module are calibrated, and the corresponding environmental sensor is selected for calibration based on the target scene; Initial environmental parameters are collected and substituted into a preset environmental and deviation compensation model to obtain the initial environmental compensation deviation. Initial state images of the plug and socket are collected, and the initial axial distance and radial offset are calculated. The final initial deviation is obtained by combining the initial environmental compensation deviation. Visual data, force data, and environmental data are collected simultaneously. Real-time force push deviation is obtained through a pre-trained adaptive nonlinear deviation and force mapping model. The environmental data is substituted into an environment and deviation compensation model to obtain real-time environmental compensation deviation. The compensated visual deviation is obtained by combining the visual data and the real-time environmental compensation deviation. Based on real-time force push deviation, real-time environmental compensation deviation, and post-compensation visual deviation, a global fusion deviation is output through a federated Kalman filter algorithm. The federated Kalman filter algorithm includes a visual sub-filter module, a force sensing sub-filter module, and an environmental compensation sub-filter module. The global fusion deviation is obtained by dynamically allocating fusion weights based on the local error covariance of each sub-module. Based on the global fusion deviation, a pre-trained LSTM deviation trend prediction model is used to output the deviation prediction value within a set time period in the future. The global fusion deviation and the predicted deviation value are compared with the preset safety threshold. If the global fusion deviation or the predicted deviation value exceeds the limit, the electric slide is triggered to perform graded adjustments.

2. The connection deviation detection method based on machine vision according to claim 1, characterized in that, The target scenarios include deep water scenarios, high temperature scenarios, and vibration scenarios. The environmental sensors include a water pressure sensor, an infrared temperature sensor, and a triaxial accelerometer. When the target scenario is a deep water scenario, a water pressure sensor is selected and calibrated using a standard water pressure device; when the target scenario is a high temperature scenario, an infrared temperature sensor is selected and calibrated using a constant temperature chamber; when the target scenario is a vibration scenario, a triaxial accelerometer is selected and calibrated using a standard vibration table. When calibrating the vision module, a standard checkerboard calibration board is used, and the camera intrinsic parameters are calculated using Zhang's calibration algorithm; When calibrating the force sensor module, a standard force of 0 to rated range is applied through a standard force loading device to establish a calibration curve of force value versus voltage.

3. The connection deviation detection method based on machine vision according to claim 2, characterized in that, The calculation formula for the environment and deviation compensation model is as follows: When the target scene is a deep-water scene, the following formula is used: ， When the target scenario is a high-temperature scenario, the following formula is used: ， When the target scene is a vibration scene, the following formula is used: ， in, To compensate for axial environmental deviation. To compensate for radial X-axis environmental deviation. To compensate for the radial y-axis environmental deviation, The hydrostatic deformation coefficient of the connector material. For real-time water pressure, The angle of action of water pressure, The coefficient of thermal expansion of the connector material. For real-time temperature, The axial length of the connector. Vibration-offset coefficient, For real-time vibration acceleration, The real-time vibration acceleration along the x-axis. This represents the real-time vibration acceleration along the y-axis.

4. The connection deviation detection method based on machine vision according to claim 1, characterized in that, The specific steps to obtain the final initial deviation are as follows: The initial state image was preprocessed, and the center of the socket positioning hole, the center of the plug end face, and the socket edge marker points were extracted using an edge detection algorithm. The coordinates of the center of the socket positioning hole, the center of the plug end face, and the socket edge marker points in the image coordinate system were recorded. Convert the coordinates in the image coordinate system to the actual physical coordinate system coordinates, and calculate the initial axial distance and initial radial offset. The final initial deviation is obtained by subtracting the initial environmental compensation deviation from the initial axial distance and initial radial offset.

5. The connection deviation detection method based on machine vision according to claim 1, characterized in that, The adaptive nonlinear deviation and force mapping model includes an input layer, a hidden layer, and an output layer. The input layer contains three neurons for inputting three vectors of force data. The hidden layer contains several neurons for converting the force data from the input layer into deviation data. The output layer contains three neurons for outputting three vectors corresponding to the real-time force deviation.

6. The connection deviation detection method based on machine vision according to claim 1, characterized in that, The federated Kalman filtering algorithm operates the Kalman filtering independently through three sub-modules, outputting local bias and the corresponding local error covariance. The global fusion bias is obtained by weighting the weighted calculation based on the local error covariance. The global fusion bias is fed back to the three sub-modules to correct the initial parameters of their local filtering. The three sub-modules include a vision sub-filtering module, a force sensing sub-filtering module, and an environmental compensation sub-filtering module, as shown in the following formula: The prediction equations for the three sub-modules are as follows: ， The update equations for the three sub-modules are as follows: ， in, , and These are the state prediction values ​​for the vision submodule, the force sensing sub-filtering module, and the environmental compensation sub-filtering module, respectively. , and The corresponding prediction error covariance, As a strong tracking factor, , and The process noise covariance of the vision submodule, force sensing sub-filtering module, and environmental compensation sub-filtering module are respectively defined. For Kalman gain, For the observation matrix, To observe the noise covariance matrix, It is the identity matrix. Observation matrix The transpose of the matrix, Let be the observation vector at the current moment. This represents the final deviation value after fusion at time k. This is the predicted value of the connector's true deviation at time k, based on the detection results at time k-1. This is the updated error covariance.

7. The connection deviation detection method based on machine vision according to claim 6, characterized in that, The formula for calculating the global fusion bias using the federated Kalman filter algorithm is as follows: ， ， in, This is a global fusion bias. To achieve global covariance fusion, , and Let be the error covariances of the vision sub-module, the force sensing sub-filter module, and the environmental compensation sub-filter module at time k, respectively. , and These are the deviation values ​​of the vision submodule, force sensing sub-filter module, and environmental compensation sub-filter module at time k, respectively.

8. The connection deviation detection method based on machine vision according to claim 1, characterized in that, The deviation prediction formula of the LSTM deviation trend prediction model is as follows: ， in, Predicting time t Timing deviation, To predict the time step, The weight matrix consists of parameters obtained by training the model using historical data, used to quantify the influence of historical features on future biases. This is the historical feature vector of the current time t, which typically contains key data from t and several previous time points. This is a bias term used to compensate for minor systematic errors during model training or data acquisition, thereby improving prediction accuracy.

9. The connection deviation detection method based on machine vision according to claim 1, characterized in that, The specific steps for triggering the electric slide to perform graded adjustments are as follows: When the global fusion deviation or deviation prediction value exceeds the preset safety threshold, the electric slide adjustment is triggered. When the excess value is less than or equal to the preset value, fine adjustment is made with small steps. When the excess value is greater than the preset value, fast correction is made with large steps. Each time the electric slide completes an adjustment, the global fusion deviation and the predicted deviation value are recalculated to determine whether they still exceed the limits. If they do, the adjustment is repeated until the global fusion deviation and the predicted deviation value are both within the preset safety threshold.

10. A connection deviation detection device used in the machine vision-based connection deviation detection method as described in any one of claims 1-9, comprising: The vision module is used to acquire images of the plug and socket; Force sensor module, used to collect contact forces during docking, including axial force sensor, x-axis force sensor and y-axis force sensor; The environmental sensor module selects a water pressure sensor, an infrared temperature sensor, or a triaxial accelerometer based on the target scenario to collect real-time environmental parameters. The control module is connected to the vision module, force sensor module, environmental sensor module and electric slide, and has a built-in preprocessing calibration unit, initial positioning unit, real-time fusion unit and predictive adjustment unit, which are used to calculate real-time deviation and send adjustment commands.

Images