A machine vision-based wire harness production quality inspection system and method

Abstract

This invention discloses a machine vision-based wire harness production quality inspection system and method, belonging to the field of industrial automation inspection technology. The system includes: acquiring wire harness data, including wire harness surface images, perspective views of the wire harness internal structure, and vibration data; preprocessing the wire harness data; performing geometric correction on the wire harness surface images; eliminating field-of-view bias in the perspective views of the wire harness internal structure; extracting features from the wire harness surface images using ResNet50 to generate a surface feature map; segmenting defects in the perspective views of the wire harness internal structure using U-Net to generate an internal feature map; and fusing the surface feature map and the internal feature map to obtain a fused feature map. This invention effectively corrects image acquisition distortion in dynamic environments by iteratively calculating the camera pose offset using state transition equations and a Kalman filter algorithm, improving the stability and accuracy of the inspection method and solving the problem of low detection accuracy in dynamic environments.

Description

Technical Field

[0001] This invention relates to the field of industrial automation inspection technology, and in particular to a machine vision-based wire harness production quality inspection system and method. Background Technology

[0002] With the rapid development of industrial automation, wire harnesses, as a key component in modern electronic equipment and automobile manufacturing, directly affect product performance and reliability due to their production quality. Traditional wire harness quality inspection methods mainly rely on manual visual inspection or simple optical inspection equipment, but these methods suffer from problems such as low efficiency, strong subjectivity, and difficulty in detecting internal defects.

[0003] In recent years, machine vision-based inspection methods, supported by high-resolution cameras and deep learning algorithms, have achieved efficient identification of surface defects. However, existing technologies mostly focus on single-modal detection (such as surface or interior only), lacking research on multi-source data fusion, which limits the comprehensiveness and accuracy of detection. Meanwhile, the issue of detection accuracy in dynamic environments remains unresolved; vibration interference can cause image acquisition distortion, affecting detection results. These two aspects highlight the importance of improving multi-source data fusion and detection accuracy in dynamic environments. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a machine vision-based method for inspecting the production quality of wire harnesses, which solves the problems of insufficient multi-source data fusion and low detection accuracy in dynamic environments.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a method for inspecting the production quality of wire harnesses based on machine vision, which includes: collecting wire harness data, including wire harness surface images, wire harness internal structure perspective views and vibration data; preprocessing the wire harness data; performing geometric correction on the wire harness surface images; and eliminating field deviations on the wire harness internal structure perspective views.

[0008] Features of the wire harness surface image are extracted using ResNet50 to generate a surface feature map. Defects in the perspective view of the internal structure of the wire harness are segmented using U-Net to generate an internal feature map. The surface feature map and the internal feature map are then fused to obtain a fused feature map.

[0009] Calculate the defect severity score and the correlation coefficient between defect data and each production parameter, and identify highly correlated production parameters. Based on the fused feature map and defect severity score, use the Gaussian distribution function to calculate the defect impact value at each grid center point, construct a defect heat map, and use Bayesian network analysis to analyze the causal relationship between production parameters and defects.

[0010] A three-dimensional Gaussian force field distribution is established centered on the defect heat map. The clamping force is updated using an anti-saturation PID algorithm. The process noise covariance matrix is ​​adjusted based on vibration data. The parameters of ResNet50 and U-Net models are updated using defect detection samples.

[0011] As a preferred embodiment of the machine vision-based wire harness production quality inspection method of the present invention, the specific steps of using Bayesian networks to analyze the causal relationship between production parameters and defects are as follows:

[0012] A time series anomaly detection algorithm is used to analyze production parameters that are determined to be highly correlated, and to identify periods of abnormal fluctuation in production parameters;

[0013] Calculate the time overlap index and construct a causal network diagram between production parameters and defects based on historical production parameter data, defect types, and equipment IDs;

[0014] The time overlap index is used as an observation variable in a Bayesian network to analyze the causal relationship between production parameters and defects.

[0015] As a preferred embodiment of the machine vision-based wire harness production quality inspection method of the present invention, wherein:

[0016] The specific steps for establishing a three-dimensional Gaussian force field distribution centered on the defect heatmap and updating the clamping force using an anti-saturation PID algorithm are as follows:

[0017] The three-dimensional coordinates of the defects are extracted from the defect heat map. A three-dimensional Gaussian force field distribution is established with the three-dimensional coordinates of the defects as the center. The force field intensity coefficient is linearly adjusted according to the defect severity score. The force field intensity coefficient is dynamically adjusted using a depth compensation factor.

[0018] The adoption of an anti-saturation PID algorithm to update the clamping force refers to using dynamic gain in the proportional stage, setting integral limiting in the integral stage, and adding low-pass filtering in the derivative stage.

[0019] As a preferred embodiment of the machine vision-based wire harness production quality inspection method of the present invention, wherein:

[0020] The specific steps for updating the ResNet50 and U-Net model parameters using defect detection samples are as follows:

[0021] Collect defect detection samples and use the backpropagation algorithm to calculate the gradient vector of the loss function for each defect detection sample across all output categories;

[0022] Calculate the gradient norm of the defect detection sample across all categories, and assign a weight to each defect detection sample;

[0023] Calculate the importance estimates of the parameters of ResNet50 and U-Net models, mark important parameters based on the importance estimates, and set protection thresholds for important parameters;

[0024] Update the ResNet50 and U-Net model parameters using stochastic gradient descent with momentum, and set the learning rate and momentum coefficient.

[0025] As a preferred embodiment of the machine vision-based wire harness production quality inspection method of the present invention, wherein:

[0026] The specific steps for fusing the surface feature map and the internal feature map to obtain the fused feature map are as follows:

[0027] ResNet50 was used as the backbone network to extract features from the surface image of the wire harness and generate a surface feature map.

[0028] U-Net was used as the backbone network to segment the internal copper wire fractures and insulation bubble defects in the internal structural perspective of the wire harness. The features of the internal structural perspective of the wire harness were extracted through an encoder-decoder structure and an internal feature map was generated.

[0029] The surface feature map and the internal feature map are concatenated along the channel dimension to generate a joint feature map, and the dynamic weights of the surface feature map and the internal feature map are calculated using the Sigmoid activation function.

[0030] Dynamic weights are used to weight and fuse the surface feature map and the internal feature map to generate a fused feature map.

[0031] As a preferred embodiment of the machine vision-based wire harness production quality inspection method of the present invention, wherein:

[0032] The specific steps for calculating the defect severity score and the correlation coefficient between defect data and various production parameters, and determining highly correlated production parameters, are as follows:

[0033] Use regression analysis to fit historical defect data;

[0034] Calculate the defect severity score and classify the defect level;

[0035] The Pearson correlation coefficient was used to calculate the correlation coefficient between defect data and various production parameters, and the highly correlated production parameters were identified.

[0036] As a preferred embodiment of the machine vision-based wire harness production quality inspection method of the present invention, wherein:

[0037] The specific steps for performing geometric correction on the surface image of the wire harness and eliminating field-of-view deviation on the perspective view of the internal structure of the wire harness are as follows:

[0038] Calculate the observed values ​​using the observation equation;

[0039] The camera position and attitude at the next time step are predicted using the state transition equation, the covariance matrix at the next time step is predicted and updated using the state transition matrix and the process noise covariance matrix, and the Kalman gain is calculated.

[0040] The camera position and attitude are updated by the difference between the observed values ​​and the camera position and attitude at the next time step, and by the Kalman gain. The covariance matrix is ​​adjusted by the Kalman gain and the observation matrix.

[0041] The camera pose offset is calculated by performing a prediction-update loop iteration;

[0042] Construct a homography matrix and perform geometric correction on the wire harness surface image;

[0043] Calculate the hand-eye calibration matrix and transform the perspective view of the internal structure of the wire harness to the coordinate system of the wire harness surface image using the hand-eye calibration matrix.

[0044] Secondly, the present invention provides a machine vision-based wire harness production quality inspection system, comprising,

[0045] The correction module collects wire harness data, including wire harness surface images, wire harness internal structure perspective views, and vibration data. It preprocesses the wire harness data, performs geometric correction on the wire harness surface images, and eliminates field-of-view deviations on the wire harness internal structure perspective views.

[0046] The fusion module uses ResNet50 to extract features from the surface image of the wire harness and generates a surface feature map. It uses U-Net to segment defects in the perspective view of the internal structure of the wire harness and generates an internal feature map. The surface feature map and the internal feature map are fused to obtain a fused feature map.

[0047] The analysis module calculates the defect severity score and the correlation coefficient between defect data and various production parameters, and identifies highly correlated production parameters. Based on the fused feature map and defect severity score, it uses the Gaussian distribution function to calculate the defect impact value at each grid center point, constructs a defect heat map, and uses Bayesian network analysis to analyze the causal relationship between production parameters and defects.

[0048] The update module establishes a three-dimensional Gaussian force field distribution centered on the defect heat map, updates the clamping force using an anti-saturation PID algorithm, adjusts the process noise covariance matrix based on vibration data, and updates the ResNet50 and U-Net model parameters using defect detection samples.

[0049] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein the computer program, when executed by the processor, implements any step of the machine vision-based wire harness production quality inspection method described in the first aspect of the present invention.

[0050] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the machine vision-based wire harness production quality inspection method as described in the first aspect of the present invention.

[0051] The beneficial effects of this invention are as follows: By iteratively calculating the camera pose offset using the state transition equation and Kalman filter algorithm, effective correction of image acquisition distortion in dynamic environments is achieved, improving the stability and accuracy of the detection method. Especially in complex production environments, it ensures high-quality data acquisition and analysis results, solving the problem of low detection accuracy in dynamic environments. By stitching together channels to generate joint feature maps, and using dynamic weights adjusted based on the vibration compensation parameter matrix, surface and internal feature maps are weighted and fused to generate the final fused feature map for defect classification. This achieves comprehensive analysis of internal and external defects in the wire harness, overcomes the limitations of single-modal detection, and solves the problem of insufficient multi-source data fusion. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the machine vision-based wire harness production quality inspection method in Example 1.

[0054] Figure 2 This is a schematic diagram of the machine vision-based wire harness production quality inspection system in Example 1.

[0055] Figure 3 This is a schematic diagram of the generation of the three-dimensional defect thermal map in Example 1.

[0056] Figure 4 This is a schematic diagram of the anti-saturation PID control algorithm in Example 1. Detailed Implementation

[0057] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0058] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0059] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0060] Example 1, referring to Figure 1 , Figure 2 , Figure 3 and Figure 4 This embodiment provides a machine vision-based method for inspecting the production quality of wire harnesses, including the following steps:

[0061] S1: Collect wire harness data, including wire harness surface image, wire harness internal structure perspective view and vibration data, preprocess the wire harness data, perform geometric correction on the wire harness surface image, and eliminate field deviation on the wire harness internal structure perspective view.

[0062] The specific steps are as follows:

[0063] Install a high-resolution industrial camera (resolution ≥ 12MP) on the inspection platform, ensuring the distance between the camera and the wire harness is 50cm and the viewing angle is 45° to acquire images of the wire harness surface. Set the camera frame rate to 30fps and automatically adjust the exposure time to ensure clear images.

[0064] It should also be noted that using a high-resolution industrial camera can capture minute defect features on the surface of the wire harness, improving the accuracy and reliability of the inspection.

[0065] A miniature X-ray machine was installed on the testing platform. The distance between the miniature X-ray machine and the wire bundle was 30cm, and the viewing angle was 90°. A perspective view of the internal structure of the wire bundle was acquired. The power of the miniature X-ray machine was set to 50kV, and the exposure time was 0.1s to ensure that the perspective view was clear.

[0066] It should also be noted that using a miniature X-ray machine for internal structural inspection can non-destructively detect defects inside the wire harness, such as broken copper wires and air bubbles in the insulation layer, ensuring the safety of product quality.

[0067] A MEMS inertial sensor is mounted on a high-resolution industrial camera mount to monitor vibration data in real time, including the vibration acceleration (range ±16g) and vibration angular velocity (range ±2000° / s) of the wire harness in three-dimensional space. The sensor sampling frequency is set to 100Hz to ensure high-precision acquisition of vibration data.

[0068] It should also be noted that by monitoring vibration data in real time and aligning it with image data, the state of the harness can be analyzed more accurately in dynamic environments, thus improving the accuracy of the detection results.

[0069] Align the wire harness surface image, the wire harness internal structure perspective view, and the vibration data according to the timestamps generated by the PTP protocol to ensure that the timestamps of each frame of image, perspective view, and vibration data are consistent.

[0070] The vibration data were subjected to Fast Fourier Transform (FFT) with a window length of 1024, a sampling frequency of 100Hz, and a frequency resolution of 0.1Hz.

[0071] The difference between the mean vibration acceleration and the gravitational acceleration under static conditions is calculated to obtain the static acceleration offset. The static acceleration offset is then integrated twice over time to obtain the static position offset.

[0072] The static rotation angle increment is obtained by integrating the mean vibration angular velocity under static conditions over time. The static rotation angle increment is converted into a quaternion increment using quaternion mapping. The current attitude is updated by quaternion multiplication, and the static quaternion attitude offset is recorded.

[0073] The static position offset and the static quaternion attitude offset are integrated into a static offset.

[0074] The camera's current position and attitude are updated using state transition equations. The input control variables are vibration acceleration and vibration angular velocity, and the process noise is Gaussian distributed. The vibration acceleration is superimposed with the process noise and integrated over the acquisition interval to update the current velocity. The current velocity is superimposed with the process noise and integrated over the acquisition interval to update the current displacement offset and update the camera's current position. The angular velocity is superimposed with the process noise and integrated over the acquisition interval to obtain the rotation angle increment and update the rotation angle. The rotation angle increment is converted into a quaternion increment using quaternion mapping, including decomposing the rotation angle increment into the rotation axis and rotation angle, and converting it into a quaternion increment according to the right-hand rule. After superimposing Gaussian noise, it is multiplied with the current attitude quaternion using quaternion multiplication to update the camera's current attitude and record the quaternion attitude offset.

[0075] The observation values ​​are calculated using the observation equation based on the current position, current attitude, and static offset of the camera. The observation noise follows a Gaussian distribution.

[0076] The camera pose offset is calculated iteratively through a prediction-update step: Based on the current vibration acceleration, vibration angular velocity, current camera position, and current camera attitude, the camera position and attitude at the next moment are predicted using the state transition equation. The covariance matrix at the next moment is predicted and updated using the state transition matrix and the process noise covariance matrix. The Kalman gain is calculated based on the predicted covariance matrix and the observation noise covariance matrix. The camera position and attitude are updated using the difference between the observed values ​​and the camera position and attitude at the next moment, and the Kalman gain. The covariance matrix is ​​updated using the Kalman gain and the observation matrix to reduce errors. The prediction-update loop is executed every 0.01 seconds to iteratively calculate the camera pose offset.

[0077] A homography matrix is ​​constructed based on the camera pose offset, including translation, rotation, and scaling parameters, to describe the geometric transformation relationship between image planes. The homography matrix is ​​used to perform geometric correction on the wire harness surface image: feature points are extracted between the wire harness surface image and the preset target plane using the ORB algorithm, and corresponding feature point pairs are matched. The homography matrix is ​​fitted using the RANSAC algorithm, and the inverse transformation of the homography matrix is ​​applied to the wire harness surface image to project the image onto the target plane, eliminating viewpoint distortion. Subpixel-level compensation is achieved by filling the transformed pixels through bilinear interpolation, ensuring that the correction error is controlled within 0.05 pixels.

[0078] Corner detection is performed on the perspective view of the internal structure of the wire harness using a checkerboard pattern. The calibration board transformation matrix is ​​then calculated using the least squares method. Based on the camera pose offset and the calibration board transformation matrix, a hand-eye calibration equation is established. The rotation and translation components of the hand-eye calibration matrix are decomposed into independent variables. The rotation component is solved using singular value decomposition (SVD). The rotation component is then substituted into the translation equation, and the translation component is solved using the least squares method. The rotation and translation components are combined to obtain the hand-eye calibration matrix. The perspective view of the internal structure of the wire harness is then transformed to the wire harness surface image coordinate system using the hand-eye calibration matrix to eliminate field-of-view bias.

[0079] The quaternion attitude offset is converted into a rotation matrix, the position offset is converted into a translation vector, and a pose transformation matrix is ​​constructed. The pose transformation matrix is ​​then combined with the camera intrinsic parameter matrix to generate a vibration compensation parameter matrix.

[0080] It should also be noted that by synchronizing and analyzing the surface images of the wire harness, the perspective view of the internal structure of the wire harness, and the vibration data, as well as calculating the camera pose offset, the accuracy of detection can be improved, thus providing a high-quality data foundation for subsequent defect identification.

[0081] S2: Use ResNet50 to extract features from the surface image of the wire harness and generate a surface feature map. Use U-Net to segment defects in the perspective view of the internal structure of the wire harness and generate an internal feature map. Then, fuse the surface feature map and the internal feature map to obtain a fused feature map.

[0082] The specific steps are as follows:

[0083] Using ResNet50 as the backbone, the top fully connected classification layer is removed, retaining the convolutional layers and residual block structure. The convolutional layers receive three-channel RGB images and output 64-channel feature maps, capturing the geometric features of the wire bundle surface (such as edges and corners). Shallow residual block groups, consisting of 3 residual units, output 256-channel feature maps, extracting local details of the wire bundle texture (such as discontinuous edges of fine scratches). Mid-layer residual block groups, consisting of 4 residual units, output 512-channel feature maps. This study analyzes the complex texture features at the interface between the tape and wire on the wire harness surface. A deep residual block group, consisting of 6 residual units, outputs a 1024-channel feature map, capturing global structural information (such as the continuous direction of long scratches or the overall deformation of loose tape). An end residual block group, consisting of 3 residual units, outputs a 2048-channel feature map, encoding high-dimensional semantic information (such as defect category and severity). This process extracts surface scratches and loose tape features from the wire harness surface image and uses convolutional layers to extract edges from the wire harness surface image. Edge and corner features are extracted from the surface image of the wire harness using shallow residual block groups. Mid-layer residual block groups are used to identify the fracture patterns of surface scratches and the local deformation of loose tape. Deep residual block groups capture global structural information (such as the continuous direction of long scratches or the overall deformation of loose tape). A global average pixel value is calculated for each channel on the 2048-channel feature map output by the end residual block group, generating a 2048-dimensional global feature vector and encoding high-level semantics (such as defect category labels). (Scratches or loose tape) The 1024-channel feature map (14×14) of the deep residual block group is upsampled by 4 times, and its size is aligned with the 56×56 feature map of the shallow residual block group. The first convolutional layer (64 channels), the shallow residual block group (256 channels) and the upsampled deep feature (1024 channels) are spliced ​​according to the channel dimension to form a 1344-channel hybrid feature map. The 1344 channels are compressed to 256 channels by 1×1 convolution to generate the surface feature map.

[0084] It should also be noted that deep learning methods can identify complex surface defect features, improving detection efficiency and accuracy.

[0085] The U-Net backbone network is used, comprising an encoder (feature extraction) and a decoder (resolution restoration). The encoder includes an input layer that receives a perspective view of the internal structure of the receiving harness; a convolutional module where each level contains two repeated convolution operations, followed by batch normalization and a ReLU activation function, and finally downsampling using 2×2 max pooling; a layer depth of 4 downsampling levels, with each level halving the resolution (512-256-128-64-32), ultimately outputting a 32×32 resolution deep feature map. The decoder includes an upsampling module that achieves a 2x resolution upsampling at each level through bilinear interpolation; and feature concatenation, which concatenates the feature maps from the encoder and the upsampled feature maps from the decoder along the channel dimension, preserving details and semantic information. The output layer uses a 1×1 convolution with a sigmoid activation function to generate a binary segmentation mask of the same size as the input, identifying defect areas and segmenting internal copper wire breaks and insulation bubble defects in the perspective view of the wire harness's internal structure. A encoder-decoder structure extracts multi-scale features from the perspective view of the wire harness's internal structure, including shallow features (capturing high-resolution details such as minute breaks at the edges of copper wires and local contours of insulation bubble), mid-scale features (identifying medium-scale structures such as discontinuities in copper wire breaks and aggregated bubble morphology), and deep features (understanding global semantic information such as the overall layout of the wire harness and the distribution density of defect areas). The shallow, mid-scale, and deep features are then concatenated to generate an internal feature map.

[0086] It should also be noted that the U-Net architecture excels at segmentation tasks in fields such as medical imaging, and its application in wire harness internal defect detection can also effectively improve the accuracy of defect localization and classification.

[0087] The surface feature map and the internal feature map are concatenated along the channel dimension to generate a joint feature map. Based on the vibration compensation parameter matrix and the joint feature map, the dynamic weights of the surface feature map and the internal feature map are calculated using the Sigmoid activation function. The elements of the vibration compensation parameter matrix are normalized to the range of -1 to 1 and expanded into a vibration parameter matrix that matches the spatial dimension of the joint feature map through a fully connected layer. The joint feature map and the vibration parameter matrix are concatenated along the channel dimension to form a fusion input. The weight matrix of the surface feature map and the internal feature map is generated using the Sigmoid activation function to adjust the contribution of the surface feature map and the internal feature map. The surface feature map and the internal feature map are weighted and fused using dynamic weights to generate a fused feature map.

[0088] It should also be noted that by dynamically adjusting the weights of different feature maps, the importance of features can be optimized according to the actual situation, thereby improving the reliability of classification results.

[0089] A fully connected layer is used to classify the fused feature map to obtain a defect category probability vector. The probability of each defect category is calculated using the Softmax function to determine the defect type.

[0090] S3: Calculate the defect severity score and the correlation coefficient between defect data and each production parameter, and determine the highly correlated production parameters. Based on the fused feature map and defect severity score, use the Gaussian distribution function (σ is 1 times the grid spacing, the Gaussian distribution covariance matrix is ​​isotropic, and μ is the coordinate of the defect point) to calculate the defect influence value of each grid center point, construct a defect heat map, and use Bayesian network analysis to analyze the causal relationship between production parameters and defects.

[0091] The specific steps are as follows:

[0092] Based on historical defect data, including surface scratch length, internal copper wire breakage, and corresponding defect severity scores, regression analysis was used to fit the historical defect data to obtain the weights for surface scratch length and internal copper wire breakage. The surface scratch length was normalized to the range of 0 to 1 by dividing it by a preset maximum surface scratch length (10 mm). The internal copper wire breakage was directly assigned a dimensionless value (1 for breakage, 0 for no breakage). The surface scratch length and internal copper wire breakage were weighted and summed to obtain the defect severity score.

[0093] It should also be noted that the defect severity score calculation not only considers the length of surface scratches, but also the internal copper wire breakage, comprehensively assessing the product's quality status and helping to accurately determine whether the product meets the standards.

[0094] Defects are classified into severity levels (fatal / critical / moderate) based on their severity scores. Defects with a severity score of 0.8 or higher are classified as fatal defects, defects with a severity score of 0.5 or higher but less than 0.8 are classified as critical defects, and defects with a severity score less than 0.5 are classified as moderate defects.

[0095] It should also be noted that clear defect classification standards facilitate rapid decision-making and are of great guiding significance for taking appropriate measures in a timely manner.

[0096] Retrieve historical defect records from SPC (Statistical Process Control), calculate the covariance between defect data and each production parameter (vibration frequency, temperature, humidity, pressure, equipment operating speed), divide the covariance by the respective standard deviation to obtain the correlation coefficient between defect data and each production parameter, and if the correlation coefficient is greater than 0.8, it is judged as highly correlated.

[0097] It should also be noted that by analyzing the relationship between defects and production parameters, key factors affecting product quality can be identified, thereby optimizing the production process.

[0098] A right-handed coordinate system is established based on the product entity. The location of the product corner feature is selected as the origin (0,0,0). The X, Y, and Z axes are set according to the length, width, and height of the product, respectively, and the coordinate unit is set to 0.1mm. The defect positions in the fused feature map are uniformly transformed to the right-handed coordinate system through the coordinate transformation matrix to obtain a three-dimensional space. The three-dimensional space is then discretized by meshing, and the product volume is divided into uniform small cubic units (e.g., 0.1mm×0.1mm×0.1mm). Based on the fused feature map and defect severity score, the defect influence value of each grid center point is calculated using a Gaussian distribution function. The defect influence values ​​of all grid center points are normalized and mapped to a color scale range of 0-255 to establish a color gradient scale. The three-dimensional scene is constructed using OpenGL (Open Graphics Library) technology, and the heat distribution is presented using semi-transparent gradient color cubes to obtain a defect heat map.

[0099] It should also be noted that constructing a three-dimensional heat map can intuitively display the distribution of defects and their severity.

[0100] Time series anomaly detection algorithms (such as STL decomposition combined with the 3σ principle) are used to analyze production parameters that are determined to be highly correlated. Abnormal fluctuation periods of production parameters that coincide with the defect occurrence period are identified. These abnormal fluctuation periods are overlaid with the defect occurrence period, and a time overlap index (overlap duration divided by total abnormal duration) is calculated. Leading and lagging time relationships are marked. Based on the historical records of production parameters, defect types, and equipment IDs, a causal network diagram of production parameters and defects is constructed. The time overlap index is used as the observed variable in a Bayesian network. Combined with the leading and lagging time relationship markings, conditional probability dependencies and d-separation tests are calculated. Combined with the rule constraints of the process knowledge base, production parameters directly related to defects (direct causes) and auxiliary factors that influence through other production parameters (indirect causes) are distinguished. Controllable key nodes (production parameters with practical operability) are marked in the causal network diagram of production parameters and defects.

[0101] It should also be noted that by using time series anomaly detection algorithms and causal network analysis, we can further explore the root causes of defects, provide a scientific basis for optimizing production processes, and help reduce the defect rate and improve production efficiency.

[0102] S4: Establish a three-dimensional Gaussian force field distribution centered on the defect heat map, update the clamping force using an anti-saturation PID algorithm, adjust the process noise covariance matrix based on vibration data, and update the ResNet50 and U-Net model parameters using defect detection samples.

[0103] The specific steps are as follows:

[0104] The three-dimensional coordinates of the defects are extracted from the defect heatmap, and a three-dimensional Gaussian force field distribution is established with the three-dimensional coordinates of the defects as the center. An isotropic action range parameter is set, the force field decays uniformly in all directions, the maximum force intensity at the center point is specified, and the force field intensity decays according to a Gaussian function. The covariance matrix of the Gaussian force field distribution (a diagonal matrix) is determined by calculating the variance of the isotropic action range parameter. The distance from each point in space to the defect center is calculated, and the distance contribution in each direction is adjusted according to the covariance matrix parameter. For example, if the action range in a certain direction is large, the distance decay in that direction is slower; the distance is substituted into the Gaussian function to calculate the force field intensity. The radius of the force field is set to 15mm, and the force field intensity coefficient is linearly adjusted according to the defect severity score: 0.9 for fatal defects, 0.6 for severe defects, and 0.3 for general defects. A depth compensation factor is used; when the defect depth exceeds 1 / 3 of the product thickness, the adjustment range is automatically reduced by 20%.

[0105] The clamping force is updated using an anti-saturation PID algorithm: the proportional element (P) adopts dynamic gain, with the gain coefficient set to 0.9 within 5mm of the defect and 0.6 within 5-10mm. The integral element (I) is set with an integral limit to prevent long-term accumulation of errors, with a limit value of ±15N. The derivative element (D) adds a low-pass filter with a cutoff frequency of 50Hz to eliminate high-frequency vibration interference. The output limit is set to [25N, 85N], and an alarm is automatically triggered if the range is exceeded.

[0106] It should also be noted that by optimizing the PID control algorithm, the clamping force can be controlled more stably, avoiding product damage or operation failure caused by excessive clamping or loosening.

[0107] The vibration data was windowed using a Hanning window and then subjected to an FFT transform to calculate the 1 / 3 octave band spectrum, yielding frequency band vibration energy distribution data (in dB or m / s). 2 The frequency band vibration energy distribution data is compared with the vibration baseline of the same period in the past 30 days to obtain the Z-score value. The vibration baseline is a reference curve established by statistical analysis of the vibration data (such as calculating the mean, standard deviation and probability distribution). The frequency band with Z-score > 3 is marked as the sensitive frequency band.

[0108] The values ​​of the process noise covariance matrix elements corresponding to the sensitive frequency bands are increased by 30-50% (linearly adjusted according to the Z-score value), while the elements of the non-sensitive frequency bands remain unchanged. An exponential smooth transition is used for the process noise covariance matrix elements corresponding to the sensitive frequency bands, with a time constant of 5 minutes.

[0109] 1000 defect detection samples were collected. Each sample included multispectral image data (8 channels, 2048×2048 resolution), 3D point cloud data (0.01mm accuracy), and manually re-inspected labels. The backpropagation algorithm was used to calculate the gradient vector of the loss function for each defect detection sample across all output categories. Then, the L2 norm (Euclidean norm) of the gradient vector for each defect detection sample was calculated to obtain the gradient norm of the defect detection sample across all categories. This norm was used to evaluate the importance of the sample to the parameter updates of the ResNet50 and U-Net models. The top 10% of defect detection samples by gradient norm were assigned a standard weight of 3, the middle 60% maintained their standard weight, and the bottom 30% were reduced to a standard weight of 0.5, with a standard weight of 1. The loss function gradient of the current batch of defect detection samples was used. The diagonal elements of the Fisher information matrix are calculated using the squared mean. The gradients of each ResNet50 and U-Net model parameter are obtained through backpropagation, squared, and then averaged in batches to obtain the importance estimate of each ResNet50 and U-Net model parameter. ResNet50 and U-Net model parameters whose importance estimate is more than 1.5 times the median are marked as important parameters. A protection threshold is set for important parameters, allowing a maximum change of 5% per update. The ResNet50 and U-Net model parameters are updated using stochastic gradient descent with momentum. The learning rate is set to 0.001 and the momentum coefficient is set to 0.9. The historical accuracy is verified every 100 iterations. If the historical accuracy drops by more than 2%, elastic weight solidification is triggered, and a Fisher regularization term is added to force the retention of important parameters.

[0110] It should also be noted that the anti-saturation PID and elastic model update dynamically adjust the noise covariance through Z-score frequency band analysis (50% improvement in sensitive frequency bands), combined with elastic weight solidification (5% update threshold + Fisher regularization), so as to avoid performance degradation caused by incremental learning of the model while ensuring that the clamping force compensation error is ≤±1.5N.

[0111] This embodiment also provides a machine vision-based wire harness production quality inspection system, including:

[0112] The correction module collects wire harness data, including wire harness surface images, wire harness internal structure perspective views, and vibration data. It preprocesses the wire harness data, performs geometric correction on the wire harness surface images, and eliminates field-of-view deviations on the wire harness internal structure perspective views.

[0113] The fusion module uses ResNet50 to extract features from the surface image of the wire harness and generates a surface feature map. It uses U-Net to segment defects in the perspective view of the internal structure of the wire harness and generates an internal feature map. The surface feature map and the internal feature map are fused to obtain a fused feature map.

[0114] The analysis module calculates the defect severity score and the correlation coefficient between defect data and various production parameters, and identifies highly correlated production parameters. Based on the fused feature map and defect severity score, it uses the Gaussian distribution function to calculate the defect impact value at each grid center point, constructs a defect heat map, and uses Bayesian network analysis to analyze the causal relationship between production parameters and defects.

[0115] The update module establishes a three-dimensional Gaussian force field distribution centered on the defect heat map, updates the clamping force using an anti-saturation PID algorithm, adjusts the process noise covariance matrix based on vibration data, and updates the ResNet50 and U-Net model parameters using defect detection samples.

[0116] This embodiment also provides a computer device applicable to the case of a machine vision-based wire harness production quality inspection method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the machine vision-based wire harness production quality inspection method proposed in the above embodiment.

[0117] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0118] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the machine vision-based wire harness production quality inspection method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0119] In summary, this invention achieves effective correction of image acquisition distortion in dynamic environments by iteratively calculating the camera pose offset using state transition equations and Kalman filtering algorithms. This improves the stability and accuracy of the detection method, especially in complex production environments, ensuring high-quality data acquisition and analysis results and solving the problem of low detection accuracy in dynamic environments. Furthermore, by stitching together channels to generate a joint feature map, and using dynamic weights adjusted based on the vibration compensation parameter matrix, the surface and internal feature maps are weighted and fused to generate the final fused feature map for defect classification. This enables comprehensive analysis of internal and external defects in the wire harness, overcoming the limitations of single-modal detection and solving the problem of insufficient multi-source data fusion.

[0120] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A machine vision-based method for inspecting the production quality of wire harnesses, characterized in that: include, Data on the wire harness is collected, including images of the wire harness surface, perspective views of the internal structure of the wire harness, and vibration data. The wire harness data is preprocessed, geometric correction is performed on the wire harness surface images, and field deviation is eliminated on the perspective views of the internal structure of the wire harness. Features of the wire harness surface image are extracted using ResNet50 to generate a surface feature map. Defects in the perspective view of the internal structure of the wire harness are segmented using U-Net to generate an internal feature map. The surface feature map and the internal feature map are then fused to obtain a fused feature map. Calculate the defect severity score and the correlation coefficient between defect data and each production parameter, and identify highly correlated production parameters. Based on the fused feature map and defect severity score, use the Gaussian distribution function to calculate the defect impact value at each grid center point, construct a defect heat map, and use Bayesian network analysis to analyze the causal relationship between production parameters and defects. A three-dimensional Gaussian force field distribution is established centered on the defect heat map. The clamping force is updated using an anti-saturation PID algorithm. The process noise covariance matrix is ​​adjusted based on vibration data. The parameters of ResNet50 and U-Net models are updated using defect detection samples.

2. The machine vision-based wire harness production quality inspection method as described in claim 1, characterized in that: The specific steps for using Bayesian networks to analyze the causal relationship between production parameters and defects are as follows: A time series anomaly detection algorithm is used to analyze production parameters that are determined to be highly correlated, and to identify periods of abnormal fluctuation in production parameters; Calculate the time overlap index and construct a causal network diagram between production parameters and defects based on historical production parameter data, defect types, and equipment IDs; The time overlap index is used as an observation variable in a Bayesian network to analyze the causal relationship between production parameters and defects.

3. The machine vision-based wire harness production quality inspection method as described in claim 1, characterized in that: The specific steps for establishing a three-dimensional Gaussian force field distribution centered on the defect heatmap and updating the clamping force using an anti-saturation PID algorithm are as follows: The three-dimensional coordinates of the defects are extracted from the defect heat map. A three-dimensional Gaussian force field distribution is established with the three-dimensional coordinates of the defects as the center. The force field intensity coefficient is linearly adjusted according to the defect severity score. The force field intensity coefficient is dynamically adjusted using a depth compensation factor. The adoption of an anti-saturation PID algorithm to update the clamping force refers to using dynamic gain in the proportional stage, setting integral limiting in the integral stage, and adding low-pass filtering in the derivative stage.

4. The machine vision-based wire harness production quality inspection method as described in claim 1, characterized in that: The specific steps for updating the ResNet50 and U-Net model parameters using defect detection samples are as follows: Collect defect detection samples and use the backpropagation algorithm to calculate the gradient vector of the loss function for each defect detection sample across all output categories; Calculate the gradient norm of the defect detection sample across all categories, and assign a weight to each defect detection sample; Calculate the importance estimates of the parameters of ResNet50 and U-Net models, mark important parameters based on the importance estimates, and set protection thresholds for important parameters; Update the ResNet50 and U-Net model parameters using stochastic gradient descent with momentum, and set the learning rate and momentum coefficient.

5. The machine vision-based wire harness production quality inspection method as described in claim 1, characterized in that: The specific steps for fusing the surface feature map and the internal feature map to obtain the fused feature map are as follows: ResNet50 was used as the backbone network to extract features from the surface image of the wire harness and generate a surface feature map. U-Net was used as the backbone network to segment the internal copper wire fractures and insulation bubble defects in the internal structural perspective of the wire harness. The features of the internal structural perspective of the wire harness were extracted through an encoder-decoder structure and an internal feature map was generated. The surface feature map and the internal feature map are concatenated along the channel dimension to generate a joint feature map, and the dynamic weights of the surface feature map and the internal feature map are calculated using the Sigmoid activation function. Dynamic weights are used to weight and fuse the surface feature map and the internal feature map to generate a fused feature map.

6. The machine vision-based wire harness production quality inspection method as described in claim 1, characterized in that: The specific steps for calculating the defect severity score and the correlation coefficient between defect data and various production parameters, and determining highly correlated production parameters, are as follows: Use regression analysis to fit historical defect data; Calculate the defect severity score and classify the defect level; The Pearson correlation coefficient was used to calculate the correlation coefficient between defect data and various production parameters, and the highly correlated production parameters were identified.

7. The machine vision-based wire harness production quality inspection method as described in claim 1, characterized in that: The specific steps for performing geometric correction on the surface image of the wire harness and eliminating field-of-view deviation on the perspective view of the internal structure of the wire harness are as follows: Calculate the observed values ​​using the observation equation; The camera position and attitude at the next time step are predicted using the state transition equation, the covariance matrix at the next time step is predicted and updated using the state transition matrix and the process noise covariance matrix, and the Kalman gain is calculated. The camera position and attitude are updated by the difference between the observed values ​​and the camera position and attitude at the next time step, and by the Kalman gain. The covariance matrix is ​​adjusted by the Kalman gain and the observation matrix. The camera pose offset is calculated by performing a prediction-update loop iteration; Construct a homography matrix and perform geometric correction on the wire harness surface image; Calculate the hand-eye calibration matrix and transform the perspective view of the internal structure of the wire harness to the coordinate system of the wire harness surface image using the hand-eye calibration matrix.

8. A machine vision-based wire harness production quality inspection system, based on the machine vision-based wire harness production quality inspection method according to any one of claims 1 to 7, characterized in that: include, The correction module collects wire harness data, including wire harness surface images, wire harness internal structure perspective views, and vibration data. It preprocesses the wire harness data, performs geometric correction on the wire harness surface images, and eliminates field-of-view deviations on the wire harness internal structure perspective views. The fusion module uses ResNet50 to extract features from the surface image of the wire harness and generates a surface feature map. It uses U-Net to segment defects in the perspective view of the internal structure of the wire harness and generates an internal feature map. The surface feature map and the internal feature map are fused to obtain a fused feature map. The analysis module calculates the defect severity score and the correlation coefficient between defect data and various production parameters, and identifies highly correlated production parameters. Based on the fused feature map and defect severity score, it uses the Gaussian distribution function to calculate the defect impact value at each grid center point, constructs a defect heat map, and uses Bayesian network analysis to analyze the causal relationship between production parameters and defects. The update module establishes a three-dimensional Gaussian force field distribution centered on the defect heat map, updates the clamping force using an anti-saturation PID algorithm, adjusts the process noise covariance matrix based on vibration data, and updates the ResNet50 and U-Net model parameters using defect detection samples.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the machine vision-based wire harness production quality inspection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the machine vision-based wire harness production quality inspection method according to any one of claims 1 to 7.

Images