Wire harness production quality detection system and method based on machine vision

Abstract

The invention discloses a wire harness production quality detection system and method based on machine vision, and relates to the technical field of industrial automatic detection, and the method comprises the steps: collecting wire harness data including a wire harness surface image, a wire harness internal structure perspective view and vibration data, preprocessing the wire harness data, carrying out the geometric correction of the wire harness surface image, and carrying out the detection of the wire harness production quality. Performing view field deviation elimination on the perspective view of the internal structure of the wire harness; extracting features of the surface image of the wire harness by using ResNet50 to generate a surface feature map, segmenting defects of the perspective view of the internal structure of the wire harness by using U-Net to generate an internal feature map, and fusing the surface feature map and the internal feature map to obtain a fused feature map. According to the method, the pose offset of the camera is iteratively calculated through the state transition equation and the Kalman filtering algorithm, effective correction of image acquisition distortion in the dynamic environment is realized, the stability and accuracy of the detection method are improved, and the problem of low detection precision in the dynamic environment is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial automation detection, and particularly to a wire harness production quality detection system and method based on machine vision. Background Art

[0002] With the rapid development of industrial automation, wire harnesses, as key components in modern electronic devices and automotive manufacturing, their production quality directly affects the performance and reliability of products. Traditional wire harness quality detection methods mainly rely on manual visual inspection or simple optical detection equipment, but these methods have problems such as low efficiency, strong subjectivity, and difficulty in detecting internal defects.

[0003] In recent years, detection methods based on machine vision, supported by high-resolution cameras and deep learning algorithms, have achieved efficient recognition of surface defects. However, existing technologies mostly focus on single-modal detection (such as only surface or internal), lacking research on multi-source data fusion, which limits the comprehensiveness and accuracy of detection. At the same time, the detection accuracy problem in a dynamic environment has not been fully solved, and vibration interference may cause image acquisition distortion, affecting the detection effect. These two aspects indicate the importance of improving multi-source data fusion and detection accuracy in a dynamic environment. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a wire harness production quality detection method based on machine vision to solve the problems of insufficient multi-source data fusion and low detection accuracy in a dynamic environment.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a wire harness production quality detection method based on machine vision, which includes collecting wire harness data, including wire harness surface images, wire harness internal structure perspective views, and vibration data, preprocessing the wire harness data, geometrically correcting the wire harness surface images, and eliminating the field-of-view deviation of the wire harness internal structure perspective views;

[0008] Using ResNet50 to extract the features of the wire harness surface images to generate surface feature maps, using U-Net to segment the defects of the wire harness internal structure perspective views to generate internal feature maps, and fusing the surface feature maps and the internal feature maps to obtain fused feature maps;

[0009] Calculate the defect severity score and the correlation coefficient between the defect data and each production parameter, determine the highly correlated production parameters, calculate the defect impact value of the center point of each grid based on the fused feature map and the defect severity score, construct a defect heat map, and use a Bayesian network to analyze the causal relationship between production parameters and defects;

[0010] Establish a three-dimensional Gaussian force field distribution centered on the defect heat map, use an anti-saturation PID algorithm to update the clamping force, adjust the process noise covariance matrix according to the vibration data, and update the ResNet50 and U-Net model parameters using the defect detection samples.

[0011] As a preferred solution of the machine vision-based wire harness production quality detection method described in the present invention, wherein: the step of using a Bayesian network to analyze the causal relationship between production parameters and defects is as follows:

[0012] Use a time series anomaly detection algorithm to analyze the production parameters determined to be highly correlated, and identify the abnormal fluctuation period of the production parameters;

[0013] Calculate the time coincidence index, and construct a causal network diagram of production parameters and defects based on the historical records of production parameters, defect types, and equipment IDs;

[0014] Use the time coincidence index as an observation variable of the Bayesian network to analyze the causal relationship between production parameters and defects.

[0015] As a preferred solution of the machine vision-based wire harness production quality detection method described in the present invention, wherein:

[0016] The step of establishing a three-dimensional Gaussian force field distribution centered on the defect heat map and using an anti-saturation PID algorithm to update the clamping force is as follows:

[0017] Extract the three-dimensional coordinates of the defect from the defect heat map, establish a three-dimensional Gaussian force field distribution centered on the three-dimensional coordinates of the defect, linearly adjust the force field intensity coefficient according to the defect severity score, and dynamically adjust the force field intensity coefficient using a depth compensation factor;

[0018] The use of an anti-saturation PID algorithm to update the clamping force means using a dynamic gain in the proportional link, setting an integral limit in the integral link, and adding a low-pass filter in the derivative link.

[0019] As a preferred solution of the machine vision-based wire harness production quality detection method described in the present invention, wherein:

[0020] The step of using the defect detection samples to update the ResNet50 and U-Net model parameters is as follows:

[0021] Collect defect detection samples, and use the backpropagation algorithm to calculate the loss function gradient vectors of each defect detection sample for all output categories;

[0022] Calculate the gradient norms of the defect detection samples for all categories, and assign weights to each defect detection sample;

[0023] Calculate the importance estimation values of the ResNet50 and U-Net model parameters, mark the important parameters according to the importance estimation values, and set protection thresholds for the important parameters;

[0024] Use the stochastic gradient descent method with momentum to update the ResNet50 and U-Net model parameters, and set the learning rate and momentum coefficient.

[0025] As a preferred solution of the wire harness production quality detection method based on machine vision according to the present invention, wherein:

[0026] The fusion of the surface feature map and the internal feature map to obtain a fusion feature map is specifically as follows:

[0027] Use ResNet50 as the backbone network to extract the features of the wire harness surface image and generate a surface feature map;

[0028] Use U-Net as the backbone network to segment the internal copper wire breakage and insulation layer bubble defects in the perspective view of the wire harness internal structure, extract the features of the wire harness internal structure perspective view through the encoder-decoder structure, and generate an internal feature map;

[0029] Stitch the surface feature map and the internal feature map in the channel dimension to generate a joint feature map, and calculate the dynamic weights of the surface feature map and the internal feature map through the Sigmoid activation function;

[0030] Use the dynamic weights to perform weighted fusion on the surface feature map and the internal feature map to generate a fusion feature map.

[0031] As a preferred solution of the wire harness production quality detection method based on machine vision according to the present invention, wherein:

[0032] The calculation of the defect severity score and the correlation coefficient between the defect data and each production parameter, and the determination of highly correlated production parameters are specifically as follows:

[0033] Use regression analysis to fit the historical defect data;

[0034] Calculate the defect severity score and divide the defect levels;

[0035] Use the Pearson correlation coefficient to calculate the correlation coefficient between the defect data and each production parameter, and determine the highly correlated production parameters.

[0036] As a preferred solution of the wire harness production quality detection method based on machine vision according to the present invention, wherein:

[0037] For geometric correction of the wire harness surface image and elimination of the field-of-view deviation of the perspective view of the wire harness internal structure, the specific steps are as follows:

[0038] Calculate the observed value using the observation equation;

[0039] Predict the camera position and attitude at the next moment using the state transition equation, predict and update the covariance matrix at the next moment using the state transition matrix and the process noise covariance matrix, and calculate the Kalman gain;

[0040] Update the camera position and attitude based on the difference between the observed value and the camera position and camera attitude at the next moment and the Kalman gain, and adjust the covariance matrix using the Kalman gain and the observation matrix;

[0041] Calculate the camera pose offset by performing the prediction-update loop iteration;

[0042] Construct the homography matrix and perform geometric correction on the wire harness surface image;

[0043] Calculate the hand-eye calibration matrix and transform the perspective view of the wire harness internal structure to the wire harness surface image coordinate system through the hand-eye calibration matrix.

[0044] In a second aspect, the present invention provides a wire harness production quality detection system based on machine vision, including,

[0045] A calibration module, which collects wire harness data, including the wire harness surface image, the perspective view of the wire harness internal structure, and vibration data, preprocesses the wire harness data, performs geometric correction on the wire harness surface image, and eliminates the field-of-view deviation of the perspective view of the wire harness internal structure;

[0046] A fusion module, which extracts the features of the wire harness surface image using ResNet50 to generate a surface feature map, segments the defects of the perspective view of the wire harness internal structure using U-Net to generate an internal feature map, and fuses the surface feature map and the internal feature map to obtain a fused feature map;

[0047] An analysis module, which calculates the defect severity score and the correlation coefficient between the defect data and each production parameter, determines the highly correlated production parameters, calculates the defect influence value of each grid center point based on the fused feature map and the defect severity score, constructs a defect heat map, and analyzes the causal relationship between the production parameters and the defects using a Bayesian network;

[0048] Update module, establish a three-dimensional Gaussian force field distribution centered on the defect heat map, use an anti-saturation PID algorithm to update the clamping force, adjust the process noise covariance matrix according to the vibration data, and update the ResNet50 and U-Net model parameters using the defect detection samples.

[0049] In a third aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the computer program is executed by the processor, any step of the machine vision-based wire harness production quality detection method described in the first aspect of the present invention is implemented.

[0050] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by the processor, any step of the machine vision-based wire harness production quality detection method described in the first aspect of the present invention is implemented.

[0051] The beneficial effects of the present invention are as follows: By iteratively calculating the pose offset of the camera through the state transition equation and the Kalman filter algorithm, effective correction of image acquisition distortion in a dynamic environment is achieved, improving the stability and accuracy of the detection method. Especially in a complex production environment, high-quality data acquisition and analysis results are ensured, and the problem of low detection accuracy in a dynamic environment is solved; By splicing through the channel dimension to generate a joint feature map, and based on the dynamic weights adjusted by the vibration compensation parameter matrix, weighted fusion of the surface and internal feature maps is performed to generate the final fusion feature map for defect classification, realizing a comprehensive analysis of internal and external defects of the wire harness, overcoming the limitations of single-modal detection, and solving the problem of insufficient multi-source data fusion. Description of the Drawings

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0053] Figure 1 It is a flowchart of the machine vision-based wire harness production quality detection method in Embodiment 1.

[0054] Figure 2 It is a schematic diagram of the machine vision-based wire harness production quality detection system in Embodiment 1.

[0055] Figure 3 It is a schematic diagram of generating a three-dimensional defect heat map in Embodiment 1

[0056] Figure 4 It is a schematic diagram of the anti-saturation PID control algorithm in Embodiment 1 Detailed implementation manners

[0057] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the detailed implementation manners of the present invention with reference to the accompanying drawings of the specification.

[0058] In the following description, many specific details are set forth to facilitate a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0059] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The appearances of "in one embodiment" in different places in this specification do not all refer to the same embodiment, nor are they separate or alternative embodiments that exclude each other with other embodiments.

[0060] Embodiment 1, referring to Figure 1 、 Figure 2 、 Figure 3 and Figure 4 , this embodiment provides a method for detecting the production quality of wire harnesses based on machine vision, including the following steps:

[0061] S1: Collect wire harness data, including the surface image of the wire harness, the perspective view of the internal structure of the wire harness, and vibration data. Preprocess the wire harness data, perform geometric correction on the surface image of the wire harness, and eliminate the field-of-view deviation of the perspective view of the internal structure of the wire harness.

[0062] The specific steps are as follows:

[0063] Install a high-resolution industrial camera (resolution ≥ 12MP) on the detection platform, ensure that the distance between the camera and the wire harness is 50 cm, the viewing angle is 45°, collect the surface image of the wire harness, set the camera frame rate to 30 fps, and automatically adjust the exposure time to ensure clear images.

[0064] It should also be noted that: Using a high-resolution industrial camera can capture the subtle defect features on the surface of the wire harness, improving the accuracy and reliability of detection.

[0065] Install a micro X-ray machine on the detection platform, the distance between the micro X-ray machine and the wire harness is 30 cm, the viewing angle is 90°, collect the perspective view of the internal structure of the wire harness, set the power of the micro X-ray machine to 50 kV, and the exposure time is 0.1 s to ensure clear perspective views.

[0066] It should also be noted that by using a micro X-ray machine for internal structure fluoroscopy inspection, defects inside the wire harness, such as broken copper wires and insulation layer bubbles, can be detected without damage, ensuring the safety of product quality.

[0067] Install a MEMS inertial sensor on the high-resolution industrial camera bracket to monitor vibration data in real time, including the vibration acceleration (range ±16g) and vibration angular velocity (range ±2000° / s) of the wire harness in three-dimensional space. The sensor sampling frequency is set to 100Hz to ensure high-precision acquisition of vibration data.

[0068] It should also be noted that by monitoring vibration data in real time and aligning it with image data, the state of the wire harness can be analyzed more precisely in a dynamic environment, improving the accuracy of the detection results.

[0069] Align the wire harness surface image, the internal structure perspective view of the wire harness, and the vibration data according to the timestamps generated by the PTP protocol to ensure that the timestamps of each frame of image, perspective view, and vibration data are consistent.

[0070] Perform a fast Fourier transform (FFT) on the vibration data. The FFT window length is 1024, the sampling frequency is 100Hz, and the frequency resolution is 0.1Hz.

[0071] Calculate the difference between the mean value of the vibration acceleration in the static state and the gravitational acceleration to obtain the static acceleration offset. Integrate the static acceleration offset with respect to time twice to obtain the static position offset.

[0072] Integrate the mean value of the vibration angular velocity in the static state with respect to time to obtain the static rotation angle increment. Use quaternion mapping to convert the static rotation angle increment into a quaternion increment, update the current attitude through quaternion multiplication, and record the static quaternion attitude offset.

[0073] Integrate the static position offset and the static quaternion attitude offset into the static offset.

[0074] Update the current position and attitude of the camera through the state transition equation. The input control quantities are the vibration acceleration and vibration angular velocity, and the process noise is Gaussian distributed: Integrate the vibration acceleration after adding the process noise with respect to the acquisition interval time to update the current velocity. Integrate the current velocity after adding the process noise with respect to the acquisition interval time to update the current displacement offset and update the current position of the camera. Integrate the angular velocity after adding the process noise with respect to the acquisition interval time to obtain the rotation angle increment and update the rotation angle. Use quaternion mapping to convert the rotation angle increment into a quaternion increment, including decomposing the rotation angle increment into the rotation axis and rotation angle, and converting it into a quaternion increment according to the right-hand rule. Multiply it with the current attitude quaternion after adding Gaussian noise through quaternion multiplication to update the current attitude of the camera and record the quaternion attitude offset.

[0075] Based on the current position of the camera, the current attitude of the camera, and the static offset, the observed value is calculated using the observation equation, and the observation noise is Gaussian distributed.

[0076] The camera pose offset is iteratively calculated through the prediction-update step: Based on the current vibration acceleration, vibration angular velocity, the current position of the camera, and the current attitude of the camera, the state transition equation is used to predict the camera position and attitude at the next moment, and the state transition matrix and the process noise covariance matrix are used to predict and update the covariance matrix at the next moment; Based on the predicted covariance matrix and the observation noise covariance matrix, the Kalman gain is calculated, and the camera position and attitude are updated through the difference between the observed value and the camera position and attitude at the next moment and the Kalman gain, and the covariance matrix is updated through the Kalman gain and the observation matrix to reduce the error; The prediction-update loop is executed every 0.01 seconds to iteratively calculate the camera pose offset.

[0077] A homography matrix is constructed based on the camera pose offset, including translation, rotation, and scaling parameters, which describes the geometric transformation relationship between image planes. The homography matrix is used to geometrically correct the image of the wire harness surface: Feature points are extracted between the image of the wire harness surface and the preset target plane through the ORB algorithm, and the corresponding feature point pairs are matched. The RANSAC algorithm is used to fit the homography matrix, and the inverse transformation of the homography matrix is applied to the image of the wire harness surface to project the image onto the target plane to eliminate the perspective distortion. Sub-pixel compensation is achieved by bilinearly interpolating the transformed pixels to ensure that the correction error is controlled within 0.05 pixels.

[0078] Corner detection is performed on the perspective view of the internal structure of the wire harness through a checkerboard and combined with the least squares method to fit and calculate the calibration plate transformation matrix. According to the camera pose offset and the calibration plate transformation matrix, the hand-eye calibration equation is established. The rotation component and translation component of the hand-eye calibration matrix are disassembled into independent variables, the rotation component is solved using the singular value decomposition (SVD), the rotation component is substituted into the translation equation and the translation component is solved using the least squares method, and the rotation component and translation component are combined to obtain the hand-eye calibration matrix. The perspective view of the internal structure of the wire harness is transformed to the image coordinate system of the wire harness surface image through the hand-eye calibration matrix to eliminate the field of view deviation.

[0079] The quaternion attitude offset is converted into a rotation matrix, the position offset is converted into a translation vector, and a pose transformation matrix is constructed. The pose transformation matrix is combined with the camera internal parameter matrix to generate a vibration compensation parameter matrix.

[0080] It should also be noted that: By synchronizing, analyzing, and processing the time of the image of the wire harness surface, the perspective view of the internal structure of the wire harness, and the vibration data, as well as calculating the camera pose offset, the detection accuracy can be improved, thereby providing a high-quality data basis for subsequent defect identification.

[0081] S2: Use ResNet50 to extract the features of the wire harness surface image, generate the surface feature map, use U-Net to segment the defects in the perspective view of the internal structure of the wire harness, generate the internal feature map, and fuse the surface feature map and the internal feature map to obtain the fused feature map.

[0082] The specific steps are as follows:

[0083] Use ResNet50 as the backbone network, remove the fully connected classification layer at the top, and retain the convolutional layer and the residual block structure. The convolutional layer receives a three-channel RGB image and outputs a 64-channel feature map to capture the geometric features (such as edges and corners) of the wire harness surface. The shallow residual block group, consisting of 3 residual units, outputs a 256-channel feature map to extract the local details of the wire harness texture (such as the discontinuous edges of small scratches). The middle residual block group consists of 4 residual units and outputs a 512-channel feature map to learn the complex texture features at the junction of the tape and the wire on the wire harness surface. The deep residual block group consists of 6 residual units and outputs a 1024-channel feature map to capture the global structure information (such as the continuous trend of long scratches or the overall deformation of tape loosening). The end residual block group, consisting of 3 residual units, outputs a 2048-channel feature map to encode high-dimensional semantic information (such as defect category and severity). Extract the surface scratch and tape loosening features of the wire harness surface image, extract the edge and corner features of the wire harness surface image through the convolutional layer, extract the texture features of the wire harness surface image through the shallow residual block group, use the middle residual block group to identify the fracture pattern of surface scratches and the local deformation of tape loosening, use the deep residual block group to capture the global structure information (such as the continuous trend of long scratches or the overall deformation of tape loosening). Calculate the global average value of the pixel values for each channel on the 2048-channel feature map output by the end residual block group to generate a 2048-dimensional global feature vector and encode the high-level semantics (such as the defect category label "scratch" or "tape loosening"). Upsample the 1024-channel feature map (14×14) of the deep residual block group by 4 times so that its size is aligned with the 56×56 feature map of the shallow residual block group. Concatenate the first convolutional layer (64 channels), the shallow residual block group (256 channels), and the upsampled deep features (1024 channels) along the channel dimension to form a 1344-channel mixed feature map. Compress the 1344 channels to 256 channels through a 1×1 convolution to generate the surface feature map.

[0084] It should also be noted that: The deep learning method can identify complex surface defect features and improve the detection efficiency and accuracy.

[0085] Using U-Net as the backbone network, which includes an encoder (feature extraction) and a decoder (resolution restoration). The encoder includes an input layer: receiving the perspective view of the internal structure of the wire harness; a convolutional module: each level contains two repeated convolutional operations, followed by batch normalization and ReLU activation function after each convolution, and finally downsampling is performed through 2×2 max pooling. The hierarchical depth: there are 4 levels of downsampling in total, and the resolution is halved at each level (512-256-128-64-32), and finally a deep feature map with a resolution of 32×32 is output; the decoder includes an upsampling module: the resolution is doubled at each level through bilinear interpolation, feature concatenation: the feature map of the same level in the encoder is concatenated with the upsampled feature map in the decoder in the channel dimension to retain detailed and semantic information, and the output layer: the last layer uses 1×1 convolution with the Sigmoid activation function to generate a binary segmentation mask with the same size as the input, identifying the defective area, segmenting the internal copper wire breakage and insulation layer bubble defects in the perspective view of the internal structure of the wire harness, and extracting multi-scale features of the perspective view of the internal structure of the wire harness through the encoder-decoder structure, including shallow features: capturing high-resolution details, such as small breaks at the edges of copper wires and local contours of insulation layer bubbles; middle features: identifying medium-scale structures, such as continuity interruptions of copper wire breaks and aggregated forms of bubbles; deep features: understanding global semantic information, such as the overall layout of the wire harness and the distribution density of defective areas. The shallow features, middle features, and deep features are concatenated to generate an internal feature map.

[0086] It should also be noted that: The U-Net architecture is good at handling segmentation tasks in fields such as medical imaging. Applying the U-Net architecture to the internal defect detection of wire harnesses can also effectively improve the accuracy of defect location and classification.

[0087] The surface feature map and the internal feature map are concatenated in the channel dimension to generate a joint feature map; based on the vibration compensation parameter matrix and the joint feature map, the dynamic weights of the surface feature map and the internal feature map are calculated through the Sigmoid activation function. The elements of the vibration compensation parameter matrix are normalized to the range of -1 to 1, and are extended to a vibration parameter matrix that matches the spatial dimension of the joint feature map through a fully connected layer. The joint feature map and the vibration parameter matrix are concatenated in the channel dimension to form a fusion input, and a weight matrix of the surface feature map and the internal feature map is generated through the Sigmoid activation function to adjust the contributions of the surface feature map and the internal feature map. The surface feature map and the internal feature map are weighted and fused using the dynamic weights to generate a fused feature map.

[0088] It should also be noted that: By dynamically adjusting the weights of different feature maps, the importance of features can be optimized according to the actual situation, thereby improving the reliability of the classification results.

[0089] Use a fully connected layer to classify the fused feature map to obtain a defect category probability vector, calculate the probabilities of each defect category through the Softmax function, and determine the defect type.

[0090] S3: Calculate the defect severity score and the correlation coefficients between the defect data and each production parameter, and determine the highly correlated production parameters. Based on the fused feature map and the defect severity score, use the Gaussian distribution function (with σ value being 1 times the grid spacing, the Gaussian distribution covariance matrix being isotropic, and μ being the coordinates of the defect point) to calculate the defect influence value at the center point of each grid, construct a defect heat map, and use a Bayesian network to analyze the causal relationship between the production parameters and the defects.

[0091] The specific steps are as follows:

[0092] Based on historical defect data including surface scratch length, internal copper wire breakage situation, and the corresponding defect severity scores, use regression analysis to fit the historical defect data to obtain the surface scratch length weight and the internal copper wire breakage weight. Use the method of dividing the surface scratch length by the preset maximum surface scratch length (10 mm) to normalize the surface scratch length to the range of 0 to 1, directly assign a dimensionless value to the internal copper wire breakage situation (1 for breakage and 0 for non-breakage), and perform a weighted sum of the surface scratch length and the internal copper wire breakage situation to obtain the defect severity score.

[0093] It should also be noted that: The calculation of the defect severity score not only considers the surface scratch length but also the internal copper wire breakage situation, comprehensively evaluates the quality status of the product, and helps to accurately judge whether the product meets the standards.

[0094] Divide the defect levels (fatal / serious / general) according to the defect severity score. Set the defects with a defect severity score greater than or equal to 0.8 as fatal defects, the defects with a defect severity score greater than or equal to 0.5 and less than 0.8 as serious defects, and the defects with a defect severity score less than 0.5 as general defects.

[0095] It should also be noted that: The clear defect classification standard is convenient for quick decision-making and has important guiding significance for taking corresponding measures in a timely manner.

[0096] Retrieve historical defect records from SPC (Statistical Process Control), calculate the covariance between the defect data and each production parameter (vibration frequency, temperature, humidity, pressure, equipment operating speed), divide the covariance by their respective standard deviations to obtain the correlation coefficients between the defect data and each production parameter. If the correlation coefficient is greater than 0.8, it is determined to be highly correlated.

[0097] It should also be noted that: By analyzing the relationship between defects and production parameters, key factors affecting product quality can be identified, and then the production process can be optimized.

[0098] A right - hand coordinate system is established with the product entity as the reference. The position of the corner point feature of the product is selected as the coordinate origin (0, 0, 0). The X, Y, and Z axes are set respectively in the length, width, and height directions of the product. The coordinate unit is set to 0.1 mm. The defect positions in the fusion feature map are uniformly transformed into the right - hand coordinate system through a coordinate transformation matrix to obtain a three - dimensional space. The three - dimensional space is subjected to grid - based discrete processing, and the product volume is divided into uniform small cube units (such as 0.1 mm×0.1 mm×0.1 mm). Based on the fusion feature map and the defect severity score, the Gaussian distribution function is used to calculate the defect influence value of each grid center point. The defect influence values of all grid center points are normalized and mapped to the color scale range of 0 - 255. A color gradient scale is established, and the OpenGL (Open Graphics Library) technology is used to construct a three - dimensional scene. The thermal distribution is presented by semi - transparent gradient - colored solid cubes to obtain a defect thermal map.

[0099] It should also be noted that: The three - dimensional thermal map can intuitively display the defect distribution and its severity.

[0100] The time - series anomaly detection algorithm (such as STL decomposition combined with the 3σ principle) is used to analyze the production parameters determined to be highly correlated, identify the abnormal fluctuation periods of production parameters that coincide with the defect occurrence periods, perform axis superposition on the abnormal fluctuation periods of production parameters and the defect occurrence periods, and calculate the time coincidence index (coincidence duration divided by the total abnormal duration). The lead and lag time relationships are marked. Based on the historical records of production parameters, defect types, and equipment IDs, a causal network diagram of production parameters and defects is constructed. The time coincidence index is used as an observed variable of the Bayesian network, combined with the lead and lag time relationship marks. By calculating the conditional probability dependence relationship and d - separation test, combined with the rule constraints of the process knowledge base, the production parameters directly related to the defects (direct causes) and the auxiliary factors (indirect causes) that affect through other production parameters are distinguished, and the controllable key nodes (production parameters with practical operability) are marked in the causal network diagram of production parameters and defects.

[0101] It should also be noted that: By using the time - series anomaly detection algorithm and causal network analysis, the root causes of defect occurrence can be further explored, providing a scientific basis for production process optimization, helping to reduce the defect rate and improve production efficiency.

[0102] S4: A three - dimensional Gaussian force field distribution is established with the defect thermal map as the center. The anti - saturation PID algorithm is used to update the clamping force. The process noise covariance matrix is adjusted according to the vibration data, and the ResNet50 and U - Net model parameters are updated using the defect detection samples.

[0103] The specific steps are as follows:

[0104] Extract the three-dimensional coordinates of the defect from the defect heat map, and establish a three-dimensional Gaussian force field distribution with the three-dimensional coordinates of the defect as the center: Set the isotropic action range parameter, the force field decays uniformly in all directions, specify the maximum force intensity at the center point, the force field intensity decays according to the Gaussian function, determine the covariance matrix of the Gaussian force field distribution by calculating the variance of the isotropic action range parameter (the covariance matrix of the Gaussian force field distribution is a diagonal matrix), calculate the distance from each point in space to the defect center, and adjust the distance contribution in each direction according to the covariance matrix parameters. For example, if the action range in a certain direction is large, the distance decay in this direction is slow, and substitute the distance into the Gaussian function to calculate the force field intensity; Set the action radius of the force field to 15 mm, linearly adjust the force field intensity coefficient according to the defect severity score, take 0.9 for critical defects, 0.6 for major defects, and 0.3 for minor defects, and adopt a depth compensation factor, when the defect depth exceeds 1 / 3 of the product thickness, automatically reduce the adjustment amplitude by 20%.

[0105] Adopt an anti-saturation PID algorithm to update the clamping force: The proportional link (P) adopts a dynamic gain, the gain coefficient is set to 0.9 within 5 mm around the defect, and 0.6 within the range of 5 - 10 mm. The integral link (I) sets an integral limit to prevent long-term cumulative errors, and the limit value is ±15 N. The derivative link (D) adds a low-pass filter, the cut-off frequency is set to 50 Hz to eliminate high-frequency vibration interference, and the output limit is set to [25 N, 85 N], and an alarm is automatically triggered when it exceeds the range.

[0106] It should also be noted that: By optimizing the PID control algorithm, the clamping force can be controlled more stably, avoiding product damage or operation failure caused by over-clamping or loosening.

[0107] Perform windowing processing on the vibration data using a Hanning window and then perform FFT transformation, calculate the 1 / 3 octave spectrum, and obtain the vibration energy distribution data in sub-bands (unit: dB or m / s 2 ); Compare the vibration energy distribution data in sub-bands with the vibration baseline in the same time period in the past 30 days to obtain the Z-score value. The vibration baseline is a reference baseline curve established through statistical analysis of vibration data (such as calculating the mean, standard deviation, and probability distribution), and mark the frequency bands with Z-score > 3 as sensitive frequency bands.

[0108] Increase the numerical values of the elements of the process noise covariance matrix corresponding to the sensitive frequency bands by 30 - 50% (linearly adjusted according to the Z-score value), keep the elements of the non-sensitive frequency bands unchanged, and use exponential smoothing for the elements of the process noise covariance matrix corresponding to the sensitive frequency bands, with the time constant set to 5 minutes.

[0109] Collect 1000 groups of defect detection samples. Each sample includes multi-spectral image data (8 channels, resolution 2048×2048), 3D point cloud data (accuracy 0.01mm), and manual re-inspection labels. Use the backpropagation algorithm to calculate the loss function gradient vector of each defect detection sample on all output categories, and then calculate the L2 norm (Euclidean norm) of the gradient vector of each defect detection sample to obtain the gradient norm of the defect detection sample on all categories, which is used to evaluate the importance of the sample for updating the ResNet50 and U-Net model parameters. Assign 3 times the standard weight to the defect detection samples with the top 10% of the gradient norms, keep the standard weight for the middle 60% of the defect detection samples, and reduce the standard weight of the last 30% of the defect detection samples to 0.5 times the standard weight. The standard weight is 1. Calculate the diagonal elements of the Fisher information matrix using the mean square of the loss function gradient of the current batch of defect detection samples. After obtaining the gradients of each ResNet50 and U-Net model parameter through backpropagation, square them and take the batch average to obtain the importance estimate of each ResNet50 and U-Net model parameter. Mark the ResNet50 and U-Net model parameters with an importance estimate exceeding 1.5 times the median as important parameters. Set a protection threshold for the important parameters, allowing a maximum change of 5% for each update. Use the stochastic gradient descent method with momentum to update the ResNet50 and U-Net model parameters. Set the learning rate to 0.001 and the momentum coefficient to 0.9. Verify the historical accuracy every 100 iterations. If the historical accuracy drops by more than 2%, trigger elastic weight consolidation and increase the Fisher regularization term to force the retention of important parameters.

[0110] It should also be noted that: The anti-saturation PID and elastic model update dynamically adjust the noise covariance through Z-score band analysis (increasing the sensitive band by 50%), combined with elastic weight consolidation (update threshold 5% + Fisher regularization), while the clamping force compensation error ≤ ±1.5N, avoiding the performance degradation of the model caused by incremental learning.

[0111] This embodiment also provides a wire harness production quality detection system based on machine vision, including:

[0112] A calibration module that collects wire harness data, including wire harness surface images, internal structure perspective views of the wire harness, and vibration data, preprocesses the wire harness data, performs geometric calibration on the wire harness surface images, and eliminates the field-of-view deviation of the internal structure perspective views of the wire harness;

[0113] A fusion module that uses ResNet50 to extract the features of the wire harness surface image to generate a surface feature map, uses U-Net to segment the defects of the internal structure perspective view of the wire harness to generate an internal feature map, and fuses the surface feature map and the internal feature map to obtain a fused feature map;

[0114] An analysis module calculates the defect severity score and the correlation coefficients between the defect data and each production parameter, determines the highly correlated production parameters, calculates the defect influence value of the center point of each grid using the Gaussian distribution function based on the fusion feature map and the defect severity score, constructs a defect heat map, and analyzes the causal association between the production parameter and the defect using a Bayesian network;

[0115] An update module establishes a three-dimensional Gaussian force field distribution centered on the defect heat map, updates the clamping force using an anti-saturation PID algorithm, adjusts the process noise covariance matrix according to the vibration data, and updates the ResNet50 and U-Net model parameters using the defect detection samples.

[0116] This embodiment also provides a computer device applicable to the case of a wire harness production quality detection method based on machine vision, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the wire harness production quality detection method based on machine vision proposed in the above embodiment.

[0117] The computer device may be a terminal. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0118] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for detecting the quality of wire harness production based on machine vision as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read-only memory (EEPROM for short), erasable programmable read-only memory (EPROM for short), programmable read-only memory (PROM for short), read-only memory (ROM for short), magnetic memory, flash memory, magnetic disk or optical disk.

[0119] In summary, the present invention: iteratively calculates the pose offset of the camera through the state transition equation and the Kalman filter algorithm, realizes the effective correction of image acquisition distortion in a dynamic environment, improves the stability and accuracy of the detection method, especially in a complex production environment, ensures high-quality data acquisition and analysis results, and solves the problem of low detection accuracy in a dynamic environment; generates a joint feature map by splicing in the channel dimension, and based on the dynamic weights adjusted by the vibration compensation parameter matrix, performs weighted fusion on the surface and internal feature maps to generate the final fusion feature map for defect classification, realizes the comprehensive analysis of internal and external defects of the wire harness, overcomes the limitations of single-modal detection, and solves the problem of insufficient multi-source data fusion.

[0120] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A quality inspection method for wire harness production based on machine vision, characterized in that: Including, Collecting harness data, including harness surface images, perspective views of the internal structure of the harness, and vibration data, preprocessing the harness data, geometrically correcting the harness surface images, and eliminating the field-of-view deviation of the perspective views of the internal structure of the harness; Using ResNet50 to extract the features of the harness surface image to generate a surface feature map, using U-Net to segment the defects in the perspective view of the internal structure of the harness to generate an internal feature map, and fusing the surface feature map and the internal feature map to obtain a fused feature map; Calculating the defect severity score and the correlation coefficient between the defect data and each production parameter, determining the highly correlated production parameters, based on the fused feature map and the defect severity score, using the Gaussian distribution function to calculate the defect influence value of each grid center point, constructing a defect heat map, and using a Bayesian network to analyze the causal relationship between the production parameters and the defects; Establishing a three-dimensional Gaussian force field distribution centered on the defect heat map, using an anti-saturation PID algorithm to update the clamping force, adjusting the process noise covariance matrix according to the vibration data, and using the defect detection samples to update the parameters of the ResNet50 and U-Net models.

2. The quality inspection method for wire harness production based on machine vision according to claim 1, wherein: The specific steps for using the Bayesian network to analyze the causal relationship between the production parameters and the defects are as follows: Using a time series anomaly detection algorithm to analyze the production parameters determined to be highly correlated, and identifying the abnormal fluctuation periods of the production parameters; Calculating the time coincidence index, and constructing a causal network diagram of the production parameters and the defects based on the historical records of the production parameters, the defect types, and the equipment ID; Using the time coincidence index as the observed variable of the Bayesian network to analyze the causal relationship between the production parameters and the defects.

3. The method for detecting the quality of wire harness production based on machine vision according to claim 1, characterized in that: The specific steps for establishing a three-dimensional Gaussian force field distribution centered on the defect heat map and using an anti-saturation PID algorithm to update the clamping force are as follows: Extracting the three-dimensional coordinates of the defects from the defect heat map, establishing a three-dimensional Gaussian force field distribution centered on the three-dimensional coordinates of the defects, linearly adjusting the force field intensity coefficient according to the defect severity score, and dynamically adjusting the force field intensity coefficient using a depth compensation factor; The use of the anti-saturation PID algorithm to update the clamping force means using a dynamic gain in the proportional link, setting an integral limit in the integral link, and adding a low-pass filter in the derivative link.

4. The method for detecting the quality of wire harness production based on machine vision according to claim 1, wherein: The specific steps for using the defect detection samples to update the parameters of the ResNet50 and U-Net models are as follows: Collecting defect detection samples, and using the backpropagation algorithm to calculate the loss function gradient vector of each defect detection sample on all output categories; Calculating the gradient norm of the defect detection samples on all categories, and assigning weights to each defect detection sample; Calculating the importance estimation values of the parameters of the ResNet50 and U-Net models, marking the important parameters according to the importance estimation values, and setting protection thresholds for the important parameters; Using the stochastic gradient descent method with momentum to update the parameters of the ResNet50 and U-Net models, and setting the learning rate and the momentum coefficient.

5. The method for detecting the quality of wire harness production based on machine vision according to claim 1, wherein: The specific steps for fusing the surface feature map and the internal feature map to obtain a fused feature map are as follows: Using ResNet50 as the backbone network to extract the features of the harness surface image and generating a surface feature map; Use U-Net as the backbone network to segment the internal copper wire breakage and insulation layer bubble defects in the perspective view of the wire harness internal structure. Extract the features of the wire harness internal structure perspective view through the encoder-decoder structure and generate an internal feature map; Concatenate the surface feature map and the internal feature map in the channel dimension to generate a combined feature map, and calculate the dynamic weights of the surface feature map and the internal feature map through the Sigmoid activation function; Use the dynamic weights to perform weighted fusion on the surface feature map and the internal feature map to generate a fused feature map.

6. The method for detecting the quality of wire harness production based on machine vision according to claim 1, wherein: The steps of calculating the defect severity score and the correlation coefficient between the defect data and each production parameter, and determining the highly correlated production parameters are as follows: Use regression analysis to fit the historical defect data; Calculate the defect severity score and divide the defect grades; Use the Pearson correlation coefficient to calculate the correlation coefficient between the defect data and each production parameter, and determine the highly correlated production parameters.

7. The method for detecting the production quality of wire harnesses based on machine vision according to claim 1, wherein: The steps of geometrically correcting the wire harness surface image and eliminating the field of view deviation of the wire harness internal structure perspective view are as follows: Use the observation equation to calculate the observed value; Use the state transition equation to predict the camera position and attitude at the next moment, use the state transition matrix and the process noise covariance matrix to predict and update the covariance matrix at the next moment, and calculate the Kalman gain; Update the camera position and attitude through the difference between the observed value and the camera position and camera attitude at the next moment and the Kalman gain, and adjust the covariance matrix through the Kalman gain and the observation matrix; Calculate the camera pose offset through the prediction-update loop iteration; Construct a homography matrix and perform geometric correction on the wire harness surface image; Calculate the hand-eye calibration matrix and convert the wire harness internal structure perspective view to the wire harness surface image coordinate system through the hand-eye calibration matrix.

8. A wire harness production quality detection system based on machine vision, based on the wire harness production quality detection method based on machine vision according to any one of claims 1 to 7, characterized in that: Including, A calibration module that collects wire harness data, including wire harness surface images, wire harness internal structure perspective views, and vibration data, preprocesses the wire harness data, geometrically corrects the wire harness surface images, and eliminates the field of view deviation of the wire harness internal structure perspective views; A fusion module that uses ResNet50 to extract the features of the wire harness surface image to generate a surface feature map, uses U-Net to segment the defects in the wire harness internal structure perspective view to generate an internal feature map, and fuses the surface feature map and the internal feature map to obtain a fused feature map; An analysis module that calculates the defect severity score and the correlation coefficient between the defect data and each production parameter, determines the highly correlated production parameters, calculates the defect influence value of each grid center point based on the fused feature map and the defect severity score, constructs a defect heat map, and uses a Bayesian network to analyze the causal relationship between production parameters and defects; An update module that establishes a three-dimensional Gaussian force field distribution centered on the defect heat map, uses an anti-saturation PID algorithm to update the clamping force, adjusts the process noise covariance matrix according to the vibration data, and updates the ResNet50 and U-Net model parameters using defect detection samples.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the machine vision-based wire harness production quality detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the machine vision-based wire harness production quality inspection method according to any one of claims 1 to 7.

Images