Wiring harness defect intelligent identification system and method based on machine vision

Abstract

The invention discloses a wire harness defect intelligent identification system and method based on machine vision, and relates to the technical field of machine vision, and the method comprises the following steps: obtaining wire harness image information, wire harness electric signals, current and temperature information through a camera and a sensor in detection equipment; performing data preprocessing on the obtained data, and performing feature extraction on the preprocessed data; through a deep learning neural network model, the model is trained by using a deep learning algorithm, the wire harness quality is judged, and color number tube matching detection is carried out; wiring harness defect recognition is divided into two different scenes, the detection process of the wiring harness defects is analyzed for each scene, a corresponding detection report is generated according to the detection result, and intelligent wiring harness defect recognition based on machine vision is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and specifically, to an intelligent recognition system and method for wire harness defects based on machine vision. Background Technique

[0002] With the rapid development of industries such as automobiles and electronics, the wire harness, as a key component connecting electrical devices, its quality and reliability are directly related to the performance and safety of the entire system. In the process of production manufacturing and quality control, the accurate identification of wire harness defects has become the core link to ensure product quality. The accuracy, efficiency, and comprehensiveness of wire harness defect identification directly affect the safety and stability of products, as well as the production efficiency and market competitiveness of enterprises.

[0003] However, traditional wire harness defect identification methods often face the following problems when dealing with complex production environments, diverse defect types, and a large number of detection tasks: First, the detection process lacks intelligent means and it is difficult to detect complex wire harness defects. Traditional detection methods usually simply check whether there are obvious damages, breaks, etc. on the appearance of the wire harness, mainly relying on manual experience judgment. However, with the continuous complexity of wire harness design and manufacturing processes, the types of wire harness defects have become increasingly diverse, and it is difficult to directly discover wire harness defects through manual visual inspection. Second, the wire harness detection lacks comprehensiveness and systematicness, without comprehensively considering factors in multiple dimensions such as the electrical performance, physical structure, and external environment of the wire harness. Traditional detection methods often only focus on a single detection index, without comprehensively analyzing the change trends and their mutual influences in multiple dimensions such as electrical signals, temperature changes, and mechanical stresses. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent recognition system and method for wire harness defects based on machine vision to solve the problems raised in the prior art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: An intelligent recognition method for wire harness defects based on machine vision, characterized in that: the method includes the following steps:

[0006] Obtain wire harness image information, wire harness electrical signals, current, and temperature information through the camera and sensors in the detection device;

[0007] Perform data preprocessing on the obtained data, and extract features from the preprocessed data;

[0008] Through a deep learning neural network model, use deep learning algorithms to train the model, judge the quality of the wire harness, and perform color number tube matching detection;

[0009] Divide the harness defect identification into two different scenarios, analyze the detection processes of harness defects for each scenario separately, and generate corresponding detection reports based on the detection results.

[0010] Obtain the harness image information, harness electrical signals, current, and temperature information through the cameras and sensors in the detection equipment. The specific steps include:

[0011] Obtain the harness pose image information at different shooting angles through the cameras in the detection equipment, denoted as P = {P1, P2,..., P n}, where P1, P2,..., P n represent the harness pose image information captured by the 1st, 2nd,..., nth cameras, and the cameras are arranged at different angles of the detection equipment. Perform data annotation on the collected harness pose image information, annotate the harness type, defect type, and key point positions, and record the image clarity and contrast at the same time;

[0012] Obtain the electrical signals, current, and temperature information of the harness by interacting the sensors in the detection equipment with the harness.

[0013] Perform data preprocessing on the obtained data, and extract features from the preprocessed data. The specific steps include:

[0014] Perform preprocessing on the collected harness pose image data P, perform image enhancement operations through histogram equalization, and use edge detection algorithms to extract the harness edge features in the image, including edges, corners, and textures;

[0015] For the electrical signal data E of the harness, obtain the preprocessed electrical signal data E' after processing by the moving average filtering method, process the current data I by the median filtering algorithm to obtain the preprocessed current data I', and process the temperature data Tm by the Kalman filtering algorithm to obtain the preprocessed temperature data Tm';

[0016] Normalize the preprocessed electrical signal data E', current data I', and temperature data Tm', and uniformly normalize these different types of data to the [0, 1] interval.

[0017] In the detection of the crimping quality of the harness terminals, by comparing and studying the shallow CNN network and the Mobile Net deep learning neural network model, use the deep learning algorithm to train the model to judge the quality of the harness. The specific steps include:

[0018] Use the deep learning algorithm to train the shallow CNN network model to judge the harness quality. The shallow CNN network model includes:

[0019] Perform data preprocessing, uniformly adjust the wire harness images of different sizes to a fixed size, and normalize the pixel values of the images, scaling the pixel values to the range of [0, 1];

[0020] Construct a shallow CNN network model, determine its parameters for the convolutional layer and fully connected layer in the shallow CNN network. For each convolutional layer, determine the number, size, and stride of the convolutional kernels. The number of neurons in the fully connected layer is adjusted according to the classification task and data situation:

[0021] Input the pixel data of the preprocessed wire harness images into the input layer of the shallow CNN network model. In the shallow CNN, design convolutional layers, and each convolutional layer is followed by a max pooling layer. The convolutional layer captures the basic texture and shape features in the image through convolutional kernels. The convolutional kernels slide on the image for convolution operations to extract different feature information. After the convolution operation, use ReLU as the activation function;

[0022] The eigenvalue in the image is transmitted to the pooling layer through the convolutional layer, which reduces the dimension of the convolutional feature map;

[0023] The fully connected layer flattens the feature map after convolution and pooling into a one-dimensional vector, performs calculations with other neurons through the connection of neurons. The fully connected layer synthesizes the extracted features for final classification judgment. The output of the fully connected layer is processed by the softmax activation function. The softmax function converts the output of the model into probability values for each category, and determines the image type based on the output probability values, thereby judging the quality status of the wire harness;

[0024] When using a deep learning algorithm to train a Mobile Net deep learning neural network model to judge the quality of wire harnesses, the construction and training process of this model is as follows:

[0025] Adjust the input layer of the Mobile Net model according to the size of the preprocessed wire harness images;

[0026] Transmit the wire harness images through the input layer to the depthwise separable convolutional layer. Among them, the depthwise separable convolutional layer consists of two parts: depthwise convolution and pointwise convolution. The depthwise convolution performs convolution operations independently for each input channel; the pointwise convolution uses the 1×1 convolution method to combine each channel output by the depthwise convolution;

[0027] After the depthwise separable convolutional layer, use batch normalization operation;

[0028] After the convolution operation, use ReLU as the activation function;

[0029] After the convolution and activation operations, use the global average pooling layer to convert the feature map into a feature vector with a fixed length;

[0030] After obtaining the feature vector of fixed length, it is mapped to the category space through a fully connected layer;

[0031] The probability distribution of the wire harness quality is output using the Softmax function in the output layer;

[0032] To optimize the complexity and performance of the model, the width multiplier and the resolution multiplier are adjusted. The width multiplier is used to control the number of channels in each layer of the model, and the resolution multiplier is used to control the resolution of the input image. The computational complexity FLOPs of the model is proportional to the square of the width multiplier multiplied by the square of the resolution multiplier. By reasonably adjusting the width multiplier and the resolution multiplier, the computational complexity of the model is controlled, and the input layer, depthwise separable convolutional layer, fully connected layer, and output layer of the Mobile Net deep learning neural network model are optimized;

[0033] Obtain the accuracy A1 of the trained shallow CNN network model on the test set;

[0034] Obtain the accuracy A2 of the trained Mobile Net deep learning neural network model on the test set;

[0035] Judge the magnitudes of the accuracies A1 and A2, and select the model with the higher accuracy as the model for wire harness terminal crimping quality detection.

[0036] In the color number tube matching detection, opencv is used for wire harness color recognition, and Tesseract-OCR and Baidu Cloud OCR are used for number tube character recognition to judge the matching status. The specific steps include:

[0037] In the color number tube matching detection, opencv is used for wire harness color recognition. Using the functions in the OpenCV library, the image is converted from the RGB color space to the HSV color space;

[0038] According to the preset value range of the wire harness color in the HSV color space, the threshold segmentation algorithm is used to process the image, and the pixel points with colors within the set range are extracted to determine the color of the wire harness;

[0039] Tesseract-OCR and Baidu Cloud OCR are used for number tube character recognition. Judge the magnitudes of the accuracies A3 and A4 of Tesseract-OCR and Baidu Cloud OCR for number tube character recognition, and select the number tube character recognition technology with the higher accuracy as the number tube character recognition technology;

[0040] Compare the recognized wire harness color and number tube characters with the preset standard data, where the standard data contains the corresponding relationship between the correct colors and number tube characters. When the recognized colors and number tube characters match a certain group in the standard data, it is determined that the color number tube matches; when there is no match, it is determined that there is an abnormality;

[0041] Integrate the selected model for detecting the crimping quality of wire harness terminals and the technology for recognizing number tube characters into the detection equipment.

[0042] Classify the wire harness defect recognition into the first type of defect recognition scenario and the second type of defect recognition scenario. Among them, the positioning of the first type of defect recognition scenario is an independent fault caused by manufacturing or material defects of the wire harness to be detected, including short circuit, open circuit, and poor contact, without relying on external environmental conditions. The positioning of the second type of defect recognition scenario is a defect caused by wire harness crossing and overlapping, installation layout, or external environment (vibration, temperature) of the wire harness to be detected, regardless of whether the quality of the wire harness itself is qualified;

[0043] The specific steps for detecting wire harness defects in the first type of defect recognition scenario include:

[0044] Use machine vision to detect the integrity of terminal crimping, verify the conductivity of the wire harness through an LCR tester, use a withstand voltage tester to detect the insulation resistance, and at the same time use machine vision to check for physical damage to the conductor;

[0045] Determine the wire harness i1 with defects in the wire harness through the integrated color number tube matching detection in the detection equipment, where i1 represents the wire harness number to be detected;

[0046] Associate the detection data with the wire harness number i1 to generate a detection report;

[0047] The specific steps for detecting wire harness defects in the second type of defect recognition scenario include:

[0048] Obtain the wire harness layout image and identify the intersection coordinates and stacking layers; obtain the vibration spectrum of the environment where the wire harness is located through a vibration sensor, and obtain the ambient temperature and humidity recorded by a temperature and humidity sensor;

[0049] Conduct electro-thermal-mechanical performance detection, identify abnormal conditions through electrical signal and current monitoring, use an oscilloscope to capture the signal noise at the wire harness intersection, and analyze the current balance of multi-parallel wire harnesses. Conduct a dynamic test of the insulation performance in a vibration environment, monitor the change of the insulation resistance in real time, use an infrared thermal imager to scan the temperature distribution at the intersection, mark the high-temperature area, and record the temperature change curve over time to analyze the thermal runaway trend;

[0050] Conduct correlation analysis of cross - overlay defects, and comprehensively judge the potential failure risk of the wire harness through multi - sensor data fusion. In terms of spatial correlation, match the high - temperature points detected by the infrared thermal imager, the abnormal points identified by current monitoring, and the cross - positions identified by vision to confirm the problem area; in terms of temporal correlation, analyze the synchronous relationship between the peak value of the vibration spectrum, the electrical signal noise, and the temperature fluctuation to identify the defect trend; based on these data, conduct defect mode determination: when the temperature at the cross - point exceeds 100 °C and the insulation resistance suddenly drops below 0.5 MΩ, there is a short - circuit risk; when the vibration frequency is synchronized with the resistance fluctuation, there is a poor - contact problem; when the stress value (measured by a strain sensor) at the cross - point exceeds the fatigue threshold of the material, there is a fracture risk;

[0051] Determine the wire harness i2 with defects in the wire harness through the matching detection of the color number tube integrated in the detection device, where i2 represents the wire harness number to be detected;

[0052] Associate the detection data with the wire harness number i2 to generate a detection report.

[0053] The correlation analysis of cross - overlay defects described above includes:

[0054] Determine all relevant risk factors, including the high - temperature points detected by the infrared thermal imager and the abnormal points identified by current monitoring. For each risk factor, calculate the distance d between it and the cross - position identified by vision j , according to the pre - set weight α j and the risk coefficient w j , calculate the contribution of each risk factor to the spatial risk assessment value according to the spatial risk assessment formula and accumulate them to obtain the final spatial risk assessment value R s , when R s exceeds the set threshold, it is considered that there is a potential failure risk in the cross - overlay area in the spatial dimension;

[0055] The spatial risk assessment formula is defined as follows:

[0056]

[0057] Among them, R s represents the risk assessment value of spatial correlation, comprehensively measuring the potential failure risk of the cross - overlay area in the spatial dimension, with a value range of [0,1], m represents the number of risk factors participating in the assessment, α j represents the weight of the j - th risk factor, reflecting the influence degree of this factor on the overall risk, d j represents the distance between the j - th risk factor and the cross - position identified by vision, max(d) represents the maximum value among the distances between all risk factors and the cross - position, used to normalize the distance value, w j$k_j$ represents the risk coefficient of the $j$-th risk factor, which is determined based on historical data and experience and reflects the likelihood of a fault caused by this factor.

[0058] Obtain the peak sequence $f$ of the vibration spectrum v , the resistance fluctuation sequence $f$ r and the temperature fluctuation sequence $f$ t , and calculate $\text{corr}(f$ v , $f$ r ), $\text{corr}(f$ v , $f$ t ), and $\text{corr}(f$ r , $f$ t ) respectively. According to the preset weights $\beta_1$, $\beta_2$, $\beta_3$, calculate the time-series risk assessment value $R$ t using the time-series risk assessment formula. When $R_t$ is greater than the set threshold, it indicates that there is a potential fault risk in the time dimension.

[0059] The time-series risk assessment formula is defined as follows:

[0060] $R$ t $=\beta_1\times\text{corr}(f$ v , $f$ r ) $+\beta_2\times\text{corr}(f$ v , $f$ t ) $+\beta_3\times\text{corr}(f$ r , $f$ t );

[0061] where $R$ t represents the risk assessment value related to time series, comprehensively measuring the potential fault risk in the cross-overlapping area in the time dimension, with a value range of $[-1, 1]$. $\beta_1$, $\beta_2$, $\beta_3$ respectively represent the weights of the correlations between vibration frequency and resistance fluctuation, vibration frequency and temperature fluctuation, and resistance fluctuation and temperature fluctuation on the overall risk assessment. $\text{corr}(x, y)$ represents the Pearson correlation coefficient between variables $x$ and $y$, which is used to measure the linear correlation degree between two variables in the time series, with a value range of $[-1, 1]$. $f$ v represents the peak sequence of the vibration spectrum, $f$ r represents the resistance fluctuation sequence, and $f$ t represents the temperature fluctuation sequence.

[0062] An intelligent wire harness defect identification system based on machine vision, the system includes a data acquisition module, a data processing module, a wire harness quality detection module and a wire harness defect detection module. The data acquisition module is used to obtain wire harness image information, wire harness electrical signals, current and temperature information through cameras and sensors in the detection equipment. The data processing module is used to perform data preprocessing on the acquired data and extract features from the preprocessed data. The wire harness quality detection module is used to judge the quality of the wire harness through a deep learning neural network model, train the model using deep learning algorithms, and perform color number tube matching detection. The wire harness defect detection module is used to divide the wire harness defect identification into two different scenarios, analyze the detection process of wire harness defects for each scenario respectively, and generate corresponding detection reports based on the detection results.

[0063] The data acquisition module includes an image acquisition unit and a sensing detection unit. The image acquisition unit is used to obtain wire harness pose image information at different shooting angles through cameras in the detection equipment. The sensing detection unit is used to obtain the electrical signals, current and temperature information of the wire harness by interacting the sensors in the detection equipment with the wire harness. The data processing module includes a preprocessing unit and a feature extraction unit. The preprocessing unit is used to preprocess the acquired wire harness pose image data P, perform image enhancement operations through histogram equalization, process electrical signals using a moving average filtering method, process current data using median filtering, and process temperature data using the Kalman filtering algorithm. The feature extraction unit is used to extract wire harness features using edge detection algorithms, including edge, corner point and texture information.

[0064] The wire harness quality detection module includes a network model training unit, a model evaluation unit and a color number tube matching detection unit. The network model training unit is used to train the model using deep learning algorithms by comparing and studying a shallow CNN network and a Mobile Net deep learning neural network model for wire harness terminal crimping quality detection. The model evaluation unit is used to obtain the accuracy rates of the trained shallow CNN network model and the Mobile Net deep learning neural network model on the test set, judge the magnitudes of the accuracy rates of the two models, and select the model with the higher accuracy rate as the model for wire harness terminal crimping quality detection. The color number tube matching detection unit is used to identify the wire harness color using opencv, identify the characters on the number tube using Tesseract-OCR and Baidu Cloud OCR, and judge the matching status.

[0065] The wire harness defect detection module includes a first - type defect recognition unit, a second - type defect recognition unit, and a report generation unit. The first - type defect recognition unit is used to detect the integrity of terminal crimping through machine vision, detect the conductivity of the wire harness in combination with an LCR tester, detect the insulation resistance using a withstand voltage tester, and check for wire harness defects using the color number tube matching technology. The second - type defect recognition unit is used to collect wire harness layout images, identify the intersection coordinates and the number of overlapping layers, collect data from vibration sensors and temperature - humidity sensors, evaluate the environmental impact, and perform correlation analysis of cross - overlap defects. The report generation unit is used to associate the detection data with the wire harness number and generate a detection report.

[0066] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0067] 1. The present invention comprehensively considers factors in multiple dimensions including the electrical performance, physical structure, and external environment of the wire harness. It obtains the appearance image information of the wire harness through a camera, collects electrical signals, current, and temperature information using sensors, performs fusion analysis on multi - dimensional data, detects physical damage to the wire harness, monitors abnormal electrical performance, and also considers the impact of the external environment on the wire harness by analyzing the change trends of data in different dimensions and their mutual influence.

[0068] 2. After the detection is completed, a detection report is generated, associating the detection data with the wire harness number for easy quality traceability. Using a deep - learning neural network model, it gets rid of the limitation of relying on manual experience. Through learning a large amount of wire harness images, electrical signals, current, and temperature data, the model extracts complex features and identifies wire harness defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a schematic flow chart of a method for intelligent identification of wire harness defects based on machine vision according to the present invention;

[0070] Figure 2 is a schematic structural diagram of a system for intelligent identification of wire harness defects based on machine vision according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0071] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0072] In the embodiment: As Figure 1 - Figure 2 shown, the present invention provides a technical solution, a method for intelligent identification of wire harness defects based on machine vision, characterized in that the method includes the following steps:

[0073] Obtain the wire harness image information, wire harness electrical signals, current, and temperature information through the cameras and sensors in the detection device;

[0074] Perform data preprocessing on the acquired data, and extract features from the preprocessed data;

[0075] Through a deep learning neural network model, use deep learning algorithms to train the model, judge the quality of the wire harness, and perform color number tube matching detection;

[0076] Divide the wire harness defect recognition into two different scenarios, analyze the detection process of wire harness defects for each scenario respectively, and generate corresponding detection reports based on the detection results.

[0077] Obtain the wire harness image information, wire harness electrical signals, current, and temperature information through the cameras and sensors in the detection device. The specific steps include:

[0078] Obtain the wire harness pose image information at different shooting angles through the cameras in the detection device, denoted as P = {P1, P2,..., P n}, where P1, P2,..., P n represent the wire harness pose image information captured by the 1st, 2nd,..., nth cameras, and the cameras are arranged at different angles of the detection device. Perform data annotation on the collected wire harness pose image information, annotate the wire harness type, defect type, and key point positions, and record the image clarity and contrast at the same time;

[0079] Obtain the electrical signals, current, and temperature information of the wire harness by interacting the sensors in the detection device with the wire harness.

[0080] Specifically, install 5 industrial cameras at different angles of the detection device, shoot the wire harness pose images from different perspectives, collect 10,000 groups of images every day, each group of images contains 5 sub-images at different angles, and perform annotation on the images during the collection process;

[0081] Obtain the electrical signals, current, and temperature information of the wire harness by interacting the sensors in the detection device with the wire harness. The acquired electrical signal data ranges from 0 - 5V, the current data ranges from 0 - 20A, and the temperature data ranges from -40°C - 125°C.

[0082] Perform data preprocessing on the acquired data, and extract features from the preprocessed data. The specific steps include:

[0083] Perform preprocessing on the collected wire harness pose image data P, perform image enhancement operations through histogram equalization, and use edge detection algorithms to extract the wire harness edge features in the image, including edges, corners, and textures;

[0084] For the electrical signal data E of the wire harness, after being processed by the moving average filtering method, the preprocessed electrical signal data E' is obtained. The median filtering algorithm is used to process the current data I to obtain the preprocessed current data I'. The Kalman filtering algorithm is used to process the temperature data Tm to obtain the preprocessed temperature data Tm'.

[0085] The preprocessed electrical signal data E', current data I' and temperature data Tm' are normalized, and these different types of data are uniformly normalized to the interval [0, 1].

[0086] Specifically, the collected wire harness pose image data is preprocessed. The histogram equalization algorithm is used to enhance the image. After processing, the average contrast of the image is increased to 0.75. The Canny edge detection algorithm is used to extract the wire harness edge features. The average number of detected edge pixels accounts for 15% of the total number of image pixels, and the average number of corner points is 500 per image. The dimension of the texture feature vector extracted by the texture analysis algorithm is 128 dimensions.

[0087] In the detection of the crimping quality of wire harness terminals, by comparing and studying the shallow CNN network and the Mobile Net deep learning neural network model, the deep learning algorithm is used to train the model to judge the quality of the wire harness. The specific steps include:

[0088] Use the deep learning algorithm to train the shallow CNN network model to judge the quality of the wire harness. The shallow CNN network model includes:

[0089] Perform data preprocessing, uniformly adjust wire harness images of different sizes to a fixed size, and normalize the pixel values of the images, scaling the pixel values to the interval [0, 1];

[0090] Construct a shallow CNN network model, determine the parameters for the convolutional layer and the fully connected layer in the shallow CNN network. For each convolutional layer, determine the number, size and stride of the convolutional kernels, and the number of neurons in the fully connected layer is adjusted according to the classification task and data situation:

[0091] Input the preprocessed wire harness image pixel data into the input layer of the shallow CNN network model. In the shallow CNN, a convolutional layer is designed, and each convolutional layer is followed by a max pooling layer. The convolutional layer captures the basic texture and shape features in the image through the convolutional kernels. The convolutional kernels slide on the image for convolutional operations to extract different feature information. After the convolutional operation, ReLU is used as the activation function;

[0092] The feature values in the image are transmitted to the pooling layer through the convolutional layer, which reduces the dimension of the convolutional feature map;

[0093] The fully connected layer flattens the feature map after convolution and pooling into a one-dimensional vector, performs calculations with other neurons through the connection of neurons, synthesizes the extracted features, and makes the final classification judgment. The output of the fully connected layer is processed by the softmax activation function, which converts the output of the model into probability values for each category, and determines the image type based on the output probability values, thereby judging the quality status of the wire harness;

[0094] Specifically, 8000 labeled wire harness image data are selected as the training set, 1000 as the validation set, and 1000 as the test set. The images are uniformly adjusted to a size of 224×224 pixels, and the pixel values are normalized. A shallow CNN network model is constructed. The number of convolution kernels in the convolutional layer is set to 32, with a size of 3×3 and a stride of 1; the number of neurons in the fully connected layer is set to 128 according to the classification task (divided into 5 categories: normal, short circuit, open circuit, poor contact, and color number tube mismatch). After 100 rounds of training, the accuracy rate A1 on the test set reaches 93%;

[0095] When using the deep learning algorithm to train the Mobile Net deep learning neural network model to judge the quality of the wire harness, the construction and training process of this model are as follows:

[0096] According to the size of the preprocessed wire harness image, adjust the input layer of the Mobile Net model;

[0097] The wire harness image is transmitted through the input layer to the depthwise separable convolutional layer. Among them, the depthwise separable convolutional layer consists of two parts: depth convolution and pointwise convolution. Depth convolution performs convolution operations independently for each input channel; pointwise convolution uses the 1×1 convolution method to combine each channel output by depth convolution;

[0098] After the depthwise separable convolutional layer, batch normalization operation is used;

[0099] After the convolution operation, ReLU is used as the activation function;

[0100] After convolution and activation operations, the global average pooling layer is used to convert the feature map into a feature vector with a fixed length;

[0101] After obtaining the feature vector with a fixed length, it is mapped to the category space through the fully connected layer;

[0102] The Softmax function is used in the output layer to output the probability distribution of the wire harness quality;

[0103] To optimize the complexity and performance of the model, adjust the width multiplier and the resolution multiplier. The width multiplier is used to control the number of channels in each layer of the model, and the resolution multiplier is used to control the resolution of the input image. The computational complexity FLOPs of the model is proportional to the square of the width multiplier multiplied by the square of the resolution multiplier. By reasonably adjusting the width multiplier and the resolution multiplier, control the computational complexity of the model, and optimize the input layer, depthwise separable convolutional layer, fully connected layer, and output layer of the Mobile Net deep learning neural network model.

[0104] Obtain the accuracy rate A1 of the trained shallow CNN network model on the test set;

[0105] Obtain the accuracy rate A2 of the trained Mobile Net deep learning neural network model on the test set;

[0106] Judge the magnitudes of the accuracy rates A1 and A2, and select the model with the higher accuracy rate as the model for detecting the crimping quality of wire harness terminals.

[0107] Specifically, according to the size of the preprocessed image, adjust the input layer of the Mobile Net model, set the width multiplier to 0.75, and the resolution multiplier to 0.8 to optimize the model complexity and performance. After 100 rounds of the same training, the accuracy rate A2 on the test set reaches 95%, and finally select the Mobile Net model as the model for detecting the crimping quality of wire harness terminals.

[0108] In the matching detection of color number tubes, use opencv for wire harness color recognition, use Tesseract-OCR and Baidu Cloud OCR for number tube character recognition, and judge the matching status. The specific steps include:

[0109] In the matching detection of color number tubes, use opencv for wire harness color recognition, and use the functions in the OpenCV library to convert the image from the RGB color space to the HSV color space;

[0110] According to the preset value range of the wire harness color in the HSV color space, use the threshold segmentation algorithm to process the image, extract the pixel points with colors within the set range, and determine the color of the wire harness;

[0111] Use Tesseract-OCR and Baidu Cloud OCR for number tube character recognition, judge the magnitudes of the accuracy rates A3 and A4 of Tesseract-OCR and Baidu Cloud OCR for number tube character recognition, and select the number tube character recognition technology with the higher accuracy rate as the number tube character recognition technology;

[0112] Compare the recognized wire harness color and the number tube characters with the pre-set standard data. The standard data contains the corresponding relationship between the correct color and the number tube characters. When the recognized color and the number tube characters match a certain group in the standard data, it is determined that the color number tube matches; when they do not match, it is determined that there is an abnormality;

[0113] Integrate the selected model for detecting the crimping quality of wire harness terminals and the number tube character recognition technology into the detection equipment.

[0114] Specifically, use OpenCV to convert the image from the RGB color space to the HSV color space. According to the pre-set value range of the wire harness color in the HSV color space (hue range [0, 30], saturation range [50, 255], value range [50, 255] representing the red wire harness), use the threshold segmentation algorithm to process the image. Among the 10,000 wire harnesses detected on the same day, the colors of 9,800 wire harnesses were accurately recognized, and the accuracy rate was 98%;

[0115] Character recognition: Use Tesseract-OCR and Baidu Cloud OCR for number tube character recognition. Recognize the images of 1,000 test wire harnesses. The accuracy rate A3 of Tesseract-OCR is 96%, and the accuracy rate A4 of Baidu Cloud OCR is 99.6%. Therefore, Baidu Cloud OCR is selected as the number tube character recognition technology. Compare the recognized wire harness color and the number tube characters with the standard data, and it is found that there are 200 wire harnesses with mismatched color number tubes.

[0116] Divide the wire harness defect recognition into the first type of defect recognition scenario and the second type of defect recognition scenario. Among them, the positioning of the first type of defect recognition scenario is an independent fault caused by the manufacturing or material defects of the wire harness to be detected, including short circuit, open circuit, and poor contact, without relying on external environmental conditions. The positioning of the second type of defect recognition scenario is a defect caused by the wire harness crossing and overlapping, installation layout, or external environment (vibration, temperature) of the wire harness to be detected, regardless of whether the quality of the wire harness itself is qualified;

[0117] For the wire harness defect detection in the first type of defect recognition scenario, the specific steps include:

[0118] Use machine vision to detect the integrity of the terminal crimping, verify the wire harness conductivity through an LCR tester, use a withstand voltage tester to detect the insulation resistance, and at the same time use machine vision to check for physical damage to the conductor;

[0119] Determine the wire harness i1 with defects through the integrated color number tube matching detection in the detection equipment, where i1 represents the wire harness number to be detected;

[0120] Associate the detection data with the wire harness number i1 to generate a detection report;

[0121] Specifically, machine vision is used to detect the integrity of terminal crimping. Through detection, it is found that there are 150 wire harnesses with incomplete terminal crimping. The LCR tester is used to verify the conductivity of the wire harnesses, and 80 wire harnesses are detected to have open circuit problems, and 20 wire harnesses have short circuit problems. The withstand voltage tester is used to detect the insulation resistance, and it is found that the insulation resistance of 50 wire harnesses is lower than the standard value. At the same time, machine vision inspection finds that the conductors of 30 wire harnesses have physical damage. The defective wire harness numbers are determined through color code tube matching detection, and the detection data is associated with the wire harness numbers to generate the first type of defect detection report;

[0122] For the wire harness defect detection in the second type of defect identification scenario, the specific steps include:

[0123] Obtain the wire harness layout image and identify the intersection coordinates and overlapping layers; obtain the vibration spectrum of the environment where the wire harness is located monitored by the vibration sensor, and obtain the environmental temperature and humidity recorded by the temperature and humidity sensor;

[0124] Conduct electro-thermal-mechanical performance detection. Abnormal conditions are identified through the monitoring of electrical signals and current. An oscilloscope is used to capture the signal noise at the wire harness intersection, and the current balance of multi-parallel wire harnesses is analyzed. The dynamic test of the insulation performance is carried out in a vibrating environment, and the change of the insulation resistance is monitored in real time. An infrared thermal imager is used to scan the temperature distribution at the intersection, mark the high-temperature area, and record the temperature change curve over time to analyze the thermal runaway trend;

[0125] Conduct correlation analysis of cross-overlapping defects. The potential failure risks of wire harnesses are comprehensively judged through multi-sensor data fusion. In terms of spatial correlation, the high-temperature points detected by the infrared thermal imager, the abnormal points identified by current monitoring, and the intersection positions identified by vision are matched to confirm the problem area; in terms of temporal correlation, the synchronous relationship between the peak value of the vibration spectrum, the electrical signal noise, and the temperature fluctuation is analyzed to identify the defect trend; based on these data, defect mode determination is carried out: when the temperature at the intersection exceeds 100 °C and the insulation resistance suddenly drops below 0.5 MΩ, there is a short circuit risk; when the vibration frequency is synchronized with the resistance fluctuation, there is a poor contact problem; when the stress value (measured by the strain sensor) at the intersection exceeds the fatigue threshold of the material, there is a fracture risk;

[0126] The defective wire harness i2 is determined through color code tube matching detection integrated in the detection equipment, where i2 represents the wire harness number to be detected;

[0127] Associate the detection data with the wire harness number i2 to generate a detection report.

[0128] Specifically, 1000 randomly selected wire harness layout images are obtained. It is identified that each wire harness has an average of 3 crossing points and an average stacking layer number of 2 layers. The vibration spectrum monitored by a vibration sensor (vibration frequency range is 0 - 100 Hz) and the ambient temperature and humidity recorded by a temperature and humidity sensor (temperature range is 20°C - 30°C, humidity range is 40% - 60%) are obtained for electro-thermal-mechanical performance detection. The dynamic insulation performance test is carried out in a vibration environment. It is found that the insulation resistance of 10 wire harnesses shows abnormal decrease during vibration. The infrared thermal imager is used to scan the temperature distribution of the crossing points, and it is found that the temperatures of 5 crossing points exceed 100°C, which are marked as high-temperature areas. Through multi-sensor data fusion for cross-overlay defect correlation analysis, according to the defect mode determination rules, it is determined that there are 5 wire harnesses with short-circuit risks, 8 wire harnesses with poor contact problems, and 3 wire harnesses with fracture risks. The wire harness numbers with defects are determined through color number tube matching detection, and the detection data is associated with the wire harness numbers to generate a second type of defect detection report.

[0129] An intelligent wire harness defect identification system based on machine vision, the system includes a data acquisition module, a data processing module, a wire harness quality detection module, and a wire harness defect detection module. The data acquisition module is used to obtain wire harness image information, wire harness electrical signals, current, and temperature information through cameras and sensors in the detection equipment. The data processing module is used to perform data preprocessing on the acquired data and extract features from the preprocessed data. The wire harness quality detection module is used to judge the wire harness quality through a deep learning neural network model and train the model using deep learning algorithms, and perform color number tube matching detection. The wire harness defect detection module is used to divide the wire harness defect identification into two different scenarios, analyze the detection processes of wire harness defects for each scenario respectively, and generate corresponding detection reports based on the detection results.

[0130] The data acquisition module includes an image acquisition unit and a sensing detection unit. The image acquisition unit is used to obtain wire harness pose image information at different shooting angles through cameras in the detection equipment. The sensing detection unit is used to obtain the electrical signals, current, and temperature information of the wire harness by interacting the sensors in the detection equipment with the wire harness. The data processing module includes a preprocessing unit and a feature extraction unit. The preprocessing unit is used to preprocess the acquired wire harness pose image data P, perform image enhancement operations through histogram equalization, process electrical signals using a moving average filtering method, process current data using a median filtering method, and process temperature data using a Kalman filtering algorithm. The feature extraction unit is used to extract wire harness features using an edge detection algorithm, including edge, corner, and texture information.

[0131] The wire harness quality detection module includes a network model training unit, a model evaluation unit, and a color number tube matching detection unit. The network model training unit is used for detecting the crimping quality of wire harness terminals. By comparing and studying the shallow CNN network and the Mobile Net deep learning neural network model, it uses the deep learning algorithm to train the model. The model evaluation unit is used to obtain the accuracy rates of the trained shallow CNN network model and the Mobile Net deep learning neural network model on the test set, judge the magnitude of the accuracy rates of the two models, and select the model with the higher accuracy rate as the model for detecting the crimping quality of wire harness terminals. The color number tube matching detection unit is used to identify the wire harness color using opencv, identify the characters on the number tube using Tesseract-OCR and Baidu Cloud OCR, and judge the matching status.

[0132] The wire harness defect detection module includes a first type of defect identification unit, a second type of defect identification unit, and a report generation unit. The first type of defect identification unit is used to detect the integrity of terminal crimping through machine vision, combine with an LCR tester to detect the conductivity of the wire harness, use a withstand voltage tester to detect the insulation resistance, and use the color number tube matching technology to check the defect situation of the wire harness. The second type of defect identification unit is used to collect the wire harness layout image, identify the intersection coordinates and the number of superimposed layers, collect the data of vibration sensors and temperature and humidity sensors, evaluate the environmental impact, and conduct an association analysis of cross-overlap defects. The report generation unit is used to associate the detection data with the wire harness number and generate a detection report.

[0133] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved.

Claims

1. An intelligent method for identifying wire harness defects based on machine vision, characterized in that: The method includes the following steps: Obtain harness image information, harness electrical signals, current, and temperature information through the cameras and sensors in the detection device; Perform data preprocessing on the acquired data, and extract features from the preprocessed data; Through a deep learning neural network model, use deep learning algorithms to train the model, judge the quality of the harness, and perform color number tube matching detection; Divide the harness defect identification into two different scenarios, analyze the detection process of harness defects for each scenario separately, and generate corresponding detection reports based on the detection results.

2. The intelligent recognition method for harness defects based on machine vision according to claim 1, wherein: Obtain harness image information, harness electrical signals, current, and temperature information through the cameras and sensors in the detection device. The specific steps include: Obtain the wire harness pose image information at different shooting angles through the camera in the detection device, denoted as P = {P1, P2,..., P n}, where P1, P2,..., P n represent the wire harness pose image information captured by the 1st, 2nd,..., nth cameras, and the cameras are arranged at different angles of the detection device. Perform data annotation on the collected wire harness pose image information, annotate the wire harness type, defect type, and key point positions, and record the image clarity and contrast at the same time; Obtain the electrical signals, current, and temperature information of the harness by interacting the sensors in the detection device with the harness.

3. The intelligent recognition method for harness defects based on machine vision according to claim 2, characterized in that: Perform data preprocessing on the acquired data, and extract features from the preprocessed data. The specific steps include: Preprocess the acquired harness pose image data P, perform image enhancement operations through histogram equalization, and use edge detection algorithms to extract the harness edge features in the image, including edges, corners, and textures; For the electrical signal data E of the harness, after processing with the moving average filtering method, obtain the preprocessed electrical signal data E'. Process the current data I with the median filtering algorithm to obtain the preprocessed current data I'. Process the temperature data Tm with the Kalman filtering algorithm to obtain the preprocessed temperature data Tm'; Normalize the preprocessed electrical signal data E', current data I', and temperature data Tm', and uniformly normalize these different types of data to the interval [0, 1].

4. The intelligent identification method for harness defects based on machine vision according to claim 3, wherein: In the detection of the crimp quality of the harness terminals, by comparing and studying the shallow CNN network and the Mobile Net deep learning neural network model, use deep learning algorithms to train the model and judge the quality of the harness; Use the shallow CNN network model to judge the quality of the harness; When using deep learning algorithms to train the Mobile Net deep learning neural network model to judge the quality of the harness, the construction and training process of the model is as follows: Adjust the input layer of the Mobile Net model according to the size of the preprocessed harness image; Transmit the harness image through the input layer to the depthwise separable convolutional layer. Among them, the depthwise separable convolutional layer consists of two parts: depth convolution and pointwise convolution. The depth convolution independently performs convolution operations for each input channel; the pointwise convolution uses the 1×1 convolution method to combine the channels output by the depth convolution; After the depthwise separable convolutional layer, use batch normalization operations; After the convolution operation, use ReLU as the activation function; After the convolution and activation operations, use the global average pooling layer to convert the feature map into a fixed-length feature vector; After obtaining the fixed-length feature vector, map it to the category space through the fully connected layer; Use the Softmax function in the output layer to output the probability distribution of the harness quality; To optimize the complexity and performance of the model, adjust the width multiplier and resolution multiplier. The width multiplier is used to control the number of channels in each layer of the model, and the resolution multiplier is used to control the resolution of the input image. The computational complexity FLOPs of the model is proportional to the square of the width multiplier multiplied by the square of the resolution multiplier. By reasonably adjusting the width multiplier and resolution multiplier, control the computational complexity of the model, and optimize the input layer, depthwise separable convolutional layer, fully connected layer, and output layer of the Mobile Net deep learning neural network model; Obtain the accuracy rate A1 of the trained shallow CNN network model on the test set; Obtain the accuracy rate A2 of the trained Mobile Net deep learning neural network model on the test set; Judge the magnitudes of the accuracy rates A1 and A2, and select the model with the higher accuracy rate as the model for detecting the crimping quality of the wire harness terminals.

5. The intelligent identification method for harness defects based on machine vision according to claim 3, characterized in that: In the detection of color number tube matching, use opencv for wire harness color recognition, use Tesseract-OCR and Baidu Cloud OCR for number tube character recognition, and judge the matching status. The specific steps include: In the detection of color number tube matching, use opencv for wire harness color recognition, and use the functions in the OpenCV library to convert the image from the RGB color space to the HSV color space; According to the preset value range of the wire harness color in the HSV color space, use the threshold segmentation algorithm to process the image, extract the pixel points with colors within the set range, and determine the color of the wire harness; Use Tesseract-OCR and Baidu Cloud OCR for number tube character recognition, judge the magnitudes of the accuracy rates A3 and A4 of Tesseract-OCR and Baidu Cloud OCR for number tube character recognition, and select the number tube character recognition technology with the higher accuracy rate as the number tube character recognition technology; Compare the recognized wire harness color and number tube characters with the preset standard data. The standard data contains the corresponding relationship between the correct color and number tube characters. When the recognized color and number tube characters match a certain group in the standard data, it is determined that the color number tube matches; when they do not match, it is determined that there is an abnormality; Integrate the selected model for detecting the crimping quality of the wire harness terminals and the number tube character recognition technology into the detection device.

6. A machine vision-based intelligent wire harness defect recognition method according to claim 5, characterized in that: Classify the wire harness defect recognition into a first type of defect recognition scenario and a second type of defect recognition scenario. Among them, the first type of defect recognition scenario locates independent faults caused by manufacturing or material defects of the wire harness to be detected, without relying on external environmental conditions, and the second type of defect recognition scenario locates defects caused by wire harness crossing and overlapping, installation layout, or external environment of the wire harness to be detected; The specific steps for detecting wire harness defects in the first type of defect recognition scenario include: Use machine vision to detect the integrity of terminal crimping, verify the wire harness conductivity through an LCR tester, use a withstand voltage tester to detect the insulation resistance, and at the same time use machine vision to check for physical damage to the conductor; Determine the harness i1 with defects by matching and detecting the color number tubes integrated in the detection device, where i1 represents the harness number to be detected; Associate the detection data with the harness number i1 to generate a detection report; For the harness defect detection in the second type of defect identification scenario, the specific steps include: Obtain the harness layout image and identify the intersection coordinates and the number of overlapping layers; obtain the vibration spectrum of the environment where the harness is located by the vibration sensor, and obtain the ambient temperature and humidity recorded by the temperature and humidity sensor; Conduct electro-thermal-mechanical performance detection, identify abnormal conditions through the monitoring of electrical signals and current, use an oscilloscope to capture the signal noise at the harness intersection, and analyze the current balance of multi-parallel harnesses. Conduct a dynamic test of the insulation performance in a vibrating environment, monitor the change of the insulation resistance in real time, use an infrared thermal imager to scan the temperature distribution at the intersection, mark the high-temperature area, and record the temperature change curve over time to analyze the thermal runaway trend; Conduct a correlation analysis of the cross-overlapping defects, comprehensively judge the potential fault risk of the harness through multi-sensor data fusion. In terms of spatial correlation, match the high-temperature points detected by the infrared thermal imager, the abnormal points identified by the current monitoring, and the intersection positions identified by vision to confirm the problem area; in terms of temporal correlation, analyze the synchronization relationship between the peak value of the vibration spectrum, the electrical signal noise, and the temperature fluctuation to identify the defect trend; determine the defect mode based on these data; Determine the harness i2 with defects by matching and detecting the color number tubes integrated in the detection device, where i2 represents the harness number to be detected; Associate the detection data with the harness number i2 to generate a detection report.

7. An intelligent wire harness defect identification system based on machine vision, characterized in that: The system includes a data acquisition module, a data processing module, a harness quality detection module, and a harness defect detection module. The data acquisition module is used to obtain harness image information, harness electrical signals, current, and temperature information through the cameras and sensors in the detection device; the data processing module is used to perform data preprocessing on the acquired data and extract features from the preprocessed data; the harness quality detection module is used to judge the harness quality through a deep learning neural network model and train the model using deep learning algorithms, and conduct color number tube matching detection; the harness defect detection module is used to divide the harness defect identification into two different scenarios, analyze the detection process of the harness defects for each scenario respectively, and generate corresponding detection reports based on the detection results.

8. The intelligent wire harness defect identification system based on machine vision according to claim 7, wherein: The data acquisition module includes an image acquisition unit and a sensing detection unit. The image acquisition unit is used to obtain the wire harness pose image information at different shooting angles through the camera in the detection device. The sensing detection unit is used to obtain the electrical signal, current, and temperature information of the wire harness by interacting the sensor in the detection device with the wire harness. The data processing module includes a preprocessing unit and a feature extraction unit. The preprocessing unit is used to preprocess the collected wire harness pose image data P, perform image enhancement operations through histogram equalization, process the electrical signal using a moving average filtering method, process the current data using median filtering, and process the temperature data using the Kalman filtering algorithm. The feature extraction unit is used to extract wire harness features using an edge detection algorithm, including edge, corner, and texture information.

9. The intelligent wire harness defect identification system based on machine vision according to claim 8, characterized in that: The wire harness quality detection module includes a network model training unit, a model evaluation unit, and a color number tube matching detection unit. The network model training unit is used for detecting the crimping quality of wire harness terminals. By comparing and studying a shallow CNN network and a Mobile Net deep learning neural network model, it uses deep learning algorithms to train the model. The model evaluation unit is used to obtain the accuracy rates of the trained shallow CNN network model and Mobile Net deep learning neural network model on the test set, judge the magnitudes of the accuracy rates of the two models, and select the model with a higher accuracy rate as the model for detecting the crimping quality of wire harness terminals. The color number tube matching detection unit is used to identify the wire harness color using opencv, identify the characters on the number tube using Tesseract-OCR and Baidu Cloud OCR, and judge the matching status.

10. The intelligent wire harness defect identification system based on machine vision according to claim 9, characterized in that: The wire harness defect detection module includes a first type of defect identification unit, a second type of defect identification unit, and a report generation unit. The first type of defect identification unit is used to detect the integrity of terminal crimping through machine vision, detect the conductivity of the wire harness in combination with an LCR tester, detect the insulation resistance using a withstand voltage tester, and check for wire harness defects using the color number tube matching technology. The second type of defect identification unit is used to collect the wire harness layout image, identify the intersection coordinates and the number of overlapping layers, collect the data of vibration sensors and temperature and humidity sensors, evaluate the environmental impact, and perform an association analysis of cross-overlapping defects. The report generation unit is used to associate the detection data with the wire harness number and generate a detection report.

Images