Visual inspection method for crimping quality of automobile wire harnesses

Abstract

The present application belongs to the field of automobile electronic product inspection, and specifically relates to a visual inspection method for the crimping quality of automobile wire harnesses, the method comprising: performing denoising, grayscale conversion and binarization processing on a wire harness image, so as to determine a white wire harness connected region; calculating the number of wire harnesses on the basis of the white wire harness connected region, and comparing same with a preset standard value; if the number of wire harnesses and the preset standard value are consistent, performing edge extraction on the wire harness image so as to determine a wire harness length; determining whether the wire harness length is within a preset size range, and if so, segmenting the wire harness image into a plurality of single-wire harness images according to a preset sequence; extracting a first-order color moment and a second-order color moment of each single-wire harness image in an HSV color space, and calculating difference values between values of the first-order color moment and the second-order color moment corresponding to each single-wire harness image and template wire harness values at a corresponding position of the single wire harness; if the difference values are within threshold ranges, cropping a wire harness crimping region in the wire harness image, and generating a binary image of the wire harness crimping region; performing defect detection on the crimping region by using a pre-trained defect instance segmentation model, so as to generate a defect mask, and calculating a compactness of the crimping region; and if there is no defect and the compactness meets a standard, completing inspection of passed wire harnesses. The method can quickly identify the number, length and color characteristics and crimping region defects of wire harnesses, thereby improving inspection efficiency and accuracy.

Description

A visual inspection method for automotive wiring harness crimping quality Technical Field

[0001] This application relates to the field of automotive electronic product testing technology, and more specifically, to a visual inspection method for automotive wiring harness crimping quality. Background Technology

[0002] In the automotive wiring harness manufacturing process, the inspection of crimping quality is a crucial step in ensuring the safety and reliability of the automotive electrical system. Currently, the inspection of automotive wiring harness crimping quality mainly employs methods such as mechanical inspection, electrical inspection, and microscopic observation. Mechanical inspection assesses the crimping strength and consistency by applying external force. Electrical inspection uses continuity tests and insulation resistance tests to determine the conductivity and insulation performance of the crimped joints. Microscopic observation uses microscopes or industrial cameras to magnify and analyze the external features of the crimped areas.

[0003] However, these traditional testing methods have significant drawbacks and limitations. First, mechanical inspection and microscopic observation typically require manual intervention, making the process cumbersome and inefficient, and difficult to meet the demands of modern automotive industry's high-volume, high-paced production. Second, electrical testing methods cannot visually assess the appearance quality of crimped joints and struggle to detect issues such as over-crimping or under-crimping, lacking a comprehensive evaluation of the multi-dimensional quality characteristics of wire harness crimping. Furthermore, traditional testing methods are susceptible to subjective human judgment or equipment precision issues, making it difficult to guarantee the consistency of test results and easily leading to misjudgments or omissions.

[0004] With the rapid development of the automotive industry, higher demands are being placed on the accuracy and efficiency of wiring harness crimping quality inspection. Therefore, developing a highly efficient, accurate, and intelligent method for inspecting automotive wiring harness crimping quality has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, this application provides a visual inspection method for automotive wiring harness crimping quality, which combines modern visual inspection with artificial intelligence technology to achieve high efficiency, intelligence and automation in automotive wiring harness crimping quality inspection.

[0006] The technical solution provided in this application is as follows:

[0007] A visual inspection method for automotive wiring harness crimping quality, which sequentially detects the core wire sequence and size, and crimping defects in the wiring harness based on a wiring harness image, the method comprising:

[0008] The wire harness image is subjected to grayscale, noise reduction, and binarization processing to determine the white wire harness connected regions of the wire harness image;

[0009] The number of wire bundles is determined based on the connected area of ​​the white wire bundle;

[0010] If the number of wire bundles is consistent with the first preset standard value, then the wire bundle image is subjected to edge extraction to obtain the edge pixels of the wire bundle, which are used to determine the length of the wire bundle;

[0011] If the length of the wire harness meets the preset size range, the wire harness image is divided into multiple single wire harness images according to the preset initial order.

[0012] Extract the first-order and second-order color moments of each single wire bundle image in the HSV color space, and compare them with the difference between the template wire bundle values ​​at the corresponding positions of the single wire bundle.

[0013] If all the differences are within the preset threshold range, then the wire harness crimping area in the wire harness image is extracted, and a binary image of the wire harness crimping area is generated.

[0014] The binary image is detected based on a pre-trained crimping region defect instance segmentation model to generate a defect mask, which is used to mark crimping defects in the wire harness crimping region and determine the compactness of the crimping region. The compactness is determined by the perimeter and area of ​​the contact surface of the crimping region.

[0015] If no crimping defects are detected and the tightness is consistent with the second preset standard value, the quality inspection of the qualified wire harness is completed.

[0016] One possible implementation involves performing grayscale, noise reduction, and binarization processing on the wire harness image, including:

[0017] The wire harness image is converted to the HSV color space, and the lightness V component is taken as the grayscale image of the wire harness image.

[0018] The grayscale image is subjected to median filtering to remove noise and smooth the image.

[0019] The denoised grayscale image is binarized to segment the wire bundle image into a black background region and a white wire bundle connected region.

[0020] In one possible implementation, determining the number of wire bundles based on the connected region of the white wire bundle includes:

[0021] An erosion operation is performed on the binarized wire harness image to remove excess noise, resulting in the final wire harness image; wherein, the white wire harness connected region of the final wire harness image consists only of wire harnesses, which is used to determine the number of wire harnesses.

[0022] In one possible implementation, edge extraction is performed on the wire harness image to obtain the edge pixels of the wire harness and determine the wire harness length, including:

[0023] Based on the Canny edge detection algorithm, edge information of the wire harness is extracted from the wire harness image to determine the edge contour of the wire harness.

[0024] Randomly select N pixels on the edge contour and perform a random Hough transform;

[0025] By statistically analyzing the peak values ​​in Hough space, the line with the most collinear points is taken as the entire line bundle L, including the straight part l1 and the curved part l2.

[0026] The length of the straight section l1 is calculated using the equation y1 of the straight line, which is the Euclidean distance between the two endpoints:

[0027]

[0028] Where, A(x) a y a ), B(x) b y b () represents the two endpoints of the line;

[0029] The length of the arc l2 is calculated using the equation of the straight line y2:

[0030]

[0031] Where c and d are the x-coordinates of the two endpoints of the arc;

[0032] The sum of the lengths of the straight portion l1 and the curved portion l2 is taken as the length of the line bundle in the image;

[0033] The actual length of the wire harness is determined based on its length in the image and a preset scaling factor.

[0034] In one possible implementation, determining the preset size range includes:

[0035] Select n different sample wire bundles and statistically analyze the actual value A of each sample wire bundle. i and measured value M i The error between u i =A i -M i and the average error The error threshold ε0 is calculated and expressed as:

[0036]

[0037] The same sample wire harness is measured m times, resulting in several measurement values ​​M′ for the same wire harness. i The systematic error ε1 is determined as follows:

[0038]

[0039] The preset size range is determined based on the error threshold ε0, the system error ε1, and the preset template value. If the difference between the wire harness length and the preset template value is within the threshold range (ε0+ε1), then the wire harness length is determined to conform to the preset size range.

[0040] One possible implementation involves extracting the first-order and second-order color moments of each individual wire bundle image in the HSV color space, including:

[0041] Extract the pixel values ​​of the single wire harness image in the H, S, and V components;

[0042] Based on the pixel values, the first-order color moment and the second-order color moment corresponding to the single line bundle image are determined; wherein, the first-order color moment is expressed as:

[0043]

[0044] In the formula, N represents the total number of pixels in a single wire harness image, and P i，j e represents the j-th pixel value of the i-th color component. i The mean of all pixels representing the i-th color component is the value of the first-order color moment.

[0045] The second-order color moment is represented as:

[0046]

[0047] In the formula, S i The variance of all pixels representing the i-th color component is the second-order color moment.

[0048] In one possible implementation, the crimped area of ​​the wire harness in the wire harness image is extracted, and a binary image of the crimped area is generated, including:

[0049] The wire harness crimping area is extracted based on an adaptive threshold segmentation algorithm;

[0050] For each pixel of the image within the wire harness crimping area, a 3×3 neighborhood window is defined with the current pixel as the center, containing the center pixel and its 8 surrounding neighboring pixels;

[0051] Calculate the mean grayscale value of all pixels within the 3×3 neighborhood window;

[0052] Using the mean value as the threshold T(x, y) for the current pixel, it is expressed as:

[0053]

[0054] In the formula, T(x, y) is the threshold of the current pixel, W is the neighborhood window centered on the current pixel, I(i, j) is the gray value of the pixel in the window, and C is a constant used to adjust the sensitivity of segmentation.

[0055] The gray value of the current pixel is compared with the calculated threshold T(x, y); if the gray value of the current pixel is greater than the threshold T(x, y), the current pixel is set to white; if the gray value of the current pixel is less than or equal to the threshold T(x, y), the current pixel is set to black.

[0056] Repeat the above steps to process each pixel of the image within the wire harness crimping area to generate a binary image of the wire harness crimping area; wherein, the white area of ​​the binary image represents the wire harness crimping area, and the black area represents the background or other non-crimping areas.

[0057] In one possible implementation, the pre-trained crimping area defect instance segmentation model is the YOLACT instance segmentation model, with ResNet50 as the backbone network.

[0058] Training the defect instance segmentation model for the crimped area includes:

[0059] A dataset of crimping defects is constructed by acquiring multiple images containing various types of crimping defects, including crack defects, abnormal crimping shape defects, crimping surface oxidation defects, and exposed core wire defects in the crimping area.

[0060] The dataset of defects in the press-fit region is input into the configured YOLACT instance segmentation model, and feature information is extracted through the backbone network, represented as follows:

[0061] F∈R C×H×W (9)

[0062] In the formula, F represents the feature map dimension, C represents the number of feature channels, H represents the height of the feature map, and W represents the width of the feature map;

[0063] The feature map dimension F is reduced by using a 1×1 convolution kernel to reduce the number of feature channels C of the feature map dimension F to 32, which is used as the number of shared mask prototypes;

[0064] The shared mask dimension P is determined as follows:

[0065] P∈R M×H×W (10)

[0066] The original predicted value C of the YOLACT instance segmentation model is obtained by applying an activation function. i After normalization, the mask weight coefficients c are obtained. im, used to characterize the contribution of the shared mask prototype to a specific target instance, is expressed as:

[0067] C i ={C i1 C i2 C iM} (11)

[0068] c im =sigmoid(C i (12)

[0069] Linear combination of shared masks generates the final instance mask. The trained defect instance segmentation model for the press-fit region is represented as:

[0070]

[0071] Where M represents the number of shared masks, P represents the final mask for the i-th instance. m (x, y) represents the m-th shared mask prototype, C im This represents the m-th weight of instance i;

[0072] The network is optimized using the binary cross-entropy loss function to enhance the ability of the crimping defect instance segmentation model to distinguish different crimping defect types, as shown below:

[0073]

[0074] In the formula, M i (x, y) is the actual mask labeled by Labelme software. It is a predictive mask.

[0075] In one possible implementation, based on a defect mask, crimping defects in the wire harness crimping area are marked and the tightness of the crimping area is determined, including:

[0076] The defect mask is used to extract its contour using OpenCV, obtaining the point set of the contour of the pressed area image, represented as:

[0077] contour={(x1, y1), (x2, y2),..., (x n y n (15)

[0078] Based on the contour, calculate the distance d between adjacent points. i , represented as:

[0079]

[0080] In the formula, (x i y i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n+1 =x1,y n+1 =y1;

[0081] Based on the distance between the adjacent points, the perimeter D of the contour is determined, and is expressed as:

[0082]

[0083] Based on the formula for calculating the area of ​​a polygon, the area A of the contact surface of the pressing region is determined, expressed as:

[0084]

[0085] In the formula, (x i y i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n+1 =x1,y n+1 =y1;

[0086] The tightness of the contact surface of the pressing area is determined by the perimeter D and the area A, and is expressed as follows:

[0087]

[0088] Compared with the prior art, the technical solution provided in this application has the following beneficial effects:

[0089] This application achieves efficient and accurate inspection of wire harness quality by combining image processing and deep learning. First, preprocessing operations such as denoising, grayscale conversion, binarization, and edge extraction on the wire harness image effectively remove image noise and clearly identify connected regions and edge features of the wire harness, providing accurate data for subsequent wire harness counts and length measurements. Second, feature extraction using first- and second-order color moments in the HSV color space, compared with template wire harness values, quickly determines whether the wire harness's color features meet standards, improving detection sensitivity and accuracy. Furthermore, a pre-trained crimping area defect instance segmentation model is used to detect crimping areas of the wire harness, accurately identifying crimping defects and generating defect masks. Simultaneously, the crimping quality is further verified by calculating the tightness of the contact surface. The entire solution not only achieves comprehensive inspection of wire harness quality but also significantly improves inspection efficiency, reduces errors and labor intensity associated with manual inspection, and possesses high practicality and reliability. It can be effectively applied to quality control in the wire harness production process, ensuring high quality and consistency of wire harness products. Attached Figure Description

[0090] Figure 1 is a schematic diagram of the structure of a visual inspection system for automotive wiring harness crimping quality provided in Embodiment 1 of this application.

[0091] Figure 2 is a flowchart of a method for detecting the core wire sequence and size of an automotive wiring harness according to Embodiment 2 of this application.

[0092] Figure 3 is a flowchart of a method for detecting the wiring sequence of an automotive wiring harness provided in Embodiment 2 of this application.

[0093] Figure 4 is a flowchart of a wire harness crimping defect detection method provided in Embodiment 2 of this application.

[0094] Figure 5 is a schematic diagram of the wire harness crimping area provided in Embodiment 2 of this application.

[0095] Figure 6 is a flowchart of a visual inspection method for automotive wiring harness crimping quality provided in Embodiment 3 of this application. Detailed Implementation

[0096] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0097] Example 1

[0098] Embodiment 1 of this application provides a structural schematic diagram of an automotive wiring harness crimping quality visual inspection system. As shown in Figure 1, the overall structure of the automotive wiring harness crimping quality visual inspection system, from the outside in, includes a frame body, a conveyor module, a visual inspection module, a display and control module, and a mobile support module. The frame body is constructed of aluminum alloy profiles, serving as the main support frame for the entire device. Its lightweight yet robust structure provides an installation foundation and a stable operating environment for other modules. The conveyor module runs through the entire device, installed in the middle of the frame body. It connects the front and rear ends of the production line via a conveyor belt, used to transport the automotive wiring harness to be inspected from the starting point to the inspection area, and after inspection, to the next process or the rejection area, achieving full automation of the inspection process.

[0099] The vision inspection module, mounted above the conveyor module, includes an industrial camera, a high-brightness LED light source, and a mounting bracket. The industrial camera is fixed in place by the bracket, with its inspection area directly facing the wire harness crimping points on the conveyor belt. The high-brightness LED light source is positioned around the camera, providing uniform and high-intensity illumination for image acquisition, ensuring clear and reliable image quality. By adjusting the height and angle of the vision module bracket, the industrial camera's inspection area can completely cover the crimping points, enabling precise inspection of the wire harness crimping quality.

[0100] The display and control module is mounted on one side of the main frame and consists of a monitor and an industrial control computer. The industrial control computer connects to the industrial camera of the vision inspection module via a data cable, responsible for receiving and processing image data. The monitor connects to the industrial control computer to display inspection results and the user interface in real time, facilitating operator monitoring of the inspection process and results. The display and control module works in conjunction with the transmission module and the vision inspection module through control signals to automate and intelligently manage the inspection process.

[0101] The mobile support module, consisting of casters, is installed at the bottom of the device to facilitate movement between different production lines. The casters give the entire device excellent flexibility and versatility, allowing it to quickly adapt to different production environments and testing requirements, thus improving equipment utilization and production efficiency.

[0102] In a specific application scenario, this embodiment first sets the process parameters and camera calibration parameters for the automotive wiring harness crimping quality. A 5-megapixel industrial camera with a resolution of 2448*2048 and a white shadowless LED high-brightness light source are selected. The wiring harness to be inspected is placed on the battery fixing platform at the beginning of the conveyor belt, and the harness is fixed by grooves on the platform. The conveyor belt runs automatically, transporting the wiring harness to be inspected to below the industrial camera. With the assistance of the light source, the industrial camera captures images of the wiring harness and transmits the images to an industrial computer. The industrial computer analyzes the images using two built-in algorithms, detecting quality parameters of the wiring harness area, including core wire sequence and size, and crimping defects. The obtained inspection information is then compared with process standards to obtain the quality inspection results of the automotive wiring harness.

[0103] Through the reasonable layout and coordinated operation of the above modules, the automotive wiring harness crimping quality visual inspection device of the present invention realizes the fully automated operation from wiring harness transportation, image acquisition, defect detection and result display, meeting the requirements of modern automotive manufacturing industry for efficient, accurate and intelligent wiring harness crimping quality inspection.

[0104] The wire harness quality inspection consists of two parts: the first part inspects the core wire sequence and dimensions, and the second part inspects for compression defects. The following sections, with reference to specific implementation examples, will elaborate on the wire harness quality inspection methods for these two parts.

[0105] Example 2

[0106] Referring to Figure 2, this is a flowchart of a method for detecting the core wire sequence and size of an automotive wiring harness according to Embodiment 2 of this application. As shown in Figure 2, the specific implementation steps of the above method include:

[0107] Step 101: Acquire a complete image of the wire harness.

[0108] Step 102: Preprocess the above wire harness image, including converting the above wire harness image into a grayscale image and performing preliminary noise reduction processing through median filtering.

[0109] Specifically, the complete wire harness image acquired by the industrial camera is an RGB color image, containing three color channels: red (R), green (G), and blue (B). In this embodiment, by converting the RGB image to the HSV color space and directly taking the lightness (V) component as a grayscale image, the brightness information of the wire harness image is preserved while the color information of the wire harness image is removed.

[0110] Furthermore, the obtained grayscale image is subjected to median filtering to remove noise and smooth the image.

[0111] Step 103: Perform binarization on the above grayscale image to segment the wire harness image into a black background region and a white wire harness connected region.

[0112] After converting the above wire harness image to the HSV color space, histograms of the image in the three components of hue (H), saturation (S), and brightness (V) are calculated respectively. The horizontal axis of the histogram represents the gray value of the pixel, and the vertical axis represents the total number of pixels with that gray value.

[0113] Because the background area is relatively large and the pixel values ​​are similar, a peak with a large peak is formed in the low grayscale part of the histogram. The line bundle area occupies a small proportion of the total image, forming a peak with a low peak in the high grayscale part of the histogram. A valley is formed between the high peak and the low peak. The grayscale value corresponding to the lowest point of the valley is obtained. The average of the lowest values ​​obtained from the three component histograms is used as the threshold, as shown in formula (1). h T s T v These are the gray values ​​corresponding to the lowest points in the valleys of the histogram for the H, S, and V components, respectively.

[0114]

[0115] The wire harness image is binarized based on the threshold T mentioned above. The gray values ​​of pixels greater than the threshold T are set to 255, and the gray values ​​of pixels less than the threshold T are set to 0, thereby segmenting the wire harness image into a black background region and a white wire harness connected region.

[0116] Step 104: Perform an erosion operation on the binarized image to remove unnecessary noise and obtain the final wire bundle image. The white connected regions of the final wire bundle image consist only of wire bundles, making it easy to count the number of white connected regions and determine the number of wire bundles.

[0117] Step 105: Determine if the number of wire harnesses meets the preset standard. If it does, proceed to step 106. If it does not meet the preset standard, the automotive wiring harness product is deemed unqualified.

[0118] Step 106: Use the Canny operator to extract the edges of the processed wire harness image to obtain the edge contour of the wire harness.

[0119] This application embodiment is based on the Canny edge detection algorithm to extract the edge information of the wire harness from the wire harness image and determine the edge contour of the wire harness, providing a basis for further wire harness structure analysis.

[0120] Specifically, the Canny edge detection algorithm includes the following steps:

[0121] This application uses the Canny edge detection algorithm to extract the edge information of the wire harness from the wire harness image, thereby determining its edge contour and laying the foundation for subsequent wire harness structure analysis.

[0122] In practical applications, edge detection is easily affected by image noise. Therefore, before performing edge detection, a Gaussian filter is first used to smooth the image to effectively reduce the impact of noise on the detection results. Then, the Sobel operator is used to calculate the gradient intensity and direction of each pixel in the image. The gradient intensity reflects the saliency of the edge, while the gradient direction indicates the direction of the edge.

[0123] To further refine the edges, a non-maximum suppression operation is performed, retaining only pixels with the largest gradient intensity while suppressing pixels that are not local maxima, resulting in sharper, narrower edges. Building upon this, the Canny edge detection algorithm also performs thresholding and edge connection. Two thresholds are used: a low threshold and a high threshold. Pixels with gradient intensities above the high threshold are considered strong edges, while those below the low threshold are suppressed. For pixels in between, they are only considered part of an edge if they are connected to strong edge pixels. Finally, edge tracking technology connects broken edges to form complete edges, thus achieving accurate extraction and depiction of line bundle edges.

[0124] Step 107: Determine the wire bundle length based on the edge pixels of the edge contour.

[0125] Specifically, H pixels are randomly selected on the edge contour of the wire bundle and subjected to a random Hough transform. By statistically analyzing the peak values ​​in the Hough space, the line with the most collinear points is found. This line represents the entire wire bundle L, including the straight portion l1 and the curved portion l2. It should be noted that H is an adaptive threshold and needs to be adjusted according to the actual application scenario. A balance should be struck between recognition accuracy and computation speed when selecting this value. A value that is too small may lead to large recognition errors, while a value that is too large will affect computation speed.

[0126] The length of the straight line l1 is calculated using the equation y1, which is the Euclidean distance between the two endpoints:

[0127]

[0128] Where, A(x) a y a ), B(x) b y b () represents the two endpoints of the line.

[0129] The length of the arc l2 is calculated using the equation of the straight line y2:

[0130]

[0131] Where c and d are the x-coordinates of the two endpoints of the arc.

[0132] The sum of the lengths of the straight portion l1 and the curved portion l2 is used as the length of the line bundle in the image, thereby enabling the determination of the actual line bundle length based on a preset scaling factor.

[0133] Step 108: Determine if the above-mentioned wire harness length conforms to the preset size range. If yes, the product is considered qualified, and the wire harness sequence test continues. If the calculated wire harness length is not within the above-mentioned preset size range, the product is considered unqualified.

[0134] Specifically, because the Hough transform can only recognize curves that can be expressed by formulas, and the actual bending parts of wire harnesses are not standard circular arcs and cannot be represented by curve formulas, there is an error between the measured value and the actual value. Before actual production, it is necessary to measure n different sample wire harnesses and statistically analyze the actual value A of each sample wire harness. i and measured value M i The error between u i =A i -M i and the average error The error threshold ε0 is calculated and expressed as:

[0135]

[0136] Then, the same sample wire harness is measured m times to obtain several measurement values ​​M′ for the same wire harness. i The systematic error ε1 of the computer, expressed by the measurement value, is:

[0137]

[0138] After calculating the actual length of the wire harness by comparing the image with the actual value, if the difference between the wire harness length and the preset template value is within the threshold range (ε0+ε1), then the wire harness length is determined to be within the preset size range, and the wire harness size is deemed to be qualified.

[0139] Referring to Figure 3, this is a flowchart of a method for detecting the wiring sequence of an automotive wiring harness according to an embodiment of this application. As shown in Figure 3, the specific implementation steps of the above method include:

[0140] Step 201: Separate each wire bundle individually from the complete wire bundle image according to the number of connected components, based on the initial order from top to bottom, using the region segmentation method, to obtain multiple single wire bundle images.

[0141] Step 202: Convert each single wire harness image from the RGB color space to the HSV color space, and extract the pixel values ​​of the single wire harness image in the H, S, and V components.

[0142] Step 203: Based on the pixel value of each color component in the single wire harness image, determine the values ​​of the first-order color moment and the second-order color moment corresponding to the single wire harness image, which are used to characterize the color features and distribution characteristics of the single wire harness image.

[0143] Specifically, the value of the first-order color moment corresponding to a single line bundle image is calculated using formula (6), and is expressed as:

[0144]

[0145] In the formula, N represents the total number of pixels in a single wire harness image, and P i，j e represents the j-th pixel value of the i-th color component. i This represents the mean of all pixels in the i-th color component, i.e., the value of the first-order color moment.

[0146] The value of the first-order color moment corresponding to a single line bundle image is calculated using formula (7), and is expressed as:

[0147]

[0148] In the formula, S i The variance of all pixels representing the i-th color component is the second-order color moment.

[0149] Step 204: Calculate the difference between the first-order and second-order color moments corresponding to a single wire bundle image and the template wire bundle value at the corresponding position, and determine whether the difference is within a preset threshold range. If it is, it indicates that the single wire bundle sequence is correct.

[0150] Step 205: Repeat steps 202 to 204, making the same judgment for each wire harness. If the wire harness sequence is correct, then the wire harness sequence in this complete image is correct.

[0151] Verified wire harnesses, including those with the correct number of wires and correct wiring sequence, are precisely matched against preset wire harness size standards. The system then displays detailed information for each compliant wire harness, including its unique color and precise dimensions. This process not only ensures the accuracy and consistency of the wire harnesses but also provides strong support for quality control and production monitoring through visualization, thereby significantly improving the automation and intelligence level of the wire harness manufacturing process and ensuring the quality and reliability of the final product.

[0152] After completing the size and wiring sequence inspection of the wire harness, the next stage is the wire harness crimping defect inspection. The purpose of this stage is to ensure that no defects occur during the crimping process, such as loose crimping or crimping damage. The following section provides a detailed description of the vision-based wire harness crimping defect detection method provided in this application, using specific embodiments.

[0153] Referring to Figure 4, this is a flowchart of a wire harness crimping defect detection method provided in Embodiment 2 of this application. As shown in Figure 4, the specific implementation steps of the above method include:

[0154] Step 301: Set the process parameters and camera calibration parameters for crimping defects, and use an adaptive threshold segmentation algorithm to extract the crimping area of ​​the wire harness.

[0155] Before performing wire harness crimping defect detection, appropriate process parameters and camera calibration parameters need to be set. Process parameters include crimping height, width, crimping speed, crimping force, and the pressure, temperature, and time required during the crimping process. These parameters need to be determined based on the material properties of the wire harness and the crimping requirements. Camera calibration parameters involve both internal and external camera parameters. Internal parameters include focal length and pixel size, while external parameters include the distance and angle between the camera and the object being photographed. Accurate calibration of these parameters ensures the accuracy of subsequent image processing.

[0156] After setting the parameters, an adaptive threshold segmentation algorithm is used to extract the crimped area. Adaptive threshold segmentation can dynamically adjust the threshold according to the local characteristics of the image, thereby achieving image segmentation and separating the crimped area from the background. See Figure 5, which is a schematic diagram of the wire harness crimped area provided in Embodiment 2 of this application.

[0157] Step 302: Obtain the binary image of the above-extracted wire harness crimping area.

[0158] In one possible approach, a 3×3 neighborhood window is defined with the current pixel as the center, containing the center pixel and its eight surrounding neighboring pixels. The mean grayscale value of all pixels within the 3×3 neighborhood window is calculated. This mean value is used as the threshold T(x, y) for the current pixel, as shown in Equation (8).

[0159]

[0160] In the formula, T(x, y) is the threshold of the current pixel, W is the neighborhood window centered on the current pixel, l(i, j) is the gray value of the pixel in the window, and C is a constant used to adjust the sensitivity of segmentation.

[0161] Furthermore, the grayscale value of the current pixel is compared with the calculated threshold T(x, y). If the grayscale value of the current pixel is greater than the threshold (i.e., I(x, y) > T(x, y)), then the pixel is set to white. Otherwise, it is set to black.

[0162] Repeat the above steps to process each pixel in the image of the wire harness crimping area, ultimately generating a binary image of the wire harness crimping area. In the binary image, white areas represent the wire harness crimping area, while black areas represent the background or other non-crimped parts. This method effectively extracts the wire harness crimping area from the original image, providing clear image data for subsequent defect detection and analysis. The adaptive threshold segmentation algorithm dynamically adjusts the threshold based on the local characteristics of the image, thus better adapting to different lighting conditions and changes in image content, improving the accuracy and robustness of segmentation.

[0163] Step 303: Based on the pre-trained segmentation model for defect instances in the crimped area, detect the binary image of the wire harness crimped area to obtain a defect mask.

[0164] In this embodiment of the application, several images containing various types of crimping defects are pre-acquired, including images of crack defects, images of abnormal crimping shapes, images of oxidation defects on the crimping surface, and images of exposed core wire defects in the crimping area. All images are labeled by Labelme software.

[0165] A dataset of defects in the crimped area was constructed based on the images described above. The YOLACT instance segmentation model was configured with ResNet50 as the backbone network and a mask count of 32. Defect categories were defined, including cracks, abnormal crimped shape, oxidation of the crimped surface, and exposed core wires in the crimped area.

[0166] The above-mentioned defect dataset in the press-fit region is input into the configured YOLACT instance segmentation model, and feature information is extracted through the backbone network, represented as follows:

[0167] F∈R C×H×W (9)

[0168] In the formula, F represents the feature map dimension, C represents the number of feature channels, H represents the height of the feature map, and W represents the width of the feature map.

[0169] Furthermore, the feature map dimension F is reduced by using a 1×1 convolution kernel to decrease the number of feature channels C in feature map dimension F to 32, which serves as the number of shared mask prototypes. The shared mask dimension P is calculated using formula (10), expressed as:

[0170] P∈R M×H×W (10)

[0171] The original predicted value C of the YOLACT instance segmentation model is obtained by applying an activation function. i After normalization, the mask weight coefficients c are obtained. im The mask weight coefficients are a set of scalar values ​​that represent the degree of contribution of the shared mask prototype to a specific target instance, as shown in equations (11) and (12), and are expressed as follows:

[0172] C i ={C i1 C i2 C iM} (11)

[0173] c im =sigmoid(C i (12)

[0174] The shared mask is linearly combined using the above coefficients to generate the final instance mask. As shown in equation (13), the trained defect instance segmentation model for the press-fit region is obtained, which is expressed as:

[0175]

[0176] Where M represents the number of shared masks, P represents the final mask for the i-th instance. m (x, y) represents the m-th shared mask prototype, C im This represents the m-th weight of instance i.

[0177] Finally, the network is optimized using the binary cross-entropy loss function, as shown in Equation (14), to improve the quality of the generated mask, making it more accurate in representing the shape and position of the object, improving the accuracy of target detection, and enhancing the ability to distinguish different types of targets, including cracks, abnormal crimping shapes, oxidation of the crimping surface, and exposed core wires in the crimping area.

[0178]

[0179] Among them, M i (x, y) is the actual mask labeled by Labelme software. It is a predictive mask.

[0180] The optimized segmentation model for crimped areas is used to detect defects in the binary image of the wire harness crimped area, resulting in a defect mask used to mark defective regions within the crimped area. Based on this defect mask, the defect type corresponding to the defective regions in the wire harness crimped area is determined, including cracks, abnormal crimping shape, oxidation of the crimping surface, and exposed core wires in the crimping area. In this embodiment, the identification results are uploaded to an industrial computer, thereby achieving automated quality monitoring and defect management. This not only improves detection efficiency and accuracy but also provides crucial data support for subsequent production process optimization and quality control, further enhancing the intelligence level and responsiveness of the production line.

[0181] Step 304: Use OpenCV to extract the contour of the above defect mask and determine the size information of the crimping area.

[0182] Specifically, the dimensions of the crimping area include the perimeter, area, and tightness of the contact surface. The specific calculation method is described below.

[0183] In this embodiment, contour information is extracted from the mask image of the wire harness crimping region using OpenCV. The contour is a set of wire harness edges in the image, which can be represented as a series of points that define the boundary of the crimping region. Let the set of points representing the contour of the crimping region image be the contour, represented as:

[0184] contour={(x1, y1), (x2, y2),..., (x n y n (15)

[0185] Based on the above contour, calculate the distance d between adjacent points. i , represented as:

[0186]

[0187] In the formula, (x iy i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n+1 =x1,y n+1 =y1.

[0188] Based on the distance between the adjacent points mentioned above, the perimeter D of the contour is determined as follows:

[0189]

[0190] The area extraction method is as follows:

[0191] Based on the formula for calculating the area of ​​a polygon, the area A of the contact surface of the pressing region is determined, expressed as:

[0192]

[0193] Among them, (x i y i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n+1 =x1,y n+1 =y1.

[0194] The tightness of the contact surface of the pressing area is determined by the perimeter and area obtained from the above calculations, and is expressed as follows:

[0195]

[0196] The quality of the crimped area can be evaluated by calculating the perimeter, area, and compactness. For example, compactness can be used as a standard for the acceptable crimped contact area. The greater the compactness of the contact surface, the more irregular the shape of the contact surface.

[0197] This embodiment extracts key dimensional information of the crimping area from the image through contour extraction and geometric calculation, and uses this information to evaluate the crimping quality, automatically detecting and analyzing the wire harness crimping area, thereby improving production efficiency and product quality.

[0198] Step 305: If no crimping defects are detected and the tightness of the crimped contact surface is qualified, the product is deemed qualified, referring to the process qualification standard.

[0199] Step 306: If a crimping defect is detected or the tightness of the crimped contact surface is not up to standard, the product is deemed unqualified.

[0200] In practical applications, the automated wire harness crimping inspection system outputs the aforementioned visual inspection results to the display and control module in real time. If the inspection results indicate that the wire harness crimping quality meets the preset process standards, the display will show the message "Meets process standards, crimping qualified." Simultaneously, this result will be automatically recorded and stored in the system database for subsequent quality tracking and analysis. This instant feedback mechanism not only improves production efficiency but also ensures consistent product quality.

[0201] Conversely, if the inspection results show defects in the wire harness crimping, such as cracks, abnormal crimping shape, oxidation of the crimping surface, or exposed core wires in the crimping area, the display will clearly indicate "Crimping defect present, crimping unqualified," and list the specific defect type in detail. This information is also stored in the system database, providing critical data to the quality control team so they can quickly take corrective action, optimize production processes, and reduce the generation of non-conforming products.

[0202] Example 3

[0203] This third embodiment is a comprehensive description of all the technical solutions provided in the second embodiment above. It elaborates in detail, in conjunction with specific embodiments, the method for detecting the core wire sequence and size and crimping defects of automotive wiring harnesses based on wiring harness images.

[0204] Referring to Figure 6, this is a flowchart of a method for inspecting the crimping quality of automotive wiring harnesses according to Embodiment 3 of this application. As shown in Figure 6, the specific implementation steps of the above method include:

[0205] Step 401: Perform grayscale, noise reduction, and binarization processing on the above wire harness image to determine the white wire harness connected region of the wire harness image.

[0206] Step 402: Based on the above-mentioned white wire harness connected area, determine the number of wire harnesses.

[0207] Step 403: If the number of wire bundles is consistent with the first preset standard value, then perform edge extraction on the wire bundle image to obtain the edge pixels of the wire bundle and determine the wire bundle length.

[0208] Step 404: If the length of the above-mentioned wire harness meets the preset size range, then the wire harness image is divided into multiple single wire harness images according to the preset initial order.

[0209] Step 405: Extract the first-order and second-order color moments of each single wire bundle image in the HSV color space, and compare them with the difference between the template wire bundle values ​​at the corresponding positions of the single wire bundle.

[0210] Step 406: If the above differences are all within the preset threshold range, then the wire harness crimping area in the wire harness image is extracted, and a binary image of the wire harness crimping area is generated.

[0211] Step 407: Detect the above binary image based on the pre-trained crimping area defect instance segmentation model, generate a defect mask, and use it to mark crimping defects in the wire harness crimping area and determine the compactness of the crimping area. The compactness is determined by the perimeter and area of ​​the contact surface of the crimping area.

[0212] Step 408: If no crimping defects are detected and the tightness is consistent with the second preset standard value, the quality inspection of the qualified wire harness is completed.

[0213] Step 409: If the number of wire harnesses is inconsistent with the first preset standard value, or if the length of the wire harness is not within the preset size range, or if the difference between the template wire harness values ​​at the corresponding positions of any wire harness is not within the preset threshold range, or if a crimping defect is detected, or if the tightness is inconsistent with the second preset standard value, then the product is deemed unqualified.

[0214] It should be noted that the specific operation methods of each step in this embodiment are the same as those in Embodiment 2 above, and will not be repeated in Embodiment 3.

[0215] Compared with the prior art, the technical solution provided in this application has the following beneficial effects:

[0216] This application achieves efficient and accurate inspection of wire harness quality by combining image processing and deep learning. First, preprocessing operations such as denoising, grayscale conversion, binarization, and edge extraction on the wire harness image effectively remove image noise and clearly identify connected regions and edge features of the wire harness, providing accurate data for subsequent wire harness counts and length measurements. Second, feature extraction using first- and second-order color moments in the HSV color space, compared with template wire harness values, quickly determines whether the wire harness's color features meet standards, improving detection sensitivity and accuracy. Furthermore, a pre-trained crimping area defect instance segmentation model is used to detect crimping areas of the wire harness, accurately identifying crimping defects and generating defect masks. Simultaneously, the crimping quality is further verified by calculating the tightness of the contact surface. The entire solution not only achieves comprehensive inspection of wire harness quality but also significantly improves inspection efficiency, reduces errors and labor intensity associated with manual inspection, and possesses high practicality and reliability. It can be effectively applied to quality control in the wire harness production process, ensuring high quality and consistency of wire harness products.

[0217] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A visual inspection method for automotive wiring harness crimping quality, characterized in that, The method involves sequentially detecting the core wire sequence and size, and detecting compression defects in the wire harness based on the wire harness image. The wire harness image is subjected to grayscale, noise reduction, and binarization processing to determine the white wire harness connected regions of the wire harness image; The number of wire harnesses is determined based on the connected area of ​​the white wire harness; If the number of wire bundles is consistent with the first preset standard value, then the wire bundle image is subjected to edge extraction to obtain the edge pixels of the wire bundle, which are used to determine the wire bundle length; If the length of the wire harness meets the preset size range, the wire harness image is divided into multiple single wire harness images according to the preset initial order. Extract the first-order and second-order color moments of each single wire bundle image in the HSV color space, and compare them with the difference between the template wire bundle values ​​at the corresponding positions of the single wire bundle. If all the differences are within the preset threshold range, then the wire harness crimping area in the wire harness image is extracted, and a binary image of the wire harness crimping area is generated. The binary image is detected based on a pre-trained crimping region defect instance segmentation model to generate a defect mask, which is used to mark crimping defects in the wire harness crimping region and determine the compactness of the crimping region. The compactness is determined by the perimeter and area of ​​the contact surface of the crimping region. If no crimping defects are detected and the tightness is consistent with the second preset standard value, the quality inspection of the qualified wire harness is completed. The pre-trained defect instance segmentation model for the press-fit region is the YOLACT instance segmentation model, with ResNet50 as the backbone network. Training the defect instance segmentation model for the crimped area includes: A dataset of crimping defects is constructed by acquiring multiple images containing various types of crimping defects, including crack defects, abnormal crimping shape defects, crimping surface oxidation defects, and exposed core wire defects in the crimping area. The dataset of defects in the press-fit region is input into the configured YOLACT instance segmentation model, and feature information is extracted through the backbone network, represented as follows: F∈R C×H×W (9) In the formula, F represents the feature map dimension, C represents the number of feature channels, H represents the height of the feature map, and W represents the width of the feature map; The feature map dimension F is reduced by using a 1×1 convolution kernel to reduce the number of feature channels C of the feature map dimension F to 32, which is used as the number of shared mask prototypes; The shared mask dimension P is determined as follows: P∈R M×H×W (10) The original predicted value C of the YOLACT instance segmentation model is obtained by applying an activation function. i After normalization, the mask weight coefficients c are obtained. im , used to characterize the contribution of the shared mask prototype to a specific target instance, is expressed as: C i ={C i1 ,C i2 ,...,C iM } (11) C im =sigmoid(C i ) (12) Linear combination of shared masks generates the final instance mask. The trained defect instance segmentation model for the press-fit region is represented as: Where M represents the number of shared masks, P represents the final mask for the i-th instance; m (x,y) represents the m-th shared mask prototype, C im This represents the m-th weight of instance i; The network is optimized using the binary cross-entropy loss function to enhance the ability of the crimping defect instance segmentation model to distinguish different crimping defect types, as shown below: In the formula, M i (x,y) is the actual mask labeled by Labelme software. It is a predictive mask.

2. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, The image of the wire harness is subjected to grayscale conversion, denoising, and binarization, including: The wire harness image is converted to the HSV color space, and the lightness V component is taken as the grayscale image of the wire harness image. The grayscale image is subjected to median filtering to remove noise and smooth the image. The denoised grayscale image is binarized to segment the wire bundle image into a black background region and a white wire bundle connected region.

3. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Based on the connected region of the white wire harness, the number of wire harnesses is determined, including: An erosion operation is performed on the binarized wire harness image to remove excess noise, resulting in the final wire harness image; wherein, the white wire harness connected region of the final wire harness image consists only of wire harnesses, which is used to determine the number of wire harnesses.

4. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Edge extraction is performed on the wire harness image to obtain the edge pixels of the wire harness and determine the wire harness length, including: Based on the Canny edge detection algorithm, edge information of the wire harness is extracted from the wire harness image to determine the edge contour of the wire harness. Randomly select N pixels on the edge contour and perform a random Hough transform; By statistically analyzing the peak values ​​in Hough space, the line with the most collinear points is taken as the entire line bundle L, including the straight part l1 and the curved part l2. The length of the straight section l1 is calculated using the equation y1 of the straight line, which is the Euclidean distance between the two endpoints: Where, A(x) a ,y a ),B(x b ,y b () represents the two endpoints of the line; The length of the arc portion l2 is calculated using the equation of the straight line y2: Where c and d are the x-coordinates of the two endpoints of the arc; The sum of the lengths of the straight portion l1 and the curved portion l2 is taken as the length of the line bundle in the image; The actual length of the wire harness is determined based on its length in the image and a preset scaling factor.

5. The visual inspection method for automotive wiring harness crimping quality according to claim 4, characterized in that, Determining the preset size range includes: Select n different sample wire bundles and statistically analyze the actual value A of each sample wire bundle. i and measured value M i The error between u i =A i -M i and the average error The error threshold ε0 is calculated and expressed as: The same sample wire harness is measured m times, resulting in several measurement values ​​M′ for the same sample wire harness. i The systematic error ε1 is determined as follows: The preset size range is determined based on the error threshold ε0, the system error ε1, and the preset template value. If the difference between the wire harness length and the preset template value is within the threshold range (ε0+ε1), then the wire harness length is determined to conform to the preset size range.

6. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Extracting the first-order and second-order color moments of each of the single-strand images in the HSV color space, including: Extract the pixel values ​​of the single wire harness image in the H, S, and V components; Based on the pixel values, the first-order color moment and the second-order color moment corresponding to the single line bundle image are determined; wherein, the first-order color moment is expressed as: In the formula, N represents the total number of pixels in a single wire harness image, and P i,j e represents the j-th pixel value of the i-th color component. i The mean of all pixels representing the i-th color component is the value of the first-order color moment. The second-order color moment is represented as: In the formula, S i The variance of all pixels representing the i-th color component is the second-order color moment.

7. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Extracting the crimped area of ​​the wire harness from the wire harness image and generating a binary image of the crimped area includes: The wire harness crimping area is extracted based on an adaptive threshold segmentation algorithm; For each pixel of the image within the wire harness crimping area, a 3×3 neighborhood window is defined with the current pixel as the center, containing the center pixel and its 8 surrounding neighboring pixels; Calculate the mean grayscale value of all pixels within the 3×3 neighborhood window; Using the mean value as the threshold T(x,y) for the current pixel, it is expressed as: In the formula, T(x, y) is the threshold of the current pixel, W is the neighborhood window centered on the current pixel, I(i, j) is the gray value of the pixel in the window, and C is a constant used to adjust the sensitivity of segmentation. The gray value of the current pixel is compared with the calculated threshold T(x,y); if the gray value of the current pixel is greater than the threshold T(x,y), the current pixel is set to white; if the gray value of the current pixel is less than or equal to the threshold T(x,y), the current pixel is set to black. Repeat the above steps to process each pixel of the image within the wire harness crimping area to generate a binary image of the wire harness crimping area; wherein, the white area of ​​the binary image represents the wire harness crimping area, and the black area represents the background or other non-crimping areas.

8. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Based on a defect mask, crimping defects in the wire harness crimping area are marked and the tightness of the crimping area is determined, including: The defect mask is used to extract its contour, and the point set of the contour of the overpressed area image is obtained, represented as: contour={(x1, y1), (x2, y2),..., (x n y n (15) Based on the contour, calculate the distance d between adjacent points. i , represented as: In the formula, (x i y i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n +1 =x1,y n +1 =y1; Based on the distance between the adjacent points, the perimeter D of the contour is determined, and is expressed as: Based on the formula for calculating the area of ​​a polygon, the area A of the contact surface of the pressing region is determined, expressed as: In the formula, (xi) ， y i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n +1 =x1,y n +1 =y1; The tightness of the contact surface of the pressing area is determined by the perimeter D and the area A, and is expressed as follows:

Images