Automobile wire harness crimping quality visual detection method

Abstract

The application relates to the field of automobile electronic product detection, and particularly relates to a visual detection method for the crimping quality of an automobile wire harness. The wire harness image is subjected to denoising, grayscale and binarization processing, and the white wire harness connected region is determined. The number of wire harnesses is calculated through the region, and is compared with a preset standard value. If the number is consistent, edge extraction is performed to determine the length of the wire harness, and whether the length is within a preset size range is judged. After the conditions are met, the image is segmented into single wire harness images in a preset order, and the first-order and second-order color moments of the HSV color space are calculated and compared with the template wire harness value. If the difference is within a threshold range, the crimping region of the wire harness is intercepted and a binary image is generated. A pre-trained defect instance segmentation model is used to detect defects in the crimping region, a defect mask is generated, and the compactness of the contact surface is calculated. If there is no defect and the compactness meets the standard, qualified wire harness detection is completed, the number, length, color features and crimping region defects of the wire harness can be quickly identified, and the detection efficiency and accuracy are significantly improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of automobile electronic product detection, in particular to a visual detection method for crimping quality of automobile wiring harness. BACKGROUND

[0002] In the production process of automobile wiring harness, the detection of crimping quality is an important link to ensure the safety and reliability of automobile electrical systems. Currently, the detection of crimping quality of automobile wiring harness mainly adopts mechanical detection, electrical detection and microscopic observation methods. Mechanical detection evaluates the crimping strength and consistency by applying external force. Electrical detection uses methods such as conduction test and insulation resistance test to judge the conductivity and insulation performance of the crimping point. Microscopic observation uses a microscope or an industrial camera to magnify and analyze the appearance features of the crimping part.

[0003] However, these traditional detection methods have obvious defects and limitations. First, mechanical detection and microscopic observation usually require human intervention, the detection process is tedious and inefficient, and it is difficult to meet the mass production and high rhythm production requirements of modern automobile industry. Second, the electrical detection method cannot directly evaluate the appearance quality of the crimping point, and it is difficult to detect problems such as excessive crimping or insufficient crimping, and lacks comprehensive evaluation of multi-dimensional quality characteristics of the crimping of the wiring harness. In addition, the traditional detection methods are affected by human subjective judgment or equipment precision, and the consistency of the detection results is difficult to guarantee, which may lead to misjudgment or missed judgment.

[0004] With the rapid development of the automobile industry, higher requirements are put forward for the precision and efficiency of the crimping quality detection of the wiring harness. Therefore, it has become an urgent problem to develop an efficient, accurate and intelligent crimping quality detection method for automobile wiring harness. SUMMARY

[0005] Therefore, the application provides a visual detection method for crimping quality of automobile wiring harness, which combines modern visual detection and artificial intelligence technology to realize efficient, intelligent and automatic detection of crimping quality of automobile wiring harness.

[0006] The technical scheme provided by the application is as follows:

[0007] A visual detection method for crimping quality of automobile wiring harness, which sequentially detects the core wire sequence and size of the wiring harness based on the wiring harness image and detects the crimping defect, the method comprising:

[0008] The wiring harness image is subjected to grayscale, denoising and binarization processing to determine the white wiring harness connected region of the wiring harness image;

[0009] Based on the white wiring harness connected region, the number of wiring harness strips is determined;

[0010] If the number of wire harnesses is consistent with the first preset standard value, edge extraction is performed on the wire harness image to obtain edge pixel points of the wire harness, which are used to determine the wire harness length;

[0011] If the wire harness length meets the preset size range, the wire harness image is segmented into a plurality of single-wire harness images according to a preset initial order;

[0012] The first-order color moment and the second-order color moment of each single-wire harness image in the HSV color space are extracted, and the difference between the values and the template wire harness value at the corresponding position of the single-wire harness is compared;

[0013] If the difference is within the preset threshold range, the wire harness crimping area in the wire harness image is intercepted, and a binary image of the wire harness crimping area is generated;

[0014] Based on the pre-trained crimping area defect instance segmentation model, the binary image is detected to generate a defect mask, which is used to mark the crimping defects in the wire harness crimping area and determine the tightness of the crimping area, the tightness being determined by the perimeter and area of the contact surface of the crimping area;

[0015] If no crimping defects are detected and the tightness is consistent with the second preset standard value, the quality detection of the qualified wire harness is completed.

[0016] In one possible implementation, the wire harness image is subjected to grayscale processing, denoising and binarization, including:

[0017] The wire harness image is converted to the HSV color space, and the lightness V component is taken as the grayscale image of the wire harness image;

[0018] The grayscale image is subjected to median filtering processing for removing noise in the image and smoothing the image;

[0019] The denoised grayscale image is subjected to binarization processing for segmenting the wire harness image into a black background area and a white wire harness connected area.

[0020] In one possible implementation, based on the white wire harness connected area, the number of wire harnesses is determined, including:

[0021] The binarized wire harness image is subjected to an erosion operation to remove redundant noise points to obtain a final wire harness image; wherein the white wire harness connected area of the final wire harness image is composed of wire harnesses only, which is used to determine the number of wire harnesses.

[0022] In one possible implementation, the wire harness image is subjected to edge extraction to obtain edge pixel points of the wire harness, and the wire harness length is determined, including:

[0023] Based on the Canny edge detection algorithm, edge information of the wire harness is extracted from the wire harness image to determine the edge contour of the wire harness.

[0024] Randomly select N pixels on the edge contour and perform a random Hough transform;

[0025] By statistically analyzing the peak values ​​in Hough space, the line with the most collinear points is taken as the entire line bundle L, including the straight part l1 and the curved part l2.

[0026] The length of the straight section l1 is calculated using the equation y1 of the straight line, which is the Euclidean distance between the two endpoints:

[0027]

[0028] Where, A(x) a y a ), B(x) b y b () represents the two endpoints of the line;

[0029] The length of the arc portion l2 is calculated using the equation of the straight line y2:

[0030]

[0031] Where c and d are the x-coordinates of the two endpoints of the arc;

[0032] The sum of the lengths of the straight portion l1 and the curved portion l2 is taken as the length of the line bundle in the image;

[0033] The actual length of the wire harness is determined based on its length in the image and a preset scaling factor.

[0034] In one possible implementation, determining the preset size range includes:

[0035] Select n different sample wire bundles and statistically analyze the actual value A of each sample wire bundle. i and measured value M i The error between u i =A i -M i and the average error The error threshold ε0 is calculated and expressed as:

[0036]

[0037] The same sample wire harness is measured m times, resulting in several measurement values ​​M′ for the same wire harness. i The systematic error ε1 is determined as follows:

[0038]

[0039] The preset size range is determined according to the error threshold ε0, the system error ε1 and a preset template value, and if a difference between the wire harness length and the preset template value is within a threshold range (ε0+ε1), it is determined that the wire harness length conforms to the preset size range.

[0040] In one possible implementation, first-order color moments and second-order color moments of each of the single-wire harness images in an HSV color space are extracted, including:

[0041] Pixel values of the single-wire harness image in H, S and V components are extracted.

[0042] First-order color moments and second-order color moments corresponding to the single-wire harness image are determined based on the pixel values, wherein the first-order color moments are represented as:

[0043]

[0044] In the formula, N represents a total number of pixels of the single-wire harness image, P i，j represents a jth pixel value of an ith color component, e i represents a mean value of all pixels of the ith color component, that is, a value of the first-order color moment.

[0045] The second-order color moments are represented as:

[0046]

[0047] In the formula, S i represents a variance of all pixels of the ith color component, that is, a second-order color moment.

[0048] In one possible implementation, a wire harness crimping area in the wire harness image is intercepted, and a binary image of the wire harness crimping area is generated, including:

[0049] The wire harness crimping area is intercepted based on an adaptive threshold segmentation algorithm.

[0050] For each pixel of the image in the wire harness crimping area, a 3*3 neighborhood window is defined with the current pixel as the center, including the center pixel and 8 adjacent pixels around the center pixel.

[0051] A mean value of gray scale values of all pixels in the 3*3 neighborhood window is calculated.

[0052] The mean value is taken as a threshold T(x, y) of the current pixel, and is represented as:

[0053]

[0054] In the formula, T(x, y) is the threshold value of the current pixel, W is the neighborhood window centered on the current pixel, I(i, j) is the gray value of the pixel in the window, C is a constant for adjusting the sensitivity of segmentation, and W is the neighborhood window centered on the current pixel.

[0055] The gray value of the current pixel is compared with the calculated threshold value T(x, y); if the gray value of the current pixel is greater than the threshold value T(x, y), the current pixel is set to white; if the gray value of the current pixel is less than or equal to the threshold value T(x, y), the current pixel is set to black.

[0056] The above steps are repeated to process each pixel of the image in the wire bundle crimping area to generate a binary image of the wire bundle crimping area; wherein the white region of the binary image represents the wire bundle crimping area, and the black region represents the background or other non-crimping area.

[0057] In one possible implementation, the pre-trained crimping area defect instance segmentation model is a YOLACT instance segmentation model, and ResNet50 is used as the backbone network.

[0058] Training the crimping area defect instance segmentation model includes:

[0059] A plurality of images containing various crimping defect types are obtained to construct a crimping area defect dataset, and the crimping defect types include crack defects, crimping shape abnormal defects, crimping surface oxidation defects, and crimping area core wire exposure defects.

[0060] The crimping area defect dataset is input into the configured YOLACT instance segmentation model, and the feature information is extracted by the backbone network, represented as:

[0061] F∈R C×H×W (9)

[0062] In the formula, F represents the feature map dimension, C represents the number of feature channels, H represents the height of the feature map, and W represents the width of the feature map.

[0063] The feature map dimension F is reduced in dimension, and the number of feature channels C of the feature map dimension F is reduced to 32 using a 1x1 convolution kernel as the number of shared mask prototypes.

[0064] The shared mask dimension P is determined, represented as:

[0065] P∈R M×H×W (10)

[0066] The original prediction value C i of the YOLACT instance segmentation model is normalized by an activation function to obtain the mask weight coefficient c im, used to represent the contribution degree of the shared mask prototype on a specific target instance, denoted as:

[0067] C i = {C i1 , C i2 ,..., C iM} (11)

[0068] c im = sigmoid(C i ) (12)

[0069] Linearly combining the shared masks to generate the final instance mask That is, the trained crimping area defect instance segmentation model, denoted as:

[0070]

[0071] Wherein, M represents the number of shared masks, represents the final mask of the i-th instance. P m (x, y) represents the m-th shared mask prototype, C im represents the m-th weight of the instance i;

[0072] Using the binary cross-entropy loss function to optimize the network, used to enhance the distinguishing ability of the crimping area defect instance segmentation model for different crimping defect types, denoted as:

[0073]

[0074] In the formula, M i (x, y) is the real mask labeled by the Labelme software, is the predicted mask.

[0075] In one possible implementation, based on the defect mask, the crimping defects in the wire harness crimping area are marked and the tightness of the crimping area is determined, including:

[0076] Using opencv to extract the contour of the defect mask, obtaining the point set of the crimping area image contour, denoted as:

[0077] contour = {(x1, y1), (x2, y2),..., (x n , y n )} (15)

[0078] Based on the contour contour, the distance d i between adjacent points is calculated, denoted as:

[0079]

[0080] In the formula, (x i y i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n+1 =x1,y n+1 =y1;

[0081] Based on the distance between the adjacent points, the perimeter D of the contour is determined, and is expressed as:

[0082]

[0083] Based on the formula for calculating the area of ​​a polygon, the area A of the contact surface of the pressing region is determined, expressed as:

[0084]

[0085] In the formula, (x i y i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n+1 =x1,y n+1 =y1;

[0086] The tightness of the contact surface of the pressing area is determined by the perimeter D and the area A, and is expressed as follows:

[0087]

[0088] Compared with the prior art, the technical solution provided in this application has the following beneficial effects:

[0089] This application achieves efficient and accurate inspection of wire harness quality by combining image processing and deep learning. First, preprocessing operations such as denoising, grayscale conversion, binarization, and edge extraction on the wire harness image effectively remove image noise and clearly identify connected regions and edge features of the wire harness, providing accurate data for subsequent wire harness counts and length measurements. Second, feature extraction using first- and second-order color moments in the HSV color space, compared with template wire harness values, quickly determines whether the wire harness's color features meet standards, improving detection sensitivity and accuracy. Furthermore, a pre-trained crimping area defect instance segmentation model is used to detect crimping areas of the wire harness, accurately identifying crimping defects and generating defect masks. The crimping quality is further verified by calculating the tightness of the contact surface. This entire solution not only achieves comprehensive inspection of wire harness quality but also significantly improves inspection efficiency, reduces errors and labor intensity associated with manual inspection, and possesses high practicality and reliability. It can be effectively applied to quality control in the wire harness production process, ensuring high quality and consistency of wire harness products. Attached Figure Description

[0090] Figure 1A structural schematic diagram of a vehicle wire harness crimping quality visual detection system is provided for the embodiment one of the present application.

[0091] Figure 2 A flow chart of a vehicle wire harness core wire sequence and size detection method is provided for the embodiment two of the present application.

[0092] Figure 3 A flow chart of a vehicle wire harness sequence detection method is provided for the embodiment two of the present application.

[0093] Figure 4 A flow chart of a wire harness crimping defect detection method is provided for the embodiment two of the present application.

[0094] Figure 5 A schematic diagram of a wire harness crimping area is provided for the embodiment two of the present application.

[0095] Figure 6 A flow chart of a vehicle wire harness crimping quality visual detection method is provided for the embodiment three of the present application. DETAILED DESCRIPTION

[0096] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0097] Embodiment one

[0098] The embodiment one of the present application provides a structural schematic diagram of a vehicle wire harness crimping quality visual detection system. As shown in the figure, Figure 1 The overall structure of the vehicle wire harness crimping quality visual detection system includes, from the outside to the inside, a rack main body, a conveying module, a visual detection module, a display and control module, and a mobile support module. The rack main body is built with aluminum alloy profiles, serving as the main support frame of the entire device, which is light and sturdy, providing a mounting base and stable operating environment for other modules. The conveying module runs through the entire device and is installed in the middle of the rack main body. It connects the front and rear ends of the production line through a conveyor belt, used to convey the vehicle wire harness to be detected from the starting end to the detection area, and then continue to convey to the next process or unqualified rejection area after detection is completed, realizing full automation of the detection process.

[0099] The visual detection module is installed above the conveying module and includes an industrial camera, a high-brightness LED light source, and a mounting bracket. The industrial camera is fixed by the mounting bracket, and its detection area is directly opposite the wire harness crimping points on the conveying belt. The high-brightness LED light source is installed around the camera to provide uniform and high-intensity illumination for image acquisition, ensuring clear and reliable image quality. By adjusting the height and angle of the visual module bracket, the detection area of the industrial camera can completely cover the crimping points, thereby achieving accurate detection of the wire harness crimping quality.

[0100] The display and control module is installed on one side of the rack body and consists of a display and an industrial control computer. The industrial control computer is connected to the industrial camera of the visual detection module through a data line and is responsible for receiving and processing image data. The display is connected to the industrial control computer and is used to display the detection results and the operation interface in real time, facilitating the monitoring of the detection process and results by the operator. The display and control module cooperates with the conveying module and the visual detection module through control signals to realize the automation and intelligentization of the detection process.

[0101] The mobile support module is installed at the bottom of the device and consists of casters for facilitating the movement of the device between different production lines. The casters make the entire device flexible and versatile, allowing it to quickly adapt to different production environments and detection requirements, thereby improving the utilization rate and production efficiency of the device.

[0102] In a specific application scenario, the process parameters and camera calibration parameters of the automobile wire harness crimping quality are first set, a five-megapixel industrial camera with a resolution of 2448*2048 is selected, and a white shadowless LED high-brightness light source is selected. The wire harness to be detected is placed on the starting end battery fixing platform of the conveying belt and is fixed by the groove on the fixing platform. The conveying belt automatically runs to transport the wire harness to be detected under the industrial camera. The industrial camera captures the wire harness to be detected with the assistance of the light source and transmits the captured image to the industrial computer. The industrial computer analyzes the image by two built-in algorithms to detect the quality parameters of the wire harness region, including the core wire sequence and size and the wire harness crimping defects, and compares the obtained detection information with the process standard to obtain the quality detection result of the automobile wire harness.

[0103] Through the reasonable layout and cooperative work of the above modules, the automobile wire harness crimping quality visual detection device of the present application realizes the full-process automation operation from wire harness conveying, image acquisition, defect detection to result display, meeting the efficient, accurate, and intelligent requirements of the modern automobile manufacturing industry for wire harness crimping quality detection.

[0104] The wire harness quality detection is divided into two parts: the first part detects the core wire sequence and size, and the second part detects the crimping defects. Next, the wire harness quality detection methods for the above two parts are described in detail in combination with the method embodiments.

[0105] Embodiment Two

[0106] Referring to Figure 2 A flow chart of a method for detecting the order and size of a wire harness core provided in Embodiment Two of the present application is shown in FIG. 2. As shown in FIG. 2, the method includes the following steps: Figure 2

[0107] Step 101: Collecting a complete wire harness image.

[0108] Step 102: Preprocessing the wire harness image, including converting the wire harness image to a grayscale image and performing preliminary denoising processing by median filtering.

[0109] Specifically, the complete wire harness image obtained by the industrial camera is an RGB color image containing three color channels of red (R), green (G), and blue (B). In the present application, the RGB image is converted to the HSV color space, and the brightness V component is directly taken as the grayscale image, which retains the brightness information of the wire harness image while removing the color information of the wire harness image.

[0110] Further, the obtained grayscale image is subjected to median filtering to remove noise in the image and smooth the image.

[0111] Step 103: Performing binarization processing on the grayscale image to segment the wire harness image into black background regions and white wire harness connected regions.

[0112] After converting the wire harness image to the HSV color space, the histograms of the image in the hue H, saturation S, and brightness V components are calculated respectively. The horizontal axis of the histogram represents the grayscale value of the pixel, and the vertical axis represents the total number of pixel points of the grayscale value.

[0113] Since the background region has a large proportion and the pixel value is close, a peak with a large peak value is formed in the low grayscale part of the histogram. The wire harness region has a small proportion of the total image, and a peak with a low peak value is formed in the high grayscale region of the histogram. A valley is formed between the high peak and the low peak, and the lowest point of the valley corresponds to the grayscale value. The minimum values obtained from the three component histograms are averaged to obtain the threshold value, as shown in equation (1), T h s v T

[0114]

[0115] ​​​According to the threshold value T, the wire harness image is binarized, the pixel gray value greater than the threshold value T is set as 255, and the pixel gray value less than the threshold value T is set as 0, so that the wire harness image is divided into a black background region and a white wire harness connected region.

[0116] In step 104, an erosion operation is performed on the binarized image to remove redundant noise points, and a final wire harness image is obtained. The white connected domain of the final wire harness image is only composed of wire harnesses, so as to facilitate the counting of the number of white connected domains and the determination of the number of wire harnesses.

[0117] In step 105, it is judged whether the number of wire harnesses meets a preset standard. If yes, step 106 is executed. If no, the automobile wire harness product is determined to be unqualified.

[0118] In step 106, an edge extraction is performed on the processed wire harness image by using a Canny operator, and an edge contour of the wire harness is obtained.

[0119] In the embodiment of the application, the edge information of the wire harness is extracted from the wire harness image based on the Canny edge detection algorithm, and the edge contour of the wire harness is determined, thereby laying a foundation for further wire harness structure analysis.

[0120] Specifically, the Canny edge detection algorithm includes the following steps:

[0121] In the embodiment of the application, the edge information of the wire harness is extracted from the wire harness image by using the Canny edge detection algorithm, thereby determining the edge contour of the wire harness, and laying a foundation for subsequent wire harness structure analysis.

[0122] In actual application, edge detection is easily disturbed by image noise, so before edge detection, the image is first smoothed by using a Gaussian filter to effectively reduce the influence of noise on the detection result. Subsequently, the gradient intensity and direction of each pixel point in the image are calculated by using a Sobel operator. The gradient intensity reflects the significant degree of the edge, and the gradient direction indicates the trend of the edge.

[0123] In order to further refine the edge, a non-maximum suppression operation is performed, that is, only the pixel point with the maximum gradient intensity is retained, and those pixel points that are not local maximum values are suppressed, so as to obtain a clearer and narrower edge. On this basis, the Canny edge detection algorithm also performs threshold detection and edge connection. Two thresholds are used, including a low threshold and a high threshold. The pixel point with a gradient intensity higher than the high threshold is regarded as a strong edge, and the pixel point with a gradient intensity lower than the low threshold is suppressed. For the pixel point between the two thresholds, only when it is connected with the strong edge pixel point, it is regarded as a part of the edge. Finally, through edge tracking technology, the disconnected edges are connected to form a complete edge, so as to realize accurate extraction and depiction of the wire harness edge.

[0124] Step 107, determining the length of the wire bundle based on the edge pixels of the edge contour.

[0125] Specifically, H pixel points are randomly selected on the edge contour of the wire bundle for random Hough transformation, and the straight line with the most collinear points is found by counting the peak values in the Hough space. The straight line with the most collinear points represents the whole wire bundle L, including the straight line part l1 and the arc line part l2. It should be noted that the above H is an adaptive threshold value, which needs to be adjusted according to the actual application scenario. When taking the value, attention should be paid to balance the recognition accuracy and the calculation speed. If the value is too small, it will lead to large recognition error. If the value is too large, it will affect the calculation speed.

[0126] The length of the straight line part l1 is calculated by the straight line equation y1, that is, the Euclidean distance between the two end points:

[0127]

[0128] Where A(x a , y a ) and B(x b , y b ) are the two end points of the straight line.

[0129] The length of the arc line part l2 is calculated by the straight line equation y2:

[0130]

[0131] Where c and d are the horizontal coordinates of the two end points of the arc line.

[0132] The sum of the lengths of the straight line part l1 and the arc line part l2 is taken as the length of the wire bundle in the image, so that the actual length of the wire bundle can be determined according to the preset proportion coefficient.

[0133] Step 108, judging whether the length of the wire bundle meets the preset size range. If yes, it is determined that the product is qualified, and the wire bundle line sequence detection is continued. If the calculated length of the wire bundle is not within the above-mentioned preset size range, it is determined that the product is unqualified.

[0134] Specifically, since Hough transformation can only identify curves that can be expressed by formulas, and the curved part of the actual wire bundle cannot be expressed by a curve formula, there is an error between the measured value and the actual value. Before actual production, n groups of different sample wire bundles need to be measured first, and the error u i between the actual value A i and the measured value M i of each group of sample wire bundles and the average value of the error i are calculated. i ​The error threshold ε0 is calculated and expressed as:

[0135]

[0136] Then, the same sample wire harness is measured m times to obtain several measurement values ​​M′ for the same wire harness. i The systematic error ε1 of the computer, expressed by the measurement value, is:

[0137]

[0138] After calculating the actual length of the wire harness by comparing the image with the actual value, if the difference between the wire harness length and the preset template value is within the threshold range (ε0+ε1), then the wire harness length is determined to be within the preset size range, and the wire harness size is deemed to be qualified.

[0139] See Figure 3 This is a flowchart illustrating a method for detecting the wiring sequence of an automotive wiring harness, provided in an embodiment of this application. Figure 3 As shown, the specific implementation steps of the above method include:

[0140] Step 201: Separate each wire bundle individually from the complete wire bundle image according to the number of connected components, based on the initial order from top to bottom, using the region segmentation method, to obtain multiple single wire bundle images.

[0141] Step 202: Convert each single wire harness image from the RGB color space to the HSV color space, and extract the pixel values ​​of the single wire harness image in the H, S, and V components.

[0142] Step 203: Based on the pixel value of each color component in the single wire harness image, determine the values ​​of the first-order color moment and the second-order color moment corresponding to the single wire harness image, which are used to characterize the color features and distribution characteristics of the single wire harness image.

[0143] Specifically, the value of the first-order color moment corresponding to a single line bundle image is calculated using formula (6), and is expressed as:

[0144]

[0145] In the formula, N represents the total number of pixels in a single wire harness image, and P i，j e represents the j-th pixel value of the i-th color component. i This represents the mean of all pixels in the i-th color component, i.e., the value of the first-order color moment.

[0146] The value of the first-order color moment corresponding to a single line bundle image is calculated using formula (7), and is expressed as:

[0147]

[0148] In the formula, S i The variance of all pixels representing the i-th color component is the second-order color moment.

[0149] Step 204: Calculate the difference between the first-order and second-order color moments corresponding to a single wire bundle image and the template wire bundle value at the corresponding position, and determine whether the difference is within a preset threshold range. If it is, it indicates that the single wire bundle sequence is correct.

[0150] Step 205: Repeat steps 202 to 204, making the same judgment for each wire harness. If the wire harness sequence is correct, then the wire harness sequence in this complete image is correct.

[0151] Verified wire harnesses, including those with the correct number of wires and correct wiring sequence, are precisely matched against preset wire harness size standards. The system then displays detailed information for each compliant wire harness, including its unique color and precise dimensions. This process not only ensures the accuracy and consistency of the wire harnesses but also provides strong support for quality control and production monitoring through visualization, thereby significantly improving the automation and intelligence level of the wire harness manufacturing process and ensuring the quality and reliability of the final product.

[0152] After completing the size and wiring sequence inspection of the wire harness, the next stage is the wire harness crimping defect inspection. The purpose of this stage is to ensure that no defects occur during the crimping process, such as loose crimping or crimping damage. The following section provides a detailed description of the vision-based wire harness crimping defect detection method provided in this application, using specific embodiments.

[0153] See Figure 4 This is a flowchart of a wire harness crimping defect detection method provided in Embodiment 2 of this application. Figure 4 As shown, the specific implementation steps of the above method include:

[0154] Step 301: Set the process parameters and camera calibration parameters for crimping defects, and use an adaptive threshold segmentation algorithm to extract the crimping area of ​​the wire harness.

[0155] Before performing wire harness crimping defect detection, appropriate process parameters and camera calibration parameters need to be set. Process parameters include crimping height, width, crimping speed, crimping force, and the pressure, temperature, and time required during the crimping process. These parameters need to be determined based on the material properties of the wire harness and the crimping requirements. Camera calibration parameters involve both internal and external camera parameters. Internal parameters include focal length and pixel size, while external parameters include the distance and angle between the camera and the object being photographed. Accurate calibration of these parameters ensures the accuracy of subsequent image processing.

[0156] After setting the parameters, an adaptive thresholding segmentation algorithm is used to extract the indented area. Adaptive thresholding segmentation dynamically adjusts the threshold based on the local characteristics of the image, thereby achieving image segmentation and separating the indented area from the background. See also... Figure 5 This is a schematic diagram of the wire harness crimping area provided in Embodiment 2 of this application.

[0157] Step 302: Obtain the binary image of the above-extracted wire harness crimping area.

[0158] In one possible approach, a 3×3 neighborhood window is defined with the current pixel as the center, containing the center pixel and its eight surrounding neighboring pixels. The mean grayscale value of all pixels within the 3×3 neighborhood window is calculated. This mean value is used as the threshold T(x, y) for the current pixel, as shown in Equation (8).

[0159]

[0160] In the formula, T(x, y) is the threshold of the current pixel, W is the neighborhood window centered on the current pixel, l(i, j) is the gray value of the pixel in the window, and C is a constant used to adjust the sensitivity of segmentation.

[0161] Furthermore, the grayscale value of the current pixel is compared with the calculated threshold T(x, y). If the grayscale value of the current pixel is greater than the threshold (i.e., I(x, y) > T(x, y)), then the pixel is set to white. Otherwise, it is set to black.

[0162] Repeat the above steps to process each pixel in the image of the wire harness crimping area, ultimately generating a binary image of the wire harness crimping area. In the binary image, white areas represent the wire harness crimping area, while black areas represent the background or other non-crimped parts. This method effectively extracts the wire harness crimping area from the original image, providing clear image data for subsequent defect detection and analysis. The adaptive threshold segmentation algorithm dynamically adjusts the threshold based on the local characteristics of the image, thus better adapting to different lighting conditions and changes in image content, improving the accuracy and robustness of segmentation.

[0163] Step 303: Based on the pre-trained segmentation model for defect instances in the crimped area, detect the binary image of the wire harness crimped area to obtain a defect mask.

[0164] In this embodiment of the application, several images containing various types of crimping defects are pre-acquired, including images of crack defects, images of abnormal crimping shapes, images of oxidation defects on the crimping surface, and images of exposed core wire defects in the crimping area. All images are labeled by Labelme software.

[0165] A dataset of defects in the crimped area was constructed based on the images described above. The YOLACT instance segmentation model was configured with ResNet50 as the backbone network and a mask count of 32. Defect categories were defined, including cracks, abnormal crimped shape, oxidation of the crimped surface, and exposed core wires in the crimped area.

[0166] The above-mentioned defect dataset in the press-fit region is input into the configured YOLACT instance segmentation model, and feature information is extracted through the backbone network, represented as follows:

[0167] F∈R C×H×W (9)

[0168] In the formula, F represents the feature map dimension, C represents the number of feature channels, H represents the height of the feature map, and W represents the width of the feature map.

[0169] Furthermore, the feature map dimension F is reduced by using a 1×1 convolution kernel to decrease the number of feature channels C in feature map dimension F to 32, which serves as the number of shared mask prototypes. The shared mask dimension P is calculated using formula (10), expressed as:

[0170] P∈R M×H×W (10)

[0171] The original predicted value C of the YOLACT instance segmentation model is obtained by applying an activation function. i After normalization, the mask weight coefficients c are obtained. im The mask weight coefficients are a set of scalar values ​​that represent the degree of contribution of the shared mask prototype to a specific target instance, as shown in equations (11) and (12), and are expressed as follows:

[0172] C i ={C i1 C i2 C iM} (11)

[0173] c im =sigmoid(C i (12)

[0174] The shared mask is linearly combined using the above coefficients to generate the final instance mask. As shown in equation (13), the trained defect instance segmentation model for the press-fit region is obtained, which is expressed as:

[0175]

[0176] Where M represents the number of shared masks, P represents the final mask for the i-th instance. m (x, y) represents the m-th shared mask prototype, C imThis represents the m-th weight of instance i.

[0177] Finally, the network is optimized using the binary cross-entropy loss function, as shown in Equation (14), to improve the quality of the generated mask, making it more accurate in representing the shape and position of the object, improving the accuracy of target detection, and enhancing the ability to distinguish different types of targets, including cracks, abnormal crimping shapes, oxidation of the crimping surface, and exposed core wires in the crimping area.

[0178]

[0179] Among them, M i (x, y) is the actual mask labeled by Labelme software. It is a predictive mask.

[0180] The optimized segmentation model for crimped areas is used to detect defects in the binary image of the wire harness crimped area, resulting in a defect mask used to mark defective regions within the crimped area. Based on this defect mask, the defect type corresponding to the defective regions in the wire harness crimped area is determined, including cracks, abnormal crimping shape, oxidation of the crimping surface, and exposed core wires in the crimping area. In this embodiment, the identification results are uploaded to an industrial computer, thereby achieving automated quality monitoring and defect management. This not only improves detection efficiency and accuracy but also provides crucial data support for subsequent production process optimization and quality control, further enhancing the intelligence level and responsiveness of the production line.

[0181] Step 304: Use OpenCV to extract the contour of the above defect mask and determine the size information of the crimping area.

[0182] Specifically, the dimensions of the crimping area include the perimeter, area, and tightness of the contact surface. The specific calculation method is described below.

[0183] In this embodiment, contour information is extracted from the mask image of the wire harness crimping region using OpenCV. The contour is a set of wire harness edges in the image, which can be represented as a series of points that define the boundary of the crimping region. Let the set of points representing the contour of the crimping region image be the contour, represented as:

[0184] contour={(x1, y1), (x2, y2),..., (x n y n (15)

[0185] Based on the above contour, calculate the distance d between adjacent points. i , is represented as:

[0186]

[0187] In the formula, (x i y i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n+1 =x1,y n+1 =y1.

[0188] Based on the distance between the adjacent points mentioned above, the perimeter D of the contour is determined as follows:

[0189]

[0190] The area extraction method is as follows:

[0191] Based on the formula for calculating the area of ​​a polygon, the area A of the contact surface of the pressing region is determined, expressed as:

[0192]

[0193] Among them, (x i y i ) represents the coordinates of the i-th point on the contour, n is the total number of contour points, and x is the coordinates of the i-th point on the contour. n+1 =x1,y n+1 =y1.

[0194] The tightness of the contact surface of the pressing area is determined by the perimeter and area obtained from the above calculations, and is expressed as follows:

[0195]

[0196] The quality of the crimped area can be evaluated by calculating the perimeter, area, and compactness. For example, compactness can be used as a standard for the acceptable crimped contact area. The greater the compactness of the contact surface, the more irregular the shape of the contact surface.

[0197] This embodiment extracts key dimensional information of the crimping area from the image through contour extraction and geometric calculation, and uses this information to evaluate the crimping quality, automatically detecting and analyzing the wire harness crimping area, thereby improving production efficiency and product quality.

[0198] Step 305: If no crimping defects are detected and the tightness of the crimped contact surface is qualified, the product is deemed qualified, referring to the process qualification standard.

[0199] Step 306: If a crimping defect is detected or the tightness of the crimped contact surface is not up to standard, the product is deemed unqualified.

[0200] In practical applications, the automated wire harness crimping inspection system outputs the aforementioned visual inspection results to the display and control module in real time. If the inspection results indicate that the wire harness crimping quality meets the preset process standards, the display will show the message "Meets process standards, crimping qualified." Simultaneously, this result will be automatically recorded and stored in the system database for subsequent quality tracking and analysis. This instant feedback mechanism not only improves production efficiency but also ensures consistent product quality.

[0201] Conversely, if the inspection results show defects in the wire harness crimping, such as cracks, abnormal crimping shape, oxidation of the crimping surface, or exposed core wires in the crimping area, the display will clearly indicate "Crimping defect present, crimping unqualified," and list the specific defect type in detail. This information is also stored in the system database, providing critical data to the quality control team so they can quickly take corrective action, optimize production processes, and reduce the generation of non-conforming products.

[0202] Example 3

[0203] This third embodiment is a comprehensive description of all the technical solutions provided in the second embodiment above. It elaborates in detail, in conjunction with specific embodiments, the method for detecting the core wire sequence and size and crimping defects of automotive wiring harnesses based on wiring harness images.

[0204] See Figure 6 This is a flowchart of a method for inspecting the crimping quality of automotive wiring harnesses provided in Embodiment 3 of this application. Figure 6 As shown, the specific implementation steps of the above method include:

[0205] Step 401: Perform grayscale, noise reduction, and binarization processing on the above wire harness image to determine the white wire harness connected region of the wire harness image.

[0206] Step 402: Based on the above-mentioned white wire harness connected area, determine the number of wire harnesses.

[0207] Step 403: If the number of wire bundles is consistent with the first preset standard value, then perform edge extraction on the wire bundle image to obtain the edge pixels of the wire bundle and determine the wire bundle length.

[0208] Step 404: If the length of the above-mentioned wire harness meets the preset size range, then the wire harness image is divided into multiple single wire harness images according to the preset initial order.

[0209] Step 405: Extract the first-order and second-order color moments of each single wire bundle image in the HSV color space, and compare them with the difference between the template wire bundle values ​​at the corresponding positions of the single wire bundle.

[0210] Step 406: If the above differences are all within the preset threshold range, then the wire harness crimping area in the wire harness image is extracted, and a binary image of the wire harness crimping area is generated.

[0211] Step 407: Detect the above binary image based on the pre-trained crimping area defect instance segmentation model, generate a defect mask, and use it to mark crimping defects in the wire harness crimping area and determine the compactness of the crimping area. The compactness is determined by the perimeter and area of ​​the contact surface of the crimping area.

[0212] Step 408: If no crimping defects are detected and the tightness is consistent with the second preset standard value, the quality inspection of the qualified wire harness is completed.

[0213] Step 409: If the number of wire harnesses is inconsistent with the first preset standard value, or if the length of the wire harness is not within the preset size range, or if the difference between the template wire harness values ​​at the corresponding positions of any wire harness is not within the preset threshold range, or if a crimping defect is detected, or if the tightness is inconsistent with the second preset standard value, then the product is deemed unqualified.

[0214] It should be noted that the specific operation methods of each step in this embodiment are the same as those in Embodiment 2 above, and will not be repeated in Embodiment 3.

[0215] Compared with the prior art, the technical solution provided in this application has the following beneficial effects:

[0216] This application achieves efficient and accurate inspection of wire harness quality by combining image processing and deep learning. First, preprocessing operations such as denoising, grayscale conversion, binarization, and edge extraction on the wire harness image effectively remove image noise and clearly identify connected regions and edge features of the wire harness, providing accurate data for subsequent wire harness counts and length measurements. Second, feature extraction using first- and second-order color moments in the HSV color space, compared with template wire harness values, quickly determines whether the wire harness's color features meet standards, improving detection sensitivity and accuracy. Furthermore, a pre-trained crimping area defect instance segmentation model is used to detect crimping areas of the wire harness, accurately identifying crimping defects and generating defect masks. The crimping quality is further verified by calculating the tightness of the contact surface. This entire solution not only achieves comprehensive inspection of wire harness quality but also significantly improves inspection efficiency, reduces errors and labor intensity associated with manual inspection, and possesses high practicality and reliability. It can be effectively applied to quality control in the wire harness production process, ensuring high quality and consistency of wire harness products.

[0217] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A visual inspection method for automotive wiring harness crimping quality, characterized in that, The method involves sequentially detecting the core wire sequence and size, and detecting compression defects in the wire harness based on the wire harness image. The wire harness image is subjected to grayscale, noise reduction, and binarization processing to determine the white wire harness connected regions of the wire harness image; The number of wire harnesses is determined based on the connected area of ​​the white wire harness; If the number of wire bundles is consistent with the first preset standard value, then the wire bundle image is subjected to edge extraction to obtain the edge pixels of the wire bundle, which are used to determine the wire bundle length; If the length of the wire harness meets the preset size range, the wire harness image is divided into multiple single wire harness images according to the preset initial order. Extract the first-order and second-order color moments of each single wire bundle image in the HSV color space, and compare them with the difference between the template wire bundle values ​​at the corresponding positions of the single wire bundle. If all the differences are within the preset threshold range, then the wire harness crimping area in the wire harness image is extracted, and a binary image of the wire harness crimping area is generated. The binary image is detected based on a pre-trained crimping region defect instance segmentation model to generate a defect mask, which is used to mark crimping defects in the wire harness crimping region and determine the compactness of the crimping region. The compactness is determined by the perimeter and area of ​​the contact surface of the crimping region. If no crimping defects are detected and the tightness is consistent with the second preset standard value, the quality inspection of the qualified wire harness is completed; The pre-trained defect instance segmentation model for the press-fit region is the YOLACT instance segmentation model, with ResNet50 as the backbone network. Training the defect instance segmentation model for the crimped area includes: A dataset of crimping defects is constructed by acquiring multiple images containing various types of crimping defects, including crack defects, abnormal crimping shape defects, crimping surface oxidation defects, and exposed core wire defects in the crimping area. The dataset of defects in the press-fit area is input into the configured YOLACT instance segmentation model, and feature information is extracted through the backbone network, represented as follows: In the formula, F represents the feature map dimension, C represents the number of feature channels, H represents the height of the feature map, and W represents the width of the feature map; The feature map dimension F is reduced by using a 1×1 convolution kernel to reduce the number of feature channels C of the feature map dimension F to 32, which is used as the number of shared mask prototypes; The shared mask dimension P is determined as follows: The original predicted value C of the YOLACT instance segmentation model is obtained by applying an activation function. i Normalization is performed to obtain the mask weight coefficients. , used to characterize the contribution of the shared mask prototype to a specific target instance, is expressed as: Linear combination of shared masks generates the final instance mask. That is, the trained segmentation model of defect instances in the press-fit region is represented as: in, Indicates the number of shared masks. This represents the final mask for the i-th instance; This represents the m-th shared mask prototype. This represents the m-th weight of instance i; The network is optimized using the binary cross-entropy loss function to enhance the ability of the crimping defect instance segmentation model to distinguish different crimping defect types, as shown below: In the formula, It is the actual mask labeled by Labelme software. It is a predictive mask.

2. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, The image of the wire harness is subjected to grayscale conversion, denoising, and binarization, including: The wire harness image is converted to the HSV color space, and the lightness V component is taken as the grayscale image of the wire harness image. The grayscale image is subjected to median filtering to remove noise and smooth the image. The denoised grayscale image is binarized to segment the wire bundle image into a black background region and a white wire bundle connected region.

3. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Based on the connected region of the white wire harness, the number of wire harnesses is determined, including: An erosion operation is performed on the binarized wire harness image to remove excess noise, resulting in the final wire harness image; wherein, the white wire harness connected region of the final wire harness image consists only of wire harnesses, which is used to determine the number of wire harnesses.

4. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Edge extraction is performed on the wire harness image to obtain the edge pixels of the wire harness and determine the wire harness length, including: Based on the Canny edge detection algorithm, edge information of the wire harness is extracted from the wire harness image to determine the edge contour of the wire harness. Randomly select N pixels on the edge contour and perform a random Hough transform; By statistically analyzing the peak values ​​in Hough space, the line with the most collinear points is taken as the entire line bundle L, including the straight part l1 and the curved part l2. The length of the straight section l1 is calculated using the equation y1 of the straight line, which is the Euclidean distance between the two endpoints: in, , The two endpoints of the line; The length of the arc portion l2 is calculated using the equation of the straight line y2: Where c and d are the x-coordinates of the two endpoints of the arc; The sum of the lengths of the straight portion l1 and the curved portion l2 is taken as the length of the line bundle in the image; The actual length of the wire harness is determined based on its length in the image and a preset scaling factor.

5. The visual inspection method for automotive wiring harness crimping quality according to claim 4, characterized in that, Determining the preset size range includes: Select n different sample wire bundles and statistically analyze the actual value A of each sample wire bundle. i and measured value M i Error between and the average error Calculate the error threshold , is represented as: Measurement of the same sample harness This yielded several measurements of the same sample wire bundle. Determine the systematic error , is represented as: According to the error threshold Systematic errors The preset size range is determined by the preset template value, and if the difference between the wire harness length and the preset template value is within the threshold range... If the length of the wire harness conforms to the preset size range, then the wire harness length is determined to be within the preset size range.

6. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Extracting the first-order and second-order color moments of each of the single-strand images in the HSV color space, including: Extract the pixel values ​​of the single wire bundle image in the H, S, and V components; Based on the pixel values, the first-order color moment and the second-order color moment corresponding to the single line bundle image are determined; wherein, the first-order color moment is expressed as: In the formula, N represents the total number of pixels in a single wire harness image, and P i,j e represents the j-th pixel value of the i-th color component. i The mean of all pixels representing the i-th color component, i.e., the value of the first-order color moment; The second-order color moment is represented as: In the formula, S i The variance of all pixels representing the i-th color component is the second-order color moment.

7. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Extracting the crimped area of ​​the wire harness from the wire harness image and generating a binary image of the crimped area includes: The wire harness crimping area is extracted based on an adaptive threshold segmentation algorithm; For each pixel of the image within the wire harness crimping area, a 3×3 neighborhood window is defined with the current pixel as the center, containing the center pixel and its 8 surrounding neighboring pixels; Calculate the mean grayscale value of all pixels within the 3×3 neighborhood window; Using the mean value as the threshold T(x,y) for the current pixel, it is expressed as: In the formula, T(x, y) is the threshold of the current pixel, W is the neighborhood window centered on the current pixel, I(i, j) is the gray value of the pixel in the window, and C is a constant used to adjust the sensitivity of segmentation. The gray value of the current pixel is compared with the calculated threshold T(x,y); if the gray value of the current pixel is greater than the threshold T(x,y), the current pixel is set to white; if the gray value of the current pixel is less than or equal to the threshold T(x,y), the current pixel is set to black. Repeat the above steps to process each pixel of the image within the wire harness crimping area to generate a binary image of the wire harness crimping area; wherein, the white area of ​​the binary image represents the wire harness crimping area, and the black area represents the background or other non-crimping areas.

8. The visual inspection method for automotive wiring harness crimping quality according to claim 1, characterized in that, Based on a defect mask, crimping defects in the wire harness crimping area are marked and the tightness of the crimping area is determined, including: The defect mask is used to extract its contour, and the point set of the contour of the overpressed area image is obtained, represented as: Based on the contour, calculate the distance d between adjacent points. i , is represented as: In the formula, Here, is the coordinate of the i-th point on the contour, and n is the total number of contour points. ; Based on the distance between the adjacent points, the perimeter D of the contour is determined, and is expressed as: Based on the formula for calculating the area of ​​a polygon, the area A of the contact surface of the pressing region is determined, expressed as: In the formula, Here, is the coordinate of the i-th point on the contour, and n is the total number of contour points. ; The tightness of the contact surface of the pressing area is determined by the perimeter D and the area A, and is expressed as follows: 。

Images