A vision-based method and system for measuring a flexible wire harness

Abstract

The application discloses a kind of based on vision's flexible wire harness measurement method and system, can realize the quick and accurate measurement of wire harness.The based on vision's flexible wire harness measurement method described in the application extracts wire harness image area by wire harness image pre-processing.Using image thinning algorithm and branch removal method based on intersection point extracts the main curve of wire harness.Then the main curve is scanned, the approximate position of buckle is obtained by the coarse positioning of buckle, the fine positioning of buckle accurately determines the buckle position, and the buckle is segmented and identified using the buckle accurate positioning result.Finally, the actual wire length between every two adjacent buckles is calculated using the method of segmented curve fitting combined with magnification factor correction.The based on vision's flexible wire harness measurement system described in the application obtains wire harness image through camera, and the image processing algorithm measures the wire harness, and a convenient and accurate flexible wire harness measurement method is provided.

Description

Technical Field

[0001] The present invention relates to the field of vision technology and measurement technology, and in particular to a vision-based measurement method and system for flexible wire harnesses. Background Art

[0002] With the rapid development of artificial intelligence, the demand for efficient and intelligent measurement technology in production and daily life is increasing. Wire harnesses are widely used in various fields, such as automobiles, electrical appliances and other fields. The manufacturing quality of wire harnesses is related to whether the wire harness can be assembled correctly and quickly, as well as the performance and service life of the product. Wire harnesses often cannot guarantee processing accuracy after primary processing. Inspection is a direct guarantee means to ensure the manufacturing quality of wire harnesses and stress-free installation. At present, it is usually necessary to manually inspect and measure wire harnesses to screen out unqualified products. However, this method is inefficient. In recent years, with the rapid development of sensor technology, image processing technology and the like, advanced detection technology is gradually being applied to quality inspection.

[0003] Visual measurement is a non-contact measurement method and is the mainstream development direction of next-generation measurement methods. When using vision to measure a wire harness, a camera first acquires an image of the wire harness, and then image processing methods are used to detect the wire harness, including checking the presence or absence of wire harness buckles and measuring the wire length between adjacent buckles of the wire harness. This method has high efficiency and measurement accuracy, can meet practical application requirements, and has great application prospects.

[0004] The structure of a wire harness is complex. There are buckles with different shapes and uncertain orientations on the wire harness, and the color of the buckles is similar to that of the wire harness conduit portion. Accurate segmentation and recognition of buckles is the premise for measuring the length of the wire harness conduit portion. In addition, the positions of wire harness buckles determine the length of the conduit portion between the buckles, which further affects whether the wire harness can be assembled correctly. The conduit portion of a wire harness is a curve with an uncertain shape, and achieving accurate measurement of the length of the conduit portion is the key point in wire harness measurement. At the same time, light reflection may occur due to the color and material of the wire harness, which is not conducive to subsequent processing. How to solve the above problems in visual measurement of wire harnesses is the key to accurate measurement of wire harnesses in practical applications. Summary of the Invention

[0005] To overcome the above-mentioned defects existing in the prior art, the object of the present invention is to provide a vision-based measurement method and system for flexible wire harnesses, which can measure wire harnesses accurately and quickly. A camera captures an image of the wire harness, then the measurement values of corresponding detection items are obtained through image processing, and the measurement values are compared with standard values. If the values are within an allowable error range, it indicates that the wire harness is qualified; otherwise, it indicates that the wire harness is unqualified.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] A vision-based measurement method for flexible wire harnesses, comprising the following steps:

[0008] S1: Preprocessing of the wire harness image. A light source is positioned below the wire harness, illuminating it upwards. The camera is positioned above the wire harness, with the wire harness centered in the camera's field of view. The image of the wire harness is then acquired through the camera. The acquired wire harness image is preprocessed to extract the relevant regions. The specific steps are as follows:

[0009] S11: Filter the wire harness image to remove noise and improve image quality;

[0010] S12: Threshold the image. Use the thresholded image to find the contours of objects in the image. Determine the region where the line bundle is located based on the size of the contours. The contour with the most points among all contours is the contour of the line bundle, as shown below:

[0011] C l =max{C1,C2,…,C n}

[0012] Where C l Indicates the outline of the wire harness, C n This represents the nth contour, where n represents the total number of contours.

[0013] S13: Obtain the smallest regular rectangle of the wire harness based on its outline, and extract the area where the wire harness is located by expanding outward by several pixels from each side of the smallest regular rectangle, thereby reducing the scope of subsequent processing.

[0014] S2: Harness Master Curve Extraction. After obtaining the harness region, the image is thinned. Due to the presence of clips, branches exist on the extracted harness skeleton. A branch removal method based on intersection points is used to remove the branches on the thinned image to obtain the harness master curve. The specific steps are as follows:

[0015] S21: Use a thinning algorithm to thin the wire bundles in the extracted wire bundle region image to obtain the skeleton of the wire bundle;

[0016] S22: Remove redundant points on the wire harness skeleton. That is, after processing, each point on the skeleton, except for the endpoints and intersections, has only two non-zero pixel values ​​in its eight-neighborhood, the endpoints have only one non-zero pixel value in their eight-neighborhood, and the intersections have three or more non-zero pixel values ​​in their eight-neighborhood.

[0017] S23: Find all intersection points based on the number of non-zero pixel values ​​in the eight neighboring regions. The determination method is as follows:

[0018] num(N8(x i ,y i ()>0)≥3,i=0,1,…,7

[0019] Where N8(x) i ,y i ) represents the point (x,y) in the eight neighborhood of the point (x,y). i ,y i The pixel value at position ) is represented by num(), where num() represents the number of pixels.

[0020] S24: Starting from each found intersection point, the point whose pixel value in its eight neighborhood is not zero is the starting point of a path. The center point is updated according to the eight neighborhood information of the point to realize scanning along different paths. The following conditions are used to determine whether to stop scanning: (1) If the scanned point is an endpoint and the number of scanned points is less than the threshold, the pixel value of all scanned points on this path is set to zero; (2) If the scanned point is another intersection point, the scanning of this path is stopped; (3) If the number of scanned points on a path exceeds the threshold, the scanning of this path is stopped; (4) After setting the points on a path to zero or after completing the scanning of all paths, the above operation is started on other intersection points.

[0021] S25: Use an iterative method to remove branches from the wire harness skeleton. If the number of intersections remains the same in the previous two iterations, it means that all branches have been removed and the main curve of the wire harness has been obtained. Save the points on the main curve sequentially from one end to the other end of the main curve.

[0022] S3: Segmentation and Recognition of Wire Harness Clips. After obtaining the main curve of the wire harness, the position of the clip is coarsely and finely positioned based on the main curve to achieve precise positioning of the clip. Then, the clip is segmented and recognized based on the positioning results. The specific steps are as follows:

[0023] S31: Perform coarse positioning of the clips on the wire harness, scanning from one end of the main curve to the other. Coarse positioning of the clips is determined by the distance from each point on the main curve to the wire harness contour. The distance to each point on the main curve is gradually increased along the normal direction at that point to determine if the wire harness contour has been reached, as shown below:

[0024] x=x0±n×cosθ

[0025] y = y0 ± n × sinθ

[0026] Where x0 and y0 are the coordinates of a point on the main curve, θ is the normal vector angle at this point, ± represents different sides, and x and y represent the coordinates of the normal vector that are n distances from (x0, y0). Based on the pixel value at point (x, y) and the threshold image, it can be determined whether the wire harness outline boundary has been reached, thus obtaining the distance value. Coarse segmentation of the wire harness clip is achieved based on the distance value of each point on the main curve, as shown in the following method:

[0027] d1>th1,or

[0028] d2>th1

[0029] Where d1 and d2 are the distances from points on the main curve to different sides of the line bundle contour, and th1 is the threshold.

[0030] S32: Perform fine positioning on the buckles of the wire harness. Using the coarse positioning position obtained in S31, identify a point on the main curve a certain distance away from the coarse positioning position in the direction away from the buckle as the starting point for fine positioning. Search for points on the wire harness contour along the normal direction from the starting point for fine positioning, and use these points as the starting search points for fine positioning. Starting from the starting search point, search along the wire harness contour in the direction of the buckle based on the eight-neighborhood information of the point. The method for determining the precise positioning point is as follows:

[0031]

[0032] Δa=|aa o |

[0033] Δa>th2

[0034] Where a o Let a be the initial normal angle. i Let be the angle of the normal vector at the initial i-th point, a be the angle of the normal vector at the search point, Δa be the difference between the two normal vector angles, and th2 be the threshold.

[0035] S33: Connect the corresponding precise positioning points of the same buckle, and use the outline of the buckle to divide the buckle;

[0036] S34: The buckle is identified using a feature fusion method, and feature F is shown below:

[0037] F = {F HOG ,F Hu ,F Length}

[0038] Where F HOG For the directional gradient histogram features of the snap-in, F Hu For the Hu-shaped feature of the snap fastener, F Length The length feature of the buckle is along the same direction as the main curve. A random forest is used to train and identify the features of different types of buckles.

[0039] S4: Measurement of wire length between wire harness clips. Based on the precise positioning results of the clips, the curve length between adjacent clips is calculated using a method combining curve fitting and amplification factor correction to obtain the wire length between the clips. The specific steps are as follows:

[0040] S41: The fitting data is obtained by using points on the main curve between adjacent snap fasteners and two extraction endpoints. Points on the main curve between adjacent snap fasteners have been partially removed from those too close to the snap fasteners. The extraction endpoints are obtained from the most accurate positioning points closest to the snap fasteners on the line. The calculation method is as follows:

[0041]

[0042]

[0043] Where (x) e ,y e (x) represents the coordinates of the extracted endpoints. p1 ,y p1 (x) represents the coordinates of the precise location point 1. p2 ,y p2 () represents the coordinates of the precise location point 2;

[0044] S42: Determine the trend of this part based on the coordinates of adjacent points on the main curve. If the x-coordinates of three adjacent points are the same, the trend of this part of the curve is defined as vertical. If the y-coordinates of three adjacent points are the same, the trend of this part of the curve is defined as horizontal. The point where the trend of the curve changes is defined as the segmentation point.

[0045] S43: Divide the main curve into segments based on the segmentation points. Transform the coordinates of the segmented curves according to their trends. For horizontally trending curves, no coordinate transformation is needed; for vertically trending curves, swap the x and y coordinates. Fit each segmented curve using a 4th-order polynomial curve fitting method. Calculate the length of each curve segment based on the fitting results, using the following method:

[0046]

[0047] Where a and b are the x-coordinates of the starting and ending points of the fitted data, and p1, p2, p3, and p4 are the fourth-order fitting coefficients. The total length l of the line between the clips... total The calculation method is as follows:

[0048]

[0049] Among them l i Let be the length of the i-th curve segment, and k be the total number of segments.

[0050] S44: Obtain the actual line length using a magnification factor correction method. Magnification factor interpolation data is obtained using a dot calibration plate with a 5mm center-to-center spacing between adjacent dots. First, the dot calibration plate is placed in the center of the camera's field of view to acquire the image. The center coordinates of the dot closest to the image center and the center coordinates of its four adjacent dots are extracted. The reference unit length is l. ref The calculation method is as follows:

[0051]

[0052] Among them l c1 , l c2 , l c3 , l c4 This represents the distance between the center of the nearest dot to the image center and the centers of its four adjacent dots. The magnification factor M at the coordinates of the other dot centers is also given. j The calculation method is as follows:

[0053]

[0054] Among them l j The unit length at the center coordinates of other points is calculated using the same method as l. ref The process is the same. Place the dot calibration plate at different positions within the camera's field of view and capture images. Obtain the magnification factor at the center coordinates of the dots on the calibration plate. Repeat this process until the positions where the dot calibration plate was placed cover the entire camera's field of view. Interpolate all the acquired magnification factor data to obtain the magnification factor for each point in the corresponding image, and the actual line length l. actual It can be obtained in the following ways:

[0055]

[0056]

[0057] Where M i is the magnification factor at the coordinates of the i-th point in the curve fitting data, and m is the number of curve fitting data.

[0058] A vision-based measurement system for flexible wire harnesses, based on the aforementioned flexible wire harness measurement method, includes an operating table, a camera, a light-diffusing plate, a light source, and a host computer. The operating table is used to fix the camera, light-diffusing plate, light source, and host computer. The light source illuminates the wire harness from bottom to top through the light-diffusing plate, with the wire harness placed on the plate to make the outline of the captured wire harness image clearer. The camera is positioned above the wire harness to capture images, and the image data is transmitted to the host computer via a network cable. The camera uses soft triggering, capturing one image at a time. After image processing, the measurement results of the wire harness are obtained. Based on the measurement results, the pass / fail status of the wire harness is judged, and finally, the pass / fail status of the wire harness is determined.

[0059] Compared with the prior art, the present invention has the following advantages:

[0060] This invention realizes a vision-based flexible wire harness measurement method and system. Backlighting is used to illuminate the wire harness, and a camera captures images of the wire harness, obtaining images with clear boundaries. The vision-based flexible wire harness measurement method overcomes the low efficiency of traditional manual measurement and offers significantly improved ease of operation, showing great application potential. Regarding the detection of wire harness clips, precise clip positioning is achieved by scanning along the main curve of the wire harness, combined with coarse and fine positioning. The clips are then segmented based on the positioning results, and feature fusion and random forest are used for clip identification. In calculating the guide tube length between adjacent clips, since the guide tube is a curve with uncertain shape and direction, a piecewise curve fitting method combined with amplification factor correction is used to measure the line length, greatly improving accuracy and efficiency. Attached Figure Description

[0061] Figure 1 This is a flowchart of a vision-based flexible wire harness measurement method;

[0062] Figure 2 This is a schematic diagram of the wire harness detection system in this invention;

[0063] Figure 3 This is a schematic diagram of the main curve extraction in this invention;

[0064] Figure 4 This is a schematic diagram of the coarse positioning scan of the wire harness buckle in this invention;

[0065] Figure 5 This is a schematic diagram of the fine positioning of the wire harness clip in this invention;

[0066] Figure 6 This is a schematic diagram of curve segmentation in this invention;

[0067] Figure 7 This is a schematic diagram of the calculation of the reference unit length in this invention. Detailed Implementation

[0068] The specific implementation of the present invention will be further described in detail below with reference to the accompanying drawings.

[0069] like Figure 1 As shown, a vision-based flexible wire harness measurement method includes the following specific steps:

[0070] S1: Preprocessing of the wire harness image. A light source is positioned below the wire harness, illuminating it upwards. The camera is positioned above the wire harness, with the wire harness centered in the camera's field of view. The image of the wire harness is then acquired through the camera. The acquired wire harness image is preprocessed to extract the relevant regions. The specific steps are as follows:

[0071] S11: Filter the wire harness image to remove noise and improve image quality;

[0072] S12: Threshold the image. Use the thresholded image to find the contours of objects in the image. Determine the region where the line bundle is located based on the size of the contours. The contour with the most points among all contours is the contour of the line bundle, as shown below:

[0073] C l =max{C1,C2,…,C n}

[0074] Where C l Indicates the outline of the wire harness, C n This represents the nth contour, where n represents the total number of contours.

[0075] S13: Obtain the smallest regular rectangle of the wire harness based on its outline, and extract the area where the wire harness is located by expanding outward by several pixels from each side of the smallest regular rectangle, thereby reducing the scope of subsequent processing.

[0076] S2: Harness Master Curve Extraction. After obtaining the harness region, the image is thinned. Due to the presence of clips, branches exist on the extracted harness skeleton. A branch removal method based on intersection points is used to remove the branches on the thinned image, obtaining the harness master curve, as shown below. Figure 3 As shown, the specific steps are as follows:

[0077] S21: Use a thinning algorithm to thin the wire bundles in the extracted wire bundle region image to obtain the skeleton of the wire bundle;

[0078] S22: Remove redundant points on the wire harness skeleton. That is, after processing, each point on the skeleton, except for the endpoints and intersections, has only two non-zero pixel values ​​in its eight-neighborhood, the endpoints have only one non-zero pixel value in their eight-neighborhood, and the intersections have three or more non-zero pixel values ​​in their eight-neighborhood.

[0079] S23: Find all intersection points based on the number of non-zero pixel values ​​in the eight neighboring regions. The determination method is as follows:

[0080] num(N8(x i ,y i ()>0)≥3,i=0,1,…,7

[0081] Where N8(x) i ,y i ) represents the point (x,y) in the eight neighborhood of the point (x,y). i ,y i The pixel value at position ) is represented by num(), where num() represents the number of pixels.

[0082] S24: Starting from each found intersection point, the point whose pixel value in its eight neighborhood is not zero is the starting point of a path. The center point is updated according to the eight neighborhood information of the point to realize scanning along different paths. The following conditions are used to determine whether to stop scanning: (1) If the scanned point is an endpoint and the number of scanned points is less than the threshold, the pixel value of all scanned points on this path is set to zero; (2) If the scanned point is another intersection point, the scanning of this path is stopped; (3) If the number of scanned points on a path exceeds the threshold, the scanning of this path is stopped; (4) After setting the points on a path to zero or after completing the scanning of all paths, the above operation is started on other intersection points.

[0083] S25: Use an iterative method to remove branches from the wire harness skeleton. If the number of intersections remains the same in the previous two iterations, it means that all branches have been removed and the main curve of the wire harness has been obtained. Save the points on the main curve sequentially from one end to the other end of the main curve.

[0084] S3: Segmentation and Recognition of Wire Harness Clips. After obtaining the main curve of the wire harness, the position of the clip is coarsely and finely positioned based on the main curve to achieve precise positioning of the clip. Then, the clip is segmented and recognized based on the positioning results. The specific steps are as follows:

[0085] S31: Perform coarse positioning on the clips of the wire harness, such as... Figure 4 As shown, the scan begins at one end of the main curve and continues to the other. The coarse positioning of the buckle is determined by the distance from each point on the main curve to the wire harness contour. Points on the main curve are then scanned upwards along their normal direction to determine if the wire harness contour has been reached, as shown below:

[0086] x=x0±n×cosθ

[0087] y = y0 ± n × sinθ

[0088] Where x0 and y0 are the coordinates of a point on the main curve, θ is the normal vector angle at this point, ± represents different sides, and x and y represent the coordinates of the normal vector that are n distances from (x0, y0). Based on the pixel value at point (x, y) and the threshold image, it can be determined whether the wire harness outline boundary has been reached, thus obtaining the distance value. Coarse segmentation of the wire harness clip is achieved based on the distance value of each point on the main curve, as shown in the following method:

[0089] d1>th1,or

[0090] d2>th1

[0091] Where d1 and d2 are the distances from points on the main curve to different sides of the line bundle contour, and th1 is the threshold.

[0092] S32: Perform fine positioning on the clips on the wire harness. Using the coarse positioning position obtained in S31, take a point on the main curve at a certain distance from the coarse positioning position in the direction away from the clip as the starting point for fine positioning. Figure 5 As shown in the left figure, the red line represents the coarse positioning position, and the blue line represents a position at a certain distance away from the buckle. The fine positioning starting point method is used to find a point on the wire harness outline and use it as the starting point for the fine positioning search, such as... Figure 5 The point is shown in the red box in the right image. Starting from the initial search point, the search proceeds along the harness contour, using the eight-neighbor information of the point towards the latching direction. The method for determining the precise location point is as follows:

[0093]

[0094] Δa=|aa o |

[0095] Δa>th2

[0096] Where a o Let a be the initial normal angle. i Let be the initial angle of the normal vector at point i, 'a' be the angle of the normal vector at the search point, Δa be the difference between the two normal angles, and th2 be the threshold. Figure 5 The point shown in the green box in the right image is the precise point;

[0097] S33: Connect the corresponding precise positioning points of the same buckle, and use the outline of the buckle to divide the buckle;

[0098] S34: The buckle is identified using a feature fusion method, and feature F is shown below:

[0099] F = {F HOG ,F Hu ,F Length}

[0100] Where F HOG For the directional gradient histogram features of the snap-in, F Hu For the Hu-shaped feature of the snap fastener, F Length The length feature of the buckle is along the same direction as the main curve. A random forest is used to train and identify the features of different types of buckles.

[0101] S4: Measurement of wire length between wire harness clips. Based on the precise positioning results of the clips, the curve length between adjacent clips is calculated using a method combining curve fitting and amplification factor correction to obtain the wire length between the clips. The specific steps are as follows:

[0102] S41: The fitting data is obtained by using points on the main curve between adjacent snap fasteners and two extraction endpoints. Points on the main curve between adjacent snap fasteners have been partially removed from those too close to the snap fasteners. The extraction endpoints are obtained from the most accurate positioning points closest to the snap fasteners on the line. The calculation method is as follows:

[0103]

[0104]

[0105] Where (x) e ,y e (x) represents the coordinates of the extracted endpoints. p1 ,y p1 (x) represents the coordinates of the precise location point 1. p2 ,y p2 () represents the coordinates of the precise location point 2;

[0106] S42: Determine the trend of this part of the curve based on the coordinates of adjacent points on the main curve. If the x-coordinates of three adjacent points are the same, the trend of this part of the curve is defined as vertical; if the y-coordinates of three adjacent points are the same, the trend of this part of the curve is defined as horizontal. The points where the trend of the curve changes are defined as segmentation points, such as... Figure 6 As shown;

[0107] S43: Divide the main curve into segments based on the segmentation points. Transform the coordinates of the segmented curves according to their trends. For horizontally trending curves, no coordinate transformation is needed; for vertically trending curves, swap the x and y coordinates. Fit each segmented curve using a 4th-order polynomial curve fitting method. Calculate the length of each curve segment based on the fitting results, using the following method:

[0108]

[0109] Where a and b are the x-coordinates of the starting and ending points of the fitted data, and p1, p2, p3, and p4 are the fourth-order fitting coefficients. The total length l of the line between the clips... total The calculation method is as follows:

[0110]

[0111] Among them l i Let be the length of the i-th curve segment, and k be the total number of segments.

[0112] S44: Obtain the actual line length using a magnification factor correction method. Magnification factor interpolation data is obtained using a dot calibration plate with a 5mm center-to-center spacing between adjacent dots. First, the dot calibration plate is placed in the center of the camera's field of view to acquire the image. The center coordinates of the dot closest to the image center and the center coordinates of its four adjacent dots are extracted. The reference unit length is l. refThe calculation method is as follows:

[0113]

[0114] Among them l c1 , l c2 , l c3 , l c4 The distances are the center of the nearest point to the image center and the centers of its four adjacent points, such as... Figure 7 As shown. The magnification factor M at the center coordinates of other dots. j The calculation method is as follows:

[0115]

[0116] Among them l j The unit length at the center coordinates of other points is calculated using the same method as l. ref The process is the same. Place the dot calibration plate at different positions within the camera's field of view and capture images. Obtain the magnification factor at the center coordinates of the dots on the calibration plate. Repeat this process until the positions where the dot calibration plate was placed cover the entire camera's field of view. Interpolate all the acquired magnification factor data to obtain the magnification factor for each point in the corresponding image, and the actual line length l. actual It can be obtained in the following ways:

[0117]

[0118]

[0119] Where M i is the magnification factor at the coordinates of the i-th point in the curve fitting data, and m is the number of curve fitting data.

[0120] like Figure 2 As shown, a vision-based measurement system for flexible wire harnesses, based on the aforementioned flexible wire harness measurement method, includes an operating table, a camera, a light-diffusing plate, a light source, and a host computer. The operating table is used to fix the camera, light-diffusing plate, light source, and host computer. The light source illuminates the wire harness from bottom to top through the light-diffusing plate, with the wire harness placed on the plate, making the outline of the captured wire harness image clearer. The camera is positioned above the wire harness to capture images, and the image data is transmitted to the host computer via a network cable. The camera uses soft triggering, capturing one image at a time. After image processing, the measurement results of the wire harness are obtained. Based on the measurement results, the pass / fail status of the wire harness is judged, and finally, the pass / fail status of the wire harness is determined.

[0121] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vision-based measurement method for flexible wire harnesses, characterized in that, Includes the following steps: S1: Preprocessing of wire harness image; The light source is placed below the wire harness to illuminate it from the top, and the camera is placed above the wire harness. The wire harness is placed in the middle of the camera's field of view, and the wire harness image is acquired through the camera; The acquired wire harness image is preprocessed to extract the area of ​​the wire harness image. S2: Harness main curve extraction; After obtaining the harness region, the image is thinned. Due to the presence of the clips, there are branches on the extracted harness skeleton. The branch removal method based on the intersection point is used to remove the branches on the thinned image to obtain the main curve of the harness. S3: Segmentation and recognition of wire harness clips; After obtaining the main curve of the wire harness, the position of the clip is coarsely and finely positioned according to the main curve of the wire harness to achieve precise positioning of the clip, and then the clip is segmented and recognized according to the positioning result. S4: Measurement of line length between wire harness clips; Based on the precise positioning results of the clips, the curve length between adjacent clips is calculated using a method combining curve fitting and amplification factor correction, thus obtaining the line length between the clips.

2. The vision-based measurement method for flexible wire harnesses according to claim 1, characterized in that, Step S1 includes the following steps: S11: Filter the wire harness image to remove noise and improve image quality; S12: Threshold the image; use the thresholded image to find the contours of objects in the image, determine the region where the line bundle is located based on the size of the contours, and the contour with the most points among all contours is the contour of the line bundle, as shown below: ； in Indicates the outline of the wire harness. Indicates the first n An outline, n Indicates the number of all contours; S13: Obtain the smallest regular rectangle of the wire harness based on its outline, and extract the area where the wire harness is located by expanding outward by several pixels from each side of the smallest regular rectangle, thereby reducing the scope of subsequent processing.

3. The vision-based measurement method for flexible wire harnesses according to claim 1, characterized in that, Step S2 includes the following steps: S21: Use a thinning algorithm to thin the wire bundles in the extracted wire bundle region image to obtain the skeleton of the wire bundle; S22: Remove redundant points on the wire harness skeleton. That is, after processing, each point on the skeleton, except for the endpoints and intersections, has only two non-zero pixel values ​​in its eight-neighborhood, the endpoints have only one non-zero pixel value in their eight-neighborhood, and the intersections have three or more non-zero pixel values ​​in their eight-neighborhood. S23: Find all intersection points based on the number of non-zero pixel values ​​in the eight-neighborhood. The determination method is as follows: ； in Point Points in the eight neighborhood Pixel value at that location, Indicates the number; S24: Starting from each found intersection point, the point whose pixel value in its eight neighborhood is not zero is the starting point of a path. The center point is updated according to the eight neighborhood information of the point to realize scanning along different paths. The following conditions are used to determine whether to stop scanning: (1) If the scanned point is an endpoint and the number of scanned points is less than the threshold, the pixel value of all scanned points on this path is set to zero; (2) If the scanned point is another intersection point, the scanning of this path is stopped; (3) If the number of scanned points on a path exceeds the threshold, the scanning of this path is stopped; (4) After setting the points on a path to zero or after completing the scanning of all paths, the above operation is started on other intersection points. S25: Use an iterative method to remove branches from the wire harness skeleton. If the number of intersections remains the same in the previous two iterations, it means that all branches have been removed and the main curve of the wire harness has been obtained. Save the points on the main curve sequentially from one end to the other end of the main curve.

4. The vision-based measurement method for flexible wire harnesses according to claim 1, characterized in that, Step S3 includes the following steps: S31: Perform coarse positioning of the clips on the wire harness, starting from one end of the main curve and scanning to the other end; the coarse positioning of the clips is determined by the distance from each point on the main curve to the wire harness outline; the distance from each point on the main curve to the normal direction at that point is gradually increased to determine whether the wire harness outline has been reached, as shown below: ； in The coordinates of a point on the main curve. θ The angle of the normal vector at this point, ± represents different sides. x, y Representation upward and Distance is n The coordinates of the point; based on the point ( x , y The pixel value at point () and the threshold image are used to determine whether the wire harness outline boundary has been reached, thus obtaining the distance value. Based on the distance value of each point on the main curve, coarse segmentation of the wire harness clip is achieved. The determination method is as follows: ； in, The distance from a point on the main curve to different sides of the line bundle profile. For the threshold; S32: Perform fine positioning on the buckles of the wire harness. Using the coarse positioning position obtained in step S31, acquire a point on the main curve at a certain distance from the coarse positioning position in the direction away from the buckle as the starting point for fine positioning. Search for points on the wire harness contour in the normal direction from the starting point for fine positioning and use these points as the starting search points for fine positioning. Starting from the starting search point, search along the wire harness contour in the direction of the buckle based on the eight-neighborhood information of the point. The method for determining the precise positioning point is as follows: ； ； ； in The initial normal angle, For the initial first i The angle of the normal at each point α Let the angle be the normal at the search point. The difference between the two normal angles. For the threshold; S33: Connect the corresponding precise positioning points of the same buckle, and use the outline of the buckle to divide the buckle; S34: The buckle is identified using a feature fusion method, featuring... F As shown below: ； in The directional gradient histogram features of the snap-fit. The Hu-shaped feature of the buckle, The length feature of the buckle is in the direction consistent with the main curve; the features of different types of buckles are trained and identified by random forest.

5. The vision-based measurement method for flexible wire harnesses according to claim 1, characterized in that, Step S4 includes the following steps: S41: The points on the main curve between adjacent snap fasteners and the two extraction endpoints are used as fitting data; some points close to the snap fasteners are removed from the main curve between adjacent snap fasteners, and the extraction endpoints are obtained from the precise positioning points that are close to the snap fasteners on the line. The calculation method is as follows: ； ； in To extract the coordinates of the endpoints, To pinpoint the coordinates of point 1, The coordinates of point 2 are to be precisely located; S42: Determine the trend of this part based on the coordinates of adjacent points on the main curve. If the x-coordinates of three adjacent points are the same, the trend of this part of the curve is defined as vertical. If the y-coordinates of three adjacent points are the same, the trend of this part of the curve is defined as horizontal. The point where the trend of the curve changes is defined as the segmentation point. S43: Divide the main curve into segments based on the segmentation points. Transform the coordinates of the segmented curves according to their trends. For horizontal trends, no coordinate transformation is needed; for vertical trends, the x and y coordinates are swapped. Fit the segmented curves using a 4th-order polynomial curve fitting method. Calculate the length of each curve segment based on the fitting results, using the following method: ； in To fit the x-coordinate values ​​of the starting and ending points of the data, The coefficients are fourth-order fitting coefficients; the total length of the wires between the clips. The calculation method is as follows: ； in For the first i The length of the curve segment k This represents the total number of segments; S44: Obtain the actual length of the line using a magnification factor correction method; obtain magnification factor interpolation data using a dot calibration plate with a center-to-center distance of 5mm between adjacent dots. First, place the dot calibration plate in the center of the camera's field of view to acquire the image, extract the center coordinates of the dot closest to the image center and the center coordinates of its four adjacent dots, and use the reference unit length. The calculation method is as follows: ； in This represents the distance between the center of the nearest dot to the image center and the centers of its four adjacent dots; the magnification factor at the coordinates of the other dot centers. The calculation method is as follows: ； in The unit length at the center coordinates of other points, its calculation method and The process is the same; place the dot calibration plate at different positions within the camera's field of view and capture images, obtaining the magnification factor at the center coordinates of the dots on the calibration plate. Repeat this process until the positions where the dot calibration plate was placed cover the entire camera's field of view; interpolate all the acquired magnification factor data to obtain the magnification factor for each point in the corresponding image and the actual length of the line. Obtained in the following way: ； ； in The first in the curve fitting data i The magnification factor at each point coordinate. m This represents the number of curve fitting data points.

6. A vision-based measurement system for flexible wire harnesses, implementing the vision-based measurement method for flexible wire harnesses as described in claim 1, characterized in that, The vision-based flexible wire harness measurement system includes an operating table, a camera, a light-diffusing plate, a light source, and a host computer. The operating table is used to fix the camera, light-diffusing plate, light source, and host computer. The light source illuminates from bottom to top through the light-diffusing plate, and the wire harness is placed on the light-diffusing plate, making the outline of the captured wire harness image clearer. The camera is placed above the wire harness to capture the wire harness image, and the image data is transmitted to the host computer via a network cable. The camera is soft-triggered, capturing one image at a time. After image processing, the measurement result of the wire harness is obtained. Based on the measurement result, the qualification of the wire harness is judged, and finally, the qualification status of the wire harness is determined.

Images