A method and system for detecting appearance and structure of automobile wiring harness

Through image processing and deep learning technology, the structure and defects of automotive wiring harness are automatically detected, solving the problem of inefficient detection caused by the complexity and flexibility of wiring harness, and achieving fast and accurate wiring harness detection.

CN118823019BActive Publication Date: 2024-11-26ZHUHAI QINCHUANG ELECTRONIC TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411299144.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-18
Publication Date
2024-11-26
Estimated Expiration
2044-09-18

AI Technical Summary

Technical Problem

The complex structure and flexible characteristics of automotive wiring harnesses lead to inefficient manual detection, prone to missed or missed inspections, and mechanical detection may damage the surface of the wiring harness, lacking contactless rapid detection methods.

Method used

By obtaining the multi-pose image of the wire harness, extracting the skeleton and segmenting it into line segments and branch nodes, combining line segments with the same color and similar width, calculating the length and width of the wire harness, matching the standard structure diagram, automatically detecting the wire harness structure and defects, and using deep learning models for Terminal and accessories detection.

Benefits of technology

The 360-degree detection of the wiring harness is realized, which avoids detection errors and omissions, and automates the detection of the surface defect of the wiring harness, improves the detection efficiency and reduces manual detection time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118823019B_ABST
    Figure CN118823019B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for detecting the appearance and structure of an automobile wiring harness, and belongs to the field of image processing. The method includes: obtaining wiring harness images of multiple postures to extract skeletons and segment to obtain segmented line segments and branch nodes; calculating the length and width of the segmented line segments, and merging the segmented line segments according to the width and color; comparing the structure and size of the obtained wiring harness structure diagram with the wiring harness standard structure diagram to obtain the structure and size detection results; segmenting the wiring harness image according to the preset detection point position and inputting the detection model to detect the terminal or accessories; performing wiring harness outer surface defect detection based on the wiring harness image to obtain the detection structure. The present invention can realize the rapid detection of the appearance and structure of automobile wiring harnesses.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for detecting the appearance and structure of an automobile wiring harness. Background Art

[0002] The wiring harness is the main network of the automobile circuit. At present, whether it is a high-end luxury car or an economical ordinary bike, the wiring harness of the car is basically the same, consisting of wires, connectors and wrapping tape. When inspecting the appearance and structure of the automobile wiring harness, due to the complexity of the automobile circuit, many branches, and a large number of connectors and accessories, manual inspection efficiency is low, which is easy to cause missed inspections or wrong inspections. In addition, due to the certain flexibility of the wiring harness, some morphological features are not easy to quantify accurately. When inspecting the length and width of the wiring harness, if the wiring harness is straightened by a mechanical device, it may cause damage to the surface of the wiring harness.

[0003] Therefore, a method is needed to quickly detect the appearance and structure of automotive wiring harnesses without contact. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a method and system for detecting the appearance and structure of an automobile wiring harness, which can realize rapid detection of the appearance and structure of an automobile wiring harness.

[0005] On the one hand, an embodiment of the present invention provides a method for detecting the appearance and structure of an automobile wiring harness, comprising the following steps: S100, obtaining wiring harness images of multiple postures of a wiring harness to be tested, extracting the skeleton of the wiring harness images to obtain the skeleton of each wiring harness image, and segmenting the skeleton to obtain segmentation line segments and branch nodes; S200, calculating the wiring harness width of each wiring harness image according to the segmentation line segments to obtain the width of each segmentation line segment, and merging all segmentation line segments with the same color, a width difference not exceeding a threshold and connected by branch nodes to obtain the final segmentation line segments and branch nodes of each wiring harness image; calculating the length of the final segmentation line segment of the wiring harness to be tested, and determining the wiring harness structure diagram to be tested according to the size of the final segmentation line segment and the branch nodes; S300, obtaining the wiring harness standard structure diagram, and determining in turn whether each wiring harness structure diagram to be tested is simply isomorphic with the wiring harness standard structure diagram, and if so, marking the current wiring harness structure diagram to be tested as correct; for all wiring harness structure diagrams to be tested that are marked as correct, segmentation line segments are separated based on length, width and branch nodes. Segments are matched, and the maximum value of all length values ​​corresponding to the segmented line segments is obtained as the length detection value of the segmented line segment, and the average value of all width values ​​corresponding to the segmented line segments is obtained as the width detection value of the segmented line segment; S400, each node in the standard structure diagram of the wire harness, the segmented line segments corresponding to each node, and the length standard value and width standard value corresponding to each segmented line segment are obtained, and the branch nodes, the segmented line segments corresponding to each node, and the length detection value and width detection value corresponding to each segmented line segment are matched and compared with any branch node in the wire harness structure diagram to be tested that is marked as correct, so as to obtain the structure and size detection results of the wire harness to be tested; S500, each detection point associated in the standard structure diagram of the wire harness is obtained, the detection point position in the wire harness image is determined according to the position corresponding to each detection, and the image of the detection point position is segmented based on the detection frame of a predetermined size to obtain a segmented image; the segmented image is input into the terminal detection model or the accessory detection model to obtain the detection result; S600, the appearance defect detection of the wire harness is performed based on the wire harness image to obtain the appearance defect detection result.

[0006] According to some embodiments of the present invention, step S200 includes: S210, downsampling the point set of the skeleton of the wire harness image to obtain the center point set of the wire harness, and obtaining the wire harness direction through two adjacent points, and obtaining two wire harness edge points in the wire harness image through a direction perpendicular to the wire harness direction, and calculating the distance between the two wire harness edge points to obtain the width of the segmentation line segment; S220, traversing all branch nodes, judging whether the width difference of any two segmentation line segments connected by the branch node is less than a threshold, if less than the threshold, obtaining whether the colors of the two segmentation line segments are the same, if the same, merging the two segmentation line segments into one segmentation line segment, and judging whether the current branch node is still connected to the segmentation line segment, if not, deleting the current branch node; S230, calculating the length of the final segmentation line segment of the wire harness to be tested, and obtaining the length of the segmentation line segment by counting the pixels of the partial single pixel skeleton corresponding to the segmentation line segment; determining the structure diagram of the wire harness to be tested according to the size of the final segmentation line segment and the branch node, and the weight of each edge of the wire harness structure diagram to be tested includes the length and width corresponding to the edge.

[0007] According to some embodiments of the present invention, the matching of segmentation line segments based on length, width and branch nodes for all wiring harness structure diagrams to be tested that are marked as correct includes: for any two wiring harness structure diagrams to be tested, traversing the branch nodes of one of the wiring harness structure diagrams to be tested, and matching the branch nodes of the other wiring harness structure diagram to be tested according to the number, length and width of the segmentation line segments connected by the branch nodes; when the number of segmentation line segments of the first branch node is equal to that of the second branch node, and for any segmentation line segment of the first branch node, there is a segmentation line segment of the second branch node whose length and width differ from the length and width of any segmentation line segment by no more than a set value, then the first branch node and the second branch node are matched.

[0008] According to some embodiments of the present invention, the step S400 includes: S410, arbitrarily selecting one of the wiring harness structure diagrams to be tested that is marked as correct as the first wiring harness structure diagram; traversing all nodes in the wiring harness standard structure diagram, and obtaining the corresponding segmentation line segment and the length standard value and width standard value corresponding to the segmentation line segment for the current node; S420, traversing the branch nodes of the first wiring harness structure diagram, and determining whether there is a branch node that matches the current node. When the number of segmentation line segments of the branch node and the current node is equal, and for any segmentation line segment of the current node, the difference between the length detection value and the width detection value of the segmentation line segment of the branch node and the length standard value and the width standard value of any segmentation line segment does not exceed the set value, then the branch node matches the current node; S430, if all nodes are matched successfully, the detection result of the wiring harness structure and size to be tested is qualified; if there is a node that fails to match, the segmentation line segment that fails the detection is output.

[0009] According to some embodiments of the present invention, the step S500 includes: S510, obtaining each detection point associated in the wire harness standard structure diagram, and determining the segmentation line segments or branch nodes associated with each detection point in the wire harness structure diagram to be tested that has completed the matching according to the segmentation line segments or branch nodes associated with each detection point; S520, obtaining the wire harness image corresponding to the wire harness structure diagram to be tested, determining the detection position of the wire harness image according to the segmentation line segments or branch nodes associated with each detection point, and performing image segmentation on the detection position of the wire harness image through a detection frame of a predetermined size to obtain a segmented image; when the detection point is associated with a segmentation line segment, obtaining the terminal position of the segmentation line segment as the detection position.

[0010] According to some embodiments of the present invention, step S500 includes: S530, obtaining component information of the current detection point, determining to input the segmented image into a terminal detection model or an accessory detection model, and comparing the obtained terminal identification information or accessory identification information with the component information of the current detection point. If they are the same, the detection is qualified.

[0011] According to some embodiments of the present invention, step S500 includes: acquiring image data of multiple types of terminal blocks collected from multiple angles to train a deep learning model to obtain a terminal detection model.

[0012] According to some embodiments of the present invention, step S500 includes: acquiring image data of multiple types of accessories collected from multiple angles to obtain an accessory image dataset; dividing the image dataset into a training set with a large number of samples and a support set with a small number of samples; training the YOLOV8 model based on the training set and the support set to obtain an accessory detection model.

[0013] According to some embodiments of the present invention, step S600 includes: S610, obtaining wire harness images of multiple postures of the wire harness to be tested, processing the pixels in the image of each detection point according to a detection frame of a predetermined size according to a fixed value, converting the grayscale value of the processed wire harness image, and performing contrast enhancement processing to obtain a preprocessed image; S620, denoising the preprocessed image based on a median filtering algorithm to obtain a denoised image; S630, performing threshold segmentation on the denoised image based on an adaptive threshold segmentation algorithm to obtain a segmented image; S640, filling holes in the segmented image, marking the target, calculating the number of target pixels, and outputting the defect position and size.

[0014] The embodiment of the present invention achieves at least the following beneficial effects: the method of the embodiment of the present invention detects by acquiring the wire harness images of multiple postures, and can detect the surface of the wire harness at 360 degrees around the wire harness in multiple postures, avoiding missing the surface defect points of the wire harness; by merging two segmented line segments of the same color, similar width and connected, the structural detection error caused by the crossing of the branch line of the wire harness is avoided; by taking the maximum value of the branch wire harness length obtained by detecting the wire harness images of several different postures as the final branch wire harness detection length, it is avoided that in a certain posture, due to the multi-layer wire harness, the branch wire harness is in the upper layer and has a height difference with the detection table, resulting in the segmented line segment corresponding to the obtained wire harness image being short, affecting the final detection result; through several detection points and detection information associated with the standard wire harness structure diagram, the terminals and accessories of the wire harness are automatically detected, avoiding manual time-consuming detection, and avoiding the occurrence of accessories missing or terminal falling off. By performing surface defect detection on the wire harness images of multiple postures, the surface defect detection of the wire harness is automatically realized.

[0015] On the other hand, an embodiment of the present invention provides an automobile wiring harness appearance and structure detection system, comprising: a first module, for obtaining wiring harness images of multiple postures of a wiring harness to be tested, performing skeleton extraction on the wiring harness images to obtain the skeleton of each wiring harness image, and segmenting the skeleton to obtain segmented line segments and branch nodes; a second module, for calculating the wiring harness width of each wiring harness image according to the segmented line segments, obtaining the width of each segmented line segment, and merging all segmented line segments with the same color, a width difference not exceeding a threshold and connected by branch nodes to obtain the final segmented line segments and branch nodes of each wiring harness image; calculating the length of the final segmented line segments of the wiring harness to be tested, and determining the wiring harness structure diagram to be tested according to the size of the final segmented line segments and the branch nodes; a third module, for obtaining a wiring harness standard structure diagram, and determining in turn whether each wiring harness structure diagram to be tested is simply isomorphic with the wiring harness standard structure diagram, and if so, marking the current wiring harness structure diagram to be tested as correct; for all wiring harness structure diagrams to be tested marked as correct, segmenting the segmented line segments based on the length, width and branch nodes Matching is performed, and the maximum value of all length values ​​corresponding to the segmented line segment is obtained as the length detection value of the segmented line segment, and the average value of all width values ​​corresponding to the segmented line segment is obtained as the width detection value of the segmented line segment; the fourth module is used to obtain each node in the wire harness standard structure diagram, the segmented line segment corresponding to each node, and the length standard value and width standard value corresponding to each segmented line segment, and match and compare with each branch node, the segmented line segment corresponding to each node, and the length detection value and width detection value corresponding to each segmented line segment in any wire harness structure diagram marked as correct to obtain the structure and size detection results of the wire harness to be tested; the fifth module is used to obtain each detection point associated in the wire harness standard structure diagram, determine the detection point position in the wire harness image according to the corresponding position of each detection, and segment the image of the detection point position based on the detection frame of a predetermined size to obtain a segmented image; the segmented image is input into the terminal detection model or the accessory detection model to obtain the detection result; the sixth module is used to perform wire harness appearance defect detection based on the wire harness image to obtain the appearance defect detection result.

[0016] The embodiment of the present invention achieves at least the following beneficial effects: the method of the embodiment of the present invention detects by acquiring the wire harness images of multiple postures, and can detect the surface of the wire harness at 360 degrees around the wire harness in multiple postures, avoiding missing the surface defect points of the wire harness; by merging two segmented line segments of the same color, similar width and connected, the structural detection error caused by the crossing of the branch line of the wire harness is avoided; by taking the maximum value of the branch wire harness length obtained by detecting the wire harness images of several different postures as the final branch wire harness detection length, it is avoided that in a certain posture, due to the multi-layer wire harness, the branch wire harness is in the upper layer and has a height difference with the detection table, resulting in the segmented line segment corresponding to the obtained wire harness image being short, affecting the final detection result; through several detection points and detection information associated with the standard wire harness structure diagram, the terminals and accessories of the wire harness are automatically detected, avoiding manual time-consuming detection, and avoiding the occurrence of accessories missing or terminal falling off. By performing surface defect detection on the wire harness images of multiple postures, the surface defect detection of the wire harness is automatically realized.

[0017] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0019] Figure 1 A schematic diagram of a method according to an embodiment of the present invention;

[0020] Figure 2 The module schematic block diagram of the system according to the embodiment of the present invention.

[0021] Reference numerals:

[0022] The first module 100 , the second module 200 , the third module 300 , the fourth module 400 , the fifth module 500 , and the sixth module 600 . DETAILED DESCRIPTION

[0023] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0024] In the description of the present invention, "several" means one or more, "more" means two or more, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0025] The wiring harness is the main network of the automobile circuit. At present, whether it is a high-end luxury car or an economical ordinary bike, the form of the automobile wiring harness is basically the same, and it is composed of wires, connectors and wrapping tape. If the automobile wiring harness is to be arranged in the car, it is necessary to draw the automobile wiring harness installation diagram. Since the wiring harness installation diagram mainly appears in the form of the wiring harness, there are fewer lines on the surrounding surface, and the expression of the connection between the components becomes its main content. In order to clearly express the color of the wire and the terminal code of the connector, it is often necessary to supplement it with the wiring harness grouping and terminal number table and the wire terminal wiring table. First of all, there are main wiring harnesses and branch wiring harnesses on the automobile wiring harness diagram. The diagram should show the composition of each wiring harness, how many branches there are on each wiring harness, how many wires there are on each branch, and what the color and stripes of the wires are. Secondly, the number of electrical appliances on the car is large and complex. In order to make the connection correct, each wiring harness connection point should be marked with the wiring code and wiring mark for easy connection. Then, since there are multiple wire harnesses, and the wire harnesses, branches and wire harnesses, or branches and electrical appliances are connected through connectors, it should be indicated how many wires are on each connector, where each wire is located in the connector wiring hole, what the shape of the connector is, and whether adjacent connectors are easy to confuse. In addition, it should be noted that the length of the wire harness includes the total length of the wire harness, the length of each branch, and the length of the interval between the two wire ends. I believe that according to the above points, the content of the wire harness in the automotive wiring harness installation diagram will basically not be deviated.

[0026] The automotive wiring harness appearance and structure detection method of the embodiment of the present invention obtains the standard structure diagram of the wiring harness based on the simplified processing of the automotive wiring harness installation diagram, which includes the wiring harness node information, line segment information and component information. The line segment information includes the standard length value, the standard width value and the color. The embodiment of the present invention obtains the standard length value and the standard width value of the wiring harness based on the qualified wiring harness and the same camera conditions as the wiring harness to be tested. It should be noted that the standard length value and the standard width value are not necessarily the actual length and width values ​​of the wiring harness, but the length value and the width value of the wiring harness image calculated under the detection conditions.

[0027] Reference Figure 1 The embodiment of the present invention provides a method for detecting the appearance and structure of an automobile wiring harness, comprising the following steps:

[0028] S100, obtaining wire harness images of multiple postures of the wire harness to be tested, extracting the skeleton of the wire harness images to obtain the skeleton of each wire harness image, and segmenting the skeleton to obtain segmented line segments and branch nodes. In this embodiment, the multiple postures can be wire harness images of multiple postures obtained by manually or mechanically winding the main wire harness of the wire harness around, so as to ensure that the surface of the wire harness is fully inspected for defects around one circle to avoid missed inspections.

[0029] S200, calculate the width of each wire harness image according to the segmented line segments, obtain the width of each segmented line segment, and merge all segmented line segments with the same color, width difference not exceeding the threshold and connected by branch nodes to obtain the final segmented line segments and branch nodes of each wire harness image; calculate the length of the final segmented line segments of the wire harness to be tested, and determine the structure diagram of the wire harness to be tested according to the size of the final segmented line segments and branch nodes. In this embodiment, in the case where the wire harness branches may cross in the current posture and cause structural detection errors, the crossed lines are distinguished by merging the segmented wire harnesses.

[0030] S300, obtain the standard structure diagram of the wiring harness, and determine in turn whether each wiring harness structure diagram to be tested is simply isomorphic with the standard structure diagram of the wiring harness. If they are isomorphic, mark the current wiring harness structure diagram to be tested as correct; for all wiring harness structure diagrams to be tested that are marked as correct, match the segmented line segments based on length, width and branch nodes, and obtain the maximum value of all length values ​​corresponding to the segmented line segments as the length detection value of the segmented line segments, and obtain the average value of all width values ​​corresponding to the segmented line segments as the width detection value of the segmented line segments. In this embodiment, since the wiring harness has multiple layers, when the branch wiring harness is in the upper layer, there is a height difference with the detection table, which may cause the segmented line segments corresponding to the obtained wiring harness image to be shorter and affect the final detection result. Therefore, this embodiment matches the segmented line segments of the wiring harness structure diagram obtained from several wiring harness images, and takes the longest value of the lengths of several segmented line segments that have completed the matching as the length detection value.

[0031] S400, obtaining each node in the standard structure diagram of the wiring harness, the segmentation line segments corresponding to each node, and the standard length value and width value corresponding to each segmentation line segment, and matching and comparing them with each branch node, the segmentation line segments corresponding to each node, and the length detection value and width detection value corresponding to each segmentation line segment in any wiring harness structure diagram to be tested marked as correct, to obtain the structure and size detection results of the wiring harness to be tested.

[0032] S500, obtaining each detection point associated with the wire harness standard structure diagram, determining the detection point position in the wire harness image according to the position corresponding to each detection, and segmenting the image of the detection point position based on a detection frame of a predetermined size to obtain a segmented image; inputting the segmented image into a terminal detection model or an accessory detection model to obtain a detection result.

[0033] S600: Perform wire harness appearance defect detection based on the wire harness image to obtain an appearance defect detection result.

[0034] In some embodiments, step S200 includes:

[0035] S210, downsampling the skeleton point set of the wire beam image to obtain the wire beam center point set, and obtaining the wire beam direction through two adjacent points, and obtaining two wire beam edge points in the wire beam image through a direction perpendicular to the wire beam direction, and calculating the distance between the two wire beam edge points to obtain the width of the segmented line segment.

[0036] S220, traverse all branch nodes, determine whether the width difference between any two segmentation line segments connected by the branch nodes is less than a threshold value, if it is less than the threshold value, obtain whether the colors of the two segmentation line segments are the same, if they are the same, merge the two segmentation line segments into one segmentation line segment, and determine whether the current branch node is still connected to a segmentation line segment, if not, delete the current branch node.

[0037] S230, calculating the length of the final segmentation line segment of the wiring harness to be tested, and obtaining the length of the segmentation line segment by counting the pixels of the partial single pixel skeleton corresponding to the segmentation line segment; determining the wiring harness structure diagram to be tested according to the size of the final segmentation line segment and the branch node, and the weight of each edge of the wiring harness structure diagram to be tested includes the length and width corresponding to the edge.

[0038] In some embodiments, for all the wiring harness structure diagrams to be tested marked as correct, matching the segmented line segments based on the length, width and branch nodes includes:

[0039] For any two wiring harness structure diagrams to be tested, the branch nodes of one of the wiring harness structure diagrams to be tested are traversed, and the branch nodes of the other wiring harness structure diagram to be tested are matched according to the number, length and width of the segmented line segments connected by the branch nodes.

[0040] When the number of segmentation line segments of the first branch node is equal to that of the second branch node, and for any segmentation line segment of the first branch node, the difference between the length and width of the segmentation line segment of the second branch node and the length and width of any segmentation line segment does not exceed a set value, then the first branch node and the second branch node match.

[0041] In some embodiments, step S400 includes:

[0042] S410, arbitrarily select one of the wiring harness structure diagrams marked as correct as the first wiring harness structure diagram; traverse all nodes in the wiring harness standard structure diagram, and obtain the corresponding segmentation line segment and the length standard value and width standard value corresponding to the segmentation line segment for the current node.

[0043] S420, traverse the branch nodes of the first wiring harness structure diagram to determine whether there is a branch node that matches the current node. When the number of segmentation line segments of the branch node and the current node is equal, and for any segmentation line segment of the current node, there is a segmentation line segment of the branch node whose length detection value and width detection value have a difference with the length standard value and width standard value of any segmentation line segment that does not exceed the set value, then the branch node matches the current node.

[0044] S430, if all nodes are matched successfully, the detection result of the structure and size of the wiring harness to be tested is qualified; if there is a node that fails to match, the segmented line segment that fails the detection is output.

[0045] In some embodiments, step S500 includes:

[0046] S510, acquiring each detection point associated with the wiring harness standard structure diagram, and determining the segmentation line segments or branch nodes associated with each detection point in the wiring harness structure diagram to be tested that have completed matching according to the segmentation line segments or nodes associated with each detection point.

[0047] S520, obtaining a wire harness image corresponding to the wire harness structure diagram to be tested, determining the detection position of the wire harness image according to the segmentation line segments or branch nodes associated with each detection point, and performing image segmentation on the detection position of the wire harness image through a detection frame of a predetermined size to obtain a segmented image; when the detection point is associated with a segmentation line segment, obtaining the terminal position of the segmentation line segment as the detection position.

[0048] In some embodiments, step S500 includes:

[0049] S530, obtaining component information of the current detection point, determining to input the segmented image into a terminal detection model or an accessory detection model, comparing the obtained terminal identification information or accessory identification information with the component information of the current detection point, and if they are the same, the detection is qualified.

[0050] In some embodiments, step S500 includes: acquiring image data of multiple types of terminal blocks collected from multiple angles to train a deep learning model to obtain a terminal detection model.

[0051] In some embodiments, step S500 includes: acquiring image data of multiple types of accessories collected from multiple angles to obtain an accessory image dataset; dividing the image dataset into a training set with a large number of samples and a support set with a small number of samples; training the YOLOV8 model based on the training set and the support set to obtain an accessory detection model.

[0052] In some embodiments, step S600 includes:

[0053] S610, obtaining wire harness images of multiple postures of the wire harness to be tested, processing pixels within the image of each detection point according to a detection frame of a predetermined size according to a fixed value, converting the processed wire harness image into grayscale value, and performing contrast enhancement processing to obtain a pre-processed image.

[0054] S620 , performing denoising on the preprocessed image based on a median filtering algorithm to obtain a denoised image.

[0055] S630 , performing threshold segmentation on the denoised image based on an adaptive threshold segmentation algorithm to obtain a segmented image.

[0056] S640, filling holes in the segmented image, marking the target, calculating the number of target pixels, and outputting the defect position and size.

[0057] Corresponding to the above-mentioned embodiments, the present invention also provides system embodiments. As for the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments.

[0058] Another aspect of the present invention provides a vehicle wiring harness appearance and structure detection system, including:

[0059] The first module 100 is used to obtain wire harness images of multiple postures of the wire harness to be tested, perform skeleton extraction on the wire harness images to obtain the skeleton of each wire harness image, and segment the skeleton to obtain segmented line segments and branch nodes.

[0060] The second module 200 is used to calculate the width of each wire harness image according to the segmentation line segments, obtain the width of each segmentation line segment, and merge all segmentation line segments with the same color, width difference not exceeding the threshold and connected by branch nodes to obtain the final segmentation line segments and branch nodes of each wire harness image; calculate the length of the final segmentation line segment of the wire harness to be tested, and determine the structure diagram of the wire harness to be tested according to the size of the final segmentation line segment and the branch nodes.

[0061] The third module 300 is used to obtain the standard structure diagram of the wiring harness, and determine in turn whether each wiring harness structure diagram to be tested is simply isomorphic with the standard structure diagram of the wiring harness. If so, the current wiring harness structure diagram to be tested is marked as correct; for all wiring harness structure diagrams to be tested that are marked as correct, the segmentation segments are matched based on the length, width and branch nodes, and the maximum value of all length values ​​corresponding to the segmentation segment is obtained as the length detection value of the segmentation segment, and the average value of all width values ​​corresponding to the segmentation segment is obtained as the width detection value of the segmentation segment.

[0062] The fourth module 400 is used to obtain each node in the standard structure diagram of the wiring harness, the segmented line segments corresponding to each node, and the standard length value and width value corresponding to each segmented line segment, and match and compare them with each branch node, the segmented line segments corresponding to each node, and the length detection value and width detection value corresponding to each segmented line segment in any wiring harness structure diagram to be tested marked as correct, to obtain the structure and size detection results of the wiring harness to be tested.

[0063] The fifth module 500 is used to obtain the associated detection points in the wire harness standard structure diagram, determine the detection point positions in the wire harness image according to the positions corresponding to each detection, and segment the image of the detection point positions based on a detection frame of a predetermined size to obtain a segmented image; input the segmented image into a terminal detection model or an accessory detection model to obtain a detection result.

[0064] The sixth module 600 is used to perform wire harness appearance defect detection based on the wire harness image to obtain an appearance defect detection result.

[0065] Although specific embodiments are described herein, those of ordinary skill in the art will recognize that many other modifications or alternative embodiments are also within the scope of the present disclosure. For example, any of the functions and / or processing capabilities described in conjunction with a particular device or component may be performed by any other device or component. In addition, although various exemplary implementations and architectures have been described according to embodiments of the present disclosure, those of ordinary skill in the art will recognize that many other modifications to the exemplary implementations and architectures described herein are also within the scope of the present disclosure.

[0066] Some aspects of the present disclosure are described above with reference to the block diagram and flow chart of the system, method, system and / or computer program product according to the exemplary embodiments.It should be understood that the combination of one or more blocks in the block diagram and the flow chart and the blocks in the block diagram and the flow chart can be realized by executing computer executable program instructions respectively.Equally, according to some embodiments, some blocks in the block diagram and the flow chart may not need to be executed in the order shown, or may not need to be executed in full.In addition, additional components and / or operations beyond those components and / or operations shown in the blocks in the block diagram and the flow chart may be present in certain embodiments.

[0067] Therefore, the blocks in the block diagrams and flow charts support the combination of devices for performing the specified functions, the combination of elements or steps for performing the specified functions, and the program instruction devices for performing the specified functions. It should also be understood that each block in the block diagrams and flow charts and the combination of blocks in the block diagrams and flow charts can be implemented by a special-purpose hardware computer system or a combination of special-purpose hardware and computer instructions that performs a specific function, element or step.

[0068] The program modules, applications, etc. described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the illustrative methods described herein) to be performed.

[0069] Software components can be encoded with any of various programming languages. An exemplary programming language can be a low-level programming language, such as an assembly language associated with a specific hardware architecture and / or an operating system platform. The software component comprising assembly language instructions may need to be converted to executable machine code by an assembler before being executed by hardware architecture and / or platform. Another exemplary programming language can be a higher-level programming language, which can be transplanted across multiple architectures. The software component comprising a higher-level programming language may need to be converted to an intermediate representation by an interpreter or a compiler before execution. Other examples of programming languages ​​include but are not limited to macro languages, shells or command languages, job control languages, scripting languages, database queries or search languages ​​or report writing languages. In one or more exemplary embodiments, the software component comprising the instruction of one of the above-mentioned programming language examples can be directly executed by an operating system or other software components, without first being converted into another form.

[0070] Software components may be stored as files or other data storage structures. Software components of similar type or related functions may be stored together, such as in a specific directory, folder, or library. Software components may be static (e.g., preset or fixed) or dynamic (e.g., created or modified at execution time).

[0071] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present invention.

Claims

1. A method for detecting the appearance and structure of automobile wiring harnesses. It is characterized in that The following steps are involved: S100, obtaining wire harness images of multiple postures of the wire harness to be tested, performing skeleton extraction on the wire harness images to obtain a skeleton of each wire harness image, and segmenting the skeleton to obtain segmentation line segments and branch nodes; S200, calculating the width of each wire harness image according to the segmented line segments to obtain the width of each segmented line segment, and merging all segmented line segments with the same color and a width difference not exceeding a threshold and connected by branch nodes to obtain the final segmented line segments and branch nodes of each wire harness image; calculating the length of the final segmented line segments of the wire harness to be tested, and determining the structure diagram of the wire harness to be tested according to the size of the final segmented line segments and the branch nodes; S300, obtaining a standard wiring harness structure diagram, determining in turn whether each wiring harness structure diagram to be tested is simply isomorphic with the standard wiring harness structure diagram, and if so, marking the current wiring harness structure diagram to be tested as correct; for all wiring harness structure diagrams to be tested that are marked as correct, matching the segmentation line segments based on length, width, and branch nodes, and obtaining the maximum value of all length values ​​corresponding to the segmentation line segments as the length detection value of the segmentation line segments, and obtaining the average value of all width values ​​corresponding to the segmentation line segments as the width detection value of the segmentation line segments; S400, obtaining each node in the standard structure diagram of the wiring harness, the segmented line segments corresponding to each node, and the length standard value and width standard value corresponding to each segmented line segment, and matching and comparing them with each branch node, the segmented line segments corresponding to each node, and the length detection value and width detection value corresponding to each segmented line segment in any wiring harness structure diagram marked as correct to obtain the wiring harness structure and size detection results to be tested; S500, acquiring each detection point associated in the wire harness standard structure diagram, determining the detection point position in the wire harness image according to the position corresponding to each detection, and segmenting the image of the detection point position based on a detection frame of a predetermined size to obtain a segmented image; inputting the segmented image into a terminal detection model or an accessory detection model to obtain a detection result; S600: Perform wire harness appearance defect detection based on the wire harness image to obtain an appearance defect detection result.

2. The method for detecting the appearance and structure of a vehicle wiring harness according to claim 1, It is characterized in that The step S200 includes: S210, performing downsampling processing on the point set of the skeleton of the wire beam image to obtain a wire beam center point set, and obtaining a wire beam direction through two adjacent points, and obtaining two wire beam edge points in the wire beam image through a direction perpendicular to the wire beam direction, and calculating the distance between the two wire beam edge points to obtain the width of the segmentation line segment; S220, traversing all branch nodes, determining whether the width difference between any two segmentation line segments connected by the branch nodes is less than a threshold value, if so, obtaining whether the colors of the two segmentation line segments are the same, if they are the same, merging the two segmentation line segments into one segmentation line segment, and determining whether the current branch node is still connected to a segmentation line segment, if not, deleting the current branch node; S230, calculating the length of the final segmented line segment of the wiring harness to be tested, and obtaining the length of the segmented line segment by counting the pixels of the partial single pixel skeleton corresponding to the segmented line segment; determining the wiring harness structure diagram to be tested according to the size of the final segmented line segment and the branch node, wherein the weight of each edge of the wiring harness structure diagram to be tested includes the length and width corresponding to the edge.

3. The method for detecting the appearance and structure of a vehicle wiring harness according to claim 1, It is characterized in that The matching of the segmented line segments based on the length, width and branch nodes for all the wiring harness structure diagrams to be tested that are marked as correct includes: For any two wiring harness structure diagrams to be tested, traverse the branch nodes of one of the wiring harness structure diagrams to be tested, and match the branch nodes of the other wiring harness structure diagram to be tested according to the number, length and width of the segmented line segments connected by the branch nodes; When the number of segmentation line segments of the first branch node is equal to that of the second branch node, and for any segmentation line segment of the first branch node, the difference between the length and width of the segmentation line segment of the second branch node and the length and width of the any segmentation line segment does not exceed a set value, then the first branch node and the second branch node match.

4. The method for detecting the appearance and structure of a vehicle wiring harness according to claim 1, It is characterized in that The step S400 includes: S410, arbitrarily selecting one of the wiring harness structure diagrams marked as correct to be tested as the first wiring harness structure diagram; traversing all nodes in the wiring harness standard structure diagram, obtaining the corresponding segmentation line segment and the length standard value and width standard value corresponding to the segmentation line segment for the current node; S420, traversing the branch nodes of the first wiring harness structure diagram, determining whether there is a branch node that matches the current node, when the number of segmentation line segments of the branch node and the current node is equal, and for any segmentation line segment of the current node, there is a segmentation line segment of the branch node, and the difference between the length detection value and the width detection value of the segmentation line segment and the length standard value and the width standard value of the any segmentation line segment does not exceed the set value, then the branch node matches the current node; S430, if all nodes are matched successfully, the detection result of the structure and size of the wiring harness to be tested is qualified; if there is a node that fails to match, the segmented line segment that fails the detection is output.

5. The automotive wiring harness appearance and structure detection method according to claim 1, It is characterized in that The step S500 includes: S510, obtaining each detection point associated in the wiring harness standard structure diagram, and determining the segmentation line segments or branch nodes associated with each detection point in the wiring harness structure diagram to be tested that has completed matching according to the segmentation line segments or nodes associated with each detection point; S520, obtaining a wire harness image corresponding to the wire harness structure diagram to be tested, determining a detection position of the wire harness image according to a segmentation line segment or a branch node associated with each detection point, and performing image segmentation on the detection position of the wire harness image through a detection frame of a predetermined size to obtain a segmented image; when the detection point is associated with a segmentation line segment, obtaining a terminal position of the segmentation line segment as the detection position.

6. The method for detecting the appearance and structure of a vehicle wiring harness according to claim 5, It is characterized in that The step S500 includes: S530, obtaining component information of the current detection point, determining to input the segmented image into a terminal detection model or an accessory detection model, comparing the obtained terminal identification information or accessory identification information with the component information of the current detection point, and if they are the same, the detection is qualified.

7. The method for detecting the appearance and structure of an automobile wiring harness according to claim 1, It is characterized in that The step S500 includes: acquiring image data of multiple types of connection terminals collected from multiple angles to train a deep learning model to obtain a terminal detection model.

8. The method for detecting the appearance and structure of a vehicle wiring harness according to claim 1, It is characterized in that The step S500 includes: acquiring image data of multiple types of accessories collected from multiple angles to obtain an accessory image dataset; dividing the image dataset into a training set with a large number of samples and a support set with a small number of samples; and training the YOLOV8 model based on the training set and the support set to obtain an accessory detection model.

9. The method for detecting the appearance and structure of a vehicle wiring harness according to claim 1, It is characterized in that The step S600 includes: S610, obtaining wire harness images of multiple postures of the wire harness to be tested, processing pixels in the image of each detection point according to a detection frame of a predetermined size according to a fixed value, performing grayscale value conversion on the processed wire harness image, and performing contrast enhancement processing to obtain a pre-processed image; S620, performing denoising on the preprocessed image based on a median filtering algorithm to obtain a denoised image; S630, performing threshold segmentation on the denoised image based on an adaptive threshold segmentation algorithm to obtain a segmented image; S640, filling holes in the segmented image, marking targets, calculating the number of target pixels, and outputting defect positions and sizes.

10. An automotive wiring harness appearance and structure detection system, It is characterized in that include: The first module is used to obtain wire harness images of multiple postures of the wire harness to be tested, perform skeleton extraction on the wire harness images to obtain the skeleton of each wire harness image, and segment the skeleton to obtain segmentation line segments and branch nodes; The second module is used to calculate the width of each wire harness image according to the segmentation line segments, obtain the width of each segmentation line segment, and merge all segmentation line segments with the same color, width difference not exceeding the threshold and connected by branch nodes to obtain the final segmentation line segment and branch node of each wire harness image; calculate the length of the final segmentation line segment of the wire harness to be tested, and determine the structure diagram of the wire harness to be tested according to the size of the final segmentation line segment and the branch node; The third module is used to obtain the standard structure diagram of the wiring harness, and determine in turn whether each wiring harness structure diagram to be tested is simply isomorphic with the standard structure diagram of the wiring harness. If they are isomorphic, the current wiring harness structure diagram to be tested is marked as correct; for all wiring harness structure diagrams to be tested that are marked as correct, the segmentation segments are matched based on the length, width and branch nodes, and the maximum value of all length values ​​corresponding to the segmentation segment is obtained as the length detection value of the segmentation segment, and the average value of all width values ​​corresponding to the segmentation segment is obtained as the width detection value of the segmentation segment; The fourth module is used to obtain each node in the standard structure diagram of the wire harness, the segmented line segments corresponding to each node, and the length standard value and width standard value corresponding to each segmented line segment, and match and compare them with each branch node, the segmented line segments corresponding to each node, and the length detection value and width detection value corresponding to each segmented line segment in any wire harness structure diagram marked as correct to obtain the structure and size detection results of the wire harness to be tested; The fifth module is used to obtain each detection point associated in the wire harness standard structure diagram, determine the detection point position in the wire harness image according to the position corresponding to each detection, and segment the image of the detection point position based on a detection frame of a predetermined size to obtain a segmented image; input the segmented image into a terminal detection model or an accessory detection model to obtain a detection result; The sixth module is used to perform wire harness appearance defect detection based on the wire harness image to obtain an appearance defect detection result.

Citation Information

Patent Citations

  • Device, system and method for detecting wiring harness

    CN101776522A

  • Image block target detection method and system based on deep learning, and medium

    CN114067097A

  • Wire harness head stripping measurement and defect detection method and system based on image segmentation

    CN116660286A

  • Method and device for automated portrayal and accurate measurement of width of structural crack

    WO2019134252A1