Method and device for measuring length of multi-branch complex cable and storage medium

Abstract

The invention provides a method for measuring the length of a multi-branch complex cable, and the method comprises the steps: obtaining a plurality of images of a to-be-measured multi-branch complex cable with different exposure attributes, enabling the images to form an image sequence, inputting the image sequence into an image surface enhancement model, and obtaining a fusion cable image; segmenting each cable branch in the fused cable image by using an SAM model and prompt information; the contour line of each cable branch obtained through segmentation is extracted, the skeleton line of each cable branch is extracted based on a parallel refinement method, the skeleton line with the missing end corresponding to each cable branch is obtained, fitting compensation is carried out on each skeleton line with the missing end, and the complete skeleton line of each cable branch is obtained; and calculating the pixel length of the complete skeleton line of each cable branch, and calculating the actual length of each cable branch according to the pixel length. The length of the multi-branch complex cable is measured based on machine vision, the measurement precision can be improved, and automation and intelligentization of the manufacturing and detection process of the complex cable are achieved.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision, and in particular relates to a method, a device and a storage medium for measuring the length of a multi-branch complex cable. Background Art

[0002] Complex multi-branch cables are mainly used in industrial fields such as automobiles and aerospace. Figure 1 As shown. Taking automobiles as an example, cables are one of the core components that connect and support various parts inside the car. At the same time, they are also the transmission channels for energy and information, providing guarantee for the stable operation of the car. The importance of cables in the modern automobile industry cannot be underestimated. They connect various subsystems in the car. If the cables cannot operate safely and stably, the impact on the car will be catastrophic. In modern cars, as the number of electronic devices increases, the functions of various subsystems in the car put forward different requirements on the structure and performance of cables. The complexity of automotive cables is also increasing, and various high-performance cables and connectors are needed. For this reason, automotive cables play an important role in vehicle manufacturing and repair and maintenance. As the structure and functions of modern industrial products such as automobiles and spacecraft become more and more complex, the cable structures used in them are also becoming more and more complex, usually in a multi-branch state. Therefore, the size measurement of cables is an indispensable and important link in their assembly and testing process. For the measurement of cables, there are mainly the following types of existing technologies:

[0003] 1. Manual measurement method

[0004] Workers need to lay the cable flat, straighten the flexible cable manually, measure the parameters of its key dimensions with a tape measure, and record the data manually. As the complexity of a single cable increases, a single measurement takes longer. If a single cable is long, it will take up a large space after being straightened, which also makes measurement difficult. In addition, the complex branching of the cable also poses challenges to its dimensional measurement. Manually completing the measurement task is difficult and time-consuming, which delays the development cycle of the cable. The repetitive measurement work also causes visual fatigue to the staff, which affects the accuracy of the dimensional measurement and reduces the reliability of the measurement results.

[0005] 2. Pulse reflection method

[0006] The use of electromagnetic wave propagation and reflection to measure cable length is related to the properties of the cable itself. Patent applications such as CN202210174914.4, CN202311063941.5, and CN202211011797.6 are all designed through the steps of transmitting signals, receiving return signals, and calculating, and finally achieve non-destructive and high-precision cable length measurement. However, the objects being measured are all long high-voltage cables in the power system, which can be regarded as measuring the length of a single cable. For complex cables with complex structures and many branches, the pulse reflection method is difficult to achieve high accuracy.

[0007] 3. Visual measurement method

[0008] The visual measurement method is a method of obtaining the cable length through image acquisition and processing. Among the various machine vision dimension measurement technologies that have been disclosed, the measurement of curved multi-branched complex cables is scarce. For example, patent CN202221266241.7 applies a visual method to measure the cable length, and patent CN201910797869.6 applies a visual method to accurately measure the cutting length of the cable, but the above patents are not suitable for the length measurement of complex cables with curved and multi-branched characteristics. The application object is quite different from the cable in the present invention, and its measured object is in motion. The cable targeted in the present invention has the characteristics of bending and multi-branching. The dimension measurement system based on machine vision mainly includes image acquisition system, image area extraction, dimension measurement algorithm and other parts. The image acquisition system includes processes such as camera selection, layout and calibration; the image area extraction part should use the image segmentation method; the dimension measurement algorithm part varies according to the characteristics of the measured object. At present, most vision-based length measurement methods use the method of measuring the distance between two points, which is not used for multi-branch complex cables with bending features. Among them, the dimension measurement algorithm with reference significance for cable dimension measurement is the skeleton line extraction algorithm. Since the skeleton line can effectively identify the important features of the object, and due to the special shape of the cable and the length measurement usually does not need to consider the cable width, and the skeleton line usually runs through the entire cable, the measurement of the skeleton line can be used as an important criterion for evaluating the cable length.

[0009] In summary, both the manual measurement method and the pulse reflection method cannot be applied due to the measurement accuracy problems caused by the bending and multi-branching characteristics of the cable. As an important technology in the field of intelligent manufacturing, the machine vision dimension measurement method has great advantages in measuring the length of multi-branched complex cables. However, due to the complex structure of the cable, the skeleton line of the commonly used skeleton line extraction algorithm is incomplete, discontinuous, and has extraction errors. Therefore, it is urgent to develop a technical solution for the length measurement of complex cables with bending and multi-branching characteristics. Summary of the invention

[0010] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0011] To this end, the present invention proposes a method, device and storage medium for measuring the length of multi-branch complex cables, which completes the measurement of the length of multi-branch complex cables based on machine vision, improves the measurement accuracy, and realizes the automation and intelligence of the complex cable manufacturing and detection process.

[0012] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0013] A first aspect of the present disclosure provides a method for measuring the length of a multi-branch complex cable, comprising:

[0014] Acquire multiple images of a multi-branch complex cable to be tested with different exposure properties, construct an image sequence and input it into an image surface enhancement model to fuse the input image sequence, enhance the surface details of the image, and obtain a fused cable image;

[0015] Segmenting each cable branch in the fused cable image using the SAM model and prompt information;

[0016] The contour lines of each cable branch obtained by segmentation are extracted respectively, and the skeleton lines of each cable branch are extracted respectively based on the parallel thinning method to obtain the missing skeleton lines corresponding to the ends of each cable branch, and the missing skeleton lines of each end are fitted and compensated respectively, and the two intersection points of each fitted skeleton line and the contour line of the corresponding cable branch are used as the two end points of the complete skeleton line of the cable branch, so as to obtain the complete skeleton line of each cable branch;

[0017] The pixel length of the complete skeleton line of each cable branch is calculated, and the actual length of each cable branch is calculated based on the pixel length.

[0018] In some embodiments, a plurality of images of a multi-branch complex cable to be tested having different exposure properties are acquired by an image acquisition device;

[0019] The image acquisition device includes an imaging unit and a lighting unit. The imaging unit includes an industrial camera. The lighting unit adopts a combination of backlighting and forward lighting. The backlighting is provided by an LED light board, and the LED light board also serves as a working plane for placing the multi-branch complex cable to be tested. The forward lighting is provided by the flash light of the industrial camera.

[0020] In some embodiments, the exposure properties include exposure angle and exposure intensity.

[0021] In some embodiments, the image surface enhancement model adopts a MEF-Net network based on unsupervised learning.

[0022] In some embodiments, a public dataset and multi-angle exposure images of cable components are used together as a training set for the image surface enhancement model.

[0023] In some embodiments, segmenting each cable branch in the fused cable image using the SAM model and the prompt information specifically includes:

[0024] S21, setting a point selection prompt, wherein the positive point is set as the cable branch that needs to be selected for segmentation, and the negative point is set as the area that does not need to be selected for segmentation;

[0025] S22, based on the point selection prompt, generate iterative segmentation results through the SAM model;

[0026] S23, if there is a complete segmentation in the iterative segmentation result, then select to save the result; if there is no complete segmentation in the iterative segmentation result, then select one of the iterative segmentation results as a basis to perform iterative prompting, that is, perform segmentation by selecting more prompting points;

[0027] S24, generating a more accurate segmentation result according to the iterative prompt of step S23;

[0028] S25. If the segmentation result obtained in step S24 does not meet the set accuracy requirement, repeat the iterative process of steps S21 to S24 until the segmentation result meets the set accuracy requirement, and save the final segmentation result.

[0029] In some embodiments, the number or method of setting the point selection prompts should meet the requirement of being able to completely segment a single cable branch.

[0030] In some embodiments, the specific steps of obtaining the complete skeleton line of each cable branch include:

[0031] S31, extracting the contour lines of each single cable branch obtained by segmentation respectively, and obtaining the continuous contour lines of each single cable branch;

[0032] S32, performing binarization and corrosion operations on the images of each cable branch obtained by segmentation, and then extracting the skeleton lines of the single cable branches respectively by a parallel thinning method to obtain the skeleton lines corresponding to the missing ends of each cable branch;

[0033] S33, using a polynomial fitting method, fitting the missing skeleton lines corresponding to the ends of each cable branch obtained in step S32 to achieve an extension effect, and drawing the fitting line of the cable skeleton line and the cable contour line in the same figure;

[0034] S34, traverse and obtain two intersection points of the cable contour line and the skeleton line fitting line, use the intersection points as two end points of the fitting line, and obtain a new fitting line, and the new fitting line is the complete skeleton line.

[0035] A second aspect of the present disclosure provides a device for measuring the length of a multi-branch complex cable, comprising:

[0036] An image acquisition module is used to obtain multiple images of a multi-branch complex cable to be tested with different exposure properties and form an image sequence;

[0037] An image fusion module, which has an image surface enhancement model, is used to fuse the input image sequence, enhance the surface details of the image, and obtain a fused cable image;

[0038] A cable branch segmentation module, used for segmenting each cable branch in the fused cable image by using a SAM model and prompt information;

[0039] The skeleton line extraction module is used to extract the contour lines of each cable branch obtained by segmentation, and extract the skeleton lines of each cable branch based on the parallel thinning method to obtain the missing skeleton lines corresponding to the ends of each cable branch, and perform fitting compensation on the missing skeleton lines at each end, and use the two intersection points of each fitted skeleton line and the contour line of the corresponding cable branch as the two end points of the complete skeleton line of the cable branch, so as to obtain the complete skeleton line of each cable branch;

[0040] The cable branch length calculation module is used to calculate the pixel length of the complete skeleton line of each cable branch, and calculate the actual length of each cable branch based on the pixel length.

[0041] A third aspect of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method for measuring the length of a multi-branch complex cable according to any embodiment of the first aspect of the present disclosure.

[0042] The present disclosure has the following features and beneficial effects:

[0043] The present invention utilizes multi-angle exposure image fusion to enhance the surface details of multi-branch complex cables, which is beneficial to subsequent image segmentation processing; an interactive image segmentation method for a single branch of a multi-branch complex cable is adopted to segment a single cable image, solving the measurement strategy problem of multi-branch complex cables; a parallel refinement algorithm is used to extract the skeleton line of a single branch and perform fitting compensation, and finally the differential element method is used to calculate the length, thereby improving the accuracy of single branch length measurement.

[0044] In summary, the present invention completes the measurement of the length of multi-branch complex cables based on machine vision, which can improve the measurement accuracy and realize the automation and intelligence of the complex cable manufacturing and detection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the structure of a multi-branch complex cable.

[0046] Figure 2 It is an overall flow chart of a method for measuring the length of a branched complex cable provided in an embodiment of the first aspect of the present disclosure.

[0047] Figure 3 It is a specific flow chart of single-branch cable segmentation based on iterative prompts in a method for measuring the length of a branched complex cable provided in an embodiment of the first aspect of the present disclosure.

[0048] Figure 4 It is a specific flow chart of extraction, fitting and compensation of a single skeleton line of a cable in a method for measuring the length of a branched complex cable provided in an embodiment of the first aspect of the present disclosure.

[0049] Figure 5 (a), (b), and (c) are respectively the renderings of skeleton line extraction, compensation process, and compensation effect of a single cable branch in this embodiment.

[0050] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the third aspect of the present disclosure. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] On the contrary, the present application covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present application as defined by the claims. Further, in order to make the public have a better understanding of the present application, some specific details are described in detail in the detailed description of the present application below. Those skilled in the art can fully understand the present application without the description of these details.

[0053] In the description of the present disclosure, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the basis or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure. In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present disclosure, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0054] In the description of the present disclosure, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.

[0055] In the present disclosure, unless otherwise expressly specified and limited, a first feature being “above” or “below” a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being “above”, “above”, and “above” a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being “below”, “below”, and “below” a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0056] See also Figure 2 The first aspect of the present disclosure provides a method for measuring the length of a multi-branch complex cable, comprising the following steps:

[0057] S1. Acquire multiple images of a multi-branch complex cable to be tested with different exposure properties, construct an image sequence and input them into an image surface enhancement model to fuse the input image sequence, enhance the surface details of the image, and obtain a fused cable image;

[0058] S2, using the SAM model and prompt information to segment each cable branch in the fused cable image obtained in step S1;

[0059] S3, respectively extracting the contour lines of each cable branch obtained by segmentation in step S2, and respectively extracting the skeleton lines of each cable branch based on the parallel thinning method, obtaining the skeleton lines corresponding to the ends of each cable branch that are missing, respectively performing fitting compensation on the skeleton lines that are missing at each end, and using the two intersection points of each skeleton line after fitting and the contour line of the corresponding cable branch as the two end points of the complete skeleton line of the cable branch, thereby obtaining the complete skeleton line of each cable branch;

[0060] S4. Calculate the pixel length of the complete skeleton line of each cable branch, and calculate the actual length of each cable branch based on the pixel length.

[0061] In some embodiments, in step S1, a plurality of images of the multi-branch complex cable to be tested having different exposure properties are acquired by an image acquisition device.

[0062] Furthermore, in this embodiment, the image acquisition device includes an imaging unit and a lighting unit. The imaging unit includes an industrial camera with an imaging resolution of 3840×2748 and an 8mm fixed-focus telecentric lens. The industrial camera is fixed by a bracket. The axis of the industrial camera is perpendicular to the horizontal plane and the object plane. The industrial camera maintains a working distance of at least 100mm from the multi-branch complex cable to be tested. The lighting unit adopts a combination of backlighting and forward lighting; the backlighting is provided by an LED light board, and the LED light board also serves as a working plane. The multi-branch complex cable to be tested is placed on it. The clear edge of the multi-branch complex cable to be tested is obtained by backlighting, which is suitable for measuring the length of the cable; the forward lighting is provided by the flash of the industrial camera, and the acquisition of surface detail images can be guaranteed by forward lighting. The camera's internal parameters are calibrated using a standard calibration plate, and the actual length of the unit pixel, that is, the pixel equivalent, is obtained.

[0063] Furthermore, the exposure attributes defined in this embodiment include exposure angle and exposure intensity, etc.

[0064] In many machine vision-based dimension measurement systems, the pre-processing of images is simple filtering or binarization. In step S1 of the present disclosure, a complex cable image fusion method based on multi-angle exposure is used as a pre-processing method for measurement to enhance the cable surface details and lay the foundation for optimizing the measurement effect. The multi-image method is an important method of image processing, and the multi-exposure fusion method is the most representative multi-image method. The basic idea is to input the same image with different exposure attributes (exposure angle, exposure intensity, etc.), and finally output a single fused image. The fused image has richer details and colors.

[0065] Preferably, in order to shorten the measurement time as much as possible, the embodiment of the present disclosure adopts the MEF-Net network based on unsupervised learning (Ma K, Duanmu Z, Zhu H, et al. Deep guided learning for fast multi-exposure image fusion [J]. IEEE Transactions on Image Processing, 2020, 29: 2808–2819.) as the image surface enhancement model, which has a particularly prominent advantage in fusion time. The MEF-Net network consists of a bilinear downsampler, a context feature prediction network (ContextAggregationNet, CAN), a guided filter and a weighted fusion module. Among them, the bilinear downsampler is used to downsample the input image sequence to obtain a low-resolution multi-angle exposure image input sequence. CAN, as the core module of the MEF-Net network, is specifically a convolutional network used to convert the low-resolution multi-angle exposure image input sequence into a corresponding weight map. The MEF-Net network needs to allow images of any size and number of exposure images, and generate feature maps of corresponding size and number. Finally, the output weight map is fused through the guided filter. In the weighted fusion module, the input original image sequence is weightedly fused with the weight map. The fusion is based on the formula:

[0066]

[0067] In the formula, Y represents the fused image, W k represents the image sequence X obtained in step S1 k ⊙ represents the weight graph of , and ⊙ represents the Hadamard product.

[0068] The optimization of the MEF-Net network is achieved through a loss function based on SSIM(x,y), which mainly considers the structural information of the image. The calculation formula is as follows:

[0069]

[0070] In the formula, x and y represent two images, μ x and μ y Represent the average intensity of the two images, σ x and σ y Represents the local variance of image x and image y, σ z represents the covariance between image x and image y, C 1 and C 2 denotes two small constants used to maintain stability.

[0071] This embodiment uses the public data set and the multi-angle exposure images of the original cable as the training set, and trains the optimal MEF-Net network as the image surface enhancement model. By inputting complex cable images exposed at multiple angles, a cable image with rich details is finally fused to improve the subsequent segmentation effect. The aspect ratio of the images before and after the fusion remains unchanged.

[0072] Considering the complexity of multi-branch cables, if the overall skeleton line is extracted, there will be problems such as extraction errors, incomplete ends, and burrs on the skeleton line. Therefore, the image segmentation method will be used to segment the single cable branch, which places extremely high requirements on the data set. Therefore, step S2 of the embodiment of the present disclosure adopts the Segment-Anything Model, SAM model (Kirillov A, Mintun E, Ravi N, et al. Segment anything[J].arXiv preprint arXiv:2304.02643,2023.), and completes the segmentation of the cable branches of the fused cable image obtained in step S1 through point selection iteration prompts.

[0073] Specifically, the SAM model is a relatively new basic model for image segmentation, which has zero-sample migration capability and strong generalization performance, and can guide the completion of image segmentation through interactive prompts. The structure of the SAM model includes a prompt decoder, a picture decoder, and a lightweight mask decoder. By inputting an image and a corresponding prompt (Prompt), the mask of the segmented image can be obtained. The prompt can be in the form of a single point, multiple points, a box, a mask or text, or a mixed prompt can be performed in multiple ways. In the embodiment of the present disclosure, multiple points are selected as the prompt method, and the main body of the cable branch is selected by the positive point, and the unnecessary area is selected by the negative point. In addition, the SAM model has an iterative prompt mechanism, that is, the segmentation mask of this time can be used as the input of the next segmentation. At the same time, through settings, one segmentation can be set to obtain one or three results, where the three results can point to three different valid objects respectively. If it is set to obtain one result, the result obtained is the one with the highest accuracy among the three results. Step S2 of the embodiment of the present disclosure is based on the SAM model, and a multi-branch complex cable interactive segmentation strategy is formulated, see Figure 3 , the interactive segmentation steps are as follows:

[0074] S21. Setting point selection prompts: Select a cable branch to be segmented by selecting points, wherein a positive point is set as a cable branch to be segmented, and a negative point is set as an area not to be segmented. A single positive point, a negative point, or multiple positive points, a negative point can be set. The number or method of point selection prompts should meet the requirement of completely segmenting a single cable branch. If the point selection prompts are insufficient, a single branch cannot be completely segmented.

[0075] S22, generating three prediction results: based on the point selection prompt, three iterative segmentation results are generated through the SAM model;

[0076] S23, save the segmentation result or iterate the prompt: If there is a complete segmentation in the iterative segmentation result, you can choose to save the result; if the segmentation is incomplete, you can select one of the results (represented in the form of a mask) as the basis for iterative prompting, that is, by selecting more prompt points (selecting points randomly) to perform segmentation.

[0077] S24, generating new prediction results: generating more accurate segmentation results according to the iterative prompts of step S23.

[0078] S25. If the segmentation result obtained in step S24 does not meet the set accuracy requirement, repeat the iterative process of steps S21 to S24 until the segmentation result meets the set accuracy requirement, and save the final segmentation result.

[0079] In some embodiments, see Figure 4 , step S3 specifically includes:

[0080] S31, extracting the contour lines of each single cable branch obtained in step S2 respectively (in this embodiment, the findContours function in OpenCV is used to extract the contour lines), and obtaining the continuous contour lines of each single cable branch;

[0081] S32, using the threshold segmentation method to binarize the image of the single cable branch obtained in step S2, and perform an erosion operation, and then use the parallel thinning method proposed by Zhang et al. to extract the skeleton lines of the single cable branches respectively, and obtain the missing skeleton lines corresponding to the ends of each cable branch, see Figure 5 In (a), the white thin line is the skeleton line with the missing end obtained in this step, and the area enclosed by the white line is the continuous contour line of the single cable branch obtained in step S31. It can be seen that the two ends of the skeleton line are slightly shorter than the two ends of the cable branch contour line;

[0082] S33, using a polynomial fitting method, fitting the missing skeleton lines corresponding to the ends of each cable branch obtained in step S32 to achieve an extension effect, and drawing the fitting line of the cable skeleton line and the cable contour line in the same figure, see Figure 5 (b), wherein the white thin line is the fitted extension line of the skeleton line of the single cable branch obtained in this step, and the area enclosed by the white line is the continuous contour line of each single cable branch obtained in step S31;

[0083] S34, traverse to obtain the two intersection points of the cable contour line and the skeleton line fitting line, use the intersection points as the two end points of the fitting line, and obtain a new fitting line. The fitting line is the complete skeleton line. Figure 5 In (c), the white thin line is the complete skeleton line of the single cable branch obtained in this step, and the area enclosed by the white line is the continuous contour line of each single cable branch obtained in step S31. It can be seen that the end of the skeleton line has been completed.

[0084] Furthermore, the parallel thinning algorithm used in step S32 was proposed by Zhang et al. in 1984 (Zhang TY, Suen CY. Afast parallel algorithm for thinning digital patterns [J]. Communications of the ACM, 1984, 27 (3): 236–239.), which mainly uses the idea of ​​iterative deletion to extract the skeleton line of the target. The steps are as follows:

[0085] S321, delete P that satisfies the following formula 1 Points, where the foreground points in the segmentation result obtained in step S2 are marked as 1, and the background points are marked as 0, N(P 1 ) indicates that 1 The number of foreground points among the eight adjacent pixels of the point is P 1 Point is evenly distributed around the circle, and the point is located at P 1 The pixels directly above, above right, right, below right, directly below, below left, left, and above left are denoted as P 2 , P 3 , P 4 , P 5 , P 6 , P 7 , P 8 and P 9 , the arrangement of each point is shown in Table 1, S(P 1 ) indicates P 1 The eight neighboring points of a point are arranged in a clockwise direction (P 2 →P 9 ) is the number of changes from background point to foreground point (0→1);

[0086]

[0087] Table 1

[0088] <![CDATA[P 9 ]]> <![CDATA[P 2 ]]> <![CDATA[P 3 ]]> <![CDATA[P 8 ]]> <![CDATA[P 1 ]]> <![CDATA[P 4 ]]> <![CDATA[P 7 ]]> <![CDATA[P 6 ]]> <![CDATA[P 5 ]]>

[0089] S322, similar to step S321, delete P that satisfies the following formula 1 point;

[0090]

[0091] S323, traverse all foreground points, repeat steps S321 and S322, and finally obtain a skeleton line with a single pixel width.

[0092] Since the refinement algorithm is based on morphological operations, the extracted skeleton line will be missing at both ends, resulting in incompleteness. That is, the skeleton lines of each cable branch extracted based on the parallel refinement method are missing at the ends. Then, the embodiment of the present disclosure compensates for the integrity of the skeleton line through polynomial fitting to obtain a complete skeleton line.

[0093] In some embodiments, step S4 uses the microelement method to calculate the pixel length of the complete skeleton line of each cable branch. The principle is to obtain the coordinates of the complete skeleton line through curve search, calculate the length of the micro segment in the curve, and finally accumulate the pixel length of the branch. The calculation formula of the micro segment is as follows:

[0094]

[0095] Among them, l i represents the length of the i-th micro-segment, Δx i , Δy i Respectively represent the width and height of the i-th micro segment, x si With y si are the horizontal and vertical coordinates of the first endpoint of the i-th micro-segment, respectively. ti With y ti They respectively represent the horizontal and vertical coordinates of the end point of the i-th micro-segment.

[0096] By sampling at reasonable intervals (the number of sampling intervals in this embodiment is set to 10, and can also be set to other sampling intervals according to accuracy requirements), the actual length of a single cable can be quickly and accurately calculated by multiplying the pixel length by the pixel equivalent. Through the segmentation cycle, the length of each branch of the cable can be finally obtained.

[0097] It can be understood that the embodiment of the present disclosure uses a mixed lighting solution of forward and back mixed lighting to build a mixed lighting image acquisition device, completes the accurate acquisition of multi-branch cable images, and obtains pixel equivalents through calibration; proposes a preprocessing method based on multi-angle exposure image fusion, obtains a fused cable image with enhanced details, rich colors, and more surface information, which is conducive to the segmentation of a single cable branch; through interactive segmentation based on iterative prompts, a single cable branch can be accurately extracted; finally, based on the skeleton line extraction fitting compensation method, and using the microelement length calculation algorithm, the actual length of a single cable branch can be accurately calculated. Through the method designed in the embodiment of the present disclosure, the accurate measurement of multi-branch complex cables can be completed, and the intelligent production and manufacturing detection link can be realized.

[0098] The second aspect of the present disclosure provides a device for measuring the length of a multi-branch complex cable, comprising:

[0099] An image acquisition module is used to obtain multiple images of a multi-branch complex cable to be tested with different exposure properties and form an image sequence;

[0100] An image fusion module, which has an image surface enhancement model, is used to fuse the input image sequence, enhance the surface details of the image, and obtain a fused cable image;

[0101] The cable branch segmentation module is used to segment each cable branch in the fused cable image using the SAM model and prompt information;

[0102] The skeleton line extraction module is used to extract the contour lines of each cable branch obtained by segmentation, and extract the skeleton lines of each cable branch based on the parallel thinning method to obtain the missing skeleton lines corresponding to the ends of each cable branch, and perform fitting compensation on the missing skeleton lines at each end, and use the two intersection points of each fitted skeleton line and the contour line of the corresponding cable branch as the two end points of the complete skeleton line of the cable branch, so as to obtain the complete skeleton line of each cable branch;

[0103] The cable branch length calculation module is used to calculate the pixel length of the complete skeleton line of each cable branch, and calculate the actual length of each cable branch based on the pixel length.

[0104] It should be noted that the aforementioned explanation of the embodiment of the method for measuring the length of a multi-branch complex cable is also applicable to the measuring device of this embodiment, and will not be repeated here.

[0105] In order to implement the above embodiment, the embodiment of the present disclosure further proposes a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to execute the method for measuring the length of a multi-branch complex cable of the above embodiment.

[0106] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. It should be noted that the electronic device in the embodiments of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, servers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0107] like Figure 6 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 to a random access memory (RAM) 103. In the RAM 103, various programs and data required for the operation of the electronic device are also stored. The processing device 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0108] Typically, the following devices may be connected to the I / O interface 105: an input device 106 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, etc.; an output device 107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 108 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 109. The communication device 109 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0109] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present embodiment includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 109, or installed from the storage device 108, or installed from the ROM 102. When the computer program is executed by the processing device 101, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0110] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0111] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0112] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method for measuring the length of a multi-branch complex cable.

[0113] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, python, and conventional procedural programming languages, such as "C-" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0115] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0116] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0117] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0118] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0119] A person of ordinary skill in the art may understand that all or part of the steps carried out in the above-mentioned embodiment method may be completed by instructing the relevant hardware through a program, and the developed program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0120] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0121] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for measuring the length of a multi-branch complex cable, characterized in that: include: Acquire multiple images of a multi-branch complex cable to be tested with different exposure properties, construct an image sequence and input it into an image surface enhancement model to fuse the input image sequence, enhance the surface details of the image, and obtain a fused cable image; Segmenting each cable branch in the fused cable image using the SAM model and prompt information; The contour lines of each cable branch obtained by segmentation are extracted respectively, and the skeleton lines of each cable branch are extracted respectively based on the parallel thinning method to obtain the missing skeleton lines corresponding to the ends of each cable branch, and the missing skeleton lines of each end are fitted and compensated respectively, and the two intersection points of each fitted skeleton line and the contour line of the corresponding cable branch are used as the two end points of the complete skeleton line of the cable branch, so as to obtain the complete skeleton line of each cable branch; The pixel length of the complete skeleton line of each cable branch is calculated, and the actual length of each cable branch is calculated based on the pixel length.

2. The measuring method according to claim 1, characterized in that: Acquire multiple images of a multi-branch complex cable to be tested with different exposure properties through an image acquisition device; The image acquisition device includes an imaging unit and a lighting unit. The imaging unit includes an industrial camera. The lighting unit adopts a combination of backlighting and forward lighting. The backlighting is provided by an LED light board, and the LED light board also serves as a working plane for placing the multi-branch complex cable to be tested. The forward lighting is provided by the flash light of the industrial camera.

3. The measuring method according to claim 1, characterized in that: The exposure attributes include exposure angle and exposure intensity.

4. The measuring method according to claim 1, characterized in that: The image surface enhancement model adopts the MEF-Net network based on unsupervised learning.

5. The measuring method according to claim 4, characterized in that: Public data sets and multi-angle exposure images of cable components are used together as training sets for the image surface enhancement model.

6. The measuring method according to claim 1, characterized in that: The segmenting of each cable branch in the fused cable image by using the SAM model and the prompt information specifically includes: S21, setting a point selection prompt, wherein the positive point is set as the cable branch that needs to be selected for segmentation, and the negative point is set as the area that does not need to be selected for segmentation; S22, based on the point selection prompt, generate iterative segmentation results through the SAM model; S23, if there is a complete segmentation in the iterative segmentation result, then select to save the result; if there is no complete segmentation in the iterative segmentation result, then select one of the iterative segmentation results as a basis to perform iterative prompting, that is, perform segmentation by selecting more prompting points; S24, generating a more accurate segmentation result according to the iterative prompt of step S23; S25. If the segmentation result obtained in step S24 does not meet the set accuracy requirement, repeat the iterative process of steps S21 to S24 until the segmentation result meets the set accuracy requirement, and save the final segmentation result.

7. The measuring method according to claim 6, characterized in that: The number or method of setting point selection prompts should meet the requirements of completely segmenting a single cable branch.

8. The measuring method according to claim 1, characterized in that: The specific steps of obtaining the complete skeleton line of each cable branch include: S31, extracting the contour lines of each single cable branch obtained by segmentation respectively, and obtaining the continuous contour lines of each single cable branch; S32, performing binarization and corrosion operations on the images of each cable branch obtained by segmentation, and then extracting the skeleton lines of the single cable branches respectively by a parallel thinning method to obtain the skeleton lines corresponding to the missing ends of each cable branch; S33, using a polynomial fitting method, fitting the missing skeleton lines corresponding to the ends of each cable branch obtained in step S32 to achieve an extension effect, and drawing the fitting line of the cable skeleton line and the cable contour line in the same figure; S34, traverse and obtain two intersection points of the cable contour line and the skeleton line fitting line, use the intersection points as two end points of the fitting line, and obtain a new fitting line, and the new fitting line is the complete skeleton line.

9. A device for measuring the length of a multi-branch complex cable, characterized in that: include: An image acquisition module is used to obtain multiple images of a multi-branch complex cable to be tested with different exposure properties and form an image sequence; An image fusion module, which has an image surface enhancement model, is used to fuse the input image sequence, enhance the surface details of the image, and obtain a fused cable image; A cable branch segmentation module, used for segmenting each cable branch in the fused cable image by using a SAM model and prompt information; The skeleton line extraction module is used to extract the contour lines of each cable branch obtained by segmentation, and extract the skeleton lines of each cable branch based on the parallel thinning method to obtain the missing skeleton lines corresponding to the ends of each cable branch, and perform fitting compensation on the missing skeleton lines at each end, and use the two intersection points of each fitted skeleton line and the contour line of the corresponding cable branch as the two end points of the complete skeleton line of the cable branch, so as to obtain the complete skeleton line of each cable branch; The cable branch length calculation module is used to calculate the pixel length of the complete skeleton line of each cable branch, and calculate the actual length of each cable branch based on the pixel length.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method for measuring the length of a multi-branch complex cable according to any one of claims 1 to 8.

Images