Defect detection method, device and equipment for cable structure and storage medium

Abstract

The invention discloses a defect detection method, device and equipment for a cable structure and a storage medium, and the method comprises the steps: collecting a plurality of images of the cable structure, the plurality of images comprising a partial overlapping region; splicing the images according to the feature information in the overlapping region to obtain a spliced image; performing geometric transformation on the spliced image based on the center line of the cable structure in the spliced image to obtain a tiled and unfolded image; and detecting the tiled and unfolded image to obtain a multi-lay-length defect detection result. According to the method and the device, multi-lay-length defect detection of each cable can be realized, and the subsequent prediction precision of the fault of the cable structure is improved.

Description

Technical Field

[0001] This application relates to the field of cable inspection technology, specifically to a method, apparatus, equipment, and storage medium for detecting defects in cable structures. Background Technology

[0002] Cables are components used in hoisting, transportation, construction, and other fields, playing a vital role in traction, transport, and load carrying. To improve durability, they are typically composed of multiple strands of rope; for example, wire rope cables are usually composed of multiple strands of wound steel wire. Of course, cables can also be made of other materials, such as synthetic materials, wound together.

[0003] To better identify potential traction or lifting risks in advance, it is often necessary to inspect for defects in cables. For example, in the case of wire ropes, it is often necessary to inspect for broken wires, wear, or corrosion (rust). However, due to limitations in actual inspection conditions, current cable inspections can mostly only detect defects in one lay length (the axial distance required for the strand or wire to rotate 360 ​​degrees around the core of the wire rope), affecting the overall inspection results. Summary of the Invention

[0004] This application provides a method, apparatus, equipment, and storage medium for detecting defects in cable structures, aiming to improve the detection effect of cable defects.

[0005] In a first aspect, this application provides a method for detecting defects in cable structures, including: Acquire multiple consecutive images of the target cable structure, wherein adjacent images in the multiple consecutive images include at least a partially overlapping area of ​​the target cable structure; The multiple consecutive images are stitched together based on the feature information within the overlapping area to obtain a stitched image of the target cable structure; Based on the centerline of the cable structure extracted from the edge recognition results of the stitched image, a geometric transformation is performed on the stitched image to obtain a flat unfolded image of the target cable structure; The unfolded image is inspected to obtain the multi-twist defect detection results of each cable in the target cable structure.

[0006] In one embodiment of this application, stitching together the multiple consecutive images based on feature information within the overlapping region to obtain a stitched image of the target cable structure includes: For adjacent first target images and second target images in the multiple consecutive images, feature points are extracted from the images. The feature points include multiple first feature points on the first target image and multiple second feature points on the second target image. Feature matching is performed on the plurality of first feature points and the plurality of second feature points to obtain an initial transformation matrix between the first target image and the second target image; Based on the registration error between feature points of each group of adjacent images after transformation by the transformation matrix, the initial transformation matrix is ​​globally optimized to obtain the target transformation matrix of each group of adjacent images. The target transformation matrix is ​​used to transform each group of adjacent images before stitching them together to obtain a stitched image of the target cable structure.

[0007] In one embodiment of this application, the step of performing feature matching on the plurality of first feature points and the plurality of second feature points to obtain an initial transformation matrix between the first target image and the second target image includes: Based on the feature information of each feature point, feature matching is performed on multiple first feature points and multiple second feature points to obtain the registration feature points of each feature point in its neighboring images; the registration feature points include nearest neighbor registration feature points and second nearest neighbor registration feature points. Based on the feature distance between each feature point and its registered feature point, a target feature point group is determined from the feature points. In each target feature point group, the first target feature point and the second target feature point are the nearest neighbor registered feature points, and the first feature distance between the first target feature point and its nearest neighbor registered feature point and the second feature distance between its second nearest neighbor registered feature point satisfy a preset condition. The initial transformation matrix between the first target image and the second target image is determined based on the target feature point group.

[0008] In one embodiment of this application, the step of performing feature matching on multiple first feature points and multiple second feature points based on the feature information of each feature point to obtain the registration feature points of each feature point in its neighboring images includes: Based on the neighborhood grayscale information of the feature points, a first fast feature of the first feature point and a second fast feature of the second feature point are extracted; and based on the neighborhood gradient information of the feature points, a first affine invariant feature of the first feature point and a second affine invariant feature of the second feature point are extracted. Based on the Hamming distance between the first fast feature and the second fast feature, and the Euclidean distance between the first affine invariant feature and the second affine invariant feature, feature matching is performed on multiple first feature points and multiple second feature points to obtain the registration feature points of each feature point in its neighboring images.

[0009] In one embodiment of this application, the method further includes: The stitched image after grayscale processing is binarized to obtain the boundary information in the stitched image; the segmentation threshold used for binarization is determined based on the statistical results of pixel values ​​within the region where the pixel is located; The boundary information is subjected to iterative erosion processing to obtain skeleton information with a preset pixel width, the skeleton information including multiple skeleton points; Based on the distance from the skeleton point to the boundary information, a target skeleton point is determined from the skeleton points; Dynamic path planning is performed on the target skeleton points to obtain the target curve, and curve fitting is performed on the target curve to obtain the center line of the cable structure in the stitched image.

[0010] In one embodiment of this application, the step of performing a geometric transformation on the stitched image based on the centerline of the cable structure extracted from the edge recognition result of the stitched image to obtain a tiled unfolded image of the target cable structure includes: The centerline is parameterized by arc length to obtain the cumulative arc length and normal vector of each pixel on the centerline. The pixels in the stitched image are remapped based on the accumulated arc length and the normal vector to perform a geometric transformation on the stitched image, thereby obtaining a flat unfolded image of the target cable structure.

[0011] In one embodiment of this application, the multi-twist defect detection results of each cable in the target cable structure are obtained by detecting the unfolded image, including: The unfolded image is divided into blocks, and multi-scale feature information of each block is obtained through a feature extraction network. Based on nonmaximum suppression, the multi-scale feature information of each block is deduplicated to obtain the target feature information of each block; The target feature information is processed by the convolutional block attention module to obtain the defect type and confidence level of each cable in the target cable structure, so as to determine the multi-twist defect detection result. The defect type includes at least one of broken wire defects, wear defects and corrosion defects.

[0012] Secondly, this application also provides a defect detection device for cable structures, comprising: The acquisition module is used to acquire multiple consecutive images of the target cable structure, wherein adjacent images in the multiple consecutive images include at least a partially overlapping area of ​​the target cable structure; The stitching module is used to stitch together the multiple consecutive images based on the feature information in the overlapping area to obtain a stitched image of the target cable structure; The transformation module is used to perform geometric transformation on the spliced ​​image based on the center line of the cable structure extracted from the edge recognition result of the spliced ​​image, so as to obtain a flat unfolded image of the target cable structure; The detection module is used to detect the unfolded image and obtain the multi-twist defect detection results of each cable in the target cable structure.

[0013] Thirdly, this application also provides a computer device, the computer device comprising: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the cable structure defect detection method as described in any of the preceding claims.

[0014] Fourthly, this application also provides a computer storage medium storing a computer program, which is loaded by a processor to execute the cable structure defect detection method as described in any of the preceding claims.

[0015] The cable structure defect detection method provided in this application acquires multiple images of the cable, including overlapping areas, and stitches these multiple consecutive images together based on the feature information of the overlapping areas to obtain a stitched image including the multi-twist pitch. After transforming the image based on the centerline, a flattened image can be obtained, thereby more accurately detecting multi-twist pitch defects in each cable and improving the accuracy of subsequent fault prediction of the cable structure. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the steps of a defect detection method for a cable structure provided in an embodiment of this application. Figure 2 This application provides a schematic flowchart of the steps for stitching together multiple consecutive images. Figure 3 This application provides a schematic flowchart illustrating the steps of feature matching of feature points to determine the transformation matrix between images. Figure 4a This is a schematic diagram illustrating the effect of a stitched image before global optimization, provided in an embodiment of this application. Figure 4b This is a schematic diagram illustrating the effect of a globally optimized stitched image provided in an embodiment of this application; Figure 5 This is a schematic flowchart illustrating the steps of processing an image to extract the centerline of a cable structure, as provided in an embodiment of this application. Figure 6 This application provides a schematic flowchart of steps for performing geometric transformations on stitched images. Figure 7 This application provides a schematic flowchart of a procedure for defect detection in a cable structure. Figure 8 A schematic diagram of the image detection results of a cable structure within a single twist pitch achieved by related technologies; Figure 9 A schematic diagram showing the defect detection results of a cable structure within multiple twist pitches, achieved by the detection method provided in the embodiments of this application; Figure 10 A schematic diagram of the complete process of a defect detection method for a cable structure provided in an embodiment of this application; Figure 11 This is a schematic diagram of a defect detection device for a cable structure provided in an embodiment of this application; Figure 12 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0021] To clearly understand the cable structure defect detection method, apparatus, equipment, and storage medium provided in the embodiments of this application, the relevant application scenarios of the cable structure defect detection method are first described below. Specifically, cable structure defect detection is generally applicable to the detection of defects such as broken wires, wear, or corrosion on cables, thereby predicting potential risks to the cable in subsequent lifting and transportation scenarios.

[0022] However, in actual working conditions, due to cost constraints, using industrial cameras to collect local images of wire ropes and using deep learning models to identify surface defects such as broken wires, wear, and corrosion can often only cover 1 to 2 lay lengths. This makes it impossible to observe the periodic distribution pattern of wire rope damage or to track the evolution of defects along the length of the wire rope. As a result, many progressive damages spanning multiple lay lengths are missed.

[0023] In order to solve the above-mentioned technical problems, this application provides a method, apparatus, device and storage medium for detecting defects in cable structures, aiming to accurately detect multi-twist defects in cables, thereby improving the accuracy of subsequent prediction of cable structure failures. Specifically, the following will be described in conjunction with specific embodiments.

[0024] Please see Figure 1 , Figure 1 This application provides a flowchart illustrating the steps of a defect detection method for a cable structure, specifically including steps S110 to S140: S110: Acquire multiple consecutive images of the target cable structure.

[0025] In one embodiment of this application, a continuous image acquisition system can be constructed using a high frame rate camera typically used in traditional industrial settings, thereby enabling continuous acquisition of multiple consecutive images of the target cable structure. Specifically, in this embodiment, adjacent images in the multiple consecutive images typically include partially overlapping areas of the target cable structure. For example, the camera can continuously acquire surface images of the wire rope at a fixed frame rate, with each image covering approximately 1-2 strand lengths to ensure detection effectiveness, and adjacent frames in the continuous acquisition typically have a 40%-60% overlap area to facilitate subsequent registration.

[0026] Of course, it should be noted that the high frame rate camera set above can be used to take pictures of the surface of the steel wire rope at a fixed angle. In general, panoramic pictures of the surface of the steel wire rope can be taken by setting cameras in multiple directions. Since the processing of the images taken by each camera is similar, this application embodiment will not repeat the description here, but only take the image taken by a single camera as an example for explanation.

[0027] Furthermore, in one embodiment of this application, to further improve the image processing effect and thus improve the accuracy of subsequent defect detection results, the image sequence acquired by the camera can be preprocessed to improve image quality. Specifically, the image preprocessing includes, but is not limited to, at least one of denoising, contrast enhancement, and color correction. Specifically, in one possible implementation, Contrast Limited Adaptive Histogram Equalization (CLAHE) can be used to enhance the local contrast of the image. Image quality assessment is implemented by setting a contrast limit of 2.0 and a grid size of 8×8, thereby eliminating blurry, overexposed, or underexposed images. In addition, the Laplacian variance of each image can be calculated as a sharpness indicator, with a threshold set to 100, to calculate the average brightness and standard deviation of the image, ensuring that they are within a reasonable range, for example, an average brightness of 100-200 and a standard deviation >20. For images that do not meet the above requirements, interpolation can be performed using preceding and following frames.

[0028] Of course, the above-mentioned implementation scheme is only one possible implementation scheme. It is also feasible to perform other preprocessing methods on the image based on the above-mentioned scheme to improve the image processing effect. This application embodiment does not limit this.

[0029] S120, the multiple consecutive images are stitched together based on the feature information in the overlapping area to obtain a stitched image of the target cable structure.

[0030] Based on the foregoing, in order to detect defects in cable structures within multiple twist pitches, in one embodiment of this application, multiple consecutive images that have overlapping areas and respectively indicate that the cable structure is within the range of 1 to 2 twist pitches are stitched together to obtain a stitched image of the cable structure with multiple twist pitches.

[0031] Specifically, stitching together multiple consecutive images can rely on feature information within overlapping areas. For example, as a possible implementation, feature points within overlapping areas can be identified, and then feature points in adjacent images can be matched to stitch adjacent images together sequentially until the final stitched image of the target cable structure is obtained.

[0032] Of course, the above-mentioned solution is only a feasible implementation. However, in actual application scenarios, the shooting of cable structures often occurs during the operation of the cable structure. During this period, the cable structure is affected by factors such as force, such as load-bearing, and the mechanical vibration often makes it difficult to identify feature points, resulting in a certain error in image registration. Continuous stitching will gradually accumulate errors, resulting in serious image offset or even failure to stitch. Actual test data shows that the cumulative error after stitching 10 frames of images can reach 20% of the image width.

[0033] To overcome the registration errors and their accumulation in the aforementioned scenarios, one embodiment of this application provides a method based on global registration optimization to improve the registration error of consecutive adjacent images, thereby enhancing the final stitched image's performance. For a clearer understanding of the above, please refer to [link to relevant documentation]. Figure 2 , Figure 2 This application provides a flowchart illustrating the steps for stitching together multiple consecutive images, specifically including steps S210 to S240: S210, for adjacent first target images and second target images in the multi-frame continuous images, extract feature points in the images.

[0034] For ease of description, in one embodiment of this application, two adjacent frames in a multi-frame continuous image, namely the first target image and the second target image, will be used as an example for illustration. For other adjacent images in a multi-frame continuous image, a similar scheme can be used to perform the same processing procedure, which will not be elaborated upon in this embodiment. The feature points include multiple first feature points on the first target image and multiple second feature points on the second target image.

[0035] Specifically, feature points in an image can be extracted through various processing methods. For example, as a possible implementation, feature points can be quickly extracted using an ORB (Oriented FAST and Rotated BRIEF, an image feature extraction algorithm that uses the FAST algorithm to detect feature points in an image and the BRIEF algorithm to generate descriptors) detector. Of course, other feature point extraction algorithms can also be used to obtain feature points in an image based on actual needs. This application does not limit this. For ease of description, in one embodiment of this application, the ORB detector is used as an example to extract feature points in an image. Specifically, an 8-layer pyramid can be set, and multiple first feature points on the first target image and multiple second feature points on the second target image can be extracted with a scaling factor of 1.2. Specifically, the number of feature points can be 2000. It should be noted that feature points extracted by the ORB detector usually have better rotation invariance and real-time performance.

[0036] S220, perform feature matching on the plurality of first feature points and the plurality of second feature points to obtain an initial transformation matrix between the first target image and the second target image.

[0037] Based on the above, by performing feature matching on the first feature point in the first target image and the second feature point in the second target image, an initial transformation matrix between the first target image and the second target image can be obtained, so that the first target image or the second target image can be matched with the corresponding second target image or the first target image before the transformation after the transformation process of the initial transformation matrix.

[0038] Specifically, to improve the matching effect of feature point matching and thus determine a more accurate transformation matrix, in one embodiment of this application, a cross-validation strategy and ratio constraint are considered to filter eligible registration feature point pairs during the registration process, thereby improving matching accuracy. For details, please refer to... Figure 3 , Figure 3 This application provides a flowchart illustrating a step for performing feature matching on feature points to determine a transformation matrix between images, specifically including steps S310 to S330: S310, based on the feature information of each feature point, perform feature matching on multiple first feature points and multiple second feature points to obtain the registration feature points of each feature point in its neighboring images.

[0039] In this embodiment of the application, the registration feature points include nearest neighbor registration feature points and second nearest neighbor registration feature points.

[0040] In one embodiment of this application, the feature information of feature points is typically used to indicate features on the attribute of the feature points. For example, taking the feature points extracted by a common ORB detector as an example, their feature information typically includes a 256-bit binary descriptor, namely BRIEF (Binary Robust Independent Elementary Features, a method of encoding the comparison result into binary bits by randomly selecting several pairs of pixel points around the feature point and then comparing the gray values ​​of each pair of pixels). Of course, in order to further improve the registration accuracy, the feature information may also include affine invariant features in addition to the aforementioned BRIEF features. That is, in one embodiment, the feature matching of multiple first feature points and multiple second feature points based on the feature information of each feature point to obtain the registration feature points of each feature point in its neighboring images includes: Based on the neighborhood grayscale information of the feature points, a first fast feature of the first feature point and a second fast feature of the second feature point are extracted; and based on the neighborhood gradient information of the feature points, a first affine invariant feature of the first feature point and a second affine invariant feature of the second feature point are extracted. Based on the Hamming distance between the first fast feature and the second fast feature, and the Euclidean distance between the first affine invariant feature and the second affine invariant feature, feature matching is performed on multiple first feature points and multiple second feature points to obtain the registration feature points of each feature point in its neighboring images.

[0041] Among them, the neighborhood grayscale information based on feature points refers to obtaining the first fast feature, namely the aforementioned 256-bit binary descriptor BRIEF feature, by randomly selecting several pairs of pixels around the feature point, comparing the grayscale values ​​of each pair of pixels, and encoding the comparison results into binary bits. The affine invariant feature (SIFT) of the feature point can then be calculated based on the neighborhood gradient information. Specifically, SIFT features are typically described using a 128-dimensional floating-point descriptor.

[0042] Building upon the aforementioned foundation, by calculating the similarity of the feature information of each feature point, we can obtain the feature distance between the feature points. It can be understood that the more similar the feature information of two feature points, i.e., the closer the feature distance between them, the more likely the two feature points are to be matched. To further improve matching accuracy, it is necessary to determine the registration feature points for each feature point in its neighboring images. These registration feature points include nearest neighbor registration feature points and second nearest neighbor registration feature points, which are the two feature points with the closest and second closest feature distances to that feature point.

[0043] Specifically, during the feature matching process, the similarity is calculated differently for different feature information. For example, in one possible implementation, the feature distance between feature points is determined based on the Hamming distance between the first fast feature and the second fast feature, as well as the Euclidean distance between the first affine invariant feature and the second affine invariant feature, thereby further obtaining the registration feature points for each feature point.

[0044] S320, determine the target feature point group from the feature points based on the feature distance between each feature point and its registered feature point.

[0045] To select more suitable registration feature point pairs, in one embodiment of this application, it is necessary to select the first target feature point and the second target feature point as nearest neighbor registration feature points from the features, and the first feature distance between the first target feature point and the nearest neighbor registration feature point and the second feature distance between the second target feature point and the next nearest neighbor registration feature point must meet preset conditions. Here, the first target feature point and the second target feature point being nearest neighbor registration feature points means that the nearest neighbor registration feature point of the first target feature point is the second target feature point in the adjacent image, and the nearest neighbor registration feature point of the second target feature point is the first target feature point in the adjacent image; that is, cross-validation is required. Furthermore, the first target feature point and the second target feature point need to satisfy a certain condition, such as exceeding a preset ratio threshold. Specifically, the ratio threshold here can be 1:0.75, meaning the second feature distance needs to be less than 75% of the first feature distance.

[0046] S330, determine the initial transformation matrix between the first target image and the second target image based on the target feature point group.

[0047] After determining the target feature point group that meets the requirements using the aforementioned scheme, the geometric transformation relationship between adjacent frames can be estimated based on the matched feature point pairs. Specifically, since the image is not yet distortion-corrected, the homography matrix can be used to describe the planar projection transformation. For example, in one embodiment, the robust RANSAC (Random Sample Consensus) algorithm can be used to estimate the homography matrix. Specifically, in the RANSAC algorithm, the inner threshold can be set to 3 pixels, and the iteration count can be 2000 for iterative processing.

[0048] Furthermore, for each pair of adjacent images, preliminary stitching can be performed after calculating the transformation matrix. For example, in one possible implementation, a bounding box of size equal to the merged size of the two images can be created, and inverse mapping can be used to avoid holes. For each pixel on the canvas, the corresponding position in the source image is found through inverse transformation. Then, in the overlapping region, a weighted average fusion solution is used, where the weight function can specifically be a linear transition or a sigmoid function.

[0049] S230, based on the registration error between feature points of each group of adjacent images after transformation by the transformation matrix, the initial transformation matrix is ​​globally optimized to obtain the target transformation matrix of each group of adjacent images.

[0050] Building upon the foregoing, after determining the transformation matrix between each group of adjacent images, in order to avoid error accumulation during the stitching process, in one embodiment of this application, the initial transformation matrix is ​​also globally optimized based on the registration error between feature points of adjacent images after transformation by the transformation matrix.

[0051] Specifically, in one embodiment of this application, the initial transformation matrix is ​​globally optimized, which can be achieved by constructing a graph optimization model, where each image is treated as a node and the transformation relationships between adjacent images are treated as edges. The energy function is defined as the sum of the registration errors of all edges, and the bundle adjustment algorithm is used to simultaneously optimize the positions of all images. At this point, the objective function E, which indicates the final goal of the global optimization, is defined to satisfy: E = Σ(i,j)∈edges ||p i -H ij *p j ||²where p i and p j H is the matched feature point. ij Let be the homography matrix of images i to j. The Levenberg-Marquardt algorithm (LM algorithm, a robust nonlinear least squares optimization algorithm) is used for iterative optimization. The calculation of the Jacobian matrix is ​​accelerated by utilizing sparsity, and the final convergence condition is set to an error change of less than 0.001 or 100 iterations. After completing the global optimization, the updated target transformation matrix is ​​obtained.

[0052] S240, transform each group of adjacent images based on the target transformation matrix and then stitch them together to obtain a stitched image of the target cable structure.

[0053] After optimizing the transformation matrix between adjacent images using the aforementioned global optimization algorithm, the globally optimized stitched image can be obtained. For details, please refer to [link to relevant documentation]. Figure 4a and Figure 4b The diagram shows the effect of stitching the images before and after global optimization. It can be seen that... Figure 4a The stitched image before global optimization shown has significant registration errors, and the final stitched image has accumulated to more than 20% of its width. Figure 4b The globally optimized stitched image shown significantly eliminates the registration error of images stitched together consecutively.

[0054] The solution provided in this application, by selecting feature point pairs that meet the requirements during image registration and stitching to determine the initial transformation matrix, and further iteratively updating the transformation matrix based on global optimization, can effectively eliminate the feature point matching error caused by cable structure vibration during image acquisition, thus preventing the accumulation of errors in the frame-by-frame stitching process. This allows the final stitched image to effectively present the panoramic performance of the cable structure within the multi-twist pitch.

[0055] S130, based on the centerline of the cable structure extracted from the edge recognition result of the stitched image, perform a geometric transformation on the stitched image to obtain a flat unfolded image of the target cable structure.

[0056] After stitching together multiple consecutive images using the aforementioned method, considering the barrel distortion that occurs on the surface of the cable structure, coupled with factors such as mechanical vibration and lens distortion, the image still exhibits complex geometric deformations, which severely affect subsequent defect detection. Therefore, in one embodiment of this application, a geometric transformation is provided based on the centerline of the cable structure extracted from the edge recognition results of the stitched image. This allows for more accurate defect detection of the target cable structure in the unfolded image after the geometric transformation.

[0057] Specifically, in one embodiment of this application, please refer to Figure 5 , Figure 5 This application provides a flowchart illustrating the steps for processing an image to extract the centerline of a cable structure, specifically including steps S510-S540: S510, perform binarization segmentation on the stitched image after grayscale processing to obtain the boundary information in the stitched image.

[0058] In one embodiment of this application, by performing binarization segmentation on the stitched image after grayscale processing, the boundary information in the stitched image can be effectively obtained. Specifically, the boundary information typically includes the boundaries of multiple steel cables in a cable structure. Furthermore, to improve the segmentation effect, in one embodiment of this application, the segmentation threshold used for binarization segmentation is determined based on the statistical results of pixel values ​​within the region where the pixel is located; that is, in this embodiment, an adaptive thresholding method is used to segment the panoramic image. Figure 2 The threshold is set to T = μ - k*σ, where μ is the local mean, σ is the standard deviation, and k = 0.2. Then, Canny edge detection is used to extract the upper and lower boundaries of the wire rope, for example, with a low threshold set to 50 and a high threshold set to 150, thus enabling the detection of major edge segments in the image via Hough transform.

[0059] S520, perform iterative erosion processing on the boundary information to obtain skeleton information with a preset pixel width.

[0060] In one embodiment of this application, based on the foregoing, it can be understood that the extracted boundary information usually includes the boundaries of multiple steel wire ropes in the cable structure. Due to factors such as distortion and vibration, abnormal deformation often occurs in some areas of the boundary. Therefore, the boundary information can be iteratively eroded until skeleton information with a preset pixel width is obtained. The skeleton information includes multiple skeleton points to indicate the center line of each steel wire rope in the cable structure.

[0061] S530, Based on the distance from the skeleton point to the boundary information, determine the target skeleton point from the skeleton points.

[0062] Building upon the aforementioned foundation, the ridges of the distance transformation, i.e., local maxima, are found based on the shortest distance from each skeleton point to the boundary. By connecting these target points using a thinning algorithm, a continuous skeleton can be formed. Furthermore, operations such as pruning can be performed on the skeleton to remove burrs and branches, while retaining the main trunk.

[0063] S540, perform dynamic path planning on the target skeleton points to obtain the target curve, and then perform curve fitting on the target curve to obtain the center line of the cable structure in the stitched image.

[0064] Based on the above, by performing dynamic path planning on the selected target skeleton points, the optimal path is defined as the weighted sum of curvature change and length. By using cubic splines to fit the skeleton points, a smooth target curve can be obtained. Specifically, this target curve is the curve used to indicate the center line of the cable structure in the stitched image.

[0065] After extracting the centerline of the cable structure using the aforementioned method, geometric changes in the image can be performed based on this centerline, thereby eliminating barrel distortion or other deformations in the edge area of ​​the wire rope in the image and obtaining the unfolded tiled image.

[0066] Specifically, geometric transformations of stitched images can be achieved using arc length parameterization and mapping tables. For details, please participate... Figure 6 , Figure 6 This diagram illustrates a step-by-step flowchart for performing geometric transformations on stitched images, specifically including steps S610-S620: S610, the center line is parameterized to obtain the cumulative arc length and normal vector of each pixel on the center line.

[0067] In one embodiment of this application, by parameterizing the arc length of the centerline, the cumulative arc length s(t) of each point on the centerline can be calculated. Furthermore, the tangent vector and normal vector of each point on the centerline can be calculated. The tangent vector can be obtained through local difference: T(t) = (P(t+δ) - P(t-δ)) / 2δ, while the normal vector can be obtained by rotating the tangent vector by 90 degrees, i.e., normal vector N(t) = (-Ty, Tx).

[0068] S620, based on the accumulated arc length and the normal vector, the pixels in the stitched image are remapped to perform a geometric transformation on the stitched image to obtain a flat unfolded image of the target cable structure.

[0069] Building upon the aforementioned foundation, a correction mapping table is created. The final output image has a width equal to the total arc length and a height 1.5 times the width of the steel wire rope. For each pixel (u, v) in the output image, the corresponding arc length position s = u is calculated. A point P(s) with arc length s and its normal vector N(s) are found on the center line. The source image coordinates are: (x, y) = P(s) + (v - height / 2) * scale * N(s). Using the remap function, a bicubic interpolation geometric transformation is performed on the image to obtain a geometrically regularized steel wire rope unfolded image.

[0070] S140, the unfolded image is inspected to obtain the multi-twist defect detection results of each cable in the target cable structure.

[0071] In the aforementioned scheme, after obtaining the unfolded image of the target cable structure, the detection model is used to detect the unfolded image, which yields the multi-twist defect detection results of each cable in the target cable structure.

[0072] Specifically, to further improve the detection performance of tiled images, one embodiment of this application also provides an implementation scheme based on a multi-scale convolutional block attention model to improve defect detection performance. For details, please refer to... Figure 7 , Figure 7 This application provides a schematic flowchart of a procedure for defect detection in a cable structure, specifically including steps S710 to S730: S710, the unfolded image is divided into blocks, and multi-scale feature information of each block is obtained through a feature extraction network.

[0073] In one embodiment of this application, multi-scale defect detection can be performed by deploying an improved YOLOv8 detection model on a tiled image. Since the corrected tiled image has a regular geometric structure, the detection accuracy can be significantly improved. Specifically, more scale layers can be added to the FPN (Feature Pyramid Network) to cover various defects ranging from micro-cracks to large-area corrosion. Furthermore, during the detection process, a sliding window can be used to divide the tiled image into blocks of 2048×2048 pixels, with a 256-pixel overlap, and then each window can be detected independently.

[0074] S720 uses nonmaximum suppression to remove duplicates from the multi-scale feature information of each block, thus obtaining the target feature information of each block.

[0075] Based on the independent detection of each window block as described above, the target feature information of each block can be obtained by setting NMS (non-maximum suppression) for deduplication.

[0076] S730, the target feature information is processed based on the convolutional block attention module to obtain the defect type and confidence level of each cable in the target cable structure, so as to determine the multi-twist defect detection result.

[0077] In one embodiment of this application, attention to defect features can be enhanced by integrating CBAM (convolutional block attention module), wherein deformable convolution can be used to adapt to the texture features of the wire rope.

[0078] Specifically, the defect types include at least one of wire breakage defects, wear defects, and corrosion defects. Different confidence levels can be set for different defect types to determine the final accurate result. For example, the confidence level for wire breakage can be set to 0.6, the confidence level for wear can be set to 0.6, and the confidence level for corrosion can be set to 0.5.

[0079] To clearly understand the detection effect achievable by the cable structure defect detection method provided in the embodiments of this application, please refer to... Figure 8 and Figure 9 , Figure 8 The image detection results of the cable structure within a single lay length achieved by related technologies show that although defects can be identified in a single frame image, the total number of broken wires in that section of the wire rope within multiple lay lengths cannot be directly calculated due to the unknown speed of the wire rope. Figure 9This shows the defect detection results of the cable structure within multiple lay lengths achieved by the detection method provided in this application. It can be seen that the solution provided in this application overcomes the problem of difficulty in identifying defects caused by traditional registration errors and structural vibrations, effectively controlling image distortion caused by the splicing process, and effectively achieving clear localization of defects in multi-lay wire ropes.

[0080] Of course, it should be noted that the above illustrations are still based on a series of images captured from a specific angle, and the final stitched image is only an image of a local surface of the wire rope within multiple lay lengths at that specific angle. For multiple sets of continuous images captured from multiple angles, the stitching method provided in this application can also be used, that is, by using global optimization, centerline recognition, and arc length parameterization to stitch together multiple sets of images captured from multiple angles. This will not be elaborated upon in the embodiments of this application.

[0081] To clearly understand the complete implementation scheme provided in the embodiments of this application, please refer to... Figure 10 , Figure 10 This is a complete flowchart illustrating a defect detection method for a cable structure provided in an embodiment of this application. It should be noted that the flowchart provided above is a feasible defect detection scheme and is not intended to limit the scope of this application. Modifications, substitutions, or deletions of some steps in the process based on the relevant descriptions provided in the foregoing embodiments should be within the scope of protection claimed in this application. Specifically, it includes the following steps: Step 1: High-speed image sequence acquisition A dedicated continuous image acquisition system was constructed, including a high frame rate camera. Acquisition parameters were set to ensure a 40-60% overlap between adjacent frames, which is crucial for successful subsequent stitching. The camera continuously acquired images of the wire rope surface at a fixed frame rate, with each image covering approximately 1-2 lay lengths.

[0082] Step 2: Image preprocessing and quality screening The acquired raw image sequence underwent preprocessing, including noise reduction, contrast enhancement, and color correction. Adaptive histogram equalization (CLAHE) was used to enhance local contrast, with a contrast limit of 2.0 and a grid size of 8×8. Image quality assessment was performed, removing blurry, overexposed, or underexposed images. The Laplacian variance of each image was calculated as a sharpness indicator, with a threshold of 100. The average brightness and standard deviation of the images were calculated to ensure they were within reasonable ranges (average brightness 100-200, standard deviation >20). For substandard images, interpolation was performed using preceding and following frames.

[0083] Step 3: Feature Extraction and Initial Matching Robust feature points are extracted from the original uncorrected image. Specifically, this invention uses an ORB feature detector to quickly extract feature points, setting the number of feature points to 2000, the number of pyramid layers to 8, and the scaling factor to 1.2. ORB features have rotation invariance and good real-time performance. SIFT features are also extracted as a supplement for accurate matching in textured regions. A 256-bit binary descriptor (ORB) or a 128-dimensional floating-point descriptor (SIFT) is calculated for each feature point. During feature matching, Hamming distance (ORB) or Euclidean distance (SIFT) is used for nearest neighbor search. A cross-validation strategy is employed, meaning that a valid match is considered only if the best match for feature point A in image 2 is B, and the best match for B in image 1 is also A. A ratio test is applied, with the distance ratio between the nearest and second nearest neighbors being less than 0.75.

[0084] Step 4: Adjacent frame registration and initial stitching Based on matched feature point pairs, the geometric transformation relationship between adjacent frames is estimated. Since the images are not yet distortion-corrected, the homography matrix H is used to describe the planar projection transformation. The RANSAC algorithm is used to robustly estimate the homography matrix, with an inlier threshold of 3 pixels and 2000 iterations. For each pair of adjacent wire rope images, the transformation matrix is ​​calculated before initial stitching. A temporary canvas is created, the size of which is the bounding box of the merged two images. Inverse mapping is used to avoid holes; for each pixel on the canvas, the corresponding position in the source image is found through inverse transformation. In overlapping regions, weighted average fusion is used, with either a linear transition or a sigmoid function as the weighting function.

[0085] Step 5: Global Registration Optimization After initial stitching, global optimization is performed to reduce accumulated errors. A graph optimization model is constructed, with each image as a node and the transformation relationships between adjacent images as edges. The energy function is defined as the sum of the registration errors of all edges. Bundle Adjustment is used to simultaneously optimize the positions of all images. The objective function is defined as: E = Σ(i,j)∈edges ||p i - H ij *p j ||²where p i and p j H is the matched feature point. ij This is the homography matrix of images i to j. The Levenberg-Marquardt algorithm is used for iterative optimization. Sparsity is utilized to accelerate the calculation of the Jacobian matrix. The convergence condition is set to an error variation of less than 0.001 or 100 iterations. After optimization, the panoramic image of the multi-twist steel wire rope is regenerated using the updated transformation matrix.

[0086] Step 6: Distortion detection and modeling of multi-ply wire rope First, the wire rope area is segmented. Then, an adaptive thresholding method is used to segment the panoramic view. Figure 2 Values ​​are assigned to a threshold T = μ - k*σ, where μ is the local mean, σ is the standard deviation, and k = 0.2. Internal voids are filled using morphological closing operations, employing 15×15 elliptical structuring elements. The largest connected component is extracted as the main body of the wire rope.

[0087] Then, the upper and lower boundaries of the wire rope were extracted. Canny edge detection was used with a low threshold of 50 and a high threshold of 150. Major edge segments were detected using Hough transform. The upper and lower boundary curves were fitted using RANSAC with a cubic polynomial model: y = ax³ + bx² + cx + d.

[0088] Step 7: Extraction based on the centerline of the skeleton The precise centerline is extracted from the segmented wire rope region, and the boundary is iteratively eroded using a morphological skeletonization algorithm until a skeleton with a single pixel width is obtained.

[0089] Perform a distance transform on the binary image to obtain the shortest distance from each pixel to the boundary. Find the ridges of the distance transform, i.e., the local maxima. Connect the ridge points using a thinning algorithm to form a continuous skeleton. Prune the skeleton to remove burrs and branches, retaining the main trunk. Optimize the path of the extracted skeleton. Use dynamic programming to find the optimal path from left to right. Define the path energy as a weighted sum of curvature change and length. Fit the skeleton points using cubic splines, with control point intervals of 20 pixels, to obtain a smooth centerline.

[0090] Step 8: Overall distortion correction of panoramic images The centerline is parameterized by arc length, and the cumulative arc length s(t) is calculated. For each point on the centerline, the tangent vector and normal vector are calculated. The tangent vector is obtained through local difference: T(t) = (P(t+δ) - P(t-δ)) / 2δ. The normal vector is the tangent vector rotated by 90 degrees: N(t) = (-Ty, Tx).

[0091] Create a correction mapping table. The output image width is the total arc length, and the height is 1.5 times the width of the wire rope. For each pixel (u, v) in the output image, calculate the corresponding arc length position s = u. Find the point P(s) with arc length s and its normal vector N(s) on the center line. The source image coordinates are: (x, y) = P(s) + (v - height / 2) * scale * N(s). Use the remap function to perform a bicubic interpolation geometric transformation on the image to obtain a geometrically regularized unfolded image of the wire rope.

[0092] Step 9: Multi-scale defect detection An improved YOLOv8 detection model was deployed on the corrected panoramic image for multi-scale defect detection. The corrected image has a regular geometric structure, which significantly improves the detection accuracy.

[0093] Specifically, more scale layers are added to the FPN (Feature Pyramid Network) to cover various defects ranging from micro-cracks to large-area corrosion. CBAM (Convolutional Block Attention Module) is integrated to enhance attention to defect features. Deformable convolutions are used to adapt to the texture features of the wire rope.

[0094] The panoramic image was divided into blocks using a sliding window, each block being 2048×2048 pixels with a 256-pixel overlap. Each window was inspected independently, and then deduplication was performed using NMS (Non-Maximum Suppression). Confidence thresholds were set for different defect types, such as 0.6 for broken wires, 0.6 for wear, and 0.5 for corrosion.

[0095] The cable structure defect detection method provided in this application acquires multiple images of the cable, including overlapping areas, and stitches these multiple consecutive images together based on the feature information of the overlapping areas to obtain a stitched image including the multi-twist pitch. After transforming the image based on the centerline, a flattened image can be obtained, thereby more accurately detecting multi-twist pitch defects in each cable and improving the accuracy of subsequent fault prediction of the cable structure.

[0096] Based on the aforementioned method for detecting defects in cable structures, this application also provides a device for detecting defects in cable structures. For details, please refer to [link to relevant documentation]. Figure 11 , Figure 11 This application provides a schematic diagram of a defect detection device for a cable structure, specifically including: The acquisition module 1110 is used to acquire multiple consecutive images of the target cable structure, wherein adjacent images in the multiple consecutive images include at least a partially overlapping area of ​​the target cable structure; The stitching module 1120 is used to stitch together the multiple consecutive images according to the feature information in the overlapping area to obtain a stitched image of the target cable structure. The transformation module 1130 is used to perform geometric transformation on the spliced ​​image based on the center line of the cable structure extracted from the edge recognition result of the spliced ​​image, so as to obtain a flat unfolded image of the target cable structure. The detection module 1140 is used to detect the unfolded image to obtain the multi-twist defect detection results of each cable in the target cable structure.

[0097] In one embodiment of this application, the stitching module 1120 is further configured to extract feature points in adjacent first target images and second target images in the multi-frame continuous images, the feature points including a plurality of first feature points on the first target image and a plurality of second feature points on the second target image; Feature matching is performed on the plurality of first feature points and the plurality of second feature points to obtain an initial transformation matrix between the first target image and the second target image; Based on the registration error between feature points of each group of adjacent images after transformation by the transformation matrix, the initial transformation matrix is ​​globally optimized to obtain the target transformation matrix of each group of adjacent images. The target transformation matrix is ​​used to transform each group of adjacent images before stitching them together to obtain a stitched image of the target cable structure.

[0098] In one embodiment of this application, the stitching module 1120 is further configured to perform feature matching on a plurality of first feature points and a plurality of second feature points based on the feature information of each feature point, to obtain registration feature points for each feature point in its neighboring images; the registration feature points include nearest neighbor registration feature points and second nearest neighbor registration feature points; Based on the feature distance between each feature point and its registered feature point, a target feature point group is determined from the feature points. In each target feature point group, the first target feature point and the second target feature point are the nearest neighbor registered feature points, and the first feature distance between the first target feature point and its nearest neighbor registered feature point and the second feature distance between its second nearest neighbor registered feature point satisfy a preset condition. The initial transformation matrix between the first target image and the second target image is determined based on the target feature point group.

[0099] In one embodiment of this application, the stitching module 1120 is further configured to extract a first fast feature of the first feature point and a second fast feature of the second feature point based on the neighborhood grayscale information of the feature points, and to extract a first affine invariant feature of the first feature point and a second affine invariant feature of the second feature point based on the neighborhood gradient information of the feature points. Based on the Hamming distance between the first fast feature and the second fast feature, and the Euclidean distance between the first affine invariant feature and the second affine invariant feature, feature matching is performed on multiple first feature points and multiple second feature points to obtain the registration feature points of each feature point in its neighboring images.

[0100] In one embodiment of this application, the transformation module 1130 is further configured to perform binarization segmentation on the stitched image after grayscale processing to obtain boundary information in the stitched image; the segmentation threshold used for binarization segmentation is determined based on the statistical results of pixel values ​​within the region where the pixel is located; The boundary information is subjected to iterative erosion processing to obtain skeleton information with a preset pixel width, the skeleton information including multiple skeleton points; Based on the distance from the skeleton point to the boundary information, a target skeleton point is determined from the skeleton points; Dynamic path planning is performed on the target skeleton points to obtain the target curve, and curve fitting is performed on the target curve to obtain the center line of the cable structure in the stitched image.

[0101] In one embodiment of this application, the transformation module 1130 is further configured to parameterize the arc length of the center line to obtain the cumulative arc length and normal vector of each pixel on the center line. The pixels in the stitched image are remapped based on the accumulated arc length and the normal vector to perform a geometric transformation on the stitched image, thereby obtaining a flat unfolded image of the target cable structure.

[0102] In one embodiment of this application, the detection module 1140 is further configured to divide the tiled image into blocks and obtain multi-scale feature information of each block through a feature extraction network; Based on nonmaximum suppression, the multi-scale feature information of each block is deduplicated to obtain the target feature information of each block; The target feature information is processed by the convolutional block attention module to obtain the defect type and confidence level of each cable in the target cable structure, so as to determine the multi-twist defect detection result. The defect type includes at least one of broken wire defects, wear defects and corrosion defects.

[0103] Specific limitations regarding the defect detection device for cable structures can be found in the limitations of the defect detection method for cable structures described above, and will not be repeated here. Each module in the aforementioned defect detection device for cable structures can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0104] In this embodiment, multiple images of the cable, including overlapping areas, are acquired and stitched together based on the feature information of the overlapping areas to obtain a stitched image including the multi-twist pitch. After transforming the image based on the centerline, a flat, unfolded image can be obtained, thereby more accurately detecting multi-twist pitch defects in each cable and improving the accuracy of subsequent fault prediction of the cable structure.

[0105] In some embodiments of this application, the defect detection method for cable structures can be implemented as a computer program, which can be implemented in, for example... Figure 12 The computer device shown is running the program. The computer device's memory can store the various program modules that make up the defect detection device for the cable structure, for example... Figure 11 The data acquisition module 1110, splicing module 1120, transformation module 1130, and detection module 1140 are shown. The computer program composed of these modules causes the processor to execute the steps in the cable structure defect detection methods of the various embodiments of this application described in this specification.

[0106] For example, Figure 12 The computer equipment shown can be used as follows Figure 11 The acquisition module 1110 in the cable structure defect detection device shown executes step S110. The computer device can execute step S120 via the splicing module 1120. The computer device can execute step S130 via the transformation module 1130. The computer device can execute step S140 via the detection module 1140. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with external computer devices via a network connection. When the computer program is executed by the processor, it implements a cable structure defect detection method.

[0107] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0108] In some embodiments of this application, a computer device is provided, including one or more processors; memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to perform the following steps: Acquire multiple consecutive images of the target cable structure, wherein adjacent images in the multiple consecutive images include at least a partially overlapping area of ​​the target cable structure; The multiple consecutive images are stitched together based on the feature information within the overlapping area to obtain a stitched image of the target cable structure; Based on the centerline of the cable structure extracted from the edge recognition results of the stitched image, a geometric transformation is performed on the stitched image to obtain a flat unfolded image of the target cable structure; The unfolded image is inspected to obtain the multi-twist defect detection results of each cable in the target cable structure.

[0109] In some embodiments of this application, a computer-readable storage medium is provided, storing a computer program that is loaded by a processor, causing the processor to perform the following steps: Acquire multiple consecutive images of the target cable structure, wherein adjacent images in the multiple consecutive images include at least a partially overlapping area of ​​the target cable structure; The multiple consecutive images are stitched together based on the feature information within the overlapping area to obtain a stitched image of the target cable structure; Based on the centerline of the cable structure extracted from the edge recognition results of the stitched image, a geometric transformation is performed on the stitched image to obtain a flat unfolded image of the target cable structure; The unfolded image is inspected to obtain the multi-twist defect detection results of each cable in the target cable structure.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0111] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above provides a detailed description of a defect detection method, apparatus, device, and storage medium for cable structures provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting defects in cable structures, characterized in that, include: Acquire multiple consecutive images of the target cable structure, wherein adjacent images in the multiple consecutive images include at least a partially overlapping area of ​​the target cable structure; The multiple consecutive images are stitched together based on the feature information within the overlapping area to obtain a stitched image of the target cable structure; Based on the centerline of the cable structure extracted from the edge recognition results of the stitched image, a geometric transformation is performed on the stitched image to obtain a flat unfolded image of the target cable structure; The unfolded image is inspected to obtain the multi-twist defect detection results of each cable in the target cable structure.

2. The method according to claim 1, characterized in that, The step of stitching together the multiple consecutive images based on feature information within the overlapping region to obtain a stitched image of the target cable structure includes: For adjacent first target images and second target images in the multiple consecutive images, feature points are extracted from the images. The feature points include multiple first feature points on the first target image and multiple second feature points on the second target image. Feature matching is performed on the plurality of first feature points and the plurality of second feature points to obtain an initial transformation matrix between the first target image and the second target image; Based on the registration error between feature points of each group of adjacent images after transformation by the transformation matrix, the initial transformation matrix is ​​globally optimized to obtain the target transformation matrix of each group of adjacent images. The target transformation matrix is ​​used to transform each group of adjacent images before stitching them together to obtain a stitched image of the target cable structure.

3. The method according to claim 2, characterized in that, The step of performing feature matching on the plurality of first feature points and the plurality of second feature points to obtain an initial transformation matrix between the first target image and the second target image includes: Based on the feature information of each feature point, feature matching is performed on multiple first feature points and multiple second feature points to obtain the registration feature points of each feature point in its neighboring images; the registration feature points include nearest neighbor registration feature points and second nearest neighbor registration feature points. Based on the feature distance between each feature point and its registered feature point, a target feature point group is determined from the feature points. In each target feature point group, the first target feature point and the second target feature point are the nearest neighbor registered feature points, and the first feature distance between the first target feature point and its nearest neighbor registered feature point and the second feature distance between its second nearest neighbor registered feature point satisfy a preset condition. The initial transformation matrix between the first target image and the second target image is determined based on the target feature point group.

4. The method according to claim 3, characterized in that, The step of performing feature matching on multiple first feature points and multiple second feature points based on the feature information of each feature point to obtain the registration feature points of each feature point in its neighboring images includes: Based on the neighborhood grayscale information of the feature points, a first fast feature of the first feature point and a second fast feature of the second feature point are extracted; and based on the neighborhood gradient information of the feature points, a first affine invariant feature of the first feature point and a second affine invariant feature of the second feature point are extracted. Based on the Hamming distance between the first fast feature and the second fast feature, and the Euclidean distance between the first affine invariant feature and the second affine invariant feature, feature matching is performed on multiple first feature points and multiple second feature points to obtain the registration feature points of each feature point in its neighboring images.

5. The method according to claim 1, characterized in that, The method further includes: The stitched image after grayscale processing is binarized to obtain the boundary information in the stitched image; the segmentation threshold used for binarization is determined based on the statistical results of pixel values ​​within the region where the pixel is located; The boundary information is subjected to iterative erosion processing to obtain skeleton information with a preset pixel width, the skeleton information including multiple skeleton points; Based on the distance from the skeleton point to the boundary information, a target skeleton point is determined from the skeleton points; Dynamic path planning is performed on the target skeleton points to obtain the target curve, and curve fitting is performed on the target curve to obtain the center line of the cable structure in the stitched image.

6. The method according to claim 1, characterized in that, The step of performing a geometric transformation on the stitched image based on the centerline of the cable structure extracted from the edge recognition results of the stitched image to obtain a flat, unfolded image of the target cable structure includes: The centerline is parameterized by arc length to obtain the cumulative arc length and normal vector of each pixel on the centerline. The pixels in the stitched image are remapped based on the accumulated arc length and the normal vector to perform a geometric transformation on the stitched image, thereby obtaining a flat unfolded image of the target cable structure.

7. The method according to any one of claims 1 to 6, characterized in that, The step of detecting the unfolded image to obtain the multi-twist defect detection results of each cable in the target cable structure includes: The unfolded image is divided into blocks, and multi-scale feature information of each block is obtained through a feature extraction network. Based on nonmaximum suppression, the multi-scale feature information of each block is deduplicated to obtain the target feature information of each block; The target feature information is processed by the convolutional block attention module to obtain the defect type and confidence level of each cable in the target cable structure, so as to determine the multi-twist defect detection result. The defect type includes at least one of broken wire defects, wear defects and corrosion defects.

8. A defect detection device for cable structures, characterized in that, include: The acquisition module is used to acquire multiple consecutive images of the target cable structure, wherein adjacent images in the multiple consecutive images include at least a partially overlapping area of ​​the target cable structure; The stitching module is used to stitch together the multiple consecutive images based on the feature information in the overlapping area to obtain a stitched image of the target cable structure; The transformation module is used to perform geometric transformation on the spliced ​​image based on the center line of the cable structure extracted from the edge recognition result of the spliced ​​image, so as to obtain a flat unfolded image of the target cable structure; The detection module is used to detect the unfolded image and obtain the multi-twist defect detection results of each cable in the target cable structure.

9. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the cable structure defect detection method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which is loaded by a processor to execute the cable structure defect detection method according to any one of claims 1 to 7.

Images