Adaptive enhancement defect detection method for passenger cableway steel wire rope surface image

Abstract

The application discloses a self-adaptive image enhancement method for passenger cableway steel wire rope defect detection, and relates to the technical field of machine vision defect detection. The application constructs a surface reflection decoupling model fusing spiral geometry prior of the steel wire rope, introduces a light flow weight auxiliary separation of environmental light, oil stain reflection and inherent texture component of the steel wire rope in combination with time sequence correlation of the image sequence, completes fidelity texture enhancement by relying on scene self-adaptive modulation vector driving dynamic convolution, simultaneously introduces a direction field consistency constraint mechanism to protect defect gradient features, and sets an adaptive balance coefficient to dynamically control the constraint loss weight. The application can inhibit various image interferences under complex working conditions, avoid distortion of small defect profiles, improve defect recognition stability, and is suitable for online visual detection scenes of the cableway steel wire rope with variable light and oil stain adhesion.

Description

Technical Field

[0001] This invention relates to the field of machine vision defect detection technology, specifically to an adaptive enhancement defect detection method for surface images of passenger cableway wire ropes. Background Technology

[0002] The steel wire rope of a passenger ropeway is a core load-bearing component of manned special equipment. It is constantly exposed to the open mountain environment, subjected to drastic changes in natural light, mountain shadows, rainwater, oil stains, and dust. Traditional inspection methods mainly rely on manual high-altitude visual inspection and offline electromagnetic flux leakage (EML) testing. Manual inspection is high-risk, and the results are highly subjective, easily missing minor wire breaks and early superficial corrosion. EML testing focuses on identifying the internal metal cross-sectional area loss of the wire rope, making it difficult to precisely distinguish various surface defects and unable to achieve continuous, all-weather online visual monitoring. In recent years, machine vision technology has been gradually applied to wire rope damage identification, but existing solutions are mostly developed for mine hoists and elevator indoor scenarios, making it difficult to adapt to the unique and complex imaging conditions of passenger ropeways. There is an urgent need for targeted algorithm solutions to improve the stability of defect detection in open-air environments.

[0003] Xia H et al. proposed a visualization and quantification approach for steel wire rope broken wires based on induction thermal imaging in their paper "Visualization and Quantification of Broken Wires in Steel Wire Ropes Based on Induction Thermography" (IEEE Sensors Journal, 2021). This approach utilizes morphological image segmentation to suppress background interference caused by uneven heating and extracts pixel features from the broken wire area to achieve quantitative damage statistics. This scheme verifies the feasibility of using image preprocessing combined with deep learning to identify steel wire rope defects. However, the entire method relies on active infrared thermal imaging equipment, resulting in high equipment deployment costs and making it difficult to directly adapt to large-scale, long-distance online inspection scenarios for passenger ropeways. Furthermore, its image decomposition approach simply distinguishes between foreground and background components, failing to consider the periodic texture interference caused by the spiral twisting structure of the steel wire rope. It also fails to utilize the spatiotemporal correlation information of continuous image sequences to separate multiple interferences such as natural light illumination and surface oil reflection. The use of fixed-parameter image enhancement strategies easily leads to over-enhancement or loss of detail in local areas. In open-air mountain scenes with continuously fluctuating lighting, early-stage defect features at minute scales are easily submerged by texture noise, resulting in significant fluctuations in detection accuracy.

[0004] Chinese invention patent application (publication number CN107290349A) discloses a detection device, system, and method for monitoring the safety status and alarm marking of cableway wire ropes. It establishes an image acquisition hardware architecture consisting of a camera and auxiliary light source, combined with a speed sensor to achieve synchronous image acquisition of the wire rope. The system extracts the wire rope region and identifies defects through a preset threshold segmentation method. This solution is the first to implement a visual inspection architecture in a passenger cableway scenario, but it has significant limitations: the entire image processing flow lacks a physical-level image component decoupling mechanism, making it impossible to effectively separate ambient light, oil reflections, and the inherent texture of the wire rope; the image enhancement module uses fixed-parameter filtering, failing to adaptively adjust the enhancement strategy based on real-time environmental conditions; enhancement processing and defect detection are independent, lacking a closed-loop collaborative training mechanism, making it prone to distorting the original contour of defects during enhancement, and difficult to distinguish between the natural texture of the rope strands and genuine minor damage. Furthermore, it suffers from significant false positives and false negatives when dealing with low-quality images acquired under varying lighting conditions in the field. Summary of the Invention

[0005] Based on the aforementioned technical problems, this application discloses an adaptive enhancement defect detection method for surface images of passenger ropeway wire ropes, comprising the following steps:

[0006] Acquire a continuous image sequence of the surface of the passenger cableway wire rope and synchronously collect environmental condition parameters;

[0007] A surface reflection decoupling model integrating the prior geometry of the steel wire rope spiral is constructed. Based on the environmental state parameters and the spatiotemporal correlation of the continuous image sequence, the ambient light component, oil stain reflection component and inherent texture component are separated from the original image.

[0008] A scene adaptive modulation vector is generated using the ambient light component and the oil stain reflection component. The dynamic convolution kernel is then weighted using the scene adaptive modulation vector. Fidelity enhancement is performed on the inherent texture component to obtain an enhanced steel wire rope surface image.

[0009] The enhanced image of the wire rope surface is input into the defect detection network, and the difference in orientation field features of the defect region before and after enhancement is calculated.

[0010] A joint loss function containing orientation field consistency constraints and detection loss is constructed to train the enhancement network and the detection network end-to-end. The orientation field consistency constraints are configured such that when the enhancement causes the principal gradient direction of the defect region to shift beyond a preset angle threshold, the enhancement intensity at the corresponding spatial location is suppressed in reverse to maintain the axial or circumferential morphological characteristics of the wire rope defect.

[0011] Preferably, a surface reflection decoupling model integrating the prior geometric geometry of the steel wire rope is constructed, specifically including:

[0012] Based on the lay pitch and diameter parameters of the wire rope, a helical texture mask is constructed. ;

[0013] Original image Decomposed into ambient light components Oil stain reflectivity and inherent texture components The decomposition process satisfies the following constraint equations:

[0014]

[0015] in, This represents element-wise multiplication. For residual terms;

[0016] Each component is solved by minimizing the following energy function:

[0017] in, This is a regularization term for illumination smoothing. For the low-rank constraint term of oil stain reflection, For consistency regularization of the spiral structure of the inherent texture, These are the corresponding weighting coefficients.

[0018] Preferably, the helical structure consistency regularization term The formula is:

[0019] in, For intrinsic texture components in pixels The gradient vector at that point, This is the unit vector representing the theoretical tangent direction of the wire rope strand at this pixel, calculated based on helical geometry priors. This indicates the transpose operation.

[0020] Preferably, generating a scene adaptive modulation vector using the ambient light component and the oil stain reflection component specifically includes:

[0021] Ambient light component Global average pooling and logarithmic transformation are performed to obtain the illumination feature vector. ;

[0022] Reflective component of oil stains Channel attention extraction is performed to obtain the oil stain feature vector. ;

[0023] The synchronously collected environmental state parameters are encoded into environmental feature vectors. ;

[0024] Will , and The concatenated vectors are input into a multilayer perceptron to generate scene-adaptive modulation vectors. :

[0025]

[0026] in, For learnable weight matrix, For bias terms, It is the Sigmoid activation function. This indicates a vector concatenation operation.

[0027] Preferably, the dynamic convolution kernel is weighted by the scene-adaptive modulation vector to perform fidelity enhancement on the intrinsic texture components, specifically including:

[0028] Define the basic set of convolution kernels ;

[0029] Using scene-adaptive modulation vectors generate Individual Core Combination Weight :

[0030]

[0031] in, For the first The learnable weight vectors corresponding to each basic convolutional kernel;

[0032] Generate dynamic convolution kernels and the inherent texture components Perform convolution enhancement:

[0033]

[0034]

[0035] in, This represents a two-dimensional convolution operation. This refers to the enhanced intrinsic texture components.

[0036] Preferably, the difference in directional field characteristics of the defect region before and after enhancement is calculated, specifically including:

[0037] For the original texture components before enhancement Images of the reinforced wire rope surface Calculate the gradient tensor fields separately;

[0038] Within the defect bounding box region output by the defect detection network, calculate the principal gradient direction angles before and after enhancement. and ;

[0039] Calculate the difference in directional field characteristics :

[0040]

[0041] in, The set of pixels within the defect bounding box. Its number of pixels.

[0042] Preferably, the orientation field consistency constraint term The formula is:

[0043]

[0044] This represents the number of defective samples in the current batch. For the first Differences in the directional field characteristics of individual defect samples For the preset angle threshold, For the defect detection network to the first The detection confidence level of a defective sample.

[0045] Preferably, the spatiotemporal correlation of consecutive image sequences is used to assist in separating the reflective component of oil stains, specifically including:

[0046] Optical flow estimation is performed on adjacent frames in a continuous image sequence to obtain the motion field. ;

[0047] Based on the uniform motion model of the steel wire rope, the theoretical motion field is calculated. ;

[0048] The sports field deviation was used as the temporal differentiation weight of the oil stain reflectivity component. :

[0049]

[0050] in, This is the tolerance parameter for motion deviation;

[0051] In the process of solving the energy function, As a component of oil stains' reflectivity The pixel-level weighting factor.

[0052] Preferably, the joint loss function The expression is:

[0053]

[0054] in, The detection loss of the defect detection network. For orientation field consistency constraints, For image reconstruction loss, and This is the balance coefficient;

[0055] The balance coefficient An adaptive adjustment strategy is adopted, and its update rule is as follows:

[0056]

[0057] in, For the first The balance coefficient of the next iteration To adjust the step size, For indicator functions, For the most recent In the next iteration The moving average value automatically increases when the moving average value does not decrease. To strengthen the directional field constraint.

[0058] Compared with the prior art, the technical solution of this application has the following technical effects:

[0059] This invention constructs a surface reflection decoupling model based on the helical geometry of steel wire rope. It can combine the spatiotemporal correlation characteristics of continuous images with environmental state parameters to achieve multi-component image separation, effectively distinguish between ambient light and oil stain reflections and the inherent texture information of steel wire rope, clarify different types of signal components in the original image, provide clean and effective basic image data for subsequent image enhancement operations, and reduce the continuous transmission of various irrelevant interference information to the downstream processing flow.

[0060] This invention generates a scene-adaptive modulation vector by jointly generating the illumination component and the oil stain reflection component. It uses the modulation vector to dynamically adjust the weight of the convolution kernel to complete the fidelity enhancement of the inherent texture component. It can autonomously adjust the enhancement execution strategy according to the local scene features of the image, and maintain the integrity of the steel wire rope's texture structure while strengthening the expression of potential defect features, thus avoiding the image feature distortion problem caused by the enhancement operation.

[0061] This invention introduces a calculation mechanism for the difference in orientation field features in the defect region before and after enhancement, and constructs an orientation field consistency constraint term to participate in the joint loss function calculation. During the training phase, it can constrain the enhancement process to maintain the stability of the gradient orientation features of the defect region, ensuring that the axial and circumferential morphology of various types of damage to the wire rope will not be distorted in the image enhancement process, and retaining the original morphological features of the defect for subsequent identification and processing.

[0062] This invention employs a joint loss function with multiple constraint terms to achieve end-to-end collaborative training of the enhancement network and the detection network, and sets an adaptive adjustment mechanism for the loss balance coefficient, enabling the network parameters to be synchronously iteratively optimized. This establishes the link between image enhancement and defect recognition, improves the adaptability of the entire detection process to outdoor multi-modal imaging conditions, and ensures the continuous and stable operation of the defect recognition process.

[0063] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0064] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0066] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0067] Figure 1 A complete flowchart of the overall algorithm for an adaptive enhancement defect detection method for passenger cableway wire ropes;

[0068] Figure 2 Schematic diagram of the internal structure for surface component extraction, modulation vector generation, and dynamic convolution for fidelity enhancement.

[0069] Figure 3 A comparison of the output effects of various existing image enhancement algorithms with the wire rope image enhancement method of this application;

[0070] Figure 4 A graph showing the trend of training iterations and validation set mAP@0.5 for different comparison algorithms;

[0071] Figure 5 A visualization comparison of the confidence levels of six ablation experiments targeting three typical types of wire rope defects in the detection frame.

[0072] Figure 6 The curves showing the synchronous changes of the mean of the downward field loss and the adaptive balance coefficient β during the training iterations are shown. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0074] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0075] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0076] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0077] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0078] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0079] Example 1 mainly describes an adaptive enhancement defect detection method for surface images of passenger cableway wire ropes, such as... Figure 1 As shown, it includes the following steps:

[0080] Acquire a continuous image sequence of the surface of the passenger cableway wire rope and synchronously collect environmental condition parameters;

[0081] A surface reflection decoupling model integrating the prior geometry of the steel wire rope spiral is constructed. Based on the environmental state parameters and the spatiotemporal correlation of the continuous image sequence, the ambient light component, oil stain reflection component and inherent texture component are separated from the original image.

[0082] A scene adaptive modulation vector is generated using the ambient light component and the oil stain reflection component. The dynamic convolution kernel is then weighted using the scene adaptive modulation vector. Fidelity enhancement is performed on the inherent texture component to obtain an enhanced steel wire rope surface image.

[0083] The enhanced image of the wire rope surface is input into the defect detection network, and the difference in orientation field features of the defect region before and after enhancement is calculated.

[0084] A joint loss function containing orientation field consistency constraints and detection loss is constructed to train the enhancement network and the detection network end-to-end. The orientation field consistency constraints are configured such that when the enhancement causes the principal gradient direction of the defect region to shift beyond a preset angle threshold, the enhancement intensity at the corresponding spatial location is suppressed in reverse to maintain the axial or circumferential morphological characteristics of the wire rope defect.

[0085] The entire computational process is built upon time-aligned multi-source data acquisition. The image acquisition unit continuously acquires two-dimensional images of the surface of the passenger cableway's steel wire rope at a fixed frame rate, forming a continuous image sequence with strong temporal correlation characteristics. Each image sequence contains no fewer than 90 temporally continuous images, and all images are uniformly scaled to a resolution of 640×640 pixels for standardized preprocessing. The environmental sensing acquisition unit synchronously outputs three environmental state parameters: ambient light intensity, ambient air humidity, and ambient temperature. The acquisition cycle is completely synchronized with the image acquisition cycle. Each frame of image is bound to the corresponding environmental state parameter through a unified timestamp. All parameters are normalized and mapped to the interval [0,1] before being input into the downstream computation module. The normalized environmental state parameters are mapped through a shallow fully connected encoding module to generate a fixed-dimensional environmental feature vector. Environmental state parameters do not directly participate in image pixel-level calculations, but rather serve as boundary constraints in the model solution process. They are used to limit the upper and lower bounds of the values ​​during the iterative solution of illumination components, thereby narrowing the optimization solution space and reducing the risk of component divergence caused by random illumination disturbances in open-air mountain scenes.

[0086] A surface reflection decoupling model incorporating prior geometric knowledge of the wire rope helix is ​​responsible for performing layered decomposition calculations of the original image imaging signal. During model initialization, the fixed geometric parameters of the wire rope at the passenger cableway site are imported, including a nominal rope diameter of 30mm, a single-layer strand lay distance of 135mm, and a strand helix lay angle of 18.5°. Based on these geometric parameters, a mapping relationship is established between the pixel coordinates of the two-dimensional image and the strand orientation in three-dimensional space. Based on this mapping relationship, each pixel is used to determine whether it is located in a strand protrusion area or a rope valley depression area, generating a binary spiral texture mask. A mask value of 1 indicates that the pixel is located in the body region of the strands, while a mask value of 0 indicates that the pixel is located in the valley region between the strands. The original image imaging follows a layered reflection model, and the pixel grayscale satisfies the constraint equation:

[0087]

[0088] In the formula, pixel coordinates The corresponding original image grayscale values, with the grayscale value range set to... ; This represents the ambient light component corresponding to the pixel location, characterizing the spatial distribution level of natural light; It is the inherent texture component of the steel wire rope, which carries the stable texture information and original defect characteristics of the strand itself; Represents the reflective component of oil stains, characterizing the instantaneous specular reflection signal formed by rainwater, dust, and oil stains; Operators Represents pixel-wise Hadamard product operation; residual term Used to accommodate quantization noise and motion blur noise from image sensors. The values ​​are stably distributed in the interval The model achieves simultaneous solution for the three types of components by iteratively minimizing the global energy function. The complete expression of the energy function is:

[0089]

[0090] The energy function consists of four independent sub-terms. The first term is the image reconstruction error term, which ensures that the original image information can be restored to the maximum extent after the components are superimposed; the second term... The third term is a light smoothing regularization term that constrains the spatial continuity of the light components and suppresses local irregular jumps in the light components; The nuclear norm, acting as a low-rank regularization constraint, is applied to the oil stain reflectivity component to adapt to the large-area, slowly varying distribution characteristics of oil stain reflectivity; weighting coefficients , , To enable configurable hyperparameters, the base values ​​are set to 0.22, 0.45, and 1.65, respectively, allowing for minor adjustments based on samples collected from different cableway sites. The regularization term for the inherent texture spiral structure consistency is written as follows:

[0091]

[0092] In the formula For intrinsic texture components in pixels The resulting two-dimensional gradient vector contains a horizontal gradient component. With vertical gradient components ; The symbol represents the unit vector of the tangent direction of the strand, pre-calculated based on the helical geometry prior. The vector direction strictly matches the extension direction of the strand at the current pixel position. This represents the vector transpose operation. This regularization term continuously constrains the gradient direction within the inherent texture to conform to the theoretical arrangement direction of the strands, thereby distinguishing the periodic inherent texture gradient of the wire rope from the abnormal gradient signals generated by broken wires, wear, and corrosion, and avoiding continuous interference from inherent texture signals in defect feature extraction.

[0093] Stable temporal motion correlations exist between adjacent frames within a continuous image sequence. This characteristic is incorporated to assist in optimizing the solution accuracy of the oil stain reflectivity component. Dense optical flow estimation is performed on consecutive images with a temporal interval of 3 frames, and the two-dimensional motion field is obtained pixel by pixel. A motion model was established based on the constraint of the uniform and continuous movement of the steel wire rope, with the axial speed of the steel wire rope fixed at 0.75 m / s. This model was then used to solve for the theoretical motion field corresponding to each pixel under conditions of no oil contamination and no reflective interference. By comparing the vector differences between the actual and theoretical sports fields, pixel-level temporal discrimination weights are constructed. The calculation formula is:

[0094]

[0095] parameter This represents the tolerance for motion deviation, with a base value set at 3.2. This represents the square operation of the L2 norm of a two-dimensional vector. As the pixel position motion deviation increases, the weights... A continuously decreasing value indicates that the signal at that location is likely a transient oil stain reflection; the weight value in areas with small motion deviations approaches 1, corresponding to a stable signal from the wire rope itself. The oil stain reflection component is solved iteratively using the energy function. Throughout the process, As a pixel-level weighting factor applied to the oil stain component-related sub-items, it achieves differentiated constraints at different pixel positions, enhancing the separation effect between temporally stable textures and instantaneous reflective interference.

[0096] like Figure 2 As shown, the ambient light component is completed. Oil stain reflectivity Inherent texture components After separating the three image components, feature extraction operations are performed on each of the three components according to the architecture process. This includes the two-dimensional ambient lighting component. Global average pooling is performed to compress the 640×640 two-dimensional feature matrix into a one-dimensional vector of length 256. Then, a logarithmic transformation is applied to compress the dynamic range of the values, generating the illumination feature vector. Targeting the reflective properties of oil stains A lightweight channel attention aggregation module is introduced to perform weighted fusion of all spatial location feature information, outputting a 256-dimensional oil stain feature vector. The normalized environmental state parameters are input into a shallow fully connected encoding module, which maps them to generate a 256-dimensional environmental feature vector. Perform vector concatenation on the three sets of equal-dimensional one-dimensional vectors to obtain a fused feature vector with dimension 768. The feature vector input is fused into a two-layer multilayer perceptron network to complete the nonlinear mapping and generate a scene-adaptive modulation vector. The complete mapping formula is as follows:

[0097]

[0098] In the formula This represents the weight matrix and bias vector of the first-layer perceptron FC1. ; This represents the weight matrix and bias vector of the second-layer perceptron FC2. ; As a nonlinear activation function in the intermediate layer; This represents the Sigmoid activation function, ensuring that all elements within the modulation vector fall within the specified range. ;symbol The vectors are concatenated along the channel dimension to output the scene-adaptive modulation vector. .

[0099] The scene-adaptive modulation vector further drives the real-time generation of dynamic convolution kernels, and the system pre-initializes the number of... A set of basic convolutional kernels with a size of 5×5 constitutes the basic convolutional kernel set. All basic convolutional kernel parameters participate in the entire network training and optimization process. An independent, learnable weight vector is assigned to each set of basic convolutional kernels. Adaptive modulation vector for the scene With weight vector Perform inner product operation The calculation results are input into the Softmax function for normalization, yielding the linear combination weights corresponding to each set of basic convolutional kernels. The calculation formula is expressed as follows:

[0100]

[0101] Normalization ensures that all weight coefficients satisfy the constraints. Based on the weight coefficients, all basic convolutional kernels are linearly weighted and summed to generate dynamic 5×5 convolutional kernels adapted to the current scene in real time. :

[0102]

[0103] Call the generated dynamic convolution kernel to the intrinsic texture components Perform a standard two-dimensional discrete convolution operation with a stride of 1 and a padding size of 2 to obtain the intrinsic texture components after fidelity enhancement. :

[0104]

[0105] Operators This refers to a two-dimensional discrete convolution operation. After the texture enhancement operation is completed, the enhanced texture component and the ambient lighting component are multiplied and fused at the pixel level to reconstruct a complete enhanced image of the wire rope surface. The reconstruction formula is:

[0106]

[0107] The reconstructed surface image of the reinforced steel wire rope is uniformly input into the downstream defect detection network to perform target recognition calculations.

[0108] After completing forward inference, the defect detection network outputs several bounding boxes of defect targets. The pixel set of the defect region is determined by the coordinates of the bounding boxes. The total number of pixels contained within the set is denoted as During the computation, the original intrinsic texture components before enhancement are read separately. Image reconstruction after enhancement For all pixel information within the defect area, gradient matrix eigenvalue decomposition is performed on the local neighborhood of each pixel. The direction of the eigenvector corresponding to the largest eigenvalue is taken as the principal gradient direction of that pixel, and the principal gradient direction angle before enhancement is obtained by solving. Enhanced principal gradient direction angle Considering the periodicity of angles, the minimum arc difference is selected to characterize the degree of directional offset. After calculating the deviation pixel by pixel, the mean value of the region is obtained to obtain the difference in directional field characteristics of the defect region. :

[0109]

[0110] Construct orientation field consistency constraints based on the differences in orientation field characteristics of single-defect samples. The complete calculation formula under the batch dimension is written as follows:

[0111]

[0112] Fixed training batch size setting , Representing the first batch Numerical differences in directional field features corresponding to each defect sample; angle threshold Preset value ; This represents the confidence score of the defect target output by the defect detection network, with a value range of [range missing]. When the average directional offset of the defect region exceeds the threshold, the constraint term generates an effective loss. At the same time, defect samples with low confidence have higher loss weights, forcing the network to prioritize the morphological stability of potential defects with low confidence.

[0113] By fusing the orientation field consistency constraint, defect detection loss, and image reconstruction loss, a joint loss function is constructed for the entire network architecture:

[0114]

[0115] In the formula The defect detection network has a built-in joint loss for classification and localization; To enhance the image reconstruction loss between the image and the original texture components; balance coefficient The initial value is 0.75, and the balance coefficient is... The value is fixed at 0.30. Balance coefficient. An adaptive iterative update mechanism is introduced to dynamically adjust the weight strength of the orientation field constraints with each training round. The update rule expression is as follows:

[0116]

[0117] In the formula Representing the The balance coefficient used in the next iteration; adjusting the step size The fixed value is 0.08; This is an indicator function; it outputs the value 1 if the condition is true and the value 0 if the condition is false. Represents the most recent consecutive The moving average of the orientation field constraint loss during each iteration. During the training phase, the joint loss function is used as the global optimization objective. The backpropagation algorithm simultaneously updates all convolutional parameters of the enhancement network, perceptron parameters, and all weight parameters of the defect detection network, achieving end-to-end collaborative optimization between the two networks. Within the network's forward inference process, the spatial distribution information of the orientation field consistency constraint term is read in real time, generating a spatial suppression mask of the same size. This mask is multiplied pixel-by-pixel with the feature map output by the enhancement network. For pixel regions where the gradient direction shows a significant shift, the enhancement amplitude is automatically reduced, continuously ensuring that the axial and circumferential geometry of various defects in the wire rope does not become distorted during image enhancement.

[0118] The entire training process uses an adaptive moment estimation optimizer, with the initial base learning rate set to... The learning rate is decayed every 4500 iterations, with a decay coefficient set to 0.82. Before all images are input into the network, data standardization is performed with random horizontal flipping and random brightness perturbation. The perturbation range is strictly controlled within a reasonable range to ensure that the distribution of training samples closely resembles the features of real images collected on-site at the passenger cableway. After the entire network completes training and convergence, it is deployed to an online detection system, continuously reading continuous image sequences and performing component decoupling, scene adaptive fidelity enhancement, and complete defect detection operations layer by layer, outputting the surface defect location results of the wire rope.

[0119] This embodiment can effectively separate interference information such as lighting and oil stain reflection in complex scenes, restore the real texture of the steel wire rope based on geometric priors and temporal constraints, achieve scene adaptive enhancement with the help of dynamic convolution, and protect the defect contour from distortion through orientation field constraints, significantly reducing the probability of missing small defects and improving the recognition stability and adaptability of the cableway steel wire rope visual inspection system under changing working conditions.

[0120] Example 2 uses the publicly available open-source dataset WireRopeDataset for simulation verification. The dataset is licensed under the CCBY 4.0 open-source license and is freely available for download and use by all researchers. This dataset uses an industrial camera to capture images of wire ropes under different lighting intensities, shooting angles, with oil contamination on the surface, and containing defects such as broken wires, surface wear, and corrosion. The original dataset contains 2742 images. All images were uniformly scaled to 640×640 pixels for standardization preprocessing, and a Labelme format annotation file is provided. The annotation categories include three typical defects: BS (broken wire), WR (surface wear), and CR (corrosion). The dataset is randomly divided into training, validation, and test sets in a 7:2:1 ratio, resulting in 1919 training images, 549 validation images, and 274 test images.

[0121] The training optimizer uses AdamW, with the base learning rate set... The batch size was set to B=12, the maximum number of iterations was set to 320, and an early stopping strategy was enabled during training. The iteration was terminated if the validation set metric showed no improvement for 35 consecutive iterations. The unified evaluation metric used in the experiment was mAP@0.5, representing the mean precision at an intersection-over-union (IoU) threshold of 0.5; P represented precision; R represented recall; F1 score was the combined precision and recall score; and FPS represented the number of frames per second. All metrics were calculated completely on independent test sets.

[0122] Baseline solutions widely used in the field of wire rope visual inspection were selected as control models. The control solutions were RetinexNet+YOLOv8s, EnlightenGAN+YOLOv8s, CLAHE+YOLOv8s, and WGAN-GP+YOLOv8s. All comparison solutions used the same dataset partitioning method and consistent training hyperparameters, only replacing the image enhancement branch module, while keeping the detection backbone network unchanged (YOLOv8s). RetinexNet achieves image enhancement based on a two-layer illumination reflection decomposition structure, but lacks geometric prior constraints and temporal component separation mechanisms. EnlightenGAN enhances the brightness of low-light images using a generative adversarial network, but cannot distinguish between oil stain reflection signals and the inherent texture of the wire rope itself. CLAHE is a traditional spatial contrast enhancement algorithm that relies on fixed mapping parameters to complete image adjustment, lacking scene-oriented adaptive adjustment capabilities. WGAN-GP optimizes the visual representation of images through adversarial loss, but lacks defect morphology protection constraints in the training process, making it prone to distortion and deformation of small defect contours during enhancement operations. All control models and the method of this application were trained sequentially until convergence. The statistical results were summarized on a unified test set and compiled into Table 1.

[0123] Table 1. Comparison of quantitative metrics for different algorithms on the Wire Rope Dataset test set.

[0124]

[0125] The quantitative statistical results clearly show the differences in the performance indicators of each model group. The traditional CLAHE enhancement method lacks an adaptive adjustment mechanism, resulting in the lowest performance indicators when faced with oil stain reflections and fluctuating lighting conditions. Deep learning-based enhancement schemes achieve stable performance improvements compared to traditional algorithms, but existing deep enhancement networks lack a spiral geometric prior component decoupling mechanism, making it difficult to effectively remove multiple interference signals, and small-scale defects are easily masked by periodic texture noise. To visually demonstrate the differences in image enhancement output effects between different algorithms, a sample image from the test set that simultaneously exhibits uneven natural light illumination, surface oil stains, and small broken wire defects was selected. All comparison algorithms were called sequentially to perform image enhancement operations, and the output results were saved. Figure 3 As shown, Figure 3 The original image, along with the CLAHE-enhanced output image, RetinexNet-enhanced image, EnlightenGAN-enhanced image, WGAN-GP-enhanced image, and the enhanced image output by the algorithm of this application, shows that the original image has a large area of ​​oil stains reflecting light in the rope valley region on the right, and the contrast of the image with small broken wire defects is low. The overall contrast improvement of the CLAHE-enhanced image is limited, and the noise in the oil stain reflection area is amplified simultaneously. The RetinexNet-enhanced image can balance the image illumination distribution to a certain extent, but it cannot eliminate the pseudo-texture features formed by oil stains. EnlightenGAN effectively improves the overall brightness of the image, but the edges of small broken wires show blurring and distortion. WGAN-GP optimizes the visual appearance of the image, but the periodic texture boundaries of the strands merge and adhere with the defect boundaries. The image output by the algorithm of this application can effectively suppress the interference signal caused by the large area of ​​oil stain reflection, maintain the continuous and smooth state of the inherent texture of the strands, and retain the complete outline of the edges of small broken wires without the problem of defect edge distortion.

[0126] After completing the quantitative index statistics and the visualization analysis of the enhancement effect, the directional field average offset of the defect area of ​​all test samples was further calculated, and the defect area before and after enhancement was calculated for each sample. Numerical values ​​were used to summarize the average directional offset of various algorithms. The average directional offset for CLAHE+YOLOv8s was 0.621 rad, for RetinexNet+YOLOv8s it was 0.547 rad, for EnlightenGAN+YOLOv8s it was 0.513 rad, and for WGAN-GP+YOLOv8s it was 0.468 rad. The average directional offset in the defective region of the algorithm in this application was reduced to 0.243 rad. The data demonstrates that the orientation field consistency constraint mechanism can stably constrain the enhancement operation without disrupting the gradient direction in defective regions. Validation set metrics for each round of all algorithms were continuously recorded, such as... Figure 4 As shown, Figure 4 The five curves, each with a distinct line shape, correspond to five different algorithms. All models show rapid increases in metrics during the first 80 rounds, and gradually converge after 180 rounds. The other comparative models have limited room for improvement in metrics after convergence. The algorithm in this application maintains a stable upward trend after 120 rounds, and its metrics are significantly higher than all the control schemes after convergence. At the same time, the training process exhibits smaller oscillations and a more stable network parameter optimization process.

[0127] Furthermore, this application verifies the impact of various core modules, constraints, and adaptive mechanisms on the final detection performance through ablation simulation. Six ablation control groups were set up in the ablation experiment. Ablation experiment group 1 removed the surface reflection decoupling model with fused spiral geometric priors and directly fed the original image into the dynamic convolutional enhancement network; ablation experiment group 2 retained the reflection decoupling module but removed the spatiotemporal correlation optical flow weight auxiliary solution branch; ablation experiment group 3 retained the component decoupling and temporal constraint branches, removed the scene adaptive modulation vector, and used a fixed convolutional kernel to perform texture enhancement operations; ablation experiment group 4 retained all the aforementioned modules but removed the orientation field consistency constraint. Ablation experiment group 5: Retained directional field constraint terms, removed equilibrium coefficients. An adaptive update mechanism is used, employing fixed constants in the joint loss calculation; Ablation experiment group 6 represents the standard scheme where all modules of this application are fully enabled. The ablation experiment dataset partitioning rules, training hyperparameters, and hardware operating environment are completely consistent with Example 2. Each ablation scheme independently completes three repeated training iterations, and the average value of the three training test indicators is taken as the final statistical result. The quantitative indicators of the ablation experiment are summarized and compiled into Table 2.

[0128] Table 2 Summary of Quantitative Test Indicators in Multi-Module Ablation Experiments

[0129]

[0130] Based on the ablation experiment data, the contribution of each independent module can be determined sequentially. The spiral geometric prior reflection decoupling module has the greatest impact on the overall performance. Lacking a component-level decoupling mechanism, it cannot effectively separate ambient lighting and oil stain reflection interference, and the continuous propagation of noise downstream directly leads to a significant decrease in detection accuracy. The temporal optical flow weighted branch can improve the accuracy of solving the oil stain reflection component and further reduce interference from false textures within the image. The scene-adaptive modulation vector-driven dynamic convolution kernel implements a scene-differentiated enhancement strategy, which provides stable performance gains compared to a fixed convolution kernel. The orientation field consistency constraint term can avoid distorting the defect morphology during enhancement operations, effectively reducing the missed detection of small-sized defects. The adaptive adjustment strategy of the balance coefficient can dynamically strengthen the orientation constraint strength according to the training process, continuously optimizing the convergence direction of network parameters. Three groups of defect samples of different scales were selected from the test set, including small broken wires, medium-length surface wear areas, and large-area rust areas. The defect detection visualization results corresponding to each ablation scheme were output, such as... Figure 5 As shown, ablation group 1 repeatedly missed small broken wires, ablation groups 2 and 3 had low confidence in the output of defects, and some wear defect boundaries in ablation group 4 were distorted. The complete solution group was able to stably output high-confidence detection frames, and the defect location boundaries were in close fit to the actual marked area.

[0131] To further quantify the dynamic changes of the orientation field constraint terms during the training iteration phase, records were generated every 10 iterations during the training process of the complete scheme. Moving average and adaptive balance coefficient Real-time values, such as Figure 6 As shown, in the initial training phase, the parameters of the augmented network are randomly initialized, and the overall degree of defect direction offset is relatively large. Maintaining a high level, the adaptive mechanism continues to improve. Numerical reinforcement constraints are strengthened; after 140 iterations, the reinforcement network gradually learns a stable texture enhancement mapping relationship, and defect gradient distortion is continuously reduced. Continued decline The values ​​slowly decreased and gradually converged to a stable range. The curve's change pattern confirms that the adaptive balance coefficient update mechanism can dynamically adjust the constraint strength according to the network's real-time learning status, avoiding insufficient constraints in the early stages of training or redundant constraints in the later stages caused by fixed weights.

[0132] This embodiment describes in detail the significant performance advantages of the existing enhanced detection combination model in scenarios with multiple lighting and oil pollution interference for passenger cableway steel wire ropes. The mechanisms of spiral geometric prior component decoupling, temporal optical flow-assisted solution, scene adaptive dynamic convolution enhancement, orientation field consistency constraint, and adaptive loss weight adjustment can all independently bring positive performance gains.

[0133] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. An adaptive enhancement defect detection method for surface images of passenger cableway wire ropes, characterized in that, Includes the following steps: Acquire a continuous image sequence of the surface of the passenger cableway wire rope and synchronously collect environmental condition parameters; A surface reflection decoupling model integrating the prior geometry of the steel wire rope spiral is constructed. Based on the environmental state parameters and the spatiotemporal correlation of the continuous image sequence, the ambient light component, oil stain reflection component and inherent texture component are separated from the original image. A scene adaptive modulation vector is generated using the ambient light component and the oil stain reflection component. The dynamic convolution kernel is then weighted using the scene adaptive modulation vector. Fidelity enhancement is performed on the inherent texture component to obtain an enhanced steel wire rope surface image. The enhanced image of the wire rope surface is input into the defect detection network, and the difference in orientation field features of the defect region before and after enhancement is calculated. A joint loss function containing orientation field consistency constraints and detection loss is constructed to train the enhancement network and the detection network end-to-end. The orientation field consistency constraints are configured such that when the enhancement causes the principal gradient direction of the defect region to shift beyond a preset angle threshold, the enhancement intensity at the corresponding spatial location is suppressed in reverse to maintain the axial or circumferential morphological characteristics of the wire rope defect.

2. The method according to claim 1, characterized in that, Constructing a surface reflection decoupling model that integrates the prior geometric geometry of the steel wire rope helix, specifically including: Based on the lay pitch and diameter parameters of the wire rope, a helical texture mask is constructed. ; Original image Decomposed into ambient light components Oil stain reflectivity and inherent texture components The decomposition process satisfies the following constraint equations: in, This represents element-wise multiplication. For residual terms; Each component is solved by minimizing the following energy function: in, This is a regularization term for illumination smoothing. For the low-rank constraint term of oil stain reflection, For consistency regularization of the spiral structure of the inherent texture, These are the corresponding weighting coefficients.

3. The method according to claim 2, characterized in that, The helical structure consistency regularization term The formula is: in, For intrinsic texture components in pixels The gradient vector at that point, This is the unit vector representing the theoretical tangent direction of the wire rope strand at this pixel, calculated based on helical geometry priors. This indicates the transpose operation.

4. The method according to claim 2, characterized in that, The scene adaptive modulation vector is generated using the ambient light component and the oil stain reflection component, specifically including: Ambient light component Global average pooling and logarithmic transformation are performed to obtain the illumination feature vector. ; Reflective component of oil stains Channel attention extraction is performed to obtain the oil stain feature vector. ; The synchronously collected environmental state parameters are encoded into environmental feature vectors. ; Will , and The concatenated vectors are input into a multilayer perceptron to generate scene-adaptive modulation vectors. : in, For learnable weight matrix, For bias terms, It is the Sigmoid activation function. This indicates a vector concatenation operation.

5. The method according to claim 4, characterized in that, The dynamic convolution kernel is weighted using the scene-adaptive modulation vector, and fidelity enhancement is performed on the intrinsic texture components, specifically including: Define the basic set of convolution kernels ; Using scene-adaptive modulation vectors generate Individual Core Combination Weight : in, For the first The learnable weight vectors corresponding to each basic convolutional kernel; Generate dynamic convolution kernels and the inherent texture components Perform convolution enhancement: in, This represents a two-dimensional convolution operation. This refers to the enhanced intrinsic texture components.

6. The method according to claim 1, characterized in that, Calculate the difference in directional field characteristics of the defect region before and after enhancement, specifically including: For the original texture components before enhancement Images of the reinforced wire rope surface Calculate the gradient tensor fields separately; Within the defect bounding box region output by the defect detection network, calculate the principal gradient direction angles before and after enhancement. and ; Calculate the difference in directional field characteristics : in, The set of pixels within the defect bounding box. Its number of pixels.

7. The method according to claim 6, characterized in that, The orientation field consistency constraint term The formula is: This represents the number of defective samples in the current batch. For the first Differences in the directional field characteristics of individual defect samples For the preset angle threshold, For the defect detection network to the first The detection confidence level of a defective sample.

8. The method according to claim 1, characterized in that, The spatiotemporal correlation of consecutive image sequences is used to assist in the separation of the reflective component of oil stains, specifically including: Optical flow estimation is performed on adjacent frames in a continuous image sequence to obtain the motion field. ; Based on the uniform motion model of the steel wire rope, the theoretical motion field is calculated. ; The sports field deviation was used as the temporal differentiation weight of the oil stain reflectivity component. : in, This is the tolerance parameter for motion deviation; In the process of solving the energy function, As a component of oil stains' reflectivity The pixel-level weighting factor.

9. The method according to claim 7, characterized in that, The joint loss function The expression is: in, The detection loss of the defect detection network. For orientation field consistency constraints, For image reconstruction loss, and This is the balance coefficient; The balance coefficient An adaptive adjustment strategy is adopted, and its update rule is as follows: in, For the first The balance coefficient of the next iteration To adjust the step size, For indicator functions, For the most recent In the next iteration The moving average value automatically increases when the moving average value does not decrease. To strengthen the directional field constraint.

Images