Steel wire rope internal and external defect comprehensive detection method based on multi-modal data fusion

Abstract

This invention relates to the field of wire rope defect detection technology, specifically disclosing a comprehensive detection method for internal and external defects in wire ropes based on multimodal data fusion. The method includes the following steps: acquiring the real-time belt speed and operating conditions of the running wire rope; synchronously acquiring electromagnetic signals and visual images of the wire rope based on a displacement triggering mechanism; establishing a spatial position mapping relationship between the multimodal data; performing adaptive drift-reduction preprocessing on the electromagnetic signals to extract internal defect features; and performing image stabilization and denoising preprocessing on the visual images to extract external defect features. This invention, by employing a displacement triggering mechanism to synchronously acquire electromagnetic signals and visual images, can ensure the spatial position mapping relationship between the two modal data when the wire rope's running speed fluctuates during actual operation, overcoming the spatial misalignment problem caused by time-based acquisition.

Description

Technical Field

[0001] This invention relates to the field of wire rope defect detection technology, specifically a comprehensive detection method for internal and external defects in wire ropes based on multimodal data fusion. Background Technology

[0002] Wire ropes play a crucial role in applications such as mine hoisting and port lifting, bearing significant loads, and their operational status directly impacts operational safety. Existing wire rope defect detection technologies typically employ either electromagnetic non-destructive testing (NDT) or machine vision inspection. Electromagnetic testing primarily identifies internal wire breakage or wear, but in actual operation, electromagnetic signals are susceptible to baseline drift due to variations in sensor gaps. Visual inspection mainly identifies surface structural anomalies, but is often affected by changes in ambient lighting and wire rope vibrations. A single detection method cannot simultaneously and accurately capture the combined internal and external defects of the wire rope.

[0003] To obtain more comprehensive defect information, some existing technologies attempt to deploy both electromagnetic and visual inspection equipment simultaneously. However, in actual operating conditions, the running speed of the wire rope is often in a state of dynamic fluctuation. Existing inspection systems typically use independent time frequencies for data acquisition, resulting in misalignment between electromagnetic signals and visual images in actual spatial positions, making it difficult to achieve strict alignment of multi-source data.

[0004] Furthermore, when processing the two types of test results, existing methods mostly employ conventional logical superposition, failing to consider the specific material characteristics of the wire rope and the differences in sensitivity of different types of defects to specific testing methods. When the judgment results of the two testing methods for the same location contradict each other, the system struggles to provide a reliable fusion decision. Simultaneously, existing testing methods typically only output the current surface or internal defect status, lacking a dynamic quantitative assessment mechanism that incorporates real-time service condition data to evaluate the overall health status and remaining service life of the wire rope. Summary of the Invention

[0005] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, the purpose of this invention is to propose a comprehensive detection method for internal and external defects in wire ropes based on multimodal data fusion, so as to predict the remaining safe life of wire ropes.

[0006] To achieve the above objectives, a first aspect of the present invention proposes a comprehensive detection method for internal and external defects in wire ropes based on multimodal data fusion, comprising:

[0007] The real-time belt speed and operating conditions of the running wire rope are obtained, and the electromagnetic signals and visual images of the wire rope are synchronously collected based on the displacement triggering mechanism to establish the spatial position mapping relationship of multimodal data.

[0008] The electromagnetic signal is subjected to adaptive drift-de-scraping preprocessing to extract internal defect features, and the visual image is subjected to image stabilization and denoising preprocessing to extract external defect features.

[0009] By calling the preset steel wire rope material-specific characteristic factors, the comprehensive confidence level of the internal defect verification corresponding to the internal defect feature and the comprehensive confidence level of the external defect verification corresponding to the external defect feature are calculated respectively.

[0010] Based on the detection sensitivity of the defect type, the comprehensive confidence of the internal defect review, and the comprehensive confidence of the external defect review, weights are dynamically allocated, and a multimodal fusion decision is made through a cross-arbitration mechanism to output the wire rope defect detection results.

[0011] Based on the defect detection results of the wire rope and the real-time service condition data, the dynamic damage degree of the wire rope is quantified to calculate the overall health status index, thereby predicting the remaining safe life and outputting the corresponding early warning level safety control instructions.

[0012] To achieve the above objectives, a second aspect of the present invention proposes a comprehensive detection system for internal and external defects of wire ropes based on multimodal data fusion, comprising:

[0013] The data acquisition and mapping module is used to acquire the real-time belt speed and working condition of the running wire rope, and synchronously acquire the electromagnetic signals and visual images of the wire rope based on the displacement triggering mechanism to establish the spatial position mapping relationship of multimodal data.

[0014] The preprocessing and feature extraction module is used to perform adaptive drift-de-scraping preprocessing on the electromagnetic signal to extract internal defect features, and to perform image stabilization and denoising preprocessing on the visual image to extract external defect features.

[0015] The confidence calculation module is used to call the preset steel wire rope material-specific characteristic factors to calculate the comprehensive confidence of internal defect verification corresponding to the internal defect features and the comprehensive confidence of external defect verification corresponding to the external defect features.

[0016] The multimodal fusion decision module is used to dynamically allocate weights based on the detection sensitivity of the defect type, the comprehensive confidence of the internal defect verification, and the comprehensive confidence of the external defect verification, and to perform multimodal fusion decision through a cross-arbitration mechanism to output the wire rope defect detection results.

[0017] The service assessment and control module is used to quantify the dynamic damage degree of the wire rope based on the defect detection results and real-time service condition data to calculate the overall health status index, and then predict the remaining safe life and output the corresponding early warning level safety control instructions.

[0018] To achieve the above objectives, a third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the above-described method for comprehensive detection of internal and external defects in wire ropes based on multimodal data fusion.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] This invention employs a displacement-triggered mechanism to simultaneously acquire electromagnetic signals and visual images. This ensures the spatial mapping relationship between the two modal data when the wire rope's running speed fluctuates during actual operation, overcoming the spatial misalignment problem caused by time-based acquisition. In the data processing stage, adaptive drift removal and image stabilization / denoising preprocessing are performed on both electromagnetic signals and visual images. The invention also utilizes wire rope material-specific characteristic factors to calculate the comprehensive confidence level of internal and external defects. Subsequently, weights are dynamically allocated based on the detection sensitivity of defect types, and a cross-arbitration mechanism is used for multimodal fusion decision-making. This fusion mechanism effectively handles the judgment conflicts of a single sensor under complex working conditions, improving the reliability of defect judgment. Finally, the defect detection results are combined with actual operating speed and service condition data to quantify the dynamic damage degree of the wire rope and calculate the overall health status index. This enables the prediction of the remaining safe life of the wire rope and the output of safety control instructions, providing continuous and clear quantitative assessment basis for equipment operation and maintenance on-site. Attached Figure Description

[0021] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0022] Figure 1 This is a flowchart illustrating the comprehensive detection method for internal and external defects of steel wire rope based on multimodal data fusion provided by the present invention.

[0023] Figure 2 This is a schematic diagram illustrating the implementation of the comprehensive detection system for internal and external defects of steel wire rope based on multimodal data fusion provided by the present invention.

[0024] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0025] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0026] The following description, with reference to the accompanying drawings, outlines a comprehensive detection method, system, and electronic device for internal and external defects in wire ropes based on multimodal data fusion, according to embodiments of the present invention.

[0027] Example 1:

[0028] Figure 1 This is a flowchart illustrating a comprehensive detection method for internal and external defects of wire ropes based on multimodal data fusion, according to an embodiment of the present invention. This embodiment provides a comprehensive detection method for internal and external defects of wire ropes based on multimodal data fusion. This method is mainly applied to large-scale special equipment with high load and high safety requirements, such as mine hoists, port cranes, and large passenger ropeways, for non-destructive online detection and health assessment of load-bearing wire ropes throughout their entire lifecycle during operation.

[0029] Specifically, the method described in this embodiment is based on a collaborative architecture. This collaborative architecture includes an edge-side real-time detection unit and a cloud-based iterative optimization unit.

[0030] In practical industrial deployment environments, the edge-side real-time detection unit is typically a physical edge computing terminal equipped with a high-performance field-programmable gate array (FPGA) and an embedded microprocessor, directly installed near the wellhead, cableway support, or crane pulley block at the wire rope operation site. This edge-side real-time detection unit is responsible for performing data acquisition, signal preprocessing, feature extraction, and multimodal fusion decision-making with high real-time requirements. The cloud-based iterative optimization unit is deployed in the enterprise data center or public cloud platform, possessing massive computing power and storage space. It is responsible for aggregating long-term operating data from multiple devices to perform offline incremental training and digital twin evolution analysis of the detection model. Through this edge-cloud collaborative architecture, not only is a microsecond-level response time for on-site defect detection guaranteed, but the algorithm model can also continuously adaptively evolve as its service time increases.

[0031] Specifically, the method in this embodiment includes the following steps:

[0032] Step 1: Synchronous acquisition of multimodal data based on displacement triggering mechanism.

[0033] The system acquires the real-time belt speed and operating conditions of the running wire rope, and synchronously collects the electromagnetic signals and visual images of the wire rope based on a displacement triggering mechanism to establish a spatial position mapping relationship of multimodal data.

[0034] For example, traditional timed data acquisition mechanisms, when faced with drastic speed fluctuations caused by the starting, acceleration, deceleration, and heavy-load uphill climbing of the wire rope, result in extremely uneven distribution of data collected within the same time interval along the physical length of the wire rope, leading to severe modal space misalignment. To overcome this technical challenge, this embodiment employs a strict displacement-triggered hardware synchronization mechanism.

[0035] Specifically, a high-precision incremental rotary encoder installed on-site is rigidly connected to the guide wheel or main shaft of the wire rope. When the wire rope is displaced, the encoder outputs a high-frequency orthogonal pulse signal.

[0036] A synchronous trigger pulse is generated by a field-programmable gate array (FPGA) and sent to an electromagnetic detection sensor and an industrial vision camera. The FPGA's high-frequency clock circuit acquires encoder pulse signals in real time to count the wire rope displacement. An internal unit displacement threshold is preset, and synchronous acquisition is triggered according to this threshold. Whenever the accumulated actual displacement of the wire rope reaches this threshold, the FPGA's hardware interrupt circuit immediately generates a synchronous trigger pulse with a nanosecond-level rising edge. This synchronous trigger pulse is sent in parallel and without delay to the analog-to-digital converter trigger pin of the electromagnetic detection sensor and the external hardware trigger interface of the industrial vision camera. Through this purely hardware-based triggering mechanism, the system establishes a one-to-one correspondence between acquisition time, wire rope displacement, electromagnetic signal segments, and visual image frames, thereby physically binding multi-source data into a unified one-dimensional spatial coordinate system.

[0037] It is important to note that the real-time changes in on-site conditions require the acquisition system to have a high degree of dynamic adaptability. Therefore, the end-side real-time detection unit will adaptively adjust the exposure parameters and sampling frame rate of the industrial vision camera, as well as the sampling frequency of the electromagnetic detection sensor, based on the real-time belt speed and operating conditions. When the acquired real-time belt speed increases significantly, the system automatically reduces the exposure parameters of the industrial vision camera to shorten the exposure time, thereby avoiding image motion blur caused by high-speed movement; at the same time, it proportionally increases the sampling frame rate of the camera and the sampling frequency of the electromagnetic detection sensor to ensure that the data point density acquired within a unit physical length remains constant, maintaining absolute consistency in spatial resolution.

[0038] Step 2: Semantic segmentation and mesh partitioning of visual images.

[0039] Before performing complex feature extraction, in order to reduce background noise interference and improve the accuracy of local defect localization, this embodiment also includes semantic segmentation and mesh partitioning steps before performing image stabilization and denoising preprocessing on the visual image.

[0040] Specifically, a semantic segmentation network is used to extract the complete area of ​​the wire rope in the visual image and divide it into a core stress area, a strand fixing area, a belt transition area, and an edge-insensitive area. Raw two-dimensional images acquired in industrial settings typically contain a large amount of invalid background. The system first invokes a lightweight semantic segmentation network deployed on the edge to classify the image pixels step by step.

[0041] Among them, the core stress zone refers to the raised apex area on the surface of the outermost strand of the wire rope. This area bears the greatest friction and extrusion when running through pulleys or drums, and is the area with the highest incidence of external wire breakage and wear. The strand fixing zone refers to the recessed gap area between adjacent strands. This area is prone to accumulating oil sludge and coal dust, and can also easily hide early fatigue microcracks. The belt transition zone is the secondary bearing surface under the specific weaving structure of the wire rope. The edge non-sensitive zone is the blurred and distorted area or pure background area at the edge of the image field of view.

[0042] Optionally, based on the pre-calibrated physical dimension mapping relationship on site, the core stress area and the rope strand fixing area are meshed. The system uses the camera calibration matrix to calculate the conversion ratio between image pixel distance and actual physical millimeters. Subsequently, based on the principle of equal area division, a virtual two-dimensional grid coordinate system is superimposed on the extracted core stress area and rope strand fixing area.

[0043] Each mesh cell is assigned a unique identifier and its physical location within the wire rope. Invalid mesh cells are removed, resulting in a set of valid mesh cells. Each mesh cell's identifier includes its corresponding image frame number, row index, and column index. The physical location information records the absolute longitudinal displacement coordinates of the mesh's center point relative to the wire rope's starting end, as well as the estimated circumferential angle. Morphological filtering algorithms remove invalid mesh cells that cross edge boundaries, are too small, or have severely missing pixels. The final set of valid mesh cells serves as the foundational spatial unit for subsequent local image registration and dynamic damage measurement.

[0044] Step 3: Two-level registration and image stabilization / denoising of the visual image.

[0045] The visual images are preprocessed with image stabilization and denoising to extract external defect features. Due to the mechanical vibration of large lifting equipment, the lateral sway of the wire rope itself, and the tensile deformation caused by tension changes, continuously acquired visual images inevitably contain complex spatial offsets and distortions. Therefore, the image stabilization and denoising preprocessing of the visual images described in this embodiment adopts the following two-stage registration mechanism:

[0046] The first-level global coarse registration involves extracting stable feature points in the core stress area and calculating global spatial transformation parameters through feature point matching to complete the geometric correction of the image. The system utilizes an accelerated robust feature extraction algorithm to extract a massive number of stable feature points at the high-contrast texture boundaries in the core stress area and performs spatial distance matching of feature vectors with the feature point set of the previous frame. Subsequently, a random sampling consensus algorithm is used to eliminate mismatched point pairs, and a global spatial transformation parameter matrix containing translation, rotation, and scaling components is fitted. This parameter matrix is ​​then used to perform rigid body transformation mapping on the current entire image, thereby offsetting the large-scale lateral viewpoint shift caused by equipment mechanical vibration and the macroscopic lateral sway of the wire rope.

[0047] The second level of mesh-level fine registration: For each effective mesh cell in the set of effective mesh cells, the corresponding local texture feature points within the mesh are extracted. Using the mesh corresponding to the initial healthy state of the wire rope as a reference, local deformation parameters are calculated to complete the mesh-level sub-pixel fine registration. Based on macroscopic alignment, due to drastic changes in tension, the wire rope may experience local elastic elongation or structural twisting. The system uses a single mesh cell as the processing core to calculate the displacement vector field of feature points within the mesh. By solving the local affine transformation matrix, the minute stretching and torsional deformations of the mesh cell are obtained as local deformation parameters. Sub-pixel level resampling is performed through bilinear interpolation, forcibly correcting the texture coordinates of the current mesh to a state that completely coincides with the initial healthy reference image, thereby eliminating artifacts. Noise is removed using a non-local mean filtering algorithm, and finally, external defect features with extremely high signal-to-noise ratio are extracted, such as broken wire ends, rust spot contours, and severely worn flat surfaces.

[0048] Step 4: Adaptive drift-de-processing of electromagnetic signals.

[0049] The electromagnetic signal undergoes adaptive drift-de-processing to extract internal defect features. Electromagnetic flux leakage detection technology is extremely sensitive to broken wires and cross-sectional area loss within the wire rope. However, in actual operation, the lateral vibration of the wire rope causes dynamic changes in the sensor lift-off gap, resulting in severe low-frequency baseline drift in the electromagnetic signal, significantly masking the true, subtle defect features. Therefore, this embodiment performs the following operations:

[0050] Specifically, the system collects circumferential multi-point gap data between the electromagnetic detection sensor and the wire rope, and predicts the baseline drift value corresponding to the current gap using a pre-calibrated polynomial fitting model. Multiple sets of high-frequency laser ranging probes are evenly distributed circumferentially on the probe shoe inside the electromagnetic detection probe. These probes synchronously output the absolute physical distance between the sensor's magnetic pole surface and the wire rope surface. The system substitutes this multi-point gap data into a polynomial fitting model pre-calibrated in a laboratory environment using different lift-off heights to calculate the predicted baseline drift value output by the Hall element due to spatial magnetic field distortion under this combination of lift-off gaps.

[0051] For example, the filter coefficients of the adaptive filter are dynamically adjusted based on the baseline drift prediction value to perform dynamic pipelined filtering on the original electromagnetic signal to eliminate the influence of baseline drift. The system employs a minimum mean square error adaptive filter. The baseline drift prediction value is input to the reference input of the adaptive filter, and the original electromagnetic signal is input to the main input. The filter iteratively updates its tap weight coefficients so that its output signal can approximate and cancel the low-frequency drift components in the original signal to the greatest extent possible, thereby outputting a preliminary correction signal with a flat baseline.

[0052] Subsequently, the filtered electromagnetic signal is subjected to multi-scale wavelet decomposition to reconstruct the signal components of the target frequency band and extract the internal defect features. Since high-frequency electromagnetic noise from mechanical vibration still exists, the system uses specific wavelet basis functions to perform multi-level decomposition on the preliminary correction signal, separating detail coefficients representing extremely high-frequency noise and approximation coefficients representing ultra-low-frequency fluctuations. Only the detail coefficients of the target frequency band that match the spatial distribution frequency of the leakage magnetic field of the internal defect are retained and reconstructed. Based on the reconstructed high-purity leakage magnetic signal, key parameters such as peak amplitude reflecting defect depth, peak span reflecting defect width, and phase change rate are extracted as internal defect features.

[0053] Step 5: Specific feature correction and overall confidence calculation.

[0054] At this stage, the system will call the preset steel wire rope material-specific characteristic factors to calculate the comprehensive confidence level of the internal defect verification corresponding to the internal defect characteristics and the comprehensive confidence level of the external defect verification corresponding to the external defect characteristics.

[0055] The diversity of steel wire ropes in manufacturing processes, surface treatments, and structural forms results in drastically different signal intensities from the same physical defect at the sensor end. Therefore, this embodiment innovatively introduces a steel wire rope material-specific characteristic factor for in-depth correction.

[0056] Specifically, the permeability correction factor and cross-sectional area loss correction factor from the wire rope material-specific characteristic factors are called in, and combined with the feature matching degree and feature weight coefficient, the weighted comprehensive confidence level of the internal defect verification is calculated. Due to the physical property differences in the base permeability of galvanized wire rope, phosphated wire rope and bright wire rope, the degree of leakage magnetic field from internal defects also changes accordingly.

[0057] The formula for calculating the overall confidence level of the internal defect review is as follows:

[0058] ;

[0059] In the formula: To assess the overall confidence level for internal defect review; This represents the number of independent internal defect features extracted within the current detection area; For the first The feature matching degree between an internal defect feature and the standard defect feature model in the database; For the first The feature weight coefficients corresponding to each internal defect feature; This is the permeability correction factor; This is the cross-sectional area loss correction factor.

[0060] By using the weighted aggregation of the above formula, the evaluation bias caused by the variation of material electromagnetic properties is effectively eliminated, so that the overall confidence level of the output truly reflects the physical severity of internal defects.

[0061] Similarly, by calling the surface roughness correction factor and fatigue damage correction factor from the wire rope material-specific characteristic factors, and combining them with the feature matching degree and feature weight coefficient, the weighted comprehensive confidence level of the external defect verification is calculated. The formula for calculating the comprehensive confidence level of the external defect verification is as follows:

[0062] ;

[0063] In the formula: To assess the overall confidence level of external defect verification; This represents the number of independent external defect features extracted within the current detection area; For the first Feature matching degree of each external defect feature; For the first Feature weight coefficients for each external defect feature; This is a surface roughness correction factor due to different twisting structures; It is a fatigue damage correction factor.

[0064] It is also important to note that using static thresholds to determine whether a defect has been identified can easily lead to early missed detections or frequent false alarms in later stages. Therefore, the system calculates dynamic thresholds for internal defects based on the service life of the wire rope and the real-time load, and calculates dynamic thresholds for external defects based on the surface condition of the wire rope and the design fatigue limit stress.

[0065] The formula for calculating the dynamic threshold for internal defects is proposed as follows:

[0066] ;

[0067] In the formula: The threshold for dynamic determination of internal defects; The initial baseline threshold set for the system; The service life of the wire rope; For the estimated design life of the wire rope; To collect the real-time load fed back by the sensor at the moment of acquisition; This is the rated load of the wire rope; and These are the time decay adjustment constant and the load-sensitive adjustment constant, respectively.

[0068] The formula clearly states that as the service time of the wire rope increases and the load it bears increases, the judgment threshold will be automatically lowered, thereby improving the system's sensitivity to potential minor defects and preventing major hidden dangers from being missed.

[0069] Similarly, the formula for calculating the dynamic judgment threshold of the external defect is proposed as follows:

[0070] ;

[0071] In the formula: The threshold for dynamic determination of external defects; The initial baseline threshold set for the system; This is the real-time surface stress estimated based on the current surface conditions; The design fatigue limit stress of the wire rope; This is the apparent sensitivity adjustment constant.

[0072] This formula ensures that the threshold for judging external defects can be adaptively reduced when the surface stress approaches the fatigue limit, thereby improving the detection rate of surface microcracks.

[0073] After calculating the confidence levels and dynamic thresholds, the system performs comparative analysis. Regions containing defect features where the overall confidence level of the internal defect review is lower than the internal defect dynamic judgment threshold, or where the overall confidence level of the external defect review is lower than the external defect dynamic judgment threshold, are marked as divergence point regions and temporarily stored locally. These divergence point regions represent suspected damage zones with weak signal features and located at the fuzzy judgment boundary, possessing extremely high value for secondary analysis.

[0074] Step Six: Multimodal Fusion Decision Making and Cross-Arbitration.

[0075] After obtaining the confidence level of a single mode, a higher level of fusion must be performed to eliminate false alarms from a single sensor. Based on the detection sensitivity of the defect type, the comprehensive confidence level of the internal defect verification, and the comprehensive confidence level of the external defect verification, weights are dynamically allocated, and a cross-arbitration mechanism is used to perform multimodal fusion decision-making, outputting the wire rope defect detection results.

[0076] Specifically, when the internal defect type and external defect type determination results corresponding to the same spatial location are inconsistent, it is determined as a conflict of dual-modal detection results, and the following three-level cross-arbitration mechanism is executed: In complex industrial sites, multimodal detection results often contradict each other.

[0077] For example, a drop of viscous lubricating oil might appear as a strong edge resembling a broken wire in a visual image, leading the visual modality to determine that the area has an external defect; however, the electromagnetic modality might not detect any leakage flux distortion at that coordinate. In this case, there is a serious conflict between the two determination types and their confidence levels. To address this situation, the system automatically enters a three-level cross-arbitration mechanism to prevent false alarms, including:

[0078] First-level confidence level arbitration: Comparing the combined confidence levels of the internal defect review and the external defect review that cause conflict, the detection result corresponding to the high-confidence modality with a confidence level difference greater than a preset confidence level difference threshold is given priority. If, in this oil spill false alarm case, the visual confidence level is only barely above the threshold, while the electromagnetic confidence level is at an extremely low noise level, and the difference between the two exceeds the preset confidence level difference threshold, the system quickly determines to accept the one with higher certainty, or determines that neither constitutes a high-certainty defect.

[0079] Secondary feature adaptation arbitration: If the confidence difference is not greater than the confidence difference threshold, priority is given based on the conflicting defect type: if it is an internal defect type, the modal detection result corresponding to the electromagnetic signal is given priority; if it is an external defect type, the modal detection result corresponding to the visual image is given priority. Electromagnetic detection is highly sensitive to changes in the internal metal cross-section based on physical laws, while visual detection has intuitive explanatory power for surface geometric distortions. Therefore, based on this principle of multi-dimensional feature complementarity, the decision weight is forcibly biased towards the dominant mode corresponding to the defect type.

[0080] Level 3 Manual Review Marking: If a determination still cannot be made after Level 1 confidence arbitration and Level 2 characteristic adaptation arbitration, the conflicting area is marked as a high-risk suspicious area, and the corresponding multimodal data is stored in a temporary database for subsequent manual review. This ensures that under complex signal interference, the system can avoid outputting false results, but instead retains a complete chain of data evidence for qualitative assessment by experienced field engineers.

[0081] Step 7: Dynamic damage quantification and overall health status assessment.

[0082] A simple list of defects cannot provide managers with intuitive guidance on equipment availability. Therefore, based on the defect detection results of the wire rope and real-time service condition data, the dynamic damage degree of the wire rope is quantified to calculate the overall health status index.

[0083] For example, based on the wire rope defect detection results, the effective mesh cells containing defects are extracted as target mesh cells. For each target mesh cell, the cumulative plastic deformation, plastic deformation concentration factor, tension non-uniformity factor, dynamic stiffness response factor, and the ratio of the real-time peak impact load to the rated load of the wire rope are obtained, and the dynamic damage degree of the corresponding target mesh cell is calculated by weighting.

[0084] The following formula is proposed for calculating the dynamic damage degree of the target mesh element:

[0085] ;

[0086] In the formula: The dynamic damage degree of the corresponding target mesh element; This represents the cumulative plastic deformation caused by long-term tension on the grid, which manifests as a reduction in the microscopic rope diameter; It is the plastic deformation concentration factor, reflecting the severity of local stress concentration; It is the tension non-uniformity coefficient, used to characterize the abnormal torsion caused by uneven stress on each strand of steel wire within the same lay pitch; The dynamic stiffness response coefficient characterizes the reduction in elasticity caused by core compression failure or drying. This refers to the real-time peak impact load of the steel wire rope. to These are the sensitivity weight coefficients for each influencing variable.

[0087] Using this formula, the system successfully maps the static physical defect morphology to the mechanical damage assessment value under stress.

[0088] Based on the dynamic damage degree of all the target mesh elements, the overall health status index is calculated by dynamically summing and normalizing the regional weight coefficient and real-time load correction coefficient corresponding to each target mesh element. The formula for calculating the overall health status index is as follows:

[0089] ;

[0090] In the formula: The overall health status index is defined as a score of 100. The total number of target grid cells identified; For the first Dynamic damage degree of each target mesh cell; This is the region weighting coefficient corresponding to the physical location of the target mesh element (for example, the defect weight in the core stress zone is much greater than that in the edge non-sensitive zone). This is the real-time load correction factor.

[0091] The index output by this formula directly reflects the overall health level of the entire wire rope in maintaining its safe load-bearing capacity at any given moment.

[0092] Step 8: Remaining safe life prediction and safety closed-loop management.

[0093] After calculating the health status index, the system will then predict the remaining safe lifespan and output corresponding safety control instructions at the warning level.

[0094] Optionally, the prediction of remaining safe life adopts the following physics- and data-driven prediction framework: Physics-driven foundation life prediction: Based on the linear fatigue cumulative damage criterion, wire rope fatigue curve, and real-time operating data, the remaining safe operating days of the foundation are calculated, and the remaining safe operating days of the foundation are corrected according to an environmental correction factor. The system uses the rainflow counting method to count the amplitude and frequency of historical stress cycles, and combines this with the SN fatigue curve provided by the wire rope manufacturer to calculate the remaining operating days of the foundation under a purely theoretical mechanical framework using the linear Miner damage accumulation rule. If the on-site environment has a severely corrosive atmosphere (such as a harbor salt spray environment), an environmental correction factor is introduced to proportionally reduce the number of days.

[0095] Data-driven evolution trend prediction: The spatiotemporal evolution features containing the overall health status index time-series data are input into a Long Short-Term Memory (LSTM) network optimized with an attention mechanism to predict the remaining safe operating days driven by data. Pure physical models are insufficient to cover all complex sudden degradation scenarios. The system extracts the expansion rate, new occurrence frequency, and decline slope of the health index over several past detection cycles as spatiotemporal evolution features. The LSTM network utilizes its gating mechanism to effectively capture long-term temporal dependencies and focuses computational resources on high-risk time-series segments with recent mutations through an attention mechanism, thereby using deep learning algorithms to predict a data-driven lifetime decline curve that changes with the actual on-site conditions.

[0096] Dual-model weighted fusion prediction: Based on the quantitative results of wire rope damage, the weights of the physical model and the data model are dynamically adjusted. The remaining safe operating days based on the foundation and the remaining safe operating days driven by the data are weighted and fused to output the remaining safe life. In the early stage of wire rope service, the damage is minimal, and the physical model has high reliability, so it is given a high weight. In the middle and late stages of service, fatigue cracks spread rapidly, and theoretical calculations are no longer accurate. The system automatically shifts the weights towards the data-driven model. The final number of days output by fusion not only has mechanical foundation support but also closely reflects the actual deterioration pattern. Based on this lifespan and the current health index, the system automatically matches a graded early warning strategy and directly outputs safety control instructions, including forced speed reduction, load limitation, or immediate shutdown, to the hoist main control system via the industrial bus.

[0097] To ensure that the system's performance does not degrade during long-term operation, this embodiment also includes a model incremental update mechanism. The edge-side real-time detection unit periodically encrypts and uploads the temporarily stored data corresponding to the divergence point regions to the cloud-based iterative optimization unit. This encrypted upload mechanism effectively prevents malicious tampering and network eavesdropping of industrial data.

[0098] The cloud-based iterative optimization unit annotates the data corresponding to the uploaded divergence point regions to expand the scene-specific dataset. Using this expanded dataset, the detection model is incrementally trained and validated, and the iterated detection model is remotely updated and distributed to the edge-side real-time detection unit. Cloud-based algorithm experts or automated annotation tools accurately classify and delineate the contours of these edge-generated divergence point data. These highly representative hard samples are merged into the original training set, driving the neural network to perform backpropagation fine-tuning. Once validated, the new weight file seamlessly replaces the old edge-side model via over-the-air (OTA) download technology. This closed-loop data feedback and model iteration mechanism endows the system with lifelong learning and adaptability in specific application scenarios.

[0099] In summary, this embodiment's method, through a pure hardware-based displacement triggering mechanism combined with a two-level fine registration algorithm, ensures strict spatial pixel-level alignment under complex vibration conditions. By deeply integrating the wire rope material properties and a three-level cross-arbitration mechanism, it significantly reduces the false alarm rate of defects under high background noise. Simultaneously, it elevates the underlying pixel features and leakage magnetic signals to a higher dimension, constructing a dual-drive life prediction model encompassing dynamic mechanical damage and LSTM temporal evolution. This solution effectively transforms traditional post-event static inspection into a new intelligent maintenance paradigm of "dynamic perception during operation, quantitative early warning, and autonomous evolution from edge to cloud," significantly improving the operational safety and scientific rigor of maintenance decisions for large-scale special equipment systems.

[0100] Example 2:

[0101] In actual mine hoisting or port lifting operations, wire ropes are not absolutely rigid bodies, but rather flexible load-bearing components made of multiple strands of steel wire twisted along a specific spatial helical trajectory. When a wire rope is subjected to a huge axial tensile load in a dynamically changing process, its internal helical structure will generate a corresponding torsional moment according to Hooke's law and spatial geometric constraints. This torsional moment will cause the wire rope body to rotate and deform around its central axis at a certain angle, which is the common phenomenon of "untwisting under stress" or "twisting under stress" in engineering.

[0102] Electromagnetic sensors deployed on-site typically employ ring arrays or fully enclosed magnetized probe structures, providing 360-degree circumferential coverage for scanning internal defects. However, industrial vision cameras, limited by their installation location and optical field of view, can usually only capture images of one side surface of the wire rope from a fixed spatial perspective. When the wire rope dynamically twists due to varying loads, external defects (such as broken wires on the surface) originally located directly opposite the center of the camera's field of view will experience lateral circumferential displacement as the wire rope rotates, deviating from their reference circumferential coordinates during no-load or static calibration. If this spatial circumferential deviation caused by mechanical torsion is not compensated for, the multimodal fusion decision module, upon receiving the detection results from the electromagnetic signal and visual image, may mistakenly identify two independent defects due to the mismatch in their spatial circumferential coordinates. This can lead to severe false alarms of ghosting or unnecessary arbitration conflicts during cross-arbitration.

[0103] Therefore, before dynamically allocating weights based on the detection sensitivity of defect type, the comprehensive confidence of internal defect verification, and the comprehensive confidence of external defect verification, this embodiment also includes a multimodal spatial circumferential alignment compensation step for the torsional deformation of wire rope under dynamic variable load conditions.

[0104] Specifically, based on the visual image that has undergone the image stabilization and denoising preprocessing, the strand texture direction features on the surface of the wire rope are extracted, and the real-time twist angle of the wire rope is calculated. In this step, the input visual image has already undergone the two-stage registration and image stabilization denoising preprocessing described in Example 1. Macroscopic mechanical vibration blurring and microscopic elastic stretching deformation in the image have been eliminated, resulting in a high signal-to-noise ratio representation of the wire rope surface's physical morphology. The most prominent structural feature of the wire rope surface is the spiral groove gap formed between adjacent strands, which appears as a highly directional, linear texture in the dark areas of the visual image. The system enhances the linear edge response at specific tilt angles by deploying an directionally adjustable two-dimensional spatial filter array in the end-side real-time detection unit and performing multi-directional convolution scanning on the preprocessed visual image.

[0105] Subsequently, the system extracts the grayscale gradient field in the image space and uses the Radon transform to map the high-gradient linear texture in the two-dimensional image space to the parameter space. Within the parameter space, by searching for local energy peaks, the system can fit the average tilt angle of all clear strand gaps within the current field of view with extremely high precision. This average tilt angle characterizes the angle between the true helical tangent of the wire rope at the current acquisition moment and under the current axial stress state and the longitudinal axis of the wire rope, and is defined as the real-time twist angle of the wire rope. The real-time twist angle is a dynamic physical quantity that fluctuates at high frequency with the stress state. Its accurate real-time extraction provides the only objective data basis for subsequent deformation compensation.

[0106] For example, the real-time twist angle Compared with the pre-calibrated reference no-load lay angle of the wire rope Perform a difference calculation to obtain the torsional angle deviation under dynamic load. In this step, the system needs to introduce a static reference benchmark, namely the wire rope reference no-load twist angle. This benchmark value is typically obtained by taking multiple static samples and calculating the arithmetic mean under the same lighting and shooting parameters, after the initial installation of the wire rope and the initial structural elongation release, when the system is under the lowest load condition (e.g., the lifting container is unloaded and at the wellhead, or the crane hook is unloaded and on the ground).

[0107] The reference no-load twist angle of the wire rope represents the intrinsic twist angle of the wire rope under no significant additional axial tension. During the continuous monitoring operation of the system, whenever a new visual image is acquired and the corresponding real-time twist angle is calculated, the system's calculation and processing module performs a high-frequency subtraction operation. By subtracting the reference no-load twist angle stored in the system from the real-time twist angle reflecting the current stress state, the system can separate the angular distortion component purely caused by the current additional dynamic load. The absolute value of this subtraction calculation directly reflects the severity of the axial tension currently experienced by the wire rope, while the sign of this value indicates whether the wire rope has undergone untwisting deformation or increased twist deformation. The system outputs this key parameter characterizing the degree of transient torsion, defined as the torsion angle deviation. .

[0108] Optionally, based on the torsion angle deviation... With the nominal outer diameter of the wire rope Calculate the circumferential compensation arc length After obtaining the deviation in the spatial angular dimension, it needs to be converted into a linear spatial displacement conforming to the two-dimensional coordinate system of the visual image. Since the visual camera captures the projection of the cylindrical surface of the wire rope onto the two-dimensional focal plane, the rotation of the wire rope body is visually represented on the image sensor as a lateral translation of surface texture and adhesion defects along a direction perpendicular to the longitudinal axis of the wire rope. To accurately quantify this translation distance, the geometric cross-sectional parameters of the wire rope under test must be considered. The system reads the wire rope specification parameters from the equipment register or initial configuration information to obtain its nominal maximum cross-sectional diameter. Combining the angular deviation output from the previous steps, the system uses the circumferential arc length mapping relationship to perform spatial geometric conversion.

[0109] The formula for calculating the circumferential compensation arc length is proposed as follows:

[0110] ;

[0111] In the formula: To compensate for the circumferential arc length; This refers to the deviation in the torsion angle. Pi is a constant. This refers to the nominal outer diameter of the wire rope.

[0112] Through rigorous derivation and calculation of the above formula, the system accurately flattens and maps the abstract three-dimensional cylinder rotation angle into the actual physical sliding distance along the circumferential tangent direction of the wire rope surface, i.e., the circumferential compensation arc length. This compensation arc length has a clear physical dimension, usually in millimeters or micrometers, providing a correction scale for subsequent reverse compensation operations in the image pixel coordinate system.

[0113] It should also be noted that the circumferential compensation arc length is utilized. The circumferential spatial coordinates of the external defect feature are rotated and corrected so that the corrected external defect feature and the internal defect feature are aligned in circumferential space before entering the multimodal fusion decision. After the external defect (e.g., a long strip of broken surface wire) is extracted by the image recognition algorithm, the external defect is assigned an initial circumferential spatial coordinate based on the current image pixel coordinate system in the vision processing module.

[0114] Due to the aforementioned torsional deformation, the initial circumferential spatial coordinates actually contain an offset artifact caused by the load. Directly using these coordinates for spatial matching with the internal defect coordinates output by the electromagnetic detection system will inevitably lead to position verification failure. Therefore, it is necessary to use the calculated physical compensation arc length for reverse cancellation. The system first uses a pre-calibrated camera intrinsic parameter matrix and distortion correction coefficients to determine the actual physical number of millimeters represented by a unit pixel in the image space. Subsequently, the circumferential compensation arc length is converted into pixel equivalents, and a reverse translation vector is applied to the initial center point coordinates or bounding box coordinates of the external defect.

[0115] To ensure that this physical compensation is strictly incorporated into the coordinate correction, the following formula is proposed for calculating the circumferential spatial coordinates of the corrected external defect feature:

[0116] ;

[0117] In the formula: The corrected circumferential spatial coordinates of the external defect features; The initial circumferential spatial coordinates of the external defect features are uncorrected. To compensate for the circumferential arc length; This is the physical size equivalent coefficient per pixel for a visual camera system.

[0118] By executing this formula algorithm, the system can map the space of the external defect, which has rotated due to force, to its theoretical spatial position under an unloaded reference state in the virtual data space using a coordinate rotation matrix. After this inverse rotation correction process, the circumferential spatial coordinates of the external defect features output by the visual modality are completely unified to the same reference frame as the electromagnetic probe, which is unaffected by rotation.

[0119] At this point, the corrected external defect features and the internal defect features extracted from the electromagnetic signals achieve high-precision spatial alignment in both longitudinal displacement and circumferential distribution. Thus, the differences and conflicts in the physical location attributes of the multimodal data are effectively compensated, paving the way for subsequent entry into the multimodal fusion decision module based on confidence and weights.

[0120] In summary, existing multimodal detection systems often assume that the object under test maintains ideal rigid linear motion during operation, relying solely on encoders for one-dimensional longitudinal distance alignment. However, under harsh conditions such as high-frequency start-stop and accelerated uphill driving, the untwisting of the wire rope under stress is a common and unavoidable physical phenomenon that cannot be completely constrained by the mechanical structure. If this phenomenon is ignored, the results of multimodal detection will lose their basis for fusion due to the drift of spatial coordinates, leading to frequent misjudgments by the system, identifying the same defect as multiple defects, and significantly reducing the accuracy of safety control commands.

[0121] The beneficial effects of the technical solution in this embodiment are as follows:

[0122] First, by directly extracting high-frequency strand texture direction features from the pre-processed visual image, the system can achieve real-time perception and accurate quantification of the three-dimensional dynamic torsional deformation of the wire rope without the need for any additional physical sensors (such as high-precision dynamic torque sensors or circumferential rotation displacement gauges), which greatly saves hardware deployment costs and improves the reliability of the system.

[0123] Second, through rigorous geometric modeling, the torsional angle deviation is transformed into a specific circumferential compensation arc length, and the external defect coordinates extracted by visual inspection are reversed and rotated to correct them. At the algorithm level, the misalignment of multi-sensor spatial observation caused by load changes is forcibly eliminated.

[0124] Third, this spatial circumferential alignment compensation mechanism, as a core pre-process of the multimodal fusion decision module, ensures that the data fed into the cross-arbitration mechanism have a high degree of physical homogeneity. This not only significantly reduces the false alarm rate of the system under complex variable load environments and reduces the data verification burden on on-site maintenance personnel due to data ghosting, but also provides a reliable input basis for the subsequent confidence weight allocation based on specific material correction factors and detection sensitivity. This greatly improves the scientific nature of the dynamic damage quantification of wire rope mesh units and the confidence interval of the remaining safe life prediction.

[0125] The overall solution forms a tight technical logic loop, from the bottom-level data space synchronization to the middle-level modal feature compensation and the top-level health status assessment. It comprehensively and deeply meets the stringent requirements of modern industry for high-precision and high-reliability online monitoring of large flexible load-bearing components.

[0126] Example 3:

[0127] In practical engineering applications such as mine shaft hoisting, ultra-long-distance passenger ropeways, or large port machinery, damage to wire ropes does not occur instantaneously. Instead, it is a continuous process in which minute defects, such as surface microcracks or internal broken wires, gradually expand and deteriorate under alternating loads, eventually evolving into severe damage. Therefore, the core task of the system is not only to identify what defects exist at the current detection moment, but also to track the extent to which specific defects have developed after months or years of service.

[0128] However, in historical comparisons across cycles, the system faces a serious problem of longitudinal spatial coordinate drift. When the wire rope is subjected to heavy load suspension and tensile alternating stress for a long time, its internal core is continuously compressed, and the metal strands undergo irreversible metallurgical creep, which in turn leads to cumulative plastic elongation of the wire rope body.

[0129] Meanwhile, due to the prevalence of oil stains, dust, and mechanical vibrations in industrial environments, incremental rotary encoders that come into frictional contact with the surface of the wire rope inevitably experience a continuous accumulation of microscopic slip during long-term measurement operations. The physical superposition of the aforementioned plastic elongation and encoder slip can lead to significant deviations in the absolute longitudinal displacement coordinates obtained from the same physical defect in different testing cycles (such as last month and this month).

[0130] Without a dynamic registration mechanism across cycles, the system may mistakenly identify historical defects with drifting coordinates as newly generated defects at the current location, resulting in a false increase in the number of defects and leading to serious false alarms of duplicate calculations.

[0131] To effectively solve this engineering challenge, the method in this embodiment includes a defect evolution tracking step for eliminating coordinate drift across detection cycles before quantifying the dynamic damage degree of the wire rope based on the defect detection results and real-time service condition data to calculate the overall health status index.

[0132] Specifically, for any current target defect output within the current detection cycle, the original electromagnetic signal fragment and visual image slice corresponding to the current target defect are extracted and fused to construct a multimodal spatiotemporal feature vector for the current target defect. After the multimodal fusion decision module outputs a confirmed defect, the system returns and calls the underlying cached data from the data acquisition and preprocessing stages.

[0133] For electromagnetic modes, the system extracts the original leakage magnetic signal segment with the physical center of the current target defect as the reference and extends a specific length range before and after it, and extracts the peak and trough amplitudes that reflect the intensity of leakage magnetic field distortion, the full width at half maximum (FWHM) of the signal that reflects the axial span of the defect, and the frequency band distribution of the signal's energy spectral density.

[0134] For visual modalities, the system accurately crops out visual image slices containing the texture shape of the current target defect based on the results of semantic segmentation and registration, and extracts the geometric aspect ratio, gray-level co-occurrence matrix features, and local binary pattern features of its boundary contours.

[0135] The system normalizes the vectors characterizing the electromagnetic field properties and the vectors characterizing the physical morphology of the optical surface, and then concatenates them in a multi-dimensional data space. Through this deep feature fusion and extraction, the system constructs a uniquely directional multimodal spatiotemporal feature vector for the current target defect. This feature vector can serve as a unique identifier for the physical defect, and because it integrates the material's internal magnetic flux properties and external optical texture, it possesses extremely high anti-counterfeiting and anti-interference capabilities.

[0136] For example, obtain the current displacement coordinates of the current target defect. Based on the pre-stored historical plastic elongation of the wire rope With encoder cumulative slip tolerance The length range of the spatial search window for the current target defect is determined; the length range is based on the current displacement coordinates. Centered on, its unilateral extension distance is .

[0137] Due to coordinate drift, the system cannot directly use absolute coordinates for a one-to-one hard comparison in the historical database. Instead, it needs to construct a dynamic and flexible search space. The system first reads the absolute count value corresponding to when the current target defect was detected, i.e., the current displacement coordinates. .

[0138] Subsequently, the system retrieves two key parameters from the equipment ledger or cloud model: one is the historical plastic elongation, which reflects the deformation characteristics of the wire rope material. This value is calculated through long-term tracking of the beginning and end markers; the other is the encoder cumulative slip tolerance, which reflects the extreme values ​​of mechanical errors in the field measurement system. These two parameters together constitute the physical extreme boundary of coordinate drift.

[0139] As the physical distance between the defect and the starting reference point for wire rope detection increases, i.e., the current displacement coordinate... The larger the value, the greater the absolute amount of drift caused by the cumulative effect of plastic elongation. The formula for calculating the unilateral extension distance of the spatial search window is proposed as follows:

[0140] ;

[0141] In the formula: The distance extended to one side of the spatial search window; The current displacement coordinates of the current target defect; The historical plastic elongation of the wire rope; This is the cumulative slip tolerance for the encoder.

[0142] Based on the calculation using the above formula, the system uses the current displacement coordinates. Using the geometric center point as the reference point, the physical length of the corresponding single-sided extension distance is extended in both the forward and reverse directions of the wire rope's operation. This allows for the precise delineation of a local one-dimensional spatial search interval within the entire rope data chain, which spans thousands of meters. This dynamically defined spatial search window fully encompasses all reasonable physical drift locations that the current target defect might experience.

[0143] Optionally, all historical defects falling within the spatial search window are traversed, and the multimodal feature similarity between the multimodal spatiotemporal feature vector of the current target defect and the historical multimodal spatiotemporal feature vectors corresponding to each historical defect is calculated.

[0144] After determining a reasonable spatial search window, the system retrieves historical defect databases from the previous detection period and even earlier periods from the cloud or local storage. To conserve computing power and reduce the false match rate, the system performs retrieval only within the spatial search window defined by the unilateral extension distance. The system sequentially reads the historical multimodal spatiotemporal feature vectors corresponding to each historical defect falling within this window.

[0145] Subsequently, in the multi-dimensional feature space, the system uses a cosine similarity algorithm or a weighted inverse Euclidean distance algorithm to calculate the spatial distance and directional angle between the multi-dimensional feature array of the current target defect and the feature array of each historical candidate object. This calculation process not only compares the intensity and distribution of the electromagnetic leakage magnetic field but also verifies the surface texture morphology at the visual image level. The calculation results of the multi-modal feature similarity reflect the degree of homology between defects detected in two different periods at the physical entity level.

[0146] It is also important to note that historical defects with the highest multimodal feature similarity and exceeding a preset homology matching threshold are selected as homology matching defects, and the current target defect is determined to be the continuous evolution result of the homology matching defect within the current detection period. The system sorts all similarity scores generated within the spatial search window in descending order.

[0147] The system first eliminates candidate objects whose similarity scores are below a preset homology matching threshold. This threshold is an empirically set confidence level lower limit used to filter out occasional similarities caused by random signal fluctuations. Among the remaining valid candidates, the system selects the historical defect with the highest multimodal feature similarity score. Once this unique mapping match is completed, the system confirms at the software logic level that although the longitudinal displacement coordinates given by these two detections differ by tens of centimeters or even meters, they are objectively the same damaged entity. Based on this, the system determines that the current target defect is not a newly occurring safety hazard, but rather the result of the continuous evolution of the homology-matched defect within the current detection period. Through this operation, the system successfully establishes an evolutionary tracking lifeline for a single specific defect on the timeline.

[0148] Specifically, the system extracts the current defect size of the current target defect and the historical defect size of the matching defects from the same source, calculates the morphological expansion gradient corresponding to the difference between the two, and introduces the morphological expansion gradient as an acceleration degradation factor into the subsequent calculation step of quantifying the dynamic damage degree of the wire rope, thereby weighting and increasing the dynamic damage degree of the wire rope containing the current target defect. After completing the merging of defects from the same source, the system further quantifies and analyzes the deterioration rate of the defects. The defect size includes the length parameter reflecting the axial span of the electromagnetic signal and the fracture gap width parameter identified by the visual image. The system extracts the current defect size measured in the current detection cycle and the historical defect size recorded in the historical database. By subtracting the physical dimensions over time, the system can clearly quantify the amount of physical expansion of the physical defect within the interval between two adjacent detection cycles.

[0149] To convert this time-based deterioration rate into a standard evaluation coefficient, the following formula is proposed for calculating the morphological expansion gradient:

[0150] ;

[0151] In the formula: Extend the gradient for morphological expansion; The current defect size for the current target defect; The historical defect size for origin-matching defects; The timestamp for executing the current detection cycle; The timestamp of the previous detection period that generated the historical defect.

[0152] The system calculates the size growth rate of the defect per unit time using this formula. This morphological expansion gradient directly reflects the local stress concentration and fatigue fracture propagation intensity of a specific damaged area under alternating loads. Subsequently, the system uses this morphological expansion gradient as an accelerated degradation factor in subsequent evaluation modules. When performing dynamic damage quantification calculations for this mesh element, the system multiplicatively weights and improves the initial damage score, which was originally based solely on the current static morphology, with this accelerated degradation factor.

[0153] For active defects with extremely large morphological expansion gradients and exhibiting a rapid tearing trend, the final output dynamic damage level will be significantly amplified; while for aging defects with expansion gradients approaching zero and exhibiting long-term passivation and stability, the damage level will be assessed at a conventional level. This weighted mechanism based on dynamic evolution gradients enables the damage quantification results to have real predictive power.

[0154] In general, existing non-destructive testing technologies for wire ropes typically employ a static coordinate comparison approach when processing data from multiple inspections across different lifecycles. Because they fail to overcome the coordinate reference drift caused by plastic elongation and encoder mechanical slippage during wire rope service, existing technologies are prone to misreporting historical defects with coordinate displacement as newly generated defects. This coordinate misalignment not only leads to frequent false alarms, significantly increasing the difficulty for maintenance personnel in troubleshooting massive amounts of repetitive reports, but more seriously, this processing method severs the evolutionary correlation of the same physical defect over time. This results in the system only outputting discrete, static slices of the defect's state, making it impossible to grasp the rate of defect deterioration and expansion, thus severely limiting the scientific rigor of lifespan prediction.

[0155] This embodiment achieves significant scheme optimization by introducing a defect evolution tracking step across detection cycles:

[0156] On the one hand, by integrating the original electromagnetic signal with visual slices to construct a multimodal spatiotemporal feature vector, and performing similarity traversal matching within a spatial search window dynamically defined by plastic elongation and slip tolerance, this solution successfully eliminates spatial misalignment interference caused by coordinate drift, achieves high-precision cross-cycle same-source tracking of defects in a single physical entity, effectively eliminates false alarms from repeated calculations in the system, and significantly improves the reliability and engineering usability of equipment monitoring reports;

[0157] On the other hand, by extracting the size difference across cycles to calculate the morphological expansion gradient and introducing it as an accelerated degradation factor feedforward into the quantification of dynamic damage, this scheme breaks the limitation of traditional assessment that relies solely on current static observations and successfully incorporates the deterioration trend over time (i.e., the acceleration of defect expansion) into the calculation framework of the overall health status index.

[0158] This enables the subsequent remaining safe life prediction module to no longer passively respond to static characteristics, but to dynamically adapt to the nonlinear accelerated degradation process of local damage. This provides a highly practical and forward-looking quantitative decision-making basis for the safety management of large critical load-bearing components in industrial sites, the formulation of preventive maintenance strategies, and the selection of the optimal replacement window for wire ropes.

[0159] Example 4:

[0160] like Figure 2 As shown in the figure, this embodiment provides a comprehensive detection system for internal and external defects of wire rope based on multimodal data fusion.

[0161] In view of the technical problems of existing single electromagnetic or visual inspection methods in the background technology, such as susceptibility to environmental interference in complex industrial sites, physical spatial misalignment of multi-source inspection data due to time and frequency acquisition mechanisms, and lack of quantitative life assessment by combining real-time service load of equipment, the comprehensive inspection system disclosed in this embodiment effectively overcomes the above defects through high-level synergy between software and hardware, and realizes closed-loop monitoring and control of the entire life cycle of wire rope.

[0162] Specifically, this embodiment discloses a comprehensive detection system for internal and external defects of wire ropes based on multimodal data fusion, comprising:

[0163] The data acquisition and mapping module is used to acquire the real-time belt speed and operating conditions of the running wire rope. Based on a displacement triggering mechanism, it synchronously acquires the electromagnetic signals and visual images of the wire rope, establishing a spatial position mapping relationship for multimodal data. The preprocessing and feature extraction module is used to perform adaptive drift-free preprocessing on the electromagnetic signals to extract internal defect features, and image stabilization and denoising preprocessing on the visual images to extract external defect features. The confidence calculation module is used to call preset wire rope material-specific characteristic factors to calculate the comprehensive confidence score of the internal defect verification corresponding to the internal defect features, and the confidence score of the... The system comprises: an external defect verification and comprehensive confidence level corresponding to the external defect characteristics; a multimodal fusion decision module, used to dynamically allocate weights based on the detection sensitivity of the defect type, the comprehensive confidence level of the internal defect verification, and the comprehensive confidence level of the external defect verification, and to perform multimodal fusion decision through a cross-arbitration mechanism to output the wire rope defect detection results; and a service assessment and control module, used to quantify the dynamic damage degree of the wire rope based on the wire rope defect detection results and real-time service condition data to calculate the overall health status index, and then predict the remaining safe life and output corresponding early warning level safety control instructions.

[0164] Specifically, in practical applications of large-scale special equipment, such as mine hoists and port quay cranes, the data acquisition and mapping module is typically physically deployed near the main shaft drum or key guide pulley block in the form of an integrated detection array. At the hardware level, this module includes a high-frequency industrial vision camera positioned on the side of the wire rope running path, an electromagnetic leakage magnetic field detection sensor assembly (such as a Hall element array) arranged in an array around the circumference of the wire rope, and a high-resolution incremental rotary encoder rigidly connected to the guide wheel spindle.

[0165] To address the data spatial misalignment problem described in the background section, this module integrates a Field-Programmable Gate Array (FPGA) chip. The FPGA chip possesses high-concurrency real-time processing capabilities, directly reading the pulse signals output by the incremental rotary encoder for displacement calculation. When the actual displacement of the wire rope reaches a set unit displacement threshold, the FPGA chip synchronously sends a hardware trigger level to the external trigger pins of the electromagnetic detection sensor and the industrial vision camera within a nanosecond delay. This purely hardware-based coordinated control forcibly transforms the acquisition reference of multimodal data from the time axis to the physical displacement axis, thereby establishing a strict physical spatial position mapping relationship between electromagnetic and visual data.

[0166] The preprocessing and feature extraction module typically serves as the core of the underlying data processing, housed within an edge-side real-time detection unit located close to the field sensors. Its physical carrier can be an industrial-grade edge computing device equipped with a digital signal processor (DSP) or an embedded graphics processing unit (GPU). This module receives massive amounts of raw signals from the underlying layer and performs parallel computations:

[0167] On the one hand, it drives the internal adaptive filtering circuit and frequency domain analysis algorithm to filter out the electromagnetic low-frequency baseline drift caused by the transverse mechanical vibration of the wire rope and the dynamic change of the lifting gap, and separates and extracts internal defect features such as leakage magnetic field distortion peaks that characterize the broken wires or cross-sectional wear inside the wire rope.

[0168] On the other hand, the image acceleration operator is invoked to perform global geometric correction and grid-level subpixel registration calculations on the acquired two-dimensional visual images, filtering out image ghosting and noise interference caused by mechanical vibration, and then extracting high signal-to-noise ratio texture features that characterize the surface fracture, rust pits or abnormal morphology of the wire rope.

[0169] The confidence calculation module includes a basic parameter database and a microprocessor unit pre-embedded in the system's non-volatile memory (such as high-speed flash memory). Given the objective differences in the physicochemical properties of steel wire ropes produced using different processes (such as bright finish, galvanizing, and phosphating) and twisting structures (such as point contact and line contact), this module performs standardized weighted calculations by calling pre-calibrated and stored steel wire rope material-specific characteristic factors (including permeability correction factors and surface roughness correction factors) and combining them with feature matching degrees. This calculation mechanism converts the raw physical parameters extracted by heterogeneous sensors into a unified-dimensional comprehensive confidence score for internal defect verification and a comprehensive confidence score for external defect verification. This hardware logic, which corrects data based on material property specificity, effectively eliminates the interference of basic material differences on signal detection rate, providing an objective and quantifiable confidence benchmark for subsequent judgment.

[0170] The multimodal fusion decision module is the logical decision-making center of the detection system. Upon receiving confidence data representing internal and external defects, it addresses scenarios where contradictory dual-modal detection results arise due to complex on-site interference. For example, oil contamination might cause the visual modality to issue a high-confidence external defect alarm, while the electromagnetic modality fails to detect magnetic leakage anomalies. This module executes a three-level cross-arbitration mechanism based on a fixed arbitration algorithm tree. The computational unit dynamically calculates and allocates decision weights by combining the objective physical sensitivity of different defect types to specific sensors (e.g., electromagnetic sensors are sensitive to metal cross-sectional area loss, and visual sensors are sensitive to surface broken wire morphology). The module's output effectively filters false alarms from single sensors, producing a high-confidence final wire rope defect detection result, effectively overcoming the limitations of single-modal detection.

[0171] The service assessment and control module primarily realizes the closed-loop transformation of detection data into safe execution actions in the industrial field. This module is equipped with standard industrial communication protocol interfaces (such as PROFINET and ModbusTCP) at the physical layer, directly maintaining real-time communication with the industrial field's Supervisory Control and Data Acquisition (SCADA) system or Programmable Logic Controller (PLC). This module reads the defect results output by the multimodal fusion decision module and simultaneously acquires real-time load data and service condition parameters such as impact frequency fed back by the PLC.

[0172] Subsequently, the internal time-series assessment network and damage accumulation model are invoked to transform the static defect size index into a quantified dynamic damage value, thereby calculating the overall health status index of the wire rope. Based on the quantified health index and the remaining safe life model predicted by the dual-drive system, this module outputs safety control commands with deterministic action directions to the hoist or crane main control system via the industrial bus, such as: "reduce the operating speed to 50% of the design belt speed", "prohibit heavy-load lifting operations", or "trigger the system emergency braking".

[0173] In summary, the system provided in this embodiment not only achieves high-precision synchronous spatial acquisition of multi-source sensors at the hardware level, but also constructs a complete closed loop in the software and logic architecture, from data preprocessing, specific characteristic correction, multimodal cross-fusion to final dynamic life assessment. Through the interactive collaboration between physical functional modules, it effectively implements various technical means to solve practical engineering problems, and ultimately achieves the beneficial effect of improving the long-term operational safety and controllability of heavy-duty special equipment.

[0174] Example 5:

[0175] Corresponding to the above embodiments, the present invention also proposes an electronic device.

[0176] like Figure 3 The diagram shows a structural schematic of an electronic device according to the present invention. The electronic device 100 includes a processor 101 and a memory 103. The processor 101 and the memory 103 are connected, for example, via a bus 102. Optionally, the electronic device 100 may further include a transceiver 104. It should be noted that in practical applications, the transceiver 104 is not limited to one unit, and the structure of this electronic device 100 does not constitute a limitation on the embodiments of the present invention.

[0177] Processor 101 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 101 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0178] Bus 102 may include a pathway for transmitting information between the aforementioned components. Bus 102 may be a PCI bus or an EISA bus, etc. Bus 102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0179] The memory 103 stores a computer program corresponding to the comprehensive detection method for internal and external defects of wire rope based on multimodal data fusion in the above embodiments of the present invention. This computer program is controlled and executed by the processor 101. The processor 101 executes the computer program stored in the memory 103 to implement the content shown in the aforementioned method embodiments.

[0180] Among them, electronic devices 100 include, but are not limited to: mobile terminals such as laptops and PADs (tablet computers) and fixed terminals such as desktop computers. Figure 3The electronic device 100 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0181] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A steel wire rope internal and external defect comprehensive detection method based on multi-modal data fusion, characterized in that, include: The real-time belt speed and operating conditions of the running wire rope are obtained, and the electromagnetic signals and visual images of the wire rope are synchronously collected based on the displacement triggering mechanism to establish the spatial position mapping relationship of multimodal data. The electromagnetic signal is subjected to adaptive drift-de-scraping preprocessing to extract internal defect features, and the visual image is subjected to image stabilization and denoising preprocessing to extract external defect features. By calling the preset steel wire rope material-specific characteristic factors, the comprehensive confidence level of internal defect verification corresponding to the internal defect characteristics and the comprehensive confidence level of external defect verification corresponding to the external defect characteristics are calculated respectively. Based on the detection sensitivity of the defect type, the comprehensive confidence of the internal defect review, and the comprehensive confidence of the external defect review, weights are dynamically allocated, and a multimodal fusion decision is made through a cross-arbitration mechanism to output the wire rope defect detection results. Based on the defect detection results of the wire rope and the real-time service condition data, the dynamic damage degree of the wire rope is quantified to calculate the overall health status index, thereby predicting the remaining safe life and outputting the corresponding early warning level safety control instructions.

2. The method according to claim 1, characterized in that, The method of synchronously acquiring electromagnetic signals and visual images of the wire rope based on a displacement triggering mechanism includes: Synchronous trigger pulses are generated by a field-programmable gate array and sent to an electromagnetic detection sensor and an industrial vision camera. The encoder pulse signal is collected in real time to count the displacement of the wire rope. Synchronous acquisition is triggered according to the preset unit displacement threshold to establish a one-to-one correspondence between the acquisition time, wire rope displacement, electromagnetic signal segments and visual image frames. Based on the real-time belt speed and operating conditions, the exposure parameters and sampling frame rate of the industrial vision camera are adaptively adjusted, as are the sampling frequency of the electromagnetic detection sensor.

3. The method according to claim 1, characterized in that, Before performing image stabilization and denoising preprocessing on the visual image, the process also includes semantic segmentation and mesh partitioning steps: The complete area of ​​the steel wire rope in the visual image is extracted by a semantic segmentation network and divided into the core stress area, the strand fixing area, the belt transition area and the edge non-sensitive area. Based on the pre-calibrated physical dimension mapping relationship on site, the core stress area and the rope strand fixing area are meshed; Each mesh cell after subdivision is assigned a unique identifier and its physical location information in the wire rope is bound to it. After removing invalid meshes, a set of valid mesh cells is obtained.

4. The method according to claim 3, characterized in that, The image stabilization and denoising preprocessing of the visual image adopts the following two-level registration mechanism: First-level global coarse registration: Extract stable feature points of the core stress area, calculate global spatial transformation parameters of the image through feature point matching, and complete the geometric correction of the image; Second-level mesh-level fine registration: For each effective mesh cell in the set of effective mesh cells, extract the corresponding local texture feature points within the mesh, and calculate the local deformation parameters based on the corresponding mesh of the initial healthy state of the wire rope to complete the mesh-level sub-pixel fine registration.

5. The method according to claim 1, characterized in that, The adaptive drift-de-scraping preprocessing of the electromagnetic signal to extract internal defect features includes: Collect circumferential multi-point gap data between the electromagnetic detection sensor and the wire rope, and predict the baseline drift value corresponding to the current gap using a pre-calibrated polynomial fitting model; The filter coefficients of the adaptive filter are dynamically adjusted based on the baseline drift prediction value to perform dynamic pipeline filtering on the original electromagnetic signal in order to eliminate the influence of baseline drift. The filtered electromagnetic signal is subjected to multi-scale wavelet decomposition to reconstruct the signal components of the target frequency band and extract the internal defect features.

6. The method according to claim 1, characterized in that, The step of calling preset wire rope material-specific characteristic factors to calculate the comprehensive confidence level of internal defect verification corresponding to the internal defect feature and the comprehensive confidence level of external defect verification corresponding to the external defect feature includes: calling the permeability correction factor and cross-sectional area loss correction factor in the wire rope material-specific characteristic factors, and combining them with feature matching degree and feature weight coefficient to calculate the comprehensive confidence level of internal defect verification; calling the surface roughness correction factor and fatigue damage correction factor in the wire rope material-specific characteristic factors, and combining them with feature matching degree and feature weight coefficient to calculate the comprehensive confidence level of external defect verification; calculating the dynamic judgment threshold of internal defects based on the service life and real-time load of the wire rope, and calculating the dynamic judgment threshold of external defects based on the surface condition and design fatigue limit stress of the wire rope; marking the defect feature regions where the comprehensive confidence level of internal defect verification is lower than the dynamic judgment threshold of internal defects, or the comprehensive confidence level of external defect verification is lower than the dynamic judgment threshold of external defects, as divergence point regions and temporarily storing them locally.

7. The method according to claim 1, characterized in that, The multimodal fusion decision-making process using a cross-arbitration mechanism, which outputs the wire rope defect detection results, includes: When the internal defect type and external defect type determination results corresponding to the same spatial location are inconsistent, it is determined as a conflict in the dual-modal detection results, and the following three-level cross-arbitration mechanism is executed: First-level confidence level arbitration: compare the overall confidence level of the internal defect review with the overall confidence level of the external defect review that cause conflict, and give priority to the detection result corresponding to the high confidence level mode whose confidence level difference is greater than the preset confidence level difference threshold; Secondary feature adaptation arbitration: If the confidence difference is not greater than the confidence difference threshold, priority is given to the conflicting defect type: if it is an internal defect type, the modal detection result corresponding to the electromagnetic signal is given priority; if it is an external defect type, the modal detection result corresponding to the visual image is given priority. Level 3 manual review marking: If the first-level confidence arbitration and the second-level characteristic adaptation arbitration still cannot determine the issue, the conflicting area will be marked as a high-risk suspicious area, and the corresponding multimodal data will be stored in a temporary storage for subsequent manual review.

8. The method according to claim 3, characterized in that, The quantification of the dynamic damage degree of the wire rope to calculate the overall health status index includes: Based on the wire rope defect detection results, the effective mesh cells containing the defects are extracted as target mesh cells; For each target mesh cell, the cumulative plastic deformation, plastic deformation concentration factor, tension non-uniformity factor, dynamic stiffness response factor, and the ratio of the real-time peak impact load of the wire rope to the rated load are obtained, and the dynamic damage degree of the corresponding target mesh cell is calculated by weighting. Based on the dynamic damage degree of all the target mesh cells, the overall health status index is calculated by dynamically summing and normalizing the regional weight coefficient and real-time load correction coefficient corresponding to each target mesh cell.

9. The method according to claim 1, characterized in that, The prediction of remaining safe lifetime adopts the following prediction framework driven by both physics and data: Physically driven foundation life prediction: Based on the linear fatigue cumulative damage criterion, wire rope fatigue curve and real-time operating data, the remaining safe operating days of the foundation are calculated, and the remaining safe operating days of the foundation are corrected according to the environmental correction factor. Data-driven evolution trend prediction: The spatiotemporal evolution features containing the time series data of the overall health status index are input into a long short-term memory network optimized by the attention mechanism to predict the remaining safe operating days driven by data. Dual-model weighted fusion prediction: The weights of the physical model and the data model are dynamically adjusted based on the quantitative results of the wire rope damage. The remaining safe operating days based on the basic model and the remaining safe operating days based on the data are weighted and fused to output the remaining safe life.

10. The method according to claim 6, characterized in that, The method is executed based on a collaborative architecture that includes an edge-side real-time detection unit and a cloud-based iterative optimization unit; The steps of adaptive drift-de-scraping preprocessing of the electromagnetic signal, image stabilization and denoising preprocessing of the visual image, and multimodal fusion decision-making are performed in the real-time detection unit on the edge side. The edge-side real-time detection unit periodically encrypts and uploads the data corresponding to the temporarily stored divergence point region to the cloud-based iterative optimization unit; The cloud-based iterative optimization unit annotates the data corresponding to the uploaded divergence point region to expand the scene-specific dataset, uses the expanded scene-specific dataset to incrementally train and validate the detection model, and distributes the iterated detection model to the edge-side real-time detection unit via remote update.

Images