Steel wire rope detection and real-time transmission method and system based on multi-modal data fusion

Abstract

The application discloses a kind of based on multimodal data fusion's steel wire rope detection and real-time transmission method and system, method includes: starting detection system, acquisition steel wire rope damage detection signal, acquisition equipment position and operating state data, construct original data set and pre-process, establish correlation matrix, after dividing processing unit extraction feature, form initial feature data set.Initial feature data set is extracted by multi-branch network damage feature and strengthens key area weight, and position, equipment state feature is fused by multimodal fusion module, generates fusion vector and classifies and identifies, and outputs structured detection result.Equipment end is classified to data, and transmission channel is distributed according to priority and dynamically adjusted parameter, and integrity and timing identifier are added, and host computer verifies data integrity and triggers abnormal supplement transmission.Host computer decodes data, reconstructs waveform, in combination with preset parameter real-time alarm, generates damage key information details.Provide reliable technical support for steel wire rope safe operation in industrial scene.

Description

Technical Field

[0001] This invention belongs to the field of industrial equipment testing technology, and more specifically, relates to a method and system for steel wire rope testing and real-time transmission based on multimodal data fusion. Background Technology

[0002] In industrial production, bridge construction, mining, and other fields, steel wire ropes are core load-bearing components, and their operating condition directly affects equipment safety and production efficiency. For a long time, steel wire ropes have been subjected to tensile forces, wear, and corrosion under complex working conditions, making them prone to damage such as broken wires, abrasion, and rust. If these damages are not detected and addressed in a timely manner, they may lead to equipment failure or even safety accidents, causing serious economic losses and safety risks.

[0003] Current wire rope inspection technologies rely on traditional manual methods that depend on the experience and judgment of inspectors. This is not only labor-intensive and inefficient, but also suffers from blind spots, making it difficult to accurately identify hidden damage and enabling real-time monitoring. Existing automated inspection technologies often focus on single-type detection signals, relying solely on single-modal data such as electromagnetic induction and visual imaging for damage assessment. They neglect the role of correlated data, such as the operating status and location information of the acquisition equipment. This results in inspection results that are easily affected by environmental interference, leading to insufficient accuracy in damage identification and an inability to comprehensively reflect the actual working condition of the wire rope.

[0004] In the data transmission stage, existing technologies lack differentiated processing mechanisms for detection data. All data adopts a uniform transmission strategy, which easily leads to wasted bandwidth resources. Moreover, in complex industrial environments, network conditions fluctuate significantly, and fixed transmission parameters are difficult to adapt to dynamic network environments, often resulting in data transmission delays and data loss, affecting the real-time performance and completeness of detection results. Furthermore, some transmission schemes lack effective data verification and retransmission mechanisms, failing to ensure the accuracy of data acquired at the receiving end, further reducing the reliability of the detection system.

[0005] With the increasing level of industrial intelligence, higher demands are being placed on the accuracy, real-time performance, and reliability of wire rope inspection. Existing inspection technologies, in terms of multi-source data integration, accurate damage identification, and stable and efficient transmission, are no longer sufficient to meet practical application needs. Therefore, developing a wire rope inspection and real-time transmission method that can integrate multimodal data, improve damage identification accuracy, and ensure stable data transmission is of significant practical importance for reducing safety risks, improving equipment maintenance efficiency, and ensuring industrial production safety. Summary of the Invention

[0006] This invention aims to address the problems of limited data dimensions, insufficient accuracy in damage identification, and poor data transmission stability in existing wire rope inspection technologies. By integrating multimodal data, optimizing feature extraction and fusion mechanisms, and establishing a hierarchical transmission and dynamic adaptation strategy, the accuracy and comprehensiveness of damage detection are improved, and the real-time and integrity of data transmission are ensured, providing reliable technical support for the safe operation and maintenance of wire ropes in industrial scenarios.

[0007] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method for steel wire rope detection and real-time transmission based on multimodal data fusion, comprising:

[0008] S1. Start the detection system, collect the detection signals, location data and equipment operation status data corresponding to the wire rope damage, and construct a multi-dimensional raw data set; after preprocessing the raw data, establish a multi-dimensional data association matrix; and divide the preprocessed data into processing units, extract the feature information of each processing unit, and form an initial feature dataset.

[0009] S2. A multi-branch feature extraction network is used to deeply extract the detection signal features corresponding to different types of wire rope damage. The feature weight of key damage areas is highlighted through a feature enhancement mechanism. A multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment status features to generate a multi-modal fusion feature vector, which is then classified and identified to output damage-related information and form structured detection result data.

[0010] S3. At the acquisition device end, the structured detection result data is processed according to priority: high-priority data retains complete features, while regular data only retains core features; at the same time, transmission channels are allocated according to data priority and transmission parameters are dynamically adjusted according to network status to ensure stable communication with the host computer; integrity verification and timing identifiers are added to the transmitted data, and the host computer confirms the integrity of the data by verifying the identifiers after receiving it, and triggers a retransmission mechanism for abnormal data.

[0011] S4. On the host computer, decode the received compressed data and reconstruct the detection waveform corresponding to the wire rope damage; combine the preset judgment parameters to perform real-time alarm judgment on the structured detection result data and generate alarm details containing key damage information.

[0012] Furthermore, the original data set in S1 specifically includes:

[0013] Assuming the detection system starts up, during the detection time... The internally collected wire rope damage detection signal is ,in, To detect the start time; To detect the current time; ; Characteristic signals corresponding to broken wires Characteristic signals corresponding to wear constitute; The amplitude-time series signal output by the sensing component in the corresponding area; The amplitude-time series signal output by the sensing component in another corresponding area;

[0014] The collected location data is , Including cumulative displacement data acquired in real time by the encoder Instantaneous velocity data calculated based on the rate of change of displacement ,in, ;

[0015] The data collected is the operating status data of the equipment. , Includes motor operating parameter vectors Battery power state vector and equipment working status indicators ;

[0016] in, , Provide power voltage to the motor. This is the motor operating current. Motor speed; battery power state vector , This is the real-time battery voltage. This represents the percentage of remaining battery power. This indicates that the equipment is operating normally. This indicates a device malfunction;

[0017] Multidimensional raw data set is defined as ,in:

[0018] ,

[0019] ,

[0020] ,

[0021] In the formula, To detect time series, It is a time variable.

[0022] Furthermore, the process of constructing the initial feature dataset in S1 is as follows:

[0023] Based on the denoised damage detection signal obtained after preprocessing , Calibrated position data and standardized operating status data , , To detect the time domain, a multi-dimensional data correlation matrix is ​​constructed. , matrix number row element is ,in, The number of sampling points. ;

[0024] Set the preset frame length ,pass Will Divided into continuous submatrices , To round up; for each submatrix , ,extract Extract the corresponding time-domain and frequency-domain features. Corresponding cumulative average displacement, The mode of the corresponding equipment operating status indicator;

[0025] The above features are concatenated to form a feature vector. , by all Composition of initial feature dataset This provides input for the multi-branch feature extraction network.

[0026] Furthermore, the feature enhancement mechanism in S2 is specifically as follows:

[0027] Different types of wire rope damage feature vectors output by a multi-branch feature extraction network , The number of damage types corresponds to the detection signal characteristics of damages such as broken wires and wear.

[0028] Define the damage feature significance evaluation function:

[0029] ,

[0030] in, The global mean of features of the same damage type. The global standard deviation, For the first Each processing unit corresponds to a damage feature component of that type; The total number of processing units;

[0031] This function quantifies the significance of each element in each damage feature vector. The larger the value, the more significant the difference between the corresponding feature and the normal state feature, which means it is more likely to correspond to a key damage area.

[0032] Simultaneously, a feature response threshold is introduced. , Determined based on statistical analysis of historical non-destructive wire rope inspection data; for each damage feature vector Perform element-by-element judgment: when When, retain the feature element and keep its original value; when When this characteristic element is subjected to suppression, that is, its value is reduced to zero, where, Indexed by the dimension of the feature vector;

[0033] The above process enhances the key damage area features while suppressing interference from ineffective background features, resulting in an enhanced damage feature vector. This provides highly recognizable core damage feature inputs for subsequent multimodal fusion processing.

[0034] Furthermore, the form of the multimodal fusion feature vector in S2 is specifically as follows:

[0035] Suppose that after the multi-branch feature extraction network is processed by the feature enhancement mechanism, the output enhancement feature vectors for different types of damage are: , The number of damage types corresponds to damage such as broken wires and wear; the calibrated position feature vector is... , For the cumulative displacement characteristics after calibration, The instantaneous velocity feature is; the standardized equipment state feature vector is... , Characteristics of motor operating status, Characteristics of the battery-powered state;

[0036] Define modal correlation factor

[0037] ,

[0038] ,

[0039] in, Let covariance function be used. The variance function quantifies the dynamic correlation strength between damage features and location features and equipment status features;

[0040] The multimodal fusion feature vector adopts a composite form of core feature dominance + related feature adaptation, defined as:

[0041] ,

[0042] in, This represents the dimension stacking operation representing different types of damage features. This represents element-wise multiplication; the first half of the fusion vector is a stacked representation of the core features of each type of damage, and the second half is an adapted representation of location features, equipment state features, and corresponding modal correlation factors, ultimately yielding... , To integrate the total dimension of features.

[0043] Furthermore, the process of dynamically adjusting transmission parameters according to network status in S3 is as follows:

[0044] Real-time acquisition of network transmission status between the device and the host computer, and extraction of network bandwidth. Transmission delay and packet loss rate Three core state parameters, among which, To monitor time variables;

[0045] Define network quality evaluation indicators Quantify the current network transmission capacity:

[0046] ,

[0047] Preset network quality threshold range ,when When the network conditions are good, the data transmission rate in the transmission parameters is set to the maximum value. Using the default data packet length ;

[0048] when That is, when the network condition is moderate, the transmission rate will be adjusted to... , The network bandwidth utilization coefficient is determined based on historical transmission data statistics, and the data packet length is appropriately reduced to... ;

[0049] when That is, when the network conditions are poor, the transmission rate will be reduced to [a lower value]. Further reduce the data packet length to At the same time, a data packet fragmentation transmission mechanism is enabled.

[0050] Furthermore, the method for adding the integrity verification identifier and timing identifier in S3 is as follows:

[0051] Integrity verification identifier added: for single structured inspection result data after hierarchical processing. , For each data sequence number, a hash algorithm is used to calculate its feature digest. ,Will As an integrity verification identifier, and Binding storage; the hash algorithm is selected from the SHA series algorithm or the CRC series verification algorithm, which uniquely maps the original data through a fixed-length feature digest to ensure that the data is not tampered with or damaged during the data transmission process;

[0052] Adding time sequence identifiers: based on the detection time domain of the acquisition device. ,extract Corresponding acquisition time ,Will Convert to standardized timestamp Serves as a timing identifier; simultaneously records Ordinal index in the overall data sequence ,Will and Combined into a composite time sequence identifier , attached to The header field serves as a unique temporal identifier for the data;

[0053] Ultimately, the format of each piece of data to be transmitted is defined as follows: This ensures both the integrity and verifiability of the data, and provides a dual timing basis for the timing calibration of the host computer.

[0054] Furthermore, in S4, the waveform distinguishes the detection signals of different regions through differential identifiers. The specific process is as follows:

[0055] The host computer decodes the damage detection signal. The system analyzes and identifies the probe acquisition area corresponding to the signal, and assigns differentiated identification rules based on the area type: for the acquired broken wire signal... The first identification rule is adopted, that is, the color of the wave-shaped line is set to red and the line width is set to no less than 2pt;

[0056] For the collected wear signals The second identification rule is adopted, that is, the waveform line color is set to blue, the line width is set to no less than 2pt, and a dashed line style is added;

[0057] Damage auxiliary signals acquired from the auxiliary sensing area The third identification rule is adopted, that is, the color of the waveform line is set to green, the line width is set to no less than 1pt, and a dotted line style is added;

[0058] Meanwhile, an identification panel is created on the waveform display interface to associate each differentiated identifier with the corresponding acquisition area and signal type. Through visually differentiated identifier settings, the detection signals from different areas can be clearly distinguished in the same waveform.

[0059] As a second aspect of the present invention, a wire rope detection and real-time transmission system based on multimodal data fusion is also provided, comprising:

[0060] The data acquisition and feature construction unit is used to start the detection system, collect detection signals, location data and equipment operating status data corresponding to wire rope damage, and construct a multi-dimensional raw data set; after preprocessing the raw data, a multi-dimensional data association matrix is ​​established; and the preprocessed data is divided into processing units, and the feature information of each processing unit is extracted to form an initial feature dataset.

[0061] The feature extraction, fusion, and recognition unit is used to extract the detection signal features corresponding to different types of wire rope damage using a multi-branch feature extraction network. The feature weight of key damage areas is highlighted through a feature enhancement mechanism. A multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment status features to generate a multi-modal fusion feature vector, which is then classified and identified to output damage-related information and form structured detection result data.

[0062] The hierarchical transmission and integrity assurance unit is used to process structured detection result data according to priority at the acquisition device end: high-priority data retains complete features, while regular data only retains core features; at the same time, it allocates transmission channels according to data priority and dynamically adjusts transmission parameters according to network status to ensure stable communication with the host computer; it adds integrity verification and timing identifiers to the transmitted data, and the host computer confirms the data integrity through the verification identifiers after receiving the data, and triggers a retransmission mechanism for abnormal data;

[0063] The waveform reconstruction and real-time alarm unit is used to decode the received compressed data on the host computer and reconstruct the detection waveform corresponding to the wire rope damage; it combines preset judgment parameters to perform real-time alarm judgment on the structured detection result data and generate alarm details containing key damage information.

[0064] As a third aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, which is executed by a processor as described in any one of the claims, a method for steel wire rope detection and real-time transmission based on multimodal data fusion.

[0065] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0066] 1. The present invention provides a method for steel wire rope detection and real-time transmission based on multimodal data fusion. By activating a data acquisition device, it simultaneously acquires detection signals corresponding to steel wire rope damage, device location data, and operational status data, constructing a multi-dimensional raw data set. After preprocessing the raw data, a multi-dimensional data association matrix is ​​established, processing units are divided, and feature information of each unit is extracted to form an initial feature dataset. This technology achieves comprehensive capture and orderly integration of multiple data types, avoiding the limitations of a single data dimension, providing richer basic information support for subsequent analysis. Simultaneously, through preprocessing and feature extraction, core effective information is filtered out, reducing redundant data interference, providing accurate and efficient input data for subsequent damage identification, and ensuring the relevance and reliability of the detection and analysis.

[0067] 2. The wire rope detection and real-time transmission method based on multimodal data fusion of the present invention employs a multi-branch feature extraction network to deeply extract detection signal features corresponding to different types of wire rope damage. A feature enhancement mechanism is used to highlight the feature weights of key damage areas. Then, a multimodal fusion module fuses the extracted damage features with calibrated position features and equipment status features to generate a multimodal fusion feature vector for classification and identification. This technology achieves accurate extraction and enhancement of different types of damage features, while breaking down information barriers between different modal data, allowing for deep integration of damage information and scene-related information. This improves the accuracy and comprehensiveness of damage identification, effectively distinguishing different types of wire rope damage and providing reliable technical support for subsequent detection result output.

[0068] 3. The multimodal data fusion-based steel wire rope detection and real-time transmission method of the present invention prioritizes and processes structured detection results data at the acquisition device, allocates transmission channels according to priority, dynamically adjusts transmission parameters based on network status, adds integrity verification and timing identifiers to the transmitted data, verifies data integrity upon receipt by the host computer, triggers a retransmission mechanism for abnormal data, and finally decodes the data, reconstructs the waveform, and issues real-time alarms based on preset parameters. This technical feature achieves efficient and stable transmission of detection data. Through hierarchical processing and dynamic parameter adjustment, it adapts to different network environments. Integrity identifiers and retransmission mechanisms ensure the accuracy and integrity of data transmission, while the real-time alarm function provides timely feedback on steel wire rope damage, offering timely and effective data support for equipment maintenance and safety assurance. Attached Figure Description

[0069] Figure 1 This is a flowchart of a wire rope detection and real-time transmission method based on multimodal data fusion according to an embodiment of the present invention.

[0070] Figure 2 This is a schematic diagram of the data acquisition device according to an embodiment of the present invention;

[0071] Figure 3 This is a schematic diagram of historical playback in an embodiment of the present invention;

[0072] Figure 4 This is a system unit diagram of an embodiment of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0074] Example 1

[0075] Please refer to Figure 1 This embodiment 1 provides a method for steel wire rope detection and real-time transmission based on multimodal data fusion, including:

[0076] S1. Start the detection system, collect the detection signals, location data and equipment operation status data corresponding to the wire rope damage, and construct a multi-dimensional raw data set; after preprocessing the raw data, establish a multi-dimensional data association matrix; and divide the preprocessed data into processing units, extract the feature information of each processing unit, and form an initial feature dataset.

[0077] S2. A multi-branch feature extraction network is used to deeply extract the detection signal features corresponding to different types of wire rope damage. The feature weight of key damage areas is highlighted through a feature enhancement mechanism. A multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment status features to generate a multi-modal fusion feature vector, which is then classified and identified to output damage-related information and form structured detection result data.

[0078] S3. At the acquisition device end, the structured detection result data is processed according to priority: high-priority data retains complete features, while regular data only retains core features; at the same time, transmission channels are allocated according to data priority and transmission parameters are dynamically adjusted according to network status to ensure stable communication with the host computer; integrity verification and timing identifiers are added to the transmitted data, and the host computer confirms the integrity of the data by verifying the identifiers after receiving it, and triggers a retransmission mechanism for abnormal data.

[0079] S4. On the host computer, decode the received compressed data and reconstruct the detection waveform corresponding to the wire rope damage; combine the preset judgment parameters to perform real-time alarm judgment on the structured detection result data and generate alarm details containing key damage information.

[0080] This embodiment 1 further elaborates on the above steps.

[0081] (1) Data acquisition and feature construction

[0082] In industrial settings, steel wire ropes serve as core load-bearing components, and the accuracy of their damage detection heavily relies on the comprehensive acquisition and effective processing of multi-dimensional data. Considering that single data points cannot fully reflect the damage to the steel wire rope and the state of the detection environment, the data acquisition equipment, upon startup, does not merely collect a single signal but formally initiates the entire process of multi-source data acquisition and feature construction, laying a solid data foundation for subsequent accurate damage identification.

[0083] Please refer to Figure 2 This example uses a probe shoe as the data acquisition device. The same probe shoe, equipped with wear-resistant sleeves of different rope diameters, can detect steel wire ropes of different diameters. Typically, one probe shoe can detect up to five different diameter steel wire ropes. Different areas of the probe shoe can collect electromagnetic induction signals corresponding to broken wires and wear, while the wear-resistant sleeve can reduce the direct friction between the probe shoe and the steel wire rope during the detection process, ensuring that the sensing components can collect amplitude-time series signals stably over a long period of time. This provides reliable hardware support for constructing steel wire rope damage detection signals in a multi-dimensional raw data set.

[0084] Within a preset detection time range—the entire period from the start of detection to the current detection time—the data acquisition device simultaneously collects three types of key data through its dedicated sensing system, ensuring a one-to-one correspondence between the data points over time. Specifically, the wire rope damage detection signal is captured by two separate areas of the target. This partitioned acquisition design is based on the actual working conditions where wire breakage and wear often occur in different parts of the wire rope: the electromagnetic induction sensor in the upper area specifically collects amplitude-time series signals related to wire breakage, while the similar sensor in the lower area focuses on the electromagnetic induction amplitude-time series signals corresponding to wear. These two types of signals together constitute the core data for damage detection.

[0085] Meanwhile, the position data of the acquisition device is obtained through a combination of encoder and calculation. The encoder collects cumulative displacement data in real time to determine the detection position, and then calculates instantaneous velocity data based on the displacement change rate. The combination of these two methods fully reflects the position and movement status of the acquisition device during the detection process, providing a basis for subsequent damage location. In addition, to ensure the reliability of the detection process and avoid the impact of abnormal equipment conditions on data accuracy, the acquisition device also simultaneously collects operating status data, including operating parameters such as motor power supply voltage, operating current, and speed; power supply status data such as real-time battery voltage and remaining power percentage; and information on normal or abnormal operating status of the equipment marked with specific identifiers. Integrating these three types of data constitutes a multi-dimensional raw data set.

[0086] In a preferred embodiment, the original data set is represented in the following way:

[0087] Assuming the detection system starts up, during the detection time... The internally collected wire rope damage detection signal is ,in, To detect the start time; To detect the current time; ; Characteristic signals corresponding to broken wires Characteristic signals corresponding to wear constitute; The amplitude-time series signal output by the sensing component in the corresponding area; The amplitude-time series signal output by the sensing component in another corresponding area;

[0088] The collected location data is , Including cumulative displacement data acquired in real time by the encoder Instantaneous velocity data calculated based on the rate of change of displacement ,in, ;

[0089] The data collected is the operating status data of the equipment. , Includes motor operating parameter vectors Battery power state vector and equipment working status indicators ;

[0090] in, , Provide power voltage to the motor. This is the motor operating current. Motor speed; battery power state vector , This is the real-time battery voltage. This represents the percentage of remaining battery power. This indicates that the equipment is operating normally. This indicates a device malfunction;

[0091] Multidimensional raw data set is defined as ,in:

[0092] ,

[0093] ,

[0094] ,

[0095] In the formula, To detect time series, It is a time variable.

[0096] Meanwhile, due to noise, errors, and dimensional differences in the raw data, direct use for analysis would reduce accuracy. Therefore, preprocessing of the raw data is necessary: ​​a denoising algorithm removes environmental and other interference noise from the damage detection signal; a calibration algorithm corrects accumulated errors in the location data; and standardization eliminates the dimensional influence of the operational status data, resulting in three purified data categories. To achieve multi-dimensional data correlation analysis, a multi-dimensional data correlation matrix is ​​constructed based on the processed data. Damage detection, location, and operational status data at the same sampling time are arranged in a row of the matrix, with the number of rows matching the number of sampling points.

[0097] Considering the difficulty of directly processing long, continuous data sequences and the ease with which local features are lost, frame-based processing is necessary. A fixed frame length is set, and the number of frames is calculated by rounding up. The correlation matrix is ​​then divided into multiple continuous sub-matrices, each corresponding to a segment of continuous detection data. For each sub-matrix, key features of each dimension of the data are extracted: time-domain and frequency-domain features are extracted from the damage detection signal to reflect damage characteristics; the cumulative displacement mean is extracted from the location data to reflect the location distribution of the detection segment; and the mode of the equipment operating status identifier is extracted from the operating status data to confirm the overall operating status of the equipment during the detection segment.

[0098] These extracted features are concatenated in a fixed order to form feature vectors. The feature vectors corresponding to all sub-matrices are combined to form the initial feature dataset. This dataset integrates the core information of multi-dimensional data and provides suitable input data for subsequent multi-branch feature extraction networks.

[0099] Specifically, the process of constructing the initial feature dataset is as follows:

[0100] Based on the denoised damage detection signal obtained after preprocessing , Calibrated position data and standardized operating status data , , To detect the time domain, a multi-dimensional data correlation matrix is ​​constructed. , matrix number row element is ,in, The number of sampling points. ;

[0101] Set the preset frame length ,pass Will Divided into continuous submatrices , To round up; for each submatrix , ,extract Extract the corresponding time-domain and frequency-domain features. Corresponding cumulative average displacement, The mode of the corresponding equipment operating status indicator;

[0102] The above features are concatenated to form a feature vector. , by all Composition of initial feature dataset This provides input for the multi-branch feature extraction network.

[0103] (2) Feature extraction, fusion and recognition

[0104] In wire rope damage detection, the characteristic signals corresponding to different types of damage often differ. Single feature extraction methods struggle to accurately capture the core information of various damage types, and the original features often contain invalid background interference, affecting subsequent recognition accuracy. Furthermore, the disconnect between damage features and contextual information such as detection location and equipment status leads to a lack of comprehensiveness in the detection results. To address these issues, targeted feature extraction, enhancement, and fusion strategies are needed to construct more discriminative comprehensive features.

[0105] Based on the initial feature dataset constructed in the early stage, a multi-branch feature extraction network is used for deep feature mining. Each branch of the network corresponds to a specific type of wire rope damage, which can accurately focus on the detection signal features corresponding to different damages such as wire breakage and wear, and realize the separate extraction of various damage features, avoiding mutual interference between different types of damage features.

[0106] After obtaining the feature vectors of different types of damage, a feature enhancement mechanism is used to highlight the feature weights of key damage regions. The specific feature enhancement mechanism is as follows:

[0107] Different types of wire rope damage feature vectors output by a multi-branch feature extraction network , The number of damage types corresponds to the detection signal characteristics of damages such as broken wires and wear.

[0108] Define the damage feature significance evaluation function:

[0109] ,

[0110] in, The global mean of features of the same damage type. The global standard deviation, For the first Each processing unit corresponds to a damage feature component of that type; The total number of processing units;

[0111] This function quantifies the significance of each element in each damage feature vector. The larger the value, the more significant the difference between the corresponding feature and the normal state feature, which means it is more likely to correspond to a key damage area.

[0112] Simultaneously, a feature response threshold is introduced. , Determined based on statistical analysis of historical non-destructive wire rope inspection data; for each damage feature vector Perform element-by-element judgment: when When, retain the feature element and keep its original value; when When this characteristic element is subjected to suppression, that is, its value is reduced to zero, where, Indexed by the dimension of the feature vector;

[0113] The above process enhances the key damage area features while suppressing interference from ineffective background features, resulting in an enhanced damage feature vector. This provides highly recognizable core damage feature inputs for subsequent multimodal fusion processing.

[0114] Considering that accurate damage identification requires not only the characteristics of the damage itself but also the integration of related information such as detection location and equipment operating status, a multimodal fusion module is introduced for feature integration. First, the various features involved in the fusion are defined: enhanced feature vectors of different types of damage, calibrated position feature vectors (including cumulative displacement and instantaneous velocity features), and standardized equipment status feature vectors (including motor operating status and battery power supply status features). To achieve effective correlation among these features, a modal correlation factor is defined. By calculating the ratio of the covariance to the square root of the product of the variances of the two types of features, the dynamic correlation strength between each damage feature and the position and equipment status features is quantified.

[0115] The multimodal fusion feature vector is constructed using a composite approach of "core feature dominance + related feature adaptation," specifically:

[0116] Suppose that after the multi-branch feature extraction network is processed by the feature enhancement mechanism, the output enhancement feature vectors for different types of damage are: , The number of damage types corresponds to damage such as broken wires and wear; the calibrated position feature vector is... , For the cumulative displacement characteristics after calibration, The instantaneous velocity feature is; the standardized equipment state feature vector is... , Characteristics of motor operating status, Characteristics of the battery-powered state;

[0117] Define modal correlation factor

[0118] ,

[0119] ,

[0120] in, Let covariance function be used. The variance function quantifies the dynamic correlation strength between damage features and location features and equipment status features;

[0121] The multimodal fusion feature vector adopts a composite form of core feature dominance + related feature adaptation, defined as:

[0122] ,

[0123] in, This represents the dimension stacking operation representing different types of damage features. This represents element-wise multiplication; the first half of the fusion vector is a stacked representation of the core features of each type of damage, and the second half is an adapted representation of location features, equipment state features, and corresponding modal correlation factors, ultimately yielding... , To integrate the total dimension of features.

[0124] After generating the multimodal fusion feature vector, a precise classification and recognition process is needed to extract key damage information and form standardized, structured detection results data. First, the fusion feature vector is input into a pre-defined classification and recognition model. This model is built on a deep learning network and trained and optimized using historical damage sample data, possessing deep analytical capabilities for multimodal fusion features. During the classification and recognition process, the model first performs feature space mapping on the fusion feature vector to further explore the intrinsic relationship between damage features and scene features. Then, through the collaborative work of a multilayer perceptron and a Softmax classifier, it achieves accurate determination of the damage type, clearly distinguishing different damage categories such as broken wires and wear.

[0125] Simultaneously, the model incorporates enhanced damage feature components from the fused features to quantitatively assess the degree of damage: by comparing the amplitude of key damage elements in the feature vector with preset damage level thresholds, the degree of damage is divided into three levels: mild, moderate, and severe. The threshold standards are determined based on wire rope safety operation and maintenance specifications and historical inspection data statistics. Furthermore, utilizing the location correlation information in the fused features, the specific location of the damage is accurately pinpointed. Combining the cumulative displacement features and instantaneous velocity features of the acquisition equipment, the axial coordinates and circumferential orientation of the wire rope corresponding to the damage are calculated, ensuring the accuracy of the damage location information.

[0126] After classification and identification, the output contains the original identification results, including core damage information such as damage type, damage severity, damage location, detection time, and equipment operating status. To facilitate subsequent transmission, storage, and parsing, the original identification results need to be structured: according to preset data field specifications, each piece of information is organized into key-value pairs of structured data. Damage type and severity are represented using standardized codes, damage location is presented as coordinate values, detection time is in timestamp format, and equipment operating status is identified by status codes.

[0127] After structuring, the data undergoes format validation to ensure the completeness of each field, consistency of data types, and reasonable logical relationships, avoiding invalid data or erroneous information. The final structured inspection results data not only contain the core characteristics of the damage but also have a unified format standard, meeting the application requirements of subsequent graded transmission, real-time alarms, data archiving, and other stages, providing comprehensive, accurate, and standardized data support for the safe operation and maintenance of wire ropes.

[0128] (3) Hierarchical transmission and integrity guarantee

[0129] In steel wire rope inspection in industrial settings, the quality of data transmission directly affects the real-time assessment and response efficiency of the host computer regarding damage. Considering the varying urgency of different damage information and the susceptibility of network conditions to interference and fluctuations in industrial environments, adopting a uniform transmission strategy may lead to delays in the transmission of critical damage information or data loss due to inadequate network adaptation. Therefore, it is necessary to construct a targeted, tiered transmission and protection mechanism.

[0130] After the structured inspection results are generated, they are first prioritized at the acquisition device. The prioritization criteria are based on the urgency and importance of the damage: inspection data involving severe damage or damage to critical areas are classified as high priority. This type of data is directly related to equipment safety and must retain complete feature information to ensure that the host computer obtains comprehensive damage details; data related to ordinary minor damage and routine inspection areas are classified as regular priority, retaining only core feature information and removing redundant content to reduce data transmission volume and improve transmission efficiency.

[0131] After tiering, dedicated transmission channels are allocated according to priority. High-priority data occupies channels with even higher transmission priority to ensure its transmission is unaffected by regular data and reaches the host computer first. Simultaneously, the network transmission status between the data acquisition device and the host computer is monitored in real time, focusing on extracting three core parameters: network bandwidth, transmission latency, and packet loss rate. These three parameters are used to comprehensively quantify the current network transmission capacity. The process of dynamically adjusting transmission parameters based on network status is as follows:

[0132] Real-time monitoring of network transmission status between the data acquisition device and the host computer, and extraction of network bandwidth. Transmission delay and packet loss rate Three core state parameters, among which, To monitor time variables;

[0133] Define network quality evaluation indicators Quantify the current network transmission capacity:

[0134] ,

[0135] Preset network quality threshold range ,when When the network conditions are good, the data transmission rate in the transmission parameters is set to the maximum value. Using the default data packet length ;

[0136] when That is, when the network condition is moderate, the transmission rate will be adjusted to... , The network bandwidth utilization coefficient is determined based on historical transmission data statistics, and the data packet length is appropriately reduced to... ;

[0137] when That is, when the network conditions are poor, the transmission rate will be reduced to [a lower value]. Further reduce the data packet length to At the same time, a data packet fragmentation transmission mechanism is enabled.

[0138] To ensure the accuracy and timing consistency of transmitted data, an integrity verification identifier and a timing identifier are added to each piece of data to be transmitted. The method for adding the integrity verification identifier and the timing identifier is as follows:

[0139] Integrity verification identifier added: for single structured inspection result data after hierarchical processing. , For each data sequence number, a hash algorithm is used to calculate its feature digest. ,Will As an integrity verification identifier, and Binding storage; the hash algorithm is selected from the SHA series algorithm or the CRC series verification algorithm, which uniquely maps the original data through a fixed-length feature digest to ensure that the data is not tampered with or damaged during the data transmission process;

[0140] Adding time sequence identifiers: based on the detection time domain of the acquisition device. ,extract Corresponding acquisition time ,Will Convert to standardized timestamp Serves as a timing identifier; simultaneously records Ordinal index in the overall data sequence ,Will and Combined into a composite time sequence identifier , attached to The header field serves as a unique temporal identifier for the data;

[0141] Ultimately, the format of each piece of data to be transmitted is defined as follows: This ensures both the integrity and verifiability of the data, and provides a dual timing basis for the timing calibration of the host computer.

[0142] Ultimately, each piece of data to be transmitted is formatted into a unified format containing structured detection results, integrity verification identifiers, and composite timing identifiers. Upon receiving the data, the host computer first verifies its integrity by comparing and calculating the integrity verification identifiers. If any anomalies such as missing, corrupted, or tampered data are detected, a retransmission mechanism is immediately triggered, sending a retransmission request to the acquisition device to ensure that critical data is not lost. Through the synergistic effect of hierarchical processing, dynamic parameter adjustment, dual identifier addition, and anomaly retransmission mechanisms, efficient, stable, and accurate transmission of detection data is achieved, providing reliable data support for subsequent waveform reconstruction and alarm judgment by the host computer.

[0143] (4) Waveform reconstruction and real-time alarm

[0144] In industrial settings, the host computer, acting as the core receiver and processor of wire rope inspection data, must transform the transmitted compressed data into intuitive inspection results and promptly report damage risks to provide a basis for maintenance decisions. Since the data undergoes hierarchical compression during transmission, the host computer must first decode the received compressed data, using a corresponding decompression algorithm to restore the original inspection signal and structured data, ensuring the accuracy of subsequent processing.

[0145] After decoding, the host computer begins reconstructing the detection waveform corresponding to the wire rope damage. Considering that signals from different acquisition areas correspond to different types of damage or auxiliary detection information, using a uniform identifier would easily lead to signal confusion and make it difficult to quickly distinguish key damage information. Therefore, a differentiated identifier strategy is introduced during waveform reconstruction, assigning specific display rules based on the signal acquisition area type.

[0146] Specifically, the waveform uses differentiated identifiers to distinguish the detection signals in different regions. The specific process is as follows:

[0147] The host computer decodes the damage detection signal. The system analyzes and identifies the probe acquisition area corresponding to the signal, and assigns differentiated identification rules based on the area type: for the acquired broken wire signal... The first identification rule is adopted, that is, the color of the wave-shaped line is set to red and the line width is set to no less than 2pt;

[0148] For the collected wear signals The second identification rule is adopted, that is, the waveform line color is set to blue, the line width is set to no less than 2pt, and a dashed line style is added;

[0149] Damage auxiliary signals acquired from the auxiliary sensing area The third identification rule is adopted, that is, the color of the waveform line is set to green, the line width is set to no less than 1pt, and a dotted line style is added;

[0150] Meanwhile, an identification panel is created on the waveform display interface to associate each differentiated identifier with the corresponding acquisition area and signal type. Through the above-mentioned visual differentiation identifier settings, the detection signals from different areas can be clearly distinguished in the same waveform diagram.

[0151] To facilitate operators' quick understanding of waveform meaning, a dedicated labeling panel was created on the waveform display interface of the host computer. Each differentiated label is associated with its corresponding acquisition area and signal type, presenting the labeling rules intuitively. Through this visual differentiation setting, detection signals from different areas can be clearly distinguished in the same waveform diagram. Operators can quickly identify the damage type and acquisition source corresponding to various signals, providing intuitive support for subsequent damage analysis.

[0152] Meanwhile, the host computer performs real-time alarm judgments on the structured inspection results data based on preset judgment parameters. These preset judgment parameters are formulated based on wire rope safety operation standards, industry specifications, and historical maintenance data, covering severity thresholds for different damage types and early warning standards for damage to critical parts. The host computer compares the key information in the structured data, such as damage type, damage degree, and damage location, with the preset parameters. If the detected damage degree reaches or exceeds the early warning threshold, or if damage occurs in a critical part, the alarm mechanism is immediately triggered.

[0153] Once the alarm mechanism is activated, the system automatically generates alarm details containing key damage information. This includes core data such as damage type, specific damage extent, precise damage location, detection time, and equipment operating status, ensuring maintenance personnel have a comprehensive understanding of the damage situation. The alarm details are presented in a standardized format, facilitating quick viewing and subsequent data archiving and analysis. This enables timely early warning and efficient response to wire rope damage, providing strong support for safe equipment operation and maintenance.

[0154] like Figure 3As shown, in a preferred embodiment, a historical playback function is also provided for the user. The historical detection data such as wire breakage and wear can be presented through waveforms with differentiated identifiers. It supports synchronous playback of multi-channel data, waveform zooming and parameter viewing, which makes it convenient for the user to trace and analyze the historical state of wire rope damage.

[0155] The application prospects of this embodiment are broad, covering fields such as industrial production, bridge construction, mining, and port hoisting that rely on wire ropes as core load-bearing components. In the inspection of large bridge cables, its multimodal data fusion technology can accurately identify hidden damage such as broken wires and wear. Combined with the flexible operation capabilities of the acquisition equipment, the entire section can be inspected without interrupting traffic, significantly reducing maintenance costs and safety risks. In heavy equipment such as mine hoisting systems and port cranes, real-time transmission and dynamic alarm functions can promptly report the damage status of wire ropes, avoiding production stoppages due to equipment failures and providing safety assurance for continuous production. It is especially suitable for long-term monitoring needs under complex and harsh working conditions.

[0156] With the increasing demands for industrial intelligence and safe production, the core advantages of this embodiment, such as multi-dimensional data integration, accurate damage identification, and stable and efficient transmission, align with the technical needs of modern operation and maintenance. Its differentiated waveform reconstruction and structured alarm detail output can help maintenance personnel quickly locate faults and develop targeted solutions, driving the transformation of wire rope inspection from "routine maintenance" to "precise prediction." Furthermore, the method's technical architecture possesses excellent scalability, adapting to different specifications of wire ropes and diverse inspection scenarios. It is expected to achieve large-scale application in areas such as special equipment safety monitoring and long-term infrastructure operation and maintenance, providing reliable technical support for safety upgrades and efficiency improvements in related industries.

[0157] Example 2

[0158] Please refer to Figure 4 This embodiment 2 provides a wire rope detection and real-time transmission system based on multimodal data fusion, including:

[0159] The data acquisition and feature construction unit is used to start the detection system, collect detection signals, location data and equipment operating status data corresponding to wire rope damage, and construct a multi-dimensional raw data set; after preprocessing the raw data, a multi-dimensional data association matrix is ​​established; and the preprocessed data is divided into processing units, and the feature information of each processing unit is extracted to form an initial feature dataset.

[0160] The feature extraction, fusion, and recognition unit is used to extract the detection signal features corresponding to different types of wire rope damage using a multi-branch feature extraction network. The feature weight of key damage areas is highlighted through a feature enhancement mechanism. A multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment status features to generate a multi-modal fusion feature vector, which is then classified and identified to output damage-related information and form structured detection result data.

[0161] The hierarchical transmission and integrity assurance unit is used to process structured detection result data according to priority at the acquisition device end: high-priority data retains complete features, while regular data only retains core features; at the same time, it allocates transmission channels according to data priority and dynamically adjusts transmission parameters according to network status to ensure stable communication with the host computer; it adds integrity verification and timing identifiers to the transmitted data, and the host computer confirms the data integrity through the verification identifiers after receiving the data, and triggers a retransmission mechanism for abnormal data;

[0162] The waveform reconstruction and real-time alarm unit is used to decode the received compressed data on the host computer and reconstruct the detection waveform corresponding to the wire rope damage; it combines preset judgment parameters to perform real-time alarm judgment on the structured detection result data and generate alarm details containing key damage information.

[0163] Example 3

[0164] This embodiment 3 also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement any step of a method for steel wire rope detection and real-time transmission based on multimodal data fusion.

[0165] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0167] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for steel wire rope detection and real-time transmission based on multimodal data fusion, characterized in that, include: S1. Start the detection system, collect the detection signals, location data and equipment operating status data corresponding to the wire rope damage, and construct a multi-dimensional raw data set; After preprocessing the original data, a multi-dimensional data association matrix is ​​established; and the preprocessed data is divided into processing units, and the feature information of each processing unit is extracted to form an initial feature dataset. S2. A multi-branch feature extraction network is used to deeply extract the detection signal features corresponding to different types of wire rope damage. The feature weight of key damage areas is highlighted through a feature enhancement mechanism. A multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment status features to generate a multi-modal fusion feature vector, which is then classified and identified to output damage-related information and form structured detection result data. S3. At the acquisition device end, the structured detection result data is processed according to priority: high-priority data retains complete features, while regular data only retains core features; at the same time, transmission channels are allocated according to data priority and transmission parameters are dynamically adjusted according to network status to ensure stable communication with the host computer; integrity verification and timing identifiers are added to the transmitted data, and the host computer confirms the integrity of the data by verifying the identifiers after receiving it, and triggers a retransmission mechanism for abnormal data. S4. Decode the received compressed data on the host computer and reconstruct the detection waveform corresponding to the wire rope damage; combine the preset judgment parameters to perform real-time alarm judgment on the structured detection result data and generate alarm details containing key damage information; The multimodal fusion feature vector in S2 is specifically in the following form: Suppose that after the multi-branch feature extraction network is processed by the feature enhancement mechanism, the output enhancement feature vectors for different types of damage are: , The number of damage types, including damage such as broken wires and wear; The calibrated position feature vector is , For the cumulative displacement characteristics after calibration, It is a characteristic of instantaneous velocity; The standardized device state feature vector is , Characteristics of motor operating status, Characteristics of the battery-powered state; Define modal correlation factor: ， ， in, Let covariance function be used. The variance function quantifies the dynamic correlation strength between damage features and location features and equipment status features; To achieve the enhancement of key damage region features through feature enhancement mechanism, while suppressing interference from ineffective background features, the enhanced damage feature vector is obtained. The multimodal fusion feature vector adopts a composite form of core feature dominance + related feature adaptation, defined as: ， in, This represents the dimension stacking operation representing different types of damage features. This represents element-wise multiplication; the first half of the fusion vector is a stacked representation of the core features of each type of damage, and the second half is an adapted representation of location features, equipment state features, and corresponding modal correlation factors, ultimately yielding... , To integrate the total dimension of features.

2. The method for steel wire rope detection and real-time transmission based on multimodal data fusion according to claim 1, characterized in that, The original data set in S1 is specifically as follows: Assuming the detection system starts up, during the detection time... The internally collected wire rope damage detection signal is ,in, To detect the start time; To detect the current time; ; Characteristic signals corresponding to broken wires Characteristic signals corresponding to wear constitute; The amplitude-time series signal output by the sensing component in the corresponding area; The amplitude-time series signal output by the sensing component in another corresponding area; The collected location data is , Including cumulative displacement data acquired in real time by the encoder Instantaneous velocity data calculated based on the rate of change of displacement ,in, ; The data collected is the operating status data of the equipment. , Includes motor operating parameter vectors Battery power state vector and equipment working status indicators ; in, , Provide power voltage to the motor. This is the motor operating current. Motor speed; battery power state vector , This is the real-time battery voltage. This represents the percentage of remaining battery power. This indicates that the equipment is operating normally. This indicates a device malfunction; Multidimensional raw data set is defined as ,in: ， ， ， In the formula, To detect time series, It is a time variable.

3. The method for steel wire rope detection and real-time transmission based on multimodal data fusion according to claim 1, characterized in that, The process of constructing the initial feature dataset in S1 is as follows: Based on the denoised damage detection signal obtained after preprocessing , Calibrated position data and standardized operating status data , , To detect the time domain, a multi-dimensional data correlation matrix is ​​constructed. , matrix number row element is ,in, The number of sampling points. ; Set the preset frame length ,pass Will Divided into continuous submatrices , To round up; for each submatrix , ,extract Extract the corresponding time-domain and frequency-domain features. Corresponding cumulative average displacement, The mode of the corresponding equipment operating status indicator; The above features are concatenated to form a feature vector. , by all Composition of initial feature dataset This provides input for the multi-branch feature extraction network.

4. The method for steel wire rope detection and real-time transmission based on multimodal data fusion according to claim 1, characterized in that, The feature enhancement mechanism in S2 is specifically as follows: Different types of wire rope damage feature vectors output by a multi-branch feature extraction network , The number of damage types corresponds to the detection signal characteristics of damages such as broken wires and wear. definition Damage feature significance evaluation function: ， in, The global mean of features of the same damage type. The global standard deviation, For the first Each processing unit corresponds to a damage feature component of that type; The total number of processing units; This function quantifies the significance of each element in each damage feature vector. The larger the value, the more significant the difference between the corresponding feature and the normal state feature, which means it is more likely to correspond to a key damage area. Simultaneously, a feature response threshold is introduced. , Determined based on statistical analysis of historical non-destructive wire rope inspection data; for each damage feature vector Perform element-by-element judgment: when When, retain the feature element and keep its original value; when When this characteristic element is subjected to suppression, that is, its value is reduced to zero, where, Indexed by the dimension of the feature vector; The above process enhances the key damage area features while suppressing interference from ineffective background features, resulting in an enhanced damage feature vector. This provides highly recognizable core damage feature inputs for subsequent multimodal fusion processing.

5. The method for steel wire rope detection and real-time transmission based on multimodal data fusion according to claim 1, characterized in that, The process of dynamically adjusting transmission parameters based on network status in S3 is as follows: Real-time monitoring of network transmission status between the data acquisition device and the host computer, and extraction of network bandwidth. Transmission delay and packet loss rate Three core state parameters, among which, To monitor time variables; Define network quality evaluation indicators Quantify the current network transmission capacity: ， Preset network quality threshold range ,when When the network conditions are good, the data transmission rate in the transmission parameters is set to the maximum value. Using the default data packet length ; when That is, when the network condition is moderate, the transmission rate will be adjusted to... , The network bandwidth utilization coefficient is determined based on historical transmission data statistics, and the data packet length is appropriately reduced to... ; when That is, when the network conditions are poor, the transmission rate will be reduced to [a lower value]. Further reduce the data packet length to At the same time, a data packet fragmentation transmission mechanism is enabled.

6. The method for steel wire rope detection and real-time transmission based on multimodal data fusion according to claim 1, characterized in that, The method for adding the integrity verification flag and the timing flag in S3 is as follows: Integrity verification identifier added: for single structured inspection result data after hierarchical processing. , For each data sequence number, a hash algorithm is used to calculate its feature digest. ,Will As an integrity verification identifier, and Binding storage; the hash algorithm is selected from the SHA series algorithm or the CRC series verification algorithm, which uniquely maps the original data through a fixed-length feature digest to ensure that the data is not tampered with or damaged during the data transmission process; Adding time sequence identifiers: based on the detection time domain of the acquisition device. ,extract Corresponding acquisition time ,Will Convert to standardized timestamp Serves as a timing identifier; simultaneously records Ordinal index in the overall data sequence ,Will and Combined into a composite time sequence identifier , attached to The header field serves as a unique temporal identifier for the data; Ultimately, the format of each piece of data to be transmitted is defined as follows: This ensures both the integrity and verifiability of the data, and provides a dual timing basis for the timing calibration of the host computer.

7. The method for steel wire rope detection and real-time transmission based on multimodal data fusion according to claim 1, characterized in that, In S4, the waveform distinguishes the detection signals in different regions using differential identifiers. The specific process is as follows: The host computer decodes the damage detection signal. The system analyzes and identifies the probe acquisition area corresponding to the signal, and assigns differentiated identification rules based on the area type: for the acquired broken wire signal... The first identification rule is adopted, that is, the color of the wave-shaped line is set to red and the line width is set to no less than 2pt; For the collected wear signals The second identification rule is adopted, that is, the waveform line color is set to blue, the line width is set to no less than 2pt, and a dashed line style is added; Damage auxiliary signals acquired from the auxiliary sensing area The third identification rule is adopted, that is, the color of the waveform line is set to green, the line width is set to no less than 1pt, and a dotted line style is added; Meanwhile, an identification panel is created on the waveform display interface to associate each differentiated identifier with the corresponding acquisition area and signal type. Through visually differentiated identifier settings, the detection signals from different areas can be clearly distinguished in the same waveform.

8. A wire rope detection and real-time transmission system based on multimodal data fusion, characterized in that, include: The data acquisition and feature construction unit is used to start the detection system, collect detection signals, location data and equipment operating status data corresponding to wire rope damage, and construct a multi-dimensional raw data set; after preprocessing the raw data, a multi-dimensional data association matrix is ​​established; and the preprocessed data is divided into processing units, and the feature information of each processing unit is extracted to form an initial feature dataset. The feature extraction, fusion, and recognition unit is used to extract the detection signal features corresponding to different types of wire rope damage using a multi-branch feature extraction network. The feature weight of key damage areas is highlighted through a feature enhancement mechanism. A multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment status features to generate a multi-modal fusion feature vector, which is then classified and identified to output damage-related information and form structured detection result data. The hierarchical transmission and integrity assurance unit is used to process structured detection result data according to priority at the acquisition device end: high-priority data retains complete features, while regular data only retains core features; at the same time, it allocates transmission channels according to data priority and dynamically adjusts transmission parameters according to network status to ensure stable communication with the host computer; it adds integrity verification and timing identifiers to the transmitted data, and the host computer confirms the data integrity through the verification identifiers after receiving the data, and triggers a retransmission mechanism for abnormal data; The waveform reconstruction and real-time alarm unit is used to decode the received compressed data on the host computer and reconstruct the detection waveform corresponding to the wire rope damage; it combines preset judgment parameters to perform real-time alarm judgment on the structured detection result data and generate alarm details containing key damage information. The multimodal fusion feature vector in the feature extraction, fusion, and recognition unit takes the following form: Suppose that after the multi-branch feature extraction network is processed by the feature enhancement mechanism, the output enhancement feature vectors for different types of damage are: , The number of damage types corresponds to damage including broken wires and wear; the calibrated position feature vector is... , For the cumulative displacement characteristics after calibration, It is a characteristic of instantaneous velocity; The standardized device state feature vector is , Characteristics of motor operating status, Characteristics of the battery-powered state; Define modal correlation factor ， ， in, Let covariance function be used. The variance function quantifies the dynamic correlation strength between damage features and location features and equipment status features; To achieve the enhancement of key damage region features through feature enhancement mechanism, while suppressing interference from ineffective background features, the enhanced damage feature vector is obtained. The multimodal fusion feature vector adopts a composite form of core feature dominance + related feature adaptation, defined as: ， in, This represents the dimension stacking operation representing different types of damage features. This represents element-wise multiplication; the first half of the fusion vector is a stacked representation of the core features of each type of damage, and the second half is an adapted representation of location features, equipment state features, and corresponding modal correlation factors, ultimately yielding... , To integrate the total dimension of features.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor as described in any one of claims 1-7: a method for detecting and transmitting steel wire ropes in real time based on multimodal data fusion.

Images