Steel wire rope detection and real-time transmission method and system based on multi-modal data fusion

Abstract

The invention discloses a multi-modal data fusion-based steel wire rope detection and real-time transmission method and system, and the method comprises the steps: starting a detection system, collecting a steel wire rope damage detection signal, collecting equipment position and operation state data, constructing an original data set, carrying out the preprocessing, building an incidence matrix, dividing a processing unit, and then extracting features, and forming an initial feature data set. And extracting damage features through a multi-branch network, strengthening the weight of a key region, fusing position and equipment state features through a multi-modal fusion module, generating fusion vectors, classifying and identifying, and outputting a structured detection result. And the acquisition equipment end performs grading processing on the data, distributes transmission channels according to priorities, dynamically adjusts parameters, adds integrity and time sequence identifiers, and the upper computer verifies the data integrity and triggers abnormal supplementary transmission. And the upper computer decodes the data, reconstructs the waveform, gives an alarm in real time in combination with preset parameters, and generates details of damage key information. Reliable technical support is provided for safe operation and maintenance of the steel wire rope in industrial scenes.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of industrial equipment detection, and more particularly relates to a steel wire rope detection and real-time transmission method and system based on multi-modal data fusion. BACKGROUND

[0002] In the fields of industrial production, bridge construction, and mine exploitation, steel wire ropes, as core bearing components, their running state is directly related to equipment safety and production efficiency. For a long time, steel wire ropes have been continuously subjected to tension, wear, corrosion, and other effects under complex working conditions, which are prone to damage such as wire breakage, wear, and corrosion. If not detected and handled in a timely manner, it may cause equipment failure and even safety accidents, resulting in serious economic losses and safety risks.

[0003] In current steel wire rope detection technology, traditional manual detection methods rely on the experience of detection personnel, which not only has high labor intensity and low detection efficiency, but also has detection blind spots, making it difficult to accurately identify hidden damage and unable to achieve real-time monitoring. Existing automated detection technologies mainly focus on single type detection signals, and only use single modal data such as electromagnetic induction and visual imaging for damage judgment, ignoring the role of associated data such as equipment running state and position information, resulting in detection results being easily affected by environmental interference, insufficient damage identification accuracy, and difficulty in fully reflecting the actual working conditions of steel wire ropes.

[0004] In the data transmission link, existing technologies lack differentiated processing mechanisms for detection data, and all data use a unified transmission strategy, which can easily waste bandwidth resources. In complex industrial environments, network state fluctuates greatly, and fixed transmission parameters are difficult to adapt to dynamic network environments, often causing data transmission delays, losses, and other problems, affecting the real-time and integrity of the detection results. At the same time, some transmission schemes do not establish effective data verification and retransmission mechanisms, which cannot ensure the accuracy of the data received by the receiving end, further reducing the reliability of the detection system.

[0005] With the improvement of industrial intelligence level, higher requirements are put forward for the accuracy, real-time performance, and reliability of steel wire rope detection. The existing detection technology has shortcomings in multi-source data integration, damage accurate identification, and stable and efficient transmission, which cannot meet the actual application requirements. Therefore, developing a steel wire rope detection and real-time transmission method that can integrate multi-modal data, improve damage identification accuracy, and ensure stable data transmission has important practical significance for reducing safety risks, improving equipment operation efficiency, and ensuring industrial production safety. SUMMARY

[0006] The present application aims to solve the problems of single data dimension, insufficient damage identification accuracy, poor data transmission stability and the like in existing steel wire rope detection technology. By integrating multi-modal data, optimizing feature extraction and fusion mechanism, establishing hierarchical transmission and dynamic adaptation strategy, the accuracy and comprehensiveness of damage detection are improved, the real-time and integrity of data transmission are ensured, and reliable technical support is provided for steel wire rope safe operation in industrial scenarios.

[0007] In view of the above defects or improvement needs of the prior art, the present application provides a steel wire rope detection and real-time transmission method based on multi-modal data fusion, comprising: S1. Start the detection system, collect the detection signals, position data and equipment running state data corresponding to the steel wire rope damage, and construct a multi-dimensional original data set; after preprocessing the original data, a multi-dimensional data correlation matrix is established; and the preprocessed data is divided into processing units, the feature information of each processing unit is extracted, and an initial feature data set is formed; S2. A multi-branch feature extraction network is used to extract the features of the detection signals corresponding to different types of steel wire rope damage in depth, and a feature enhancement mechanism is used to highlight the feature weights of the key damage areas; a multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment state features, generate a multi-modal fusion feature vector and classify it, output the damage related information, and form a structured detection result data; S3. The structured detection result data is processed according to priority at the acquisition device end: high priority data retains complete features, and regular data retains only core features; at the same time, transmission channels are allocated according to data priority and transmission parameters are dynamically adjusted according to network state to ensure stable communication with the host computer; integrity verification marks and time sequence marks are added to the transmission data, and the host computer confirms the data integrity through the verification marks after receiving, and triggers the retransmission mechanism for abnormal data; S4. The compressed data received at the host computer end is decoded and the detection waveform corresponding to the steel wire rope damage is reconstructed; the structured detection result data is judged in real time according to the preset judgment parameters, and the alarm details containing the key information of the damage are generated.

[0008] Further, the original data set in S1 is specifically: After the detection system is started, the steel wire rope damage detection signals collected within the detection time are , wherein is the detection start time; is the current detection time; ; The feature signal corresponding to the broken wire is , and the feature signal corresponding to the wear is ; amplitude-time series signal output by a corresponding area sensing component; amplitude-time series signal output by another corresponding area sensing component; collected position data is , includes cumulative displacement data collected by the encoder in real time and instantaneous speed data calculated based on the displacement rate wherein, ; collected equipment operating state data is , includes a motor operating parameter vector , a battery power supply state vector and an equipment working state identifier ; wherein, , is the motor supply voltage, is the motor operating current, is the motor speed; the battery power supply state vector , is the real-time voltage of the battery, is the proportion of the remaining battery capacity; indicates normal equipment operation, indicates equipment abnormality; a multi-dimensional raw data set is defined as wherein: , , , in the formula, is the detection time series, is the time variable.

[0009] Further, the construction process of the initial feature data set in S1 is: based on the denoised damage detection signal obtained after preprocessing , the calibrated position data and the standardized operating state data , , is the detection time domain, a multi-dimensional data correlation matrix is constructed , the first row element of the matrix is wherein, is the number of sampling points, ; a preset frame length is set , and the matrix is divided into ​ divided into one continuous sub-matrix , is rounded up; for each sub-matrix , extract corresponding time domain features and frequency domain features, extract corresponding cumulative displacement mean, corresponding device running state identifier mode; The above features are spliced to form a feature vector , which is composed of all initial feature data set , which provides input for the multi-branch feature extraction network.

[0010] Further, the feature enhancement mechanism in S2 is specifically: Based on the different types of steel wire damage feature vectors output by the multi-branch feature extraction network , The number of damage types, corresponding to the detection signal features of damage such as wire breakage and wear; Define the damage feature saliency evaluation function: , Among them, is the global mean of the same damage type feature, is the global standard deviation, is the damage feature component corresponding to the th processing unit; is the total number of processing units; Through this function, the saliency of each element in the damage feature vector is quantified. The larger the value, the more significant the difference between the corresponding feature and the normal state feature, that is, the more likely it is to correspond to a key damage area; At the same time, introduce the feature response threshold , Based on historical non-damage steel wire detection data statistics; for each damage feature vector Element-by-element judgment: when , the feature element is retained and its original value is kept; when , the feature element is suppressed, i.e. its value is zero, wherein is the feature vector dimension index; Through the above process, the key damage area features are highlighted and the invalid background feature interference is suppressed, and the enhanced damage feature vector is obtained, which provides high-recognizability core damage feature input for subsequent multi-modal fusion processing.

[0011] Further, the form of the multi-modal fusion feature vector in S2 is specifically: Let the multi-branch feature extraction network be processed by the feature enhancement mechanism, and the output of the different types of damage enhancement feature vectors be , for the number of damage types, corresponding to broken wires, wear and tear, etc. The calibrated position feature vector is , for the cumulative displacement feature after calibration, for the instantaneous speed feature; The standardized equipment state feature vector is , for the motor operating state feature, for the battery power supply state feature; Define the modal correlation factor , , wherein, is the covariance function, is the variance function, which quantifies the dynamic correlation strength of the damage feature and the position feature, the equipment state feature; The multi-modal fusion feature vector adopts a composite form of core feature dominance + correlation feature adaptation, and is defined as: , wherein, represents the dimension stacking operation of different types of damage features, represents the element-wise multiplication operation; The first half of the fusion vector is the stacking representation of the core features of each type of damage, and the second half is the adaptive representation of the position feature, the equipment state feature and the corresponding modal correlation factor, and finally , is the total dimension of the fusion feature.

[0012] Further, the process of dynamically adjusting the transmission parameters according to the network state in S3 is: Real-time acquisition of the network transmission state between the device and the host computer, extraction of the network bandwidth , transmission delay and packet loss rate three core state parameters, wherein, is the monitoring time variable; Define the network quality evaluation index quantify the current network transmission capacity: , preset network quality threshold interval , when , i.e. the network state is good, the data transmission rate in the transmission parameter is set to the maximum value , adopt the default data packet length ; When , i.e. the network state is medium, the transmission rate is adjusted to , is the network bandwidth utilization coefficient, determined based on historical transmission data statistics, and the data packet length is appropriately reduced to ; When , i.e. the network state is poor, the transmission rate is reduced to , and the data packet length is further reduced to , and the data packet fragmentation transmission mechanism is enabled.

[0013] Further, the method for adding integrity verification identification and timing identification in S3 is: Integrity verification identification addition: to the single structured detection result data processed in stages , is the data serial number, and a feature digest is calculated using a hash algorithm , and is taken as the integrity verification identification and stored in combination with ; wherein the hash algorithm is selected from the SHA series algorithm or the CRC series check algorithm, and the original data is uniquely mapped through a fixed-length feature digest to ensure that the data is not tampered with or damaged during transmission; Timing identification addition: based on the detection time domain of the collection device , the collection time corresponding to is extracted, and is converted into a standardized timestamp as the timing identification; at the same time, the order index of in the overall data sequence is recorded , and is combined with as a composite timing identification , which is attached to the header field of to achieve the timing unique identification of the data; Finally, the format of each data to be transmitted is defined as , which not only guarantees the data integrity and verifiability, but also provides double timing basis for the timing calibration of the upper computer.

[0014] Further, in S4, the waveform is distinguished from the detection signals in different regions through differential identification, and the specific process is: The upper computer analyzes the decoded damage detection signal , identifies the probe collection area corresponding to the signal, and allocates differential identification rules based on the region type: for the collected broken wire signal , the first identification rule is adopted, that is, the color of the wave-shaped line is red, and the line width is not less than 2pt; The wear signal collected , the second identification rule is adopted, that is, the color of the wave-shaped line is blue, the line width is not less than 2pt, and a dashed line pattern is additionally arranged; The auxiliary damage signal collected by the auxiliary sensing area , the third identification rule is adopted, that is, the color of the wave-shaped line is green, the line width is not less than 1pt, and a dot-dashed line pattern is additionally arranged; Meanwhile, an identification instruction panel is established on the wave-shaped display interface, the differentiated identification is associated with the corresponding collection area and signal type, and the clear differentiation of different area detection signals in the same wave-shaped diagram is completed through the visual differentiated identification setting.

[0015] As a second aspect of the present application, a steel wire rope detection and real-time transmission system based on multi-modal data fusion is also provided, comprising: A data acquisition and feature construction unit is used to start the detection system, collect the detection signals, position data and equipment running state data corresponding to the steel wire rope damage, construct a multi-dimensional original data set, establish a multi-dimensional data correlation matrix after preprocessing the original data, and divide the preprocessed data into processing units, extract the feature information of each processing unit, and form an initial feature data set; A feature extraction and fusion and recognition unit is used to adopt a multi-branch feature extraction network to respectively extract the features of the detection signals corresponding to different types of steel wire rope damage, highlight the feature weights of the key damage areas through a feature enhancement mechanism, introduce a multi-modal fusion module to fuse the extracted damage features with the calibrated position features and equipment state features, generate a multi-modal fusion feature vector and classify and recognize it, output the damage related information, and form a structured detection result data; A hierarchical transmission and integrity guarantee unit is used to process the structured detection result data according to the priority at the collection equipment end: the high-priority data retains complete features, and the regular data only retains core features; meanwhile, transmission channels are allocated according to the data priority, and transmission parameters are dynamically adjusted according to the network state to guarantee stable communication with the upper computer; integrity verification marks and time sequence marks are added to the transmission data, the upper computer confirms the data integrity after receiving, and triggers a retransmission mechanism for abnormal data; A wave reconstruction and real-time alarm unit is used to decode the received compressed data at the upper computer end, reconstruct the detection wave corresponding to the steel wire rope damage, and make real-time alarm judgment on the structured detection result data according to the preset judgment parameters, and generate alarm details containing key damage information.

[0016] As a third aspect of the present application, a computer readable storage medium is also provided, which stores a computer program executed by a processor to implement any of the above methods of steel wire rope detection and real-time transmission based on multi-modal data fusion.

[0017] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects: 1. The method of steel wire rope detection and real-time transmission based on multi-modal data fusion of the present application comprehensively captures and orderly integrates multiple types of data by starting the acquisition device, synchronously collecting detection signals corresponding to steel wire rope damage, position data and running state data of the acquisition device, and constructing a multi-dimensional original data set. After preprocessing the original data, a multi-dimensional data correlation matrix is established, processing units are divided, and feature information of each unit is extracted to form an initial feature data set. This technical feature avoids the limitations of a single data dimension, provides more abundant basic information support for subsequent analysis, filters out core effective information through preprocessing and feature extraction, reduces redundant data interference, provides accurate and efficient input data for subsequent damage identification, and ensures the pertinence and reliability of detection analysis.

[0018] 2. The method of steel wire rope detection and real-time transmission based on multi-modal data fusion of the present application precisely extracts and strengthens different types of damage features by using a multi-branch feature extraction network to respectively deeply extract detection signal features corresponding to different types of steel wire rope damage, and by means of a feature strengthening mechanism to highlight the feature weights of key damage areas. Then, the extracted damage features, calibrated position features and device state features are fused by a multi-modal fusion module to generate a multi-modal fusion feature vector and perform classification and identification. This technical feature precisely extracts and strengthens different types of damage features, breaks down the information barriers of each modal data, deeply fuses damage information and scene-related information, improves the accuracy and comprehensiveness of damage identification, and can effectively distinguish different types of steel wire rope damage, providing reliable technical support for subsequent detection result output.

[0019] 3. The method of steel wire rope detection and real-time transmission based on multi-modal data fusion of the present application efficiently and stably transmits detection data by hierarchically processing structured detection result data at the acquisition device end according to priority, assigning transmission channels according to priority, dynamically adjusting transmission parameters according to network state, adding integrity verification identifiers and time sequence identifiers to the transmission data, verifying data integrity after reception by the upper computer, triggering a retransmission mechanism for abnormal data, and finally decoding data by the upper computer, reconstructing waveforms, and combining preset parameters to perform real-time alarm. This technical feature efficiently and stably transmits detection data, adapts to different network environments through hierarchical processing and dynamic parameter adjustment, integrity identifiers and the retransmission mechanism ensure the accuracy and integrity of data transmission, and the real-time alarm function provides timely and effective data support for equipment maintenance and safety protection. Attached Figure Description

[0020] Figure 1 This is a flowchart of a wire rope detection and real-time transmission method based on multimodal data fusion according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the data acquisition device according to an embodiment of the present invention; Figure 3 This is a schematic diagram of historical playback in an embodiment of the present invention; Figure 4 This is a system unit diagram of an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0022] Example 1 Please refer to Figure 1 This embodiment 1 provides a method for steel wire rope detection and real-time transmission based on multimodal data fusion, including: S1. Start the detection system, collect the detection signals, location data and equipment operation status data corresponding to the wire rope damage, and construct a multi-dimensional raw data set; after preprocessing the raw data, establish a multi-dimensional data association matrix; and divide the preprocessed data into processing units, extract the feature information of each processing unit, and form an initial feature dataset. S2. A multi-branch feature extraction network is used to deeply extract the detection signal features corresponding to different types of wire rope damage. The feature weight of key damage areas is highlighted through a feature enhancement mechanism. A multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment status features to generate a multi-modal fusion feature vector, which is then classified and identified to output damage-related information and form structured detection result data. S3. At the acquisition device end, the structured detection result data is processed according to priority: high-priority data retains complete features, while regular data only retains core features; at the same time, transmission channels are allocated according to data priority and transmission parameters are dynamically adjusted according to network status to ensure stable communication with the host computer; integrity verification and timing identifiers are added to the transmitted data, and the host computer confirms the integrity of the data by verifying the identifiers after receiving it, and triggers a retransmission mechanism for abnormal data. S4. The received compressed data is decoded at the host computer end to reconstruct the detection waveform corresponding to the steel wire damage; the structured detection result data is combined with the preset judgment parameters to perform real-time alarm judgment, and alarm details containing key information of the damage are generated.

[0023] This embodiment 1 further expands on the above steps.

[0024] (1) Data acquisition and feature construction In an industrial scenario, as a core bearing component, the accuracy of steel wire damage detection highly depends on the comprehensive capture and effective processing of multi-dimensional data. Considering that a single data cannot fully reflect the steel wire damage and the detection environment state, the acquisition device is not only used to collect a single signal after starting, but also to start the whole process of multi-source data acquisition and feature construction, which lays a solid data foundation for subsequent accurate damage identification.

[0025] Please refer to Figure 2 In this example, a probe shoe is used as the acquisition device. Different wear-resistant sleeves with different rope diameters can be used to detect steel wires of different diameters. Usually, one probe shoe can correspond to the detection of up to five different diameters of steel wires. Different regions of the probe shoe can collect electromagnetic induction signals corresponding to broken wires and wear, respectively. The wear-resistant sleeve can reduce the direct friction between the probe shoe and the steel wire during detection, ensuring the long-term stable acquisition of amplitude-time sequence signals by the sensing component, and providing reliable hardware support for constructing steel wire damage detection signals in the multi-dimensional original data set.

[0026] The acquisition device synchronously collects three types of key data within the preset detection time range, i.e. the entire time period from the detection start time to the current detection time, through the special sensing system carried, ensuring that each data corresponds one by one in the time dimension. Among them, the steel wire damage detection signal is captured by the upper and lower two regions of the probe target. This partitioned acquisition design is based on the actual working condition that broken wires and wear often occur in different parts of the steel wire: the electromagnetic induction sensor in the upper half area is specially designed to collect amplitude-time sequence signals related to broken wires, and the same type of sensor in the lower half area focuses on electromagnetic induction amplitude-time sequence signals corresponding to wear. These two types of signals together constitute the core data of damage detection.

[0027] Meanwhile, the position data of the collection device is obtained by combining the encoder with the calculation, the encoder collects the cumulative displacement data in real time to determine the detection position, and the instantaneous speed data is further calculated according to the displacement change rate, so that the position and movement state of the collection device in the detection process are completely reflected, and the damage position is provided for subsequent positioning. In addition, in order to ensure the reliability of the detection process and avoid affecting the accuracy of the data due to the abnormal state of the device itself, the collection device also synchronously collects the running state data, including the motor power supply voltage, working current, rotating speed and other operating parameters, the real-time voltage and residual capacity proportion of the battery and the normal or abnormal working state information of the device marked with a specific identifier. After the integration of the three types of data, a multi-dimensional original data set is formed.

[0028] In the preferred embodiment, the original data set is represented as follows: Suppose that after the detection system is started, the steel wire damage detection signal collected by the collection device within the detection time is , wherein is the detection starting time; is the current detection time; ; is composed of the characteristic signal corresponding to the broken wire and the characteristic signal corresponding to the wear ; is the amplitude-time sequence signal output by the corresponding regional sensing component; is the amplitude-time sequence signal output by another corresponding regional sensing component; The position data collected by the collection device is , including the cumulative displacement data collected by the encoder in real time and the instantaneous speed data calculated based on the displacement change rate , wherein ; The running state data of the collection device is , including the motor operating parameter vector , the battery power supply state vector and the device working state identifier ; wherein , is the motor power supply voltage, is the motor working current, is the motor rotating speed; the battery power supply state vector , is the real-time voltage of the battery, is the residual capacity proportion of the battery; indicates the normal operation of the device, an abnormality of the device is indicated; A multi-dimensional raw data set is defined as wherein: , , , In the formula, is a detection time series, is a time variable.

[0029] At the same time, due to the problems such as noise, error and dimension difference in the raw data, direct use for analysis will reduce the accuracy, so the raw data needs to be preprocessed: through the denoising algorithm to remove the environmental interference noise in the damage detection signal, through the calibration algorithm to correct the cumulative error in the position data, through the standardization processing to eliminate the dimension influence of the running state data, to obtain the purified three types of data. In order to realize the correlation analysis of multi-dimensional data, a multi-dimensional data correlation matrix is constructed based on the processed data, and the damage detection, position and running state data at the same sampling time are arranged in a row of the matrix, and the number of matrix rows is consistent with the number of sampling points.

[0030] Considering that continuous long sequence data is difficult to process directly and easy to lose local features, it needs to be processed in frames: set a fixed frame length, calculate the frame number by rounding up, divide the correlation matrix into multiple continuous sub-matrices, and each sub-matrix corresponds to the data of a continuous detection process. For each sub-matrix, the key features of each dimension data are extracted: time domain and frequency domain features are extracted from the damage detection signal to reflect the damage characteristics, the cumulative displacement mean is extracted from the position data to reflect the position distribution of the detection, and the device working state identification mode is extracted from the running state data to confirm the overall running state of the device during the detection.

[0031] These extracted features are spliced in a fixed order to form a feature vector, and the feature vectors corresponding to all sub-matrices are combined to form an initial feature data set, which integrates the core information of multi-dimensional data and provides adaptive input data for the subsequent multi-branch feature extraction network.

[0032] Specifically, the construction process of the initial feature data set is: Based on the denoised damage detection signal , the calibrated position data and the standardized running state data , , is the detection time domain, a multi-dimensional data correlation matrix is constructed , the first row elements of the matrix are , wherein, for the sampling point number, ; set a preset frame length , by dividing into continuous sub-matrices , is the ceiling; for each sub-matrix , extract the corresponding time domain features and frequency domain features, extract the corresponding cumulative displacement mean, the corresponding device running state identifier mode; splice the above features to form a feature vector , consisting of all initial feature data set , provide input for the multi-branch feature extraction network.

[0033] (2) Feature extraction fusion and recognition In steel wire rope damage detection, different types of damage often have different characteristic signals. Single feature extraction method is difficult to accurately capture the core information of various damages, and the original features often contain invalid background interference, which affects the subsequent recognition accuracy. At the same time, the separation of damage features and scene information such as detection position and device state also leads to the lack of comprehensiveness of the detection results. To solve these problems, targeted feature extraction, strengthening and fusion strategies are needed to build more comprehensive features.

[0034] Based on the initial feature data set constructed in the early stage, a multi-branch feature extraction network is used for deep feature mining. Each branch of the network corresponds to a specific type of steel wire rope damage, which can accurately focus on the detection signal features corresponding to different damages such as wire breakage and wear, and realize the extraction of various damage features respectively, avoiding the mutual interference between different types of damage features.

[0035] After obtaining the feature vectors of different types of damage, the feature weight of the key damage area is highlighted through the feature strengthening mechanism. The feature strengthening mechanism is as follows: Based on the different types of steel wire rope damage feature vectors output by the multi-branch feature extraction network , is the number of damage types, corresponding to the detection signal features of damages such as wire breakage and wear; Define the damage feature saliency evaluation function: , where, is the global mean of the same damage type feature, is the global standard deviation, For the first Each processing unit corresponds to a damage feature component of that type; The total number of processing units; This function quantifies the significance of each element in each damage feature vector. The larger the value, the more significant the difference between the corresponding feature and the normal state feature, which means it is more likely to correspond to a key damage area. Simultaneously, a feature response threshold is introduced. , Determined based on statistical analysis of historical non-destructive wire rope inspection data; for each damage feature vector Perform element-by-element judgment: when When, retain the feature element and keep its original value; when When this characteristic element is subjected to suppression, that is, its value is reduced to zero, where, Indexed by the dimension of the feature vector; The above process enhances the key damage area features while suppressing interference from ineffective background features, resulting in an enhanced damage feature vector. This provides highly recognizable core damage feature inputs for subsequent multimodal fusion processing.

[0036] Considering that accurate damage identification requires not only the characteristics of the damage itself but also the integration of related information such as detection location and equipment operating status, a multimodal fusion module is introduced for feature integration. First, the various features involved in the fusion are defined: enhanced feature vectors of different types of damage, calibrated position feature vectors (including cumulative displacement and instantaneous velocity features), and standardized equipment status feature vectors (including motor operating status and battery power supply status features). To achieve effective correlation among these features, a modal correlation factor is defined. By calculating the ratio of the covariance to the square root of the product of the variances of the two types of features, the dynamic correlation strength between each damage feature and the position and equipment status features is quantified.

[0037] The multimodal fusion feature vector is constructed using a composite approach of "core feature dominance + related feature adaptation," specifically: Suppose that after the multi-branch feature extraction network is processed by the feature enhancement mechanism, the output enhancement feature vectors for different types of damage are: , The number of damage types corresponds to damage such as broken wires and wear; the calibrated position feature vector is... , For the cumulative displacement characteristics after calibration, The instantaneous velocity feature is; the standardized equipment state feature vector is... , Characteristics of motor operating status, Characteristics of the battery-powered state; define the modal correlation factor , , wherein, is a covariance function, is a variance function, quantifying the dynamic correlation strength of damage features and location features, equipment state features; The multi-modal fusion feature vector adopts a composite form of core feature dominance + correlation feature adaptation, and is defined as: , wherein, represents a dimension stacking operation of different types of damage features, represents an element-wise multiplication operation; the first half of the fusion vector is a stacked representation of the core features of each type of damage, and the second half is an adapted representation of the location features, equipment state features and corresponding modal correlation factors, to finally obtain , is the total dimension of the fusion features.

[0038] After generating the multi-modal fusion feature vector, the damage key information needs to be extracted through a precise classification and recognition process to form standardized structured detection result data. First, the fusion feature vector is input into a pre-set classification and recognition model, which is constructed based on a deep learning network and optimized through historical damage sample data training, and has a deep analysis capability for multi-modal fusion features. In the classification and recognition process, the model first maps the fusion feature vector in the feature space to further mine the internal correlation between damage features and scene features, and then cooperates with the multi-layer perceptron and Softmax classifier to realize accurate judgment of the damage type and clearly distinguish different damage categories such as broken wires and wear.

[0039] At the same time, the model combines the reinforced damage feature components in the fusion features to quantitatively evaluate the damage degree: by comparing the amplitude of the key damage elements in the feature vector with the preset damage level threshold, the damage degree is divided into three levels of mild, moderate and severe, and the threshold standard is determined based on the steel wire rope safety operation specification and historical detection data statistics. In addition, the position correlation information in the fusion features is used to accurately locate the specific position of the damage, and the cumulative displacement feature and instantaneous speed feature of the collection equipment are combined to calculate the axial coordinate and circumferential direction of the damage corresponding to the steel wire rope, ensuring the accuracy of the damage position information.

[0040] After the classification recognition is completed, the original recognition result containing the core information of the damage is output, covering the damage type, damage degree, damage location, detection time, equipment running state and other key data items. In order to facilitate subsequent transmission, storage and analysis, the original recognition result needs to be structured: according to the preset data field specification, each item of information is arranged into structured data in the form of key-value pairs, wherein the damage type and degree are represented by standardized codes, the damage location is presented by coordinate values, the detection time is in the format of timestamp, and the equipment running state is identified by a status code.

[0041] After the structured processing, the data is format-verified to ensure that each field information is complete, the data type is consistent, and the logical relationship is reasonable, avoiding the existence of invalid data or incorrect information. The structured detection result data formed finally not only contains the core feature information of the damage, but also has a unified format standard, which can meet the application requirements of subsequent hierarchical transmission, real-time alarm, data archiving and other links, and provide comprehensive, accurate and standardized data support for the safe operation and maintenance of the steel wire rope.

[0042] (3) Hierarchical transmission and integrity protection In the detection of steel wire ropes in industrial scenarios, the transmission quality of detection data directly affects the real-time judgment and response efficiency of the host computer to the damage situation. Considering that the urgency of different damage information differs, and the network state in industrial environments is easily disturbed and fluctuates, if a unified transmission strategy is adopted, it may cause delay in the transmission of key damage information, or data loss due to improper network adaptation, so a targeted hierarchical transmission and protection mechanism needs to be built.

[0043] After the structured detection result data is generated, the priority classification is first performed at the acquisition device end. The classification standard is based on the urgency and importance of the damage: the detection data related to severe damage and critical part damage are classified as high priority, which directly relates to the safety of the equipment, and needs to retain complete feature information to ensure that the host computer obtains comprehensive damage details; the data related to ordinary mild damage and conventional detection areas are classified as regular priority, which only retains core feature information and removes redundant content to reduce data transmission volume and improve transmission efficiency.

[0044] After the classification is completed, the dedicated transmission channel is allocated according to the priority, and the high-priority data occupies the channel with higher transmission priority to ensure that it is not disturbed by regular data during transmission and reaches the host computer first. At the same time, the network transmission state between the acquisition device and the host computer is monitored in real time, and three core parameters, i.e. network bandwidth, transmission delay and packet loss rate, are extracted to quantitatively evaluate the current network transmission capacity. Real-time monitoring of the network transmission state between the acquisition device and the host computer, extracting network bandwidth , transmission delay and packet loss rate Three core state parameters, wherein, is a monitoring time variable; Define network quality evaluation index Quantify the current network transmission capacity: , Pre-set network quality threshold interval When , the data transmission rate in the transmission parameter is set to the maximum value , the default data packet length is adopted ; When , the network state is medium, the transmission rate is adjusted to , is the network bandwidth utilization coefficient, which is determined based on historical transmission data statistics, and the data packet length is appropriately reduced to ; When , the network state is poor, the transmission rate is reduced to , and the data packet length is further reduced to , and the data packet fragmentation transmission mechanism is enabled.

[0045] In order to ensure the accuracy and timing consistency of the transmission data, an integrity verification identifier and a timing identifier are added to each piece of data to be transmitted. The method of adding integrity verification identifier and timing identifier is: Integrity verification identifier addition: to the single structured detection result data after hierarchical processing , is the data serial number, and the feature digest is calculated by using the hash algorithm , the is taken as the integrity verification identifier, and is stored with ; wherein the hash algorithm is selected from the SHA series algorithm or the CRC series check algorithm, which uniquely maps the original data through the fixed length feature digest, ensuring that the data is not tampered with or damaged during transmission; Timing identifier addition: based on the detection time domain of the collection device , the corresponding collection time is extracted , the is converted into a standardized timestamp as the timing identifier; at the same time, the order index of in the overall data sequence is recorded , the is combined with as a composite timing identifier , which is attached to the head field of the packet, to realize the time sequence unique identification of data; Finally, the format of each piece of data to be transmitted is defined as , which not only guarantees data integrity verification, but also provides double timing basis for upper computer timing calibration.

[0046] Finally, each piece of data to be transmitted forms a unified format containing structured detection results, integrity verification identifier and composite timing identifier. After receiving the data, the upper computer first compares and calculates through the integrity verification identifier to confirm whether the data is complete and correct; if it finds abnormal conditions such as data loss, damage or tampering, it will immediately trigger the retransmission mechanism and send a retransmission request to the acquisition device to ensure that critical data is not lost. Through the synergistic effect of hierarchical processing, dynamic parameter adjustment, double identifier addition and abnormal retransmission mechanism, efficient, stable and accurate transmission of detection data is realized, providing reliable data guarantee for subsequent waveform reconstruction and alarm judgment of the upper computer.

[0047] (4) Waveform reconstruction and real-time alarm In the industrial scene, the upper computer, as the core of receiving and processing steel wire detection data, needs to convert the compressed data transmitted into intuitive detection results and feedback the damage risk in time to provide basis for operation and maintenance decision. Since the data is compressed in stages during transmission, the upper computer first needs to decode the received compressed data to restore the original detection signal and structured data through the corresponding decompression algorithm to ensure the accuracy of subsequent processing.

[0048] After decoding, the upper computer starts to reconstruct the detection waveform corresponding to the steel wire damage. Considering that the signals of different acquisition areas correspond to different types of damage or auxiliary detection information, if a unified identifier is used, it is easy to cause signal confusion and it is difficult to quickly distinguish key damage information. Therefore, in the waveform reconstruction process, a differentiated identification strategy is introduced, and exclusive display rules are assigned based on the type of signal acquisition area.

[0049] Specifically, the waveform distinguishes the detection signals of different areas through differentiated identification, and the specific process is as follows: The upper computer analyzes the decoded damage detection signal , identifies the probe acquisition area corresponding to the signal, and assigns differentiated identification rules based on the area type: for the collected broken wire signal , the first identification rule is adopted, that is, the color of the waveform line is set to red and the line width is not less than 2pt; for the collected wear signal , the second identification rule is adopted, that is, the color of the waveform line is set to blue, the line width is not less than 2pt, and a dashed line style is added; for the damage auxiliary signal collected in the auxiliary sensing area , the third identification rule is adopted, that is, the waveform line color is set to green, the line width is set to not less than 1pt, and the dash-dot line pattern is added; At the same time, an identification explanation panel is established in the waveform display interface, and each differentiated identification is associated with a corresponding acquisition area and signal type. Through the above visual differentiated identification setting, clear differentiation of detection signals in different areas in the same waveform graph is completed.

[0050] To facilitate the operator to quickly understand the waveform meaning, an identification explanation panel is specially established in the waveform display interface of the upper computer, each differentiated identification is associated with a corresponding acquisition area and signal type, and the identification rule is intuitively presented. Through this visual differentiated setting, the detection signals in different areas are clearly differentiated in the same waveform graph, and the operator can quickly identify the damage types and acquisition sources of various signals, thereby providing intuitive support for subsequent damage analysis.

[0051] At the same time, the upper computer combines the preset determination parameters to perform real-time alarm judgment on the structured detection result data. The preset determination parameters are formulated based on the steel wire rope safety operation standard, industry specifications and historical operation and maintenance data, and cover the severity threshold of different damage types, the early warning standard of key part damage, etc. The upper computer compares the key information such as damage type, damage degree and damage position in the structured data with the preset parameters one by one, and if the damage degree reaches or exceeds the early warning threshold or damage occurs at the key part, the alarm mechanism is triggered immediately.

[0052] After the alarm mechanism is started, the system automatically generates alarm details containing key damage information, which covers core data such as damage type, specific damage degree, accurate damage position, detection time and equipment running state, so that the operation and maintenance personnel can fully master the damage situation. The alarm details are presented in a standardized format, which is convenient for quick viewing and can be used for subsequent data archiving and analysis, realizes timely early warning and efficient response to steel wire rope damage, and provides strong guarantee for safe operation and maintenance of equipment.

[0053] As shown in Figure 3 In a preferred embodiment, a history playback function is also provided for the user, which can present historical detection data such as broken wires and wear through differentiated identification waveforms, supports multi-channel data synchronous playback, waveform zooming and parameter viewing, and facilitates the user to trace and analyze the historical state of the steel wire rope damage.

[0054] The application prospect of the embodiment covers a wide range of fields such as industrial production, bridge construction, mine exploitation, and port hoisting, which rely on steel wire ropes as core bearing components. In large bridge cable detection, the multi-modal data fusion technology can accurately identify hidden damages such as broken wires and wear, and combined with the flexible operation capability of the collection equipment, the full-section detection can be completed without interrupting traffic, greatly reducing the operation and maintenance cost and safety risk; in mine hoisting systems, port cranes and other heavy equipment, real-time transmission and dynamic alarm functions can timely feedback the damage state of the steel wire rope, avoid production stagnation caused by equipment failure, and provide safety protection for continuous production, especially suitable for long-term monitoring needs in complex and harsh working conditions.

[0055] With the continuous improvement of industrial intelligence and safety production requirements, the multi-dimensional data integration, accurate damage identification, and stable and efficient transmission of the embodiment meet the technical needs of modern operation and maintenance. The differentiated identification waveform reconstruction and structured alarm detail output can help operation and maintenance personnel quickly locate faults and develop targeted treatment plans, promoting the transformation of steel wire rope detection from "periodic maintenance" to "accurate prediction". At the same time, the technical architecture of this method has good scalability and can adapt to different specifications of steel wire ropes and various detection scenarios, and is expected to form large-scale applications in the fields of special equipment safety monitoring and infrastructure long-term operation and maintenance, providing reliable technical support for the safety upgrade and efficiency improvement of related industries.

[0056] Embodiment 2 Please refer to Figure 4 Embodiment 2 provides a steel wire rope detection and real-time transmission system based on multi-modal data fusion, comprising: A data acquisition and feature construction unit is used to start the detection system, acquire detection signals, position data and equipment operation state data corresponding to steel wire rope damage, and construct a multi-dimensional original data set; a multi-dimensional data correlation matrix is established after preprocessing the original data; and the preprocessed data is divided into processing units, the feature information of each processing unit is extracted, and an initial feature data set is formed; A feature extraction and fusion and identification unit is used to adopt a multi-branch feature extraction network to respectively extract the features of the detection signals corresponding to different types of steel wire rope damage in depth, and highlight the feature weights of the key damage areas through a feature enhancement mechanism; a multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment state features, generate a multi-modal fusion feature vector, and classify and identify it, output damage related information, and form a structured detection result data; The hierarchical transmission and integrity guarantee unit is used for hierarchical processing of the structured detection result data according to the priority at the acquisition device end: the high-priority data retains complete features, and the conventional data retains only core features; meanwhile, transmission channels are allocated according to the data priority, and transmission parameters are dynamically adjusted according to the network state to guarantee stable communication with the upper computer; the integrity verification identifier and the time sequence identifier are added to the transmission data, and the upper computer confirms the data integrity through the verification identifier after receiving the data, and triggers the retransmission mechanism for abnormal data; The waveform reconstruction and real-time alarm unit is used for decoding the received compressed data at the upper computer end, reconstructing the detection waveform corresponding to the steel wire damage, and generating alarm details containing key information of the damage through real-time alarm judgment of the structured detection result data according to the preset judgment parameters.

[0057] Embodiment 3 The embodiment 3 also provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement any step of the steel wire detection and real-time transmission method based on multi-modal data fusion.

[0058] The computer readable storage medium can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0059] For the computer readable storage medium provided in the present application, refer to the above method embodiments, and the present application will not be repeated here.

[0060] Those skilled in the art can easily understand that the above description is only a preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A steel wire rope detection and real-time transmission method based on multi-modal data fusion, characterized in that, Comprising: S1. Start the detection system, collect the detection signal corresponding to the steel wire damage, position data and equipment running state data, and construct a multi-dimensional original data set; After preprocessing the original data, a multi-dimensional data correlation matrix is established; and the preprocessed data is divided into processing units, the feature information of each processing unit is extracted, and an initial feature data set is formed; S2. A multi-branch feature extraction network is used to extract the features of the detection signals corresponding to different types of steel wire damage in depth, and the feature weights of the key damage areas are highlighted through a feature enhancement mechanism; a multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment state features, generate a multi-modal fusion feature vector, and classify and identify it to output damage-related information and form a structured detection result data; S3. The structured detection result data is processed according to priority at the acquisition equipment end: high-priority data retains complete features, and regular data only retains core features; at the same time, transmission channels are allocated according to data priority and transmission parameters are dynamically adjusted according to network status to ensure stable communication with the upper computer; integrity verification marks and time sequence marks are added to the transmission data, and the upper computer confirms data integrity through the verification marks after receiving them, and triggers a retransmission mechanism for abnormal data; S4. The compressed data received at the upper computer end is decoded and the detection waveform corresponding to the steel wire damage is reconstructed; the structured detection result data is judged in real time according to the preset judgment parameters to generate alarm details containing key damage information.

2. The steel wire rope detection and real-time transmission method based on multi-modal data fusion according to claim 1, characterized in that, The original data set in S1 is specifically: The detection system is started, and the steel wire damage detection signal collected in the detection time The steel wire damage detection signal collected in the detection time Wherein, is the detection starting time; is the current detection time; ; The feature signal corresponding to the broken wire and the feature signal corresponding to the wear; ; is the amplitude-time sequence signal output by the corresponding area sensing component; is the amplitude-time sequence signal output by another corresponding area sensing component; The position data collected are , comprise cumulative displacement data collected in real time by the encoder and instantaneous speed data calculated on the basis of the rate of change of displacement wherein ; The collection device runs state data for , including motor operation parameter vector , battery power supply state vector and device working state identification ; wherein, , is the motor supply voltage, is the motor operating current, is the motor speed; battery supply state vector , is the real-time voltage of the battery, is the remaining battery capacity ratio; represents normal operation of the device, represents an abnormality of the device; A multi-dimensional raw data set is defined as wherein: ， ， ， In the formula, is a time series, is a time variable.

3. The method according to claim 1, characterized in that, The construction process of the initial feature data set in S1 is: based on the denoised damage detection signal obtained after preprocessing , calibrated position data and standardized operating status data , , is the detection time domain, a multi-dimensional data correlation matrix is constructed , the elements of the first row of the matrix are , wherein is the number of sampling points, ; Setting a preset frame length , by dividing into continuous sub-matrices , is rounded up; for each sub-matrix , , extracting corresponding time domain features and frequency domain features, extracting corresponding cumulative displacement mean, corresponding device running state identifier mode; The above features are spliced to form a feature vector consisting of all composing an initial feature dataset to provide input for the multi-branch feature extraction network.

4. The method according to claim 1, characterized in that, The feature enhancement mechanism in S2 is specifically: Different types of steel wire rope damage feature vectors output based on a multi-branch feature extraction network , For the number of damage types, the detection signal features corresponding to broken wires, wear and other damages Definition Damage feature saliency evaluation function: ， in, The global mean of features of the same damage type. The global standard deviation is... For the first Each processing unit corresponds to a damage feature component of that type; The total number of processing units; This function quantifies the saliency of each element in the damage feature vector, and the larger the value, the more significant the difference between the corresponding feature and the normal state feature, i.e. the more likely it is to correspond to a key damage area; Simultaneously introducing a feature response threshold , Based on historical non-damaged steel wire rope detection data statistics determination; for each damage feature vector Element-by-element judgment: when , the feature element is retained and its original value is kept; when , the feature element is suppressed, i.e. its value is set to zero, wherein, is the feature vector dimension index; Through the above process, the key damage area features are highlighted and strengthened, and invalid background feature interference is suppressed, obtaining a strengthened damage feature vector , providing a high-identifiability core damage feature input for subsequent multi-modal fusion processing.

5. The method according to claim 1, wherein, The form of the multi-modal fusion feature vector in S2 is specifically: The output of the multi-branch feature extraction network after being processed by the feature enhancement mechanism is different types of damage enhanced feature vectors , is the number of damage types, corresponding to broken wires, wear and tear and other damage; the calibrated position feature vector is , is the cumulative displacement feature after calibration, is the instantaneous speed feature; The standardized equipment state feature vector is , is a motor operating state feature, is a battery supply state feature; Definition of modal correlation factor ， ， wherein, is a covariance function, is a variance function, quantifying the dynamic strength of association of the damage feature with the location feature, the device state feature; The multi-modal fusion feature vector adopts a composite form of core feature dominance and associated feature adaptation, defined as: ， wherein, dimension stacking operation representing different types of injury characteristics, represents an element-wise multiplication operation; the front half of the fusion vector is the stacked representation of the core features of each type of injury, and the back half is the adaptive representation of the position features, device state features, and corresponding modal correlation factors, and finally , is the total dimension of the fusion features.

6. The method of claim 1, wherein, The process of dynamically adjusting transmission parameters according to network status in S3 is: Real-time monitoring of network transmission state between the acquisition device and the host computer, extracting network bandwidth , transmission delay and packet loss rate Three core state parameters, wherein, to monitor time variable; Defining network quality evaluation indicators Quantifying current network transmission capacity: ， Pre-set network quality threshold interval When The data transmission rate in the transmission parameter is set as the maximum value when the network state is excellent The default data packet length is adopted ; When , the transmission rate is adjusted to , is the network bandwidth utilization coefficient, determined based on historical transmission data statistics, and the data packet length is appropriately reduced to ; When , the transmission rate is reduced to , the data packet length is further reduced to , and the data packet fragmentation transmission mechanism is enabled.

7. The method according to claim 1, wherein, The method of adding integrity verification marks and time sequence marks in S3 is: Integrity verification identifier addition: add the single structured detection result data after hierarchical processing , is a data serial number, and a hash algorithm is used to calculate a feature digest thereof , is stored as an integrity verification identifier, and is bound with ; wherein the hash algorithm is selected from a SHA series algorithm or a CRC series check algorithm, and a fixed-length feature digest uniquely maps the original data, so as to ensure that the data is not tampered with or damaged during data transmission Adding time sequence identifiers: based on the detection time domain of the acquisition device. ,extract Corresponding acquisition time ,Will Convert to standardized timestamp Serves as a timing identifier; simultaneously records Ordinal index in the overall data sequence ,Will and Combined into a composite time sequence identifier , attached to The header field serves as a unique temporal identifier for the data; Finally, the format of each piece of data to be transmitted is defined as , which not only guarantees data integrity and verifiability, but also provides double timing basis for timing calibration of the upper computer.

8. The method according to claim 1, characterized in that, The specific process of distinguishing different regions of detection signals by waveform through differentiation is: The host computer decodes the damage detection signal The signal corresponding to the probe acquisition area is identified, and a differentiated identification rule is assigned based on the area type: for the collected broken wire signal , a first identification rule is adopted, that is, the color of the wave-shaped line is red, and the line width is not less than 2pt; to the collected wear signal , a second identification rule is adopted, that is, the waveform line color is set to blue, the line width is set to not less than 2pt, and a dashed line pattern is added; Assistance signals for damage acquired by an auxiliary sensing area a third identification rule is adopted, i.e. the color of the waveform line is set as green, the line width is set as not less than 1pt, and a dotted line pattern is added; At the same time, an identification explanation panel is established on the waveform display interface, and each differentiated identification is associated with the corresponding acquisition area and signal type, and the clear distinction of different region detection signals in the same waveform graph is completed through the visual differentiation of the identification setting.

9. A steel wire rope detection and real-time transmission system based on multi-modal data fusion, characterized in that, Comprising: A data acquisition and feature construction unit is used to start the detection system, collect the detection signal corresponding to the steel wire damage, position data and equipment running state data, and construct a multi-dimensional original data set; after preprocessing the original data, a multi-dimensional data correlation matrix is established; and the preprocessed data is divided into processing units, the feature information of each processing unit is extracted, and an initial feature data set is formed; The feature extraction fusion and recognition unit is used to extract the features of the detection signals corresponding to different types of steel wire rope damage by using a multi-branch feature extraction network, and highlight the feature weights of the key damage areas through a feature enhancement mechanism; a multi-modal fusion module is introduced to fuse the extracted damage features with the calibrated position features and equipment state features, generate a multi-modal fusion feature vector, and classify and identify the feature vector to output damage-related information and form structured detection result data; The hierarchical transmission and integrity guarantee unit is used to process the structured detection result data according to the priority at the acquisition device end: the high-priority data retains complete features, and the regular data retains only core features; meanwhile, transmission channels are allocated according to the data priority, and transmission parameters are dynamically adjusted according to the network state to guarantee stable communication with the host computer; integrity verification marks and time sequence marks are added to the transmission data, and the host computer confirms the data integrity through the verification marks after receiving the data, and triggers a retransmission mechanism for abnormal data; The waveform reconstruction and real-time alarm unit is used to decode the received compressed data at the host computer end, reconstruct the detection waveform corresponding to the steel wire rope damage, and generate alarm details containing key damage information by combining preset judgment parameters for real-time alarm judgment of the structured detection result data.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to perform the steel wire rope detection and real-time transmission method based on multi-modal data fusion according to any one of claims 1-8.

Images