Special equipment safety monitoring method based on Internet of Things

Abstract

The invention discloses a special equipment safety monitoring method based on the Internet of Things, and provides a multi-mode sensing data acquisition, noise removal, data standardization and label completion method. Rare abnormal samples are screened through unsupervised clustering and anomaly detection, and diverse pseudo-abnormal data are expanded through a conditional generative adversarial network. And deep representation learning, multi-modal feature mapping and migration fusion are further applied to construct a multi-modal collaborative abnormal feature set, and the multi-modal collaborative abnormal feature set is incorporated into an actively optimized security event discrimination model to realize dynamic rule adaptive evolution.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of safety monitoring of special equipment based on the Internet of Things, and in particular to a safety monitoring method for special equipment based on the Internet of Things. BACKGROUND

[0002] The technical field of safety monitoring of special equipment based on the Internet of Things and intelligent safety event identification, in particular, a safety monitoring method for special equipment based on the Internet of Things, is an important development direction in the field of equipment state monitoring, industrial Internet and intelligent operation and maintenance in recent years. With the large-scale application of special equipment in the energy, chemical industry, transportation and other industries, intelligent perception of safe operation and abnormal early warning have higher requirements. The mainstream solution is mostly based on a distributed Internet of Things sensor network, which collects device temperature, vibration, current and other multi-source signals, and uses big data analysis, unsupervised learning and expert knowledge base to realize identification and discrimination of abnormal events. In recent years, deep learning, multi-modal fusion and multi-source data-driven anomaly detection and fault identification models have achieved preliminary results in industrial Internet of Things applications, and have strengthened the identification ability of common fault patterns and known abnormal working conditions. Some systems also combine rule adaptation, safety event trend prediction, and continuously learn and optimize the discrimination algorithm based on historical data to promote the intelligent development of safety monitoring.

[0003] However, the existing technology still faces obvious bottlenecks when facing extreme environments, complex structures and new rare abnormal events. The current adaptive discrimination rules mostly rely on historical sample data and known abnormal patterns, and respond slowly to new abnormal patterns caused by extreme working conditions or equipment evolution. For example, when the device operating environment changes dramatically, the structure is upgraded or rare multi-modal abnormal coupling occurs, the traditional rule migration based on historical labels has limited ability to expand new discrimination boundaries in real time, resulting in frequent missed and misjudged events, and the overall robustness of the system is insufficient. In addition, in the existing anomaly detection process, the sample class distribution is highly uneven, and the rare abnormal data is extremely scarce, and the sample enhancement and generalization ability is insufficient, which further limits the system's early warning coverage of unknown threats. The feature transfer and scene generalization ability between multi-modal sensor data is not strong, which makes it difficult to infer the cooperative or interactive faults of single modal anomalies, affecting the comprehensive capture of abnormal conditions under complex conditions. Public documents show that traditional intelligent rule optimization methods often rely on simple threshold dynamic adjustment or a small amount of expert intervention, and it is difficult to realize automatic cross-modal transfer of abnormal features, dynamic expansion of abnormal sample space and intelligent discrimination of different structure working conditions. SUMMARY

[0004] The present application provides a safety monitoring method for special equipment based on the Internet of Things to solve the above technical problems.

[0005] The technical solution of the present application is as follows: A safety monitoring method for special equipment based on the Internet of Things, comprising: S1: Collecting multi-modal sensor raw data from multiple physical locations within a special equipment, the multi-modal sensors including temperature sensors, vibration sensors and current sensors, and synchronously recording environmental labels and structural parameter information of each sampling point to form a multi-modal data set with spatial, working condition and structural attributes.

[0006] S2: Performing noise removal, data format standardization and working condition label completion processing on different sensor data in the multi-modal data set respectively to obtain standardized multi-modal data sequences with consistent time alignment and annotation information.

[0007] S3: Based on the standardized multi-modal data sequences, using unsupervised clustering and anomaly detection algorithms to filter out suspected rare abnormal sample segments, and distributing the abnormal samples according to the collection location, environmental label and structural parameter to form a difference abnormal sample distribution table.

[0008] S4: For each type of abnormality in the difference abnormal sample distribution table, input the corresponding original multi-modal data into the generative adversarial network (GAN) to perform virtual abnormal sample generation operation, expand the abnormal data set to form a pseudo abnormal sample set with diversified working conditions and structural change characteristics.

[0009] S5: Inputting the multi-modal abnormal samples in the expanded abnormal data set into the representation learning model respectively to extract key abnormal feature parameters in temperature, current and vibration modalities, establishing feature mapping relationship of each modality abnormal representation for cross-modality feature fusion.

[0010] S6: Based on the established modality feature mapping relationship, performing cross-modality feature transfer and fusion operation to form a multi-modal collaborative abnormal feature set, covering structural differences, working condition labels and abnormal features of different spatial positions.

[0011] S7: Incorporating the multi-modal collaborative abnormal feature set into the active learning process of the safety event discrimination model, using dynamic self-adaptive optimization algorithm to update and calibrate the discrimination rules regularly, automatically generating dynamic safety event discrimination rules adapted to various structures, working conditions and scenarios.

[0012] S8: After updating each round of safety event discrimination rules, based on the combination of simulation deduction and actual equipment backtesting, evaluating the discrimination sensitivity and generalization performance of the dynamic safety event discrimination rules under different working conditions, structures and new abnormal patterns.

[0013] S9: If the evaluation shows that the prediction accuracy of the dynamic safety event discrimination rules driven by the multi-modal collaborative abnormal feature set under rare, extreme or new abnormal scenarios does not meet the set standard, automatically adjusting the abnormal sample generation strategy, feature transfer parameters or discrimination model structure according to the evaluation feedback until the performance requirements are met.

[0014] The application provides a special equipment safety monitoring method based on Internet of Things, which has the following beneficial effects: (1) The application is aimed at the core bottleneck of the existing safety monitoring system that the dynamic discrimination rule is not suitable for new abnormal patterns and is not sensitive to extreme or evolving abnormalities. The generative adversarial network (GAN) is innovatively introduced to realize virtual expansion of extreme and rare abnormal samples. Combined with feature migration and fusion of multi-modal (temperature, vibration, current, etc.) sensing data, the generalization and sensitive identification ability of the system to sample deficiency or historical unseen abnormal situations is greatly improved. Compared with the traditional method relying on simple historical data backtracking and static rule construction, the application can improve the detection rate of new abnormal events by 10-30%, effectively reducing the missed detection blind area of the model under unknown working conditions.

[0015] (2) In order to overcome the problem of rigid model rule and limited generalization ability in the prior art, the application adopts active learning and self-adaptive optimization algorithm (genetic optimization, particle swarm optimization, reinforcement learning, etc.) to continuously update and reconstruct the parameter space of the discrimination rule. Whenever new abnormalities and environmental variations occur, the system can automatically correct the abnormal sample generation mechanism, feature mapping and discrimination parameters according to the feedback data, realizing seamless tracking and self-evolution of the discrimination model to equipment working conditions, structure and environmental changes.

[0016] (3) The innovative data flow processing chain covers high-precision acquisition of multi-physical location and multi-modal original signals, three-layer classification of space-working condition-structure, and multiple normalization processes of noise elimination, standardization and label integrity. With the help of conditional generative adversarial network (cGAN), high-consistency and diversified pseudo abnormal samples can be generated under multi-dimensional working condition space, not only expanding the richness of original abnormal data, but also greatly alleviating the restriction of rare abnormal samples in engineering practice on model training, improving the training convergence speed of the discrimination model and data utilization.

[0017] (4) Through standardized data flow, representation learning, feature uniform coding, maximum mean difference minimization and other advanced methods, the application deeply extracts and aligns temperature, current, vibration and other multi-modal abnormal parameters, constructs high-order correlation mapping between features and cross-modal migration mechanism. Through typical correlation analysis, mutual information measurement and adversarial domain adaptation, the application realizes the collaborative fusion of abnormal features under complex equipment structure, spatial position and working condition changes, effectively improving the organization and discrimination ability of the system to complex and combined abnormal patterns. BRIEF DESCRIPTION OF DRAWINGS

[0018] Fig. 1 The application provides a flowchart of a special equipment safety monitoring method based on Internet of Things. Fig. 2A sub-flow chart of a special equipment safety monitoring method based on the Internet of Things according to the present application; Fig. 3 Another sub-flow chart of a special equipment safety monitoring method based on the Internet of Things according to the present application. DETAILED DESCRIPTION

[0019] In order to make the objects and advantages of the present application more clear and obvious, the present application will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0020] The preferred implementation methods of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these implementation methods are only used to explain the technical principles of the present application, and do not limit the protection scope of the present application.

[0021] As used herein, the singular forms "a", "an" and "the" can also include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "comprise / contain" or "have" and the like specify the presence of stated features, integers, steps, operations, components, parts or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, parts or combinations thereof. At the same time, the term "and / or" used in the specification includes any and all combinations of the related listed items.

[0022] Please refer to Figs. 1-3 As shown in the drawings, a special equipment safety monitoring method based on the Internet of Things comprises: S1: Collecting multi-modal sensor raw data from multiple physical positions in the special equipment, the multi-modal sensor comprising a temperature sensor, a vibration sensor and a current sensor, and synchronously recording environmental labels and structural parameter information of each sampling point to form a multi-modal data set with spatial, working condition and structural attributes.

[0023] S2: Performing noise removal, data format standardization and working condition label completion processing on different sensor data in the multi-modal data set respectively to obtain standardized multi-modal data sequences with consistent time sequence alignment and labeling information.

[0024] S3: Based on the standardized multi-modal data sequences, using unsupervised clustering and anomaly detection algorithms to screen out suspected rare abnormal sample segments, and distributing and classifying the abnormal samples according to the collection positions, environmental labels and structural parameters to form a difference abnormal sample distribution table.

[0025] S4: For each type of abnormality in the difference abnormal sample distribution table, input the corresponding original multi-modal data into the adversarial generation network (GAN) to perform virtual abnormal sample generation operation, and expand the abnormal data set to form a pseudo abnormal sample set with diversified working conditions and structural change characteristics.

[0026] S5: Input the multi-modal abnormal samples in the expanded abnormal data set into the representation learning model respectively, extract the key abnormal feature parameters in the temperature, current and vibration modes, and establish the feature mapping relationship of each modal abnormal representation for cross-modal feature fusion.

[0027] S6: Based on the established modal feature mapping relationship, perform cross-modal feature transfer and fusion operation to form a multi-modal collaborative abnormal feature set, which covers structural differences, working condition labels and abnormal features of different spatial positions.

[0028] S7: Incorporate the multi-modal collaborative abnormal feature set into the active learning process of the safety event discrimination model, and use a dynamic self-adaptive optimization algorithm to update and calibrate the discrimination rules regularly to automatically generate dynamic safety event discrimination rules that adapt to various structures, working conditions and scenes.

[0029] S8: After updating each round of safety event discrimination rules, evaluate the discrimination sensitivity and generalization performance of the dynamic safety event discrimination rules under different working conditions, structures and new abnormal modes based on the combination of simulation deduction and actual device backtesting.

[0030] S9: If the evaluation shows that the prediction accuracy of the dynamic safety event discrimination rules driven by the multi-modal collaborative abnormal feature set under rare, extreme or new abnormal scenarios does not meet the set standard, automatically adjust the abnormal sample generation strategy, feature transfer parameters or discrimination model structure based on the evaluation feedback until the performance requirements are met.

[0031] The step S1: Collecting multi-modal sensor original data from multiple physical positions in the special equipment, the multi-modal sensor including temperature sensor, vibration sensor and current sensor, and synchronously recording the environmental label and structure parameter information of each sampling point to form a multi-modal data set with spatial, working condition and structure attributes. Specifically, it includes: S1.1: Deploy multi-modal sensor devices on each key monitoring unit of the special equipment, including temperature sensors, vibration sensors, and current sensors, and realize unified identification and configuration of each physical position sensor node based on the industrial bus communication protocol to obtain a complete spatial distribution multi-modal sensor configuration table.

[0032] The key monitoring units of the special equipment are taken as the execution object, including the main structure node, the core power component and the related system interface.

[0033] Adopting multi-modal sensor deployment strategy, dedicated temperature sensor, vibration sensor and current sensor are arranged for each monitoring unit to realize full-quantity signal capture of equipment running temperature, mechanical vibration and power response.

[0034] Further, through unified configuration process based on industrial bus communication protocol (such as Modbus-TCP, CANopen, Profibus, etc.), node unique physical address is allocated on equipment control bus or packet communication link to complete ID registration and space mapping of each sensor node.

[0035] Further, sensor node parameter issuing and activation are performed, and according to controlled master station or slave station protocol logic, sampling period, signal channel number and sensor physical parameters (such as sensitivity, range, calibration factor) are configured for each type of sensor node arranged.

[0036] Further, based on network automatic discovery and topology scanning mechanism, sensor node connection state, specification parameters and activation identification of all physical positions on the bus are periodically polled and detected, and idle / online / fault node state information is summarized.

[0037] Through unified node configuration and metadata collection, all node entity information and key parameters (space position, sampling channel, node ID, sensor type, activation state, etc.) are filed in real time to form a structured spatial distribution multi-modal sensor configuration table.

[0038] Through the above sensor deployment and configuration operation, the mapping of the sensor nodes of the special equipment at multiple physical positions is converted into a multi-modal sensor configuration table with clear spatial distribution and unified parameter list, realizing the hardware and logic basis for subsequent efficient signal acquisition and multi-source data fusion.

[0039] For example, for a certain type of high-pressure boiler equipment, five key monitoring units are selected, including the surface of the pot body, the outlet of the furnace, the water inlet, the main motor seat, and the safety valve. Each unit is equipped with a Pt100 type temperature sensor (sensitivity: 0.00385 Ω / °C, range: -50~400°C), an IEPE type acceleration vibration sensor (sensitivity: 100 mV / g, range: ±10g), and a closed Hall current sensor (range: 0~50A, frequency response: DC~20kHz). Using the Modbus-TCP protocol, the sensor node IDs T01~T05, V01~V05, and C01~C05 are manually or automatically assigned on the gateway console, and the sampling rates are set to 1Hz, 250Hz, and 10Hz, respectively. Initialize node parameters, activate all sensor nodes, and import sensor detailed parameters (including location identification, signal number, activation status, and sampling rate) into the main control system through configuration download function. The system periodically detects and updates the online sensor list automatically through the broadcast discovery protocol, and synchronously alarms and marks the fault status when abnormal disconnection occurs. Finally, the spatial distribution multi-modal sensor configuration table is exported, including sensor ID, sensor type, physical location, sampling rate, and communication status information. Field operation data shows that the single node acquisition configuration information accuracy rate is 100% after arrangement, and the node active state monitoring response time is less than 100ms on average, laying a solid foundation for subsequent high-quality acquisition and signal labeling.

[0040] S1.2: Based on the multi-modal sensor configuration table, periodically perform real-time signal acquisition operations on temperature sensors, vibration sensors, and current sensors, and record the corresponding spatial position label for each sampling time to obtain multi-modal sensor raw signal sequences with time and space labels.

[0041] The input is a sensor node based on the configured spatial distribution multi-modal sensor configuration table, which includes temperature sensors, vibration sensors, and current sensors. The node information covers physical location identification, sampling channel number, sensor type, and sampling rate parameters.

[0042] A periodic task scheduling module (parameter configuration: sampling period is set according to sensor type, e.g. temperature 1Hz, vibration 250Hz, current 10Hz) is used to trigger raw signal acquisition operations on all temperature, vibration, and current sensor nodes registered in the configuration table based on automated data acquisition logic, achieving real-time acquisition of full signals from each physical monitoring unit of special equipment.

[0043] Further, through a high-precision local timestamp synchronization mechanism (time reference source: master system time synchronization, synchronized at each batch acquisition), the acquisition time of each multi-modal signal is strictly recorded, establishing a one-to-one mapping relationship between physical space node ID and global timestamp to distinguish different spatial positions and their time sequence states.

[0044] Further, by using a spatial tag automatic insertion program, the original signal data stream obtained each time is connected to its corresponding spatial position tag by analyzing the node physical position field in the sensor configuration table, realizing real-time labeling of signal spatial properties.

[0045] Further, by using a spatial tag automatic insertion program, the original signal data stream obtained each time is connected to its corresponding spatial position tag by analyzing the node physical position field in the sensor configuration table, realizing real-time labeling of signal spatial properties.

[0046] Through the above algorithm and data processing, the original signal stream of each multi-source sensor is converted into a spatially distributed, high temporal consistency multi-modal sensor original signal sequence with time and space labels, establishing a high-quality data basis for subsequent environment / structure label fusion and multi-dimensional feature extraction.

[0047] For example, in the high-pressure boiler monitoring scene, for main structure nodes T01, V01, C01 (corresponding to temperature, vibration, and current sensors at the same physical point, with sampling periods of 1 s, 4 ms, and 0.1 s), the scheduling program issues batch collection tasks. At t=10:00:00, the controller collects temperature data (120.3°C) for T01, vibration acceleration (0.045g) for V01, and current value (13.6A) for C01 in parallel. All three are labeled with node ID, spatial position tag "main structure node 1", and global synchronization timestamp 10:00:00. In continuous operation, the collection program collects signals from each node in an adaptive queue scheduling manner, and each thousand collection completes the main control system time synchronization. After analysis and testing, the collection signal timing alignment error is <5ms, and the spatial labeling accuracy is 100%. The final output data format is: {“Node ID”: T01, “Collection Type”: Temperature, “Physical Location”: Main Structure Node 1, “Timestamp”: 10:00:00, “Original Value”: 120.3}, etc., realizing efficient and accurate collection of multi-modal signal data and time and space label fusion, providing a solid foundation for subsequent associated working condition and structure parameter analysis.

[0048] S1.3: Combine device factory configuration files with local embedded sensor modules to synchronously collect and archive environment labels (such as temperature and humidity, workload status, etc.) and structure parameter information (such as key component models, connection methods, etc.), forming structure parameter label and working condition environment label data streams.

[0049] S1.4: Based on the multi-modal sensor raw signal sequence with space-time tags, structural parameter tags and working condition environment tag data stream, a unified data modeling framework (such as a multi-dimensional data frame assembly process based on an IoT message queue) is used for multi-level data fusion to generate a preliminary structured multi-modal acquisition data set.

[0050] S1.5: Based on the preliminary structured multi-modal acquisition data set, a data consistency verification algorithm is applied to compare the time series coverage, label integrity and abnormal value distribution under each physical location and sensor modality, and output a high-quality multi-modal data set that has passed the integrity and consistency verification, as the input basis for subsequent normalization conversion and feature engineering.

[0051] The step S2: respectively performing noise removal, data format standardization and working condition label completion processing on different sensor data in the multi-modal data set to obtain a normalized multi-modal data sequence with consistent time sequence alignment and label information. Specifically, it includes: S2.1: The median filtering algorithm and the adaptive threshold filtering algorithm are used to remove noise from the temperature sensor raw data in the multi-modal data set to obtain the denoised temperature time series signal, which provides clean basic data for subsequent data format standardization.

[0052] The input is a high-quality multi-modal data set collected through an Internet of Things architecture, where the temperature sensor raw signal covers different physical locations of special equipment and corresponding space-time, working condition and structural attribute tags.

[0053] The median filtering algorithm (window length n, configured according to the sampling rate and actual working noise characteristics) is used to replace the temperature sensor raw time series data in a sliding window to automatically suppress isolated peak noise and short-term observation anomalies, improving the anti-disturbance ability of the signal basis.

[0054] Further, the adaptive threshold filtering algorithm (parameter settings based on: temperature fluctuation normal range of environmental label statistics, dynamically calculated threshold T) is used to remove multi-scale outliers from the slowly varying anomalies or slowly drifting noise in the median filtered signal. The adaptive threshold filtering criterion is: set the temperature difference ΔT of adjacent sampling points as the threshold, dynamically compare the size relationship of T, and only keep the data points that meet the normal physical change speed.

[0055] The adaptive filtering threshold of the temperature signal is dynamically calculated using the following formula:

[0056] where, represents the historical temperature mean under a specific environmental label, is the temperature standard deviation under the same label, The adjustment coefficient is set according to the working condition characteristics of the equipment.

[0057] Further, a data index structure based on the spatial position of the sampling points and the working condition label is constructed, and the temperature signals after multiple filtering are grouped and archived to realize one-to-one correspondence merging of the temperature time series signals, spatial physical positions and environmental states.

[0058] Through redundancy check and sampling period coverage check, the consistency of the processed temperature signal time series integrity is determined, and the breakpoints or multiple redundant fragments caused by sensor failure or channel abnormality are screened out.

[0059] Through the above chain algorithm processing, the original temperature sensor time series data is converted into a denoised temperature time series signal with sufficient abnormal noise suppression, clear spatial attribution and excellent time series integrity, which realizes the provision of clean basic data for subsequent multi-modal signal format standardization and feature extraction operations.

[0060] For example, in the actual application of high-pressure boiler safety monitoring, the temperature sensor sampling period is 1s, and the median filter window length is 5 points. Random spikes caused by electromagnetic interference exist in the original signal, and after median filtering, the average spike frequency is reduced from 24 times per hour to less than 1 time. For the working condition with an environmental label of "high temperature area, load 80%", the historical temperature mean is 120°C, the standard deviation is 3.2°C, and , the adaptive threshold T is calculated as . For each sampling point, if , it is identified as noise removal. After the above filtering process, the temperature data fluctuation in actual observation tends to be smooth, the signal noise point detection rate is improved to more than 98%, and the breakpoint redundancy check pass rate reaches 100%. The output denoised temperature time series signal can stably reflect the equipment running condition, and provide a solid data foundation for the subsequent multi-modal denoising and standardization processing flow.

[0061] S2.2: Based on the denoised temperature time series signal, the original data of the vibration sensor and the original data of the current sensor are respectively processed by short-time Fourier transform and autoregressive sliding mean method to remove independent noise, and the multi-modal denoised signal stream is output, thereby laying a solid foundation for further data format unification.

[0062] S2.3: Taking the multi-modal denoised signal stream as input, using the data format standardization module to perform sampling rate resampling, dimension unification and data structure alignment on the temperature, vibration and current three modes respectively, obtaining time series synchronized and format consistent multi-modal standardized data stream, providing a unified data framework for label completion.

[0063] S2.4: For the multi-modal standardized data stream synchronized in time and consistent in format, the unannotated or missing working condition labels of the data are automatically annotated by using environmental labels and structural parameter information through a label completion algorithm based on correlation analysis, realizing full-coverage annotation of multi-modal data sequences.

[0064] S2.5: The multi-modal data sequence with full-coverage annotation is input into a data integrity verification module, and the continuity, consistency and label accuracy of the data are comprehensively checked through sequence missing detection and redundancy anomaly detection algorithms, and the standardized multi-modal data sequence meeting the high consistency requirement is output, laying a solid foundation for subsequent anomaly sample screening and analysis.

[0065] The step S3: Based on the standardized multi-modal data sequence, suspected rare abnormal sample segments are screened out by using unsupervised clustering and anomaly detection algorithms, and the abnormal samples are distributed and classified according to the collection position, environmental label and structural parameter, forming a difference abnormal sample distribution table. As shown in Fig. 2 , specifically comprising: S3.1: For the input standardized multi-modal data sequence, windowing division processing is performed based on time series segmentation algorithm to obtain structured time period multi-modal data segments, which are used for subsequent clustering and anomaly analysis.

[0066] S3.2: For the windowed structured time period multi-modal data segments, unsupervised clustering algorithm (such as density peak clustering, DBSCAN, etc.) is used for high-dimensional feature clustering division to discover the inherent working condition and structural state distribution cluster of multi-modal data, and obtain multi-modal clustering labels of each time period.

[0067] The input data is the structured time period multi-modal data segment obtained after S3.1 time series segmentation processing, wherein each segment contains synchronous signals of temperature, vibration, current and other modalities, as well as associated spatial position, working condition label and structural parameter annotation.

[0068] Density peak clustering algorithm (parameters include distance metric method Euclidean distance, adaptive cutoff radius d_c and density threshold p_c) is used to construct local density atlas in high-dimensional space for each time period multi-modal data segment, realizing quantitative judgment of natural clustering trend of multi-modal features.

[0069] Further, through local density and minimum distance calculation in the density peak clustering algorithm, the density and distance of each data segment are calculated as follows:

[0070] wherein, is the high-dimensional feature distance between segment i and segment j. is the truncation distance, is the Heaviside step function , , otherwise ).

[0071]

[0072] where, is the minimum distance from the ith segment to any denser segment.

[0073] By selecting the highest joint score of several data segments in the density-distance distribution as the initial clustering center, the clustering label of each structured time period multi-modal data segment is obtained by recursive merging and assigning according to the high-dimensional feature distance between segments.

[0074] Combined with the modal space distribution of multi-modal data itself, DBSCAN algorithm (parameters: minimum sample number minPts, neighborhood radius ε) can be selected to automatically identify high-density working condition clusters and spatial structure clusters based on the density connectivity of high-dimensional multi-modal features, and the adaptability to non-spherical distribution and abnormal clusters is improved. In the DBSCAN algorithm, each segment belongs to a high-density class within the neighborhood of the core point, and isolated segments or low-density neighborhoods are aggregated as noise or special abnormal classes.

[0075] The multi-modal feature cross-validation of the clustering label result is performed, and the significant differences of working conditions and structures of each modal data in the same period are evaluated to assist in calibrating the accuracy of clustering division and improve the representation ability of clustering labels to the actual device operating state and working condition structure differences.

[0076] Through the above unsupervised clustering algorithm processing, each windowed multi-modal period data segment is assigned a high-dimensional clustering label, and the automatic discovery of the internal working condition and structure state distribution cluster of multi-modal data is realized, which lays a feature foundation for subsequent anomaly detection and abnormal sample screening.

[0077] For example, in the application of a special pressure vessel safety monitoring system, the input data is a multi-modal period data segment divided by a 1-minute sliding window, each segment containing synchronous features of temperature (all 20Hz sampling), vibration (all 200Hz sampling), and current (all 10Hz sampling), and having spatial labels, working condition labels, and structure parameters. The density peak clustering algorithm is used to set the truncation radius d_c as the 20% quantile of the distance distribution between each modal feature spectrum, and the initial density threshold ρ_c = 10 is obtained. The Euclidean distance of all segment pairs is calculated, and the local density based on the truncation radius is calculated , the density-distance product of each segment is calculated by the formula above, and the segment with the maximum density-distance product is selected as the cluster center. Further, the DBSCAN algorithm is applied, and the parameters are set as ε = 0.8 (standardized by distance distribution) and minPts = 8, to automatically detect high-density clustering areas. After clustering, 6 multi-modal working condition-structure fusion clusters are obtained, of which 2 are dominant normal working clusters, 1 is a high-power high-load abnormal cluster, and the rest are small clusters and isolated points in special structural sections. The clustering labels are verified by modal cross verification (testing the variability of each sensor signal under different working conditions), and the false negative rate is controlled within 3%, and the abnormal detection rate is improved to 92%. The output of this step is the high-dimensional clustering label of each time period multi-modal data segment, which provides accurate basic division for the next step of anomaly detection and difference anomaly classification.

[0078] S3.3: Based on the multi-modal clustering labels of each time period, combined with anomaly detection algorithms (such as Isolation Forest and Principal Component Outlier Detection), time period multi-modal data segments that do not belong to the dominant class and show strong deviation from normal distribution are determined and screened out, and suspected rare abnormal sample groups are preliminarily locked.

[0079] The input is multi-modal time period data segments based on normalized processing, including temperature, vibration, current, and other multi-source sensor signals, as well as spatial, working condition, structural, and environmental label features.

[0080] The Isolation Forest (Isolation Forest, parameters: number of trees n_estimators, feature subsampling ratio max_features) in the anomaly detection algorithm is used to score the high-dimensional feature vectors of each multi-modal data segment, and to identify window samples that show strong deviation from the mainstream distribution.

[0081] Further, the Principal Component Outlier Detection (Principal Component Outlier Detection, parameters: cumulative contribution rate th_cum, anomaly score threshold th_outlier) is used to statistically analyze the principal component score distribution after dimensionality reduction in each time period, and to output abnormal labels for segments that deviate from the normal distribution range of principal components.

[0082] Further, the output results of Isolation Forest and Principal Component Outlier Detection are fused, and based on the multi-modal clustering label distribution, edge group samples that do not belong to the dominant class are determined, and time period segments with high anomaly degree and significant principal component residual are labeled.

[0083] The anomaly score Score(x) of data segment x in Isolation Forest is calculated by the following formula:

[0084] wherein, the average path length of sample x in the isolated tree group, the normalization constant corresponding to the data set size n.

[0085] In principal component anomaly detection, Mahalanobis distance D is used to describe the abnormality of samples in the principal component space:

[0086] wherein, the principal component mean vector, the principal component covariance matrix.

[0087] Further, the time segments with abnormal scores exceeding the preset threshold value screened out by the above algorithm are mapped to the respective corresponding multi-modal clustering labels and metadata structures, to realize clear distinction between abnormal windows and normal categories.

[0088] Through the abnormality detection processing mode, the structured multi-modal time period data obtained in the previous step is labeled into three categories of strong abnormality, weak abnormality and mainstream category, to preliminarily lock the suspected rare abnormal sample group, and realize automatic screening and intensive focus of multi-modal abnormal samples.

[0089] For example, in the operation monitoring scene of a high-pressure pump station of special equipment, the standardized 30-dimensional multi-modal feature window segment is sampled with a window sliding period of 5s. The isolated forest parameter n_estimators is set to 100, and max_features is 1.0. In principal component analysis, the first 6 principal components with a cumulative contribution rate of more than 98% are retained, and the abnormal score threshold (Score(x)>0.65 and Mahalanobis distance D(x)>3) is calculated for each batch of data segments. In actual operation, in continuous one-week data collection, an average of 4 strong abnormal segments and 11 weak abnormal segments are locked for every thousand time windows, and the mainstream normal category segment accounts for 98.5%. The above abnormal segments are concentrated in the bearing area of the pump body, the structure parameter is "enhanced type", and the working condition label is mostly "start-up segment" or "load reduction switching segment", which is highly correlated with the rare bearing failure in the actual maintenance record. The number of suspected rare abnormal sample groups successfully output finally is consistent with the artificial interpretation rate of 96.7%, which significantly improves the reliability and engineering adaptability of non-participatory anomaly detection.

[0090] S3.4: For the suspected rare abnormal sample group, the acquisition position metadata, environmental label and structure parameter metadata are used for classification and aggregation processing, to obtain a multi-modal abnormal sample distribution subset with spatial layering, environmental label layering and structure type layering.

[0091] The input is a group of suspected rare abnormal samples screened out by an anomaly detection algorithm (such as Isolation Forest, Principal Component Anomaly Detection), and each abnormal sample is attached with location metadata, environmental label and structure parameter metadata.

[0092] Using a spatial label aggregation method, each abnormal sample is subjected to spatial layering operation according to the collection location metadata, and the abnormal samples are assigned to the corresponding physical area or equipment structure level, realizing high spatial resolution anomaly archiving.

[0093] Further, based on the environmental label metadata, an environmental layering algorithm is applied to re-group the abnormal samples in the same spatial level according to the discrete values of environmental working condition variables (such as temperature and humidity, load state, etc.), establish an environmental label layered abnormal subset, and realize the working condition state decomposition and classification of abnormal distribution.

[0094] Further, through a structure parameter analysis method, the structure parameter metadata (such as key component model, equipment connection mode, etc.) is used to aggregate the abnormal samples under the above spatial-environmental joint layering, and the abnormal sample groups with the same model, structure form and assembly process are archived, completing the structure type layering.

[0095] Further, for the sample set that has completed spatial layering, environmental layering and structure layering, a multi-layer nested data grouping index structure is constructed, each subset is assigned a clear spatial level number, environmental label number and structure parameter identification, and a multi-modal abnormal sample distribution subset with three-layered attributes is realized.

[0096] Through the above classification and aggregation processing method, the original suspected rare abnormal sample group is multi-dimensionally labeled and structured layered, providing a high-recognizability data basis for classification statistics, feature migration modeling and abnormal pattern tracing.

[0097] Exemplary, in the special equipment pressure vessel safety monitoring system, the input is the data segment determined by the isolated forest algorithm with a rare abnormal probability greater than 0.95, wherein each segment is associated with a space label (such as "upper main cylinder area", "lower feeding area"), an environment label (such as "high temperature area, load 100%", "low temperature area, empty load"), and a structure parameter (such as "model X1", "welding type Y-2"). When the space is layered, all abnormal samples are divided into three areas A, B, and C according to the space label. When the environment is layered, for the samples in each space area, they are subdivided into different groups according to the environment label. When the structure is layered, in the space-environment two-level subset, the model X1 group and the model X2 group are further subdivided according to the structure parameter. Finally, each subset such as (space area A-high temperature-model X1) contains N abnormal samples, and the entire multi-layer index table covers the complete space, environment, and structure three-layer layering. The actual output of the multi-modal abnormal sample distribution subset is combined into 54 groups, each with a sample size of 5-23, which can be used for subsequent GAN sample generation and abnormal feature cross-modal migration research, realizing efficient and fine support for abnormal distribution space environment structure attribution.

[0098] S3.5: According to the classification type, the distribution characteristics and the number of abnormal samples under each layering are counted and the abnormal occurrence conditions are coded, and finally a difference abnormal sample distribution table with collection position, environment label and structure parameter annotation is formed, which is used as the input of the pseudo abnormal sample generation and feature migration model.

[0099] The step S4: for each type of abnormality in the difference abnormal sample distribution table, input the corresponding original multi-modal data into the generative adversarial network (GAN) to perform virtual abnormal sample generation operation, and expand the abnormal data set to form a pseudo abnormal sample set with diversified working conditions and structure change characteristics. As shown in Fig. 3 , specifically comprising: S4.1: For each type of abnormality in the difference abnormal sample distribution table, extract the associated original multi-modal data (including temperature sensor data, vibration sensor data and current sensor data), and identify the working condition label, space position label and structure parameter information, to obtain a structured multi-modal abnormal data set as the input basis for training the generative adversarial network (GAN).

[0100] The input data is the difference abnormal sample distribution table output by step S3.5, and each type of abnormality is associated with original multi-modal sensor data, including temperature sensor signal, vibration sensor signal and current sensor signal, as well as working condition label, space position label and structure parameter information.

[0101] The layered data extraction method (parameters: category layered conditions are collection position, working condition label, and structure parameter) is adopted to realize batch processing of all category abnormalities in the distribution table, and each category abnormality is separately extracted to extract the corresponding original multi-modal time sequence data segment.

[0102] Further, through the multi-label mapping algorithm (parameters: spatial position uniqueness constraint, label compliance judgment function), the three-dimensional sensing signal of each original data segment is one-to-one bound with its working condition label, spatial position label, and structure parameter information to generate a multi-channel labeled time sequence feature matrix.

[0103] Further, the feature normalization processing method (parameters: each modal feature normalization range, time synchronization window length) is adopted to uniformly normalize and align the extracted temperature, vibration, and current multi-modal signals, and assemble them into structured multi-modal data sequences according to the time window.

[0104] Further, through the metadata structured archiving algorithm (parameters: metadata fields include: collection time, spatial position, working condition type, and structure parameter), the above unified time sequence multi-modal data and its label are standardized and packaged to integrate into a structured multi-modal abnormal data set, which has complete working condition, structure, and spatial metadata annotations.

[0105] Through the above chain algorithm process, the original multi-modal data associated with each category abnormality in the difference abnormal sample distribution table is converted into a structured multi-modal abnormal data set, realizing the high consistency multi-modal feature input required for the training of the generative adversarial network (GAN), and providing a complete data basis for subsequent virtual abnormal sample generation, feature modeling, and distribution control.

[0106] Exemplarily, in the safety monitoring project of the special elevator drive unit, the input data is the motor abnormality category distribution table output in step S3.5, and the “high temperature / vibration composite abnormality” category is obtained through hierarchical condition screening. The associated original sensor data covers temperature (sampling frequency 10 Hz, range -20 to 120°C), vibration (sampling frequency 200 Hz, range ±8g), and current (sampling frequency 25 Hz, range 0-100A). Using a multi-label mapping algorithm, the three-way signals, spatial label (drive motor 1#), working condition label (high load start), and structure parameter (bearing type B6204) of each abnormal period segment are assembled into a 256-frame labeled feature time sequence matrix. In the feature normalization process, the temperature signal is standardized to 0~1, the vibration signal is converted to mean zero standard deviation normalization, and the current signal is linearly normalized to the [0,1] interval. A structured multi-modal protocol package is assembled with a 5-second window. Finally, through a metadata structured archiving algorithm, 20 groups of structured multi-modal abnormal data sets of “high temperature / vibration composite abnormality—drive motor 1#—high load start—bearing type B6204” are uniformly packaged and output. The data integrity and label consistency are automatically checked by the system and reach 100%. The structured multi-modal abnormal data set is used as the input of the generative adversarial network (GAN) model, which effectively improves the quality of subsequent virtual abnormal sample generation and the model's self-adaptive generalization ability.

[0107] S4.2: Based on the structured multi-modal abnormal data set, a conditional generative adversarial network (Conditional GAN, cGAN) is used to embed features and model latent variables of the input multi-modal abnormal samples, to learn the abnormal distribution rules under various working conditions, spatial structures and environmental attributes, and to provide modeling support for subsequent high-quality abnormal sample generation.

[0108] The input is the structured multi-modal abnormal data set processed in step S4.1, which includes temperature, vibration and current sensor signals, as well as working condition labels, spatial position labels and structure parameter annotations.

[0109] Using a feature embedding method, a feature encoder network is constructed to map high-dimensional features of the original multi-modal abnormal samples, and the time series signals of each sensor modality, spatial labels and structure parameters are uniformly embedded into the latent feature space to realize co-space representation of multi-modal data.

[0110] Further, using a conditional generative adversarial network (Conditional GAN, cGAN, parameter configuration including conditional input dimension, latent space dimension z, number of encoder and decoder layers, and loss function type), the working condition label, spatial position label and structure parameter are used as conditional variables Input into the generator and discriminator modules to conditionally model the abnormal sample distribution based on multi-modal feature encoding.

[0111] The following feature embedding expression and latent variable joint modeling method is adopted:

[0112] wherein, is the original multi-modal time series data, is the conditional label, is the multi-modal feature encoding function, is the high-dimensional embedding feature vector.

[0113] Through the generator of the cGAN and the discriminator co-training, the following objective function is optimized:

[0114] wherein, is the multi-modal feature embedding vector, is the latent space noise vector, is the working condition, space, structure condition, is the generator mapping, is the discriminator.

[0115] Using the above latent feature modeling mechanism, the generator and discriminator parameters are dynamically optimized within the training period, gradually approaching the true multi-modal anomaly sample distribution law under various working conditions, structures, and spatial distributions, enhancing the accuracy and generalization ability of conditional generation.

[0116] Through the conditional feature embedding method and the GAN latent variable consistency modeling method, the structured multi-modal anomaly data set is converted into a joint representation of the conditional multi-modal latent distribution parameter, providing modeling support for subsequent sampling of diversified pseudo-anomaly samples and evaluating their quality, and realizing automatic capture and efficient modeling of anomaly distribution under different working conditions, positions, and structure attributes.

[0117] Exemplarily, in a special pipeline equipment safety monitoring scene, the input is a set of layered screened abnormal samples, the temperature signal sampling rate is 50 Hz (512 points per abnormal window), the vibration signal sampling rate is 500 Hz (1024 points per window), the current signal sampling rate is 20 Hz (128 points per window), and each sample is labeled with a spatial label (three grades), a working condition label (high, medium, and low load), and a structure parameter (model A / B / C). The feature encoder uses a three-branch time convolutional network, the output feature vector dimension is 128, the working condition and structure condition are encoded as 16-dimensional discrete vectors, and the latent space z dimension is 40. Both the cGAN generator and the discriminator use four-layer fully connected networks, and the optimization loss is the joint of the conditional adversarial loss and the feature reconstruction loss. After 2000 steps of training iteration, the overlap degree of the embedded feature distribution generated under each condition category and the t-SNE mapping of the original abnormal sample reaches more than 94%. The output is a multi-modal abnormal latent distribution parameter for all working conditions, structures, and spatial layering conditions, which provides a high-consistency modeling basis for the next step of conditional sampling and virtual abnormal sample batch generation, effectively supporting data enhancement and high-availability sample expansion.

[0118] S4.3: Use the multi-modal abnormal distribution parameter modeled by the latent feature variable to perform conditional sampling and virtually synthesize multi-modal abnormal data through the generator of the generative adversarial network (GAN), output pseudo-abnormal samples with diversified distribution in working conditions, spatial positions, and structure attributes, and expand the types and quantities of original abnormal data.

[0119] The input data is a set of multi-modal abnormal distribution parameters modeled by the latent feature variable of the conditional generative adversarial network (Conditional GAN, cGAN), which represents the high-dimensional distribution characteristics of abnormal samples in each category in the dimensions of working conditions, spatial positions, and structure attributes.

[0120] The conditional sampling method (parameters: working condition label, spatial position label, and structure parameter constraint) is used to perform conditional random sampling on the latent variable vector according to the specified working condition, space, and structure condition in the high-dimensional latent space of each category of abnormal feature distribution, realizing the diversity expansion of abnormal sample working conditions and spatial attributes.

[0121] Further, the generator network (parameters: multi-modal signal generation layer number, conditional input channel number, and generation loss weight, etc.) of the generative adversarial network (GAN) is used to synthesize multi-modal data for the sampled latent variable vector, realize the virtual abnormal data generation of temperature, vibration, and current modal signals, and the generated data automatically inherits the working condition, spatial, and structure label attributes.

[0122] Further, through diversity enhancement strategies (such as latent space perturbation, label replacement, etc., parameters: perturbation amplitude ε, label perturbation probability p), the latent variables of conditional sampling are perturbed and the labels are reorganized, expanding the distribution coverage of generated data and improving the diversity performance of virtual abnormal samples under extreme working conditions, edge space and rare structural parameters.

[0123] Further, a dynamic batch generation mechanism (parameters: number of samples per category batch B, sampling rounds R) is used to generate B groups of virtual abnormal samples under each working condition-space-structure combination, and the generator parameters are reused for continuous iteration of sampling and data synthesis, gradually increasing the number and modal richness of abnormal data of each category.

[0124] Through the above chain conditional sampling and generation method, the multi-modal abnormal distribution parameters modeled by latent feature variables are converted into pseudo abnormal samples with diversified distribution in working conditions, spatial positions and structural attributes, significantly expanding the types and quantities of original abnormal data, and realizing the comprehensive coverage of virtual abnormal sample library for new and complex abnormal types.

[0125] For example, in the monitoring application of high-pressure pumps of special equipment, for the "high temperature-high vibration-main bearing fault" category abnormality, the latent feature distribution parameters modeled by cGAN are input, the working condition label is set to "high load operation / over-set temperature", the spatial position label is set to "main pump bearing area", and the structural parameter is set to "enhanced bearing A2". In the conditional sampling stage, the working condition label condition vector is encoded, the spatial and structural parameters are normalized and mapped, the latent variable perturbation amplitude ε = 0.12, the perturbation probability p = 0.3, the batch processing parameter B = 30, and the sampling rounds R = 5. In the generator network, the temperature signal generation layer uses a 3-layer fully connected network (output length 50), the vibration signal generation layer uses a 2-layer convolutional network (output length 300), and the current signal generation layer uses a single-layer fully connected mapping (output length 20). Under each working condition-space-structure combination, 30 groups of virtual abnormal multi-modal time series signals are automatically synthesized, and a total of 150 (30x5) pseudo abnormal samples with complete labels are output. The adaptive diversity enhancement strategy ensures that the abnormal samples have sufficient performance in the extreme high temperature interval, abnormal vibration spectrum peak and current distorted waveform. The finally generated pseudo abnormal samples are subjected to subsequent authenticity discrimination and consistency screening, enriching the original "high temperature-high vibration" abnormal database and improving the recognition ability of subsequent discrimination models for rare samples. The system backtest abnormal detection rate is improved by more than 12%, and the generalization accuracy is significantly improved.

[0126] S4.4: The generated multi-modal pseudo-anomaly samples are subjected to authenticity discrimination and quality scoring by the discriminator. Through feature consistency comparison with the distribution of real anomaly samples, high-quality pseudo-anomaly samples with high discrimination scores and reasonable attributes are selected to form an expanded anomaly dataset, improving the effectiveness and generalization ability of data enhancement.

[0127] S4.5: The high-quality expanded anomaly dataset is integrated with the original structured multi-modal anomaly dataset to generate a final pseudo-anomaly sample set with multi-dimensional attribute annotations such as working conditions, spatial structures, and environmental labels, providing diverse and supplementary data support for subsequent anomaly feature parameter extraction and modal feature fusion operations of the representation learning model.

[0128] The step S5: The multi-modal anomaly samples in the expanded anomaly dataset are input into the representation learning model to extract key abnormal feature parameters in the temperature, current, and vibration modalities, establish feature mapping relationships for each modal anomaly representation, and perform cross-modal feature fusion. Specifically, it includes: S5.1: For temperature modal anomaly samples in the expanded anomaly dataset, a deep representation learning network (such as an autoencoder or variational autoencoder) is used to perform multi-layer embedded feature extraction, achieving the acquisition of key abnormal parameters (such as temperature fluctuation amplitude, rise and fall rate, temperature control abnormal window, etc.) in the temperature modality to obtain temperature modality key abnormal feature parameters.

[0129] S5.2: Based on the temperature modality key abnormal feature parameters, a time series convolutional neural network (TCN) is applied to the current modality anomaly samples in the expanded anomaly dataset to perform signal parameter normalization and redundancy removal, extracting core current modality key abnormal feature parameters such as current peak value, distortion coefficient, and period change rate.

[0130] The input conditions are the current modality anomaly samples in the expanded anomaly dataset and the temperature modality key abnormal feature parameters obtained in step S5.1.

[0131] A normalization preprocessing method (parameters: maximum and minimum value range of current signal, standardization scale) is used to achieve amplitude normalization and time series synchronization of the current modality anomaly signal.

[0132] Further, a time series convolutional neural network (Temporal Convolutional Network, TCN) (parameters: convolution kernel size, network layer number, sliding window length, residual connection strategy) is used to perform multi-layer time domain feature extraction on the normalized current time series signal, achieving information capture of local abnormal changes and global trend fluctuations in the current modality signal.

[0133] Furthermore, signal parameter normalization and redundancy feature suppression methods (parameters: standard deviation threshold, principal component retention ratio) are used to normalize the multidimensional current feature parameters initially output by TCN, and a redundancy detection algorithm is applied to remove highly correlated or meaningless feature subsets, thereby reducing the feature space dimension and improving the generalization ability of the extracted features.

[0134] Furthermore, using a sliding window peak detection algorithm (parameters: window length, step width, peak detection threshold) and a distortion coefficient calculation algorithm, the peak value is extracted from the preprocessed time-series current feature sequence. The calculation expression is as follows:

[0135] in, To detect the current signal within the window, This represents the sliding window interval.

[0136] Furthermore, by defining the distortion coefficient based on the following formula:

[0137] in, The amplitude of the fundamental component. For the first amplitude of second harmonic components The total harmonic number is used to obtain the total harmonic distortion coefficient of the current signal.

[0138] Furthermore, using the periodic change rate calculation method, the time-domain current signal is divided into multiple equal-length periods, and the peak value change of each period is measured using the following formula: , in, For the first Peak of the cycle This represents the total number of cycles.

[0139] Through the above chain-like technical process, hierarchical feature extraction of current mode anomaly samples is achieved, and key anomaly characteristic parameters such as current peak value, total harmonic distortion coefficient, and periodic variation rate are obtained.

[0140] By employing temporal convolutional neural network feature extraction, normalization and redundancy removal, peak detection, and distortion coefficient and period change rate quantization, the current mode anomaly sample results in the extended anomaly dataset are transformed into core current mode key anomaly feature parameters that can be used for cross-modal feature fusion. This enables accurate characterization of abnormal current behavior of equipment under abnormal operating conditions and provides a key foundation for subsequent multimodal anomaly characterization and feature synchronous mapping.

[0141] For example, for the abnormal prediction scene of the main drive motor of a certain type of special heavy-duty elevator, the input is 300 current mode abnormal time series labeled as "high load + periodic jitter" working condition in the extended abnormal data set, with 512 sampling points per segment and a current range of 0~80A. The mean-variance normalization method is used to standardize the input signal to a distribution with a mean of 0 and a variance of 1. The TCN feature extraction network is configured with 3 layers, a convolution kernel size of 7, and 128 output channels. Residual connections are used to prevent gradient vanishing. The TCN output is correlated and reduced in dimension, retaining only the top 10 mutual information features. The peak detection window is set to 50 points with a step of 50, and the threshold is 3σ. The average peak value extracted is 2.5 times higher than the normal working condition. The THD coefficient is calculated by FFT decomposition, which is 31% in the abnormal state (normal value is 8%). The average periodicity change rate is 0.21 (normal working condition is 0.04). The final output is three key feature parameters: current peak value, distortion coefficient, and periodicity change rate. Each sample forms a feature vector as the current mode abnormal representation parameter input to the downstream feature mapping model. The verification results show that the extracted features improve the accuracy of abnormal discrimination in complex structures and switching conditions to 93%, and the robustness and discrimination ability of the features are significantly better than those of artificial experience parameters.

[0142] S5.3: With the current mode key abnormal feature parameters as input, Fourier spectrum analysis and LSTM network joint modeling are applied to the vibration mode abnormal samples in the extended abnormal data set to detect feature frequency band energy concentration, time domain pulse amplitude, and short time energy fluctuation to obtain vibration mode key abnormal feature parameters.

[0143] S5.4: Comprehensive temperature mode key abnormal feature parameters, current mode key abnormal feature parameters, and vibration mode key abnormal feature parameters, apply typical correlation analysis (CCA) and mutual information measurement algorithm to calculate the high-order correlation mapping relationship between the key features of each mode, and determine the coupling rules of the modal abnormal representation features under the constraints of structural parameters and working condition labels.

[0144] S5.5: With the modal abnormal representation feature coupling rules as input, a cross-modal feature mapping table is constructed to form a multi-modal abnormal representation feature mapping relationship with temperature mode key abnormal feature parameters, current mode key abnormal feature parameters, and vibration mode key abnormal feature parameters as nodes, providing a standard mapping basis for subsequent feature migration and multi-modal collaborative abnormal feature set construction.

[0145] The step S6: Based on the established modal feature mapping relationship, perform cross-modal feature migration and fusion operation to form a multi-modal collaborative abnormal feature set covering structural differences, working condition labels, and abnormal features in different spatial positions. Specifically, it includes: S6.1: Perform feature uniform coding processing on the input established modal feature mapping relationship to obtain a multi-modal feature vector space, ensuring that the temperature modal feature parameters, current modal feature parameters and vibration modal feature parameters have consistent structural representation and scale specification.

[0146] S6.2: Based on the multi-modal feature vector space, use a feature alignment algorithm (such as the maximum mean difference minimization method) to align the distribution of modal feature parameters under each physical position, structure parameter and working condition label, to realize the consistency of the mutual mapping relationship of multi-modal abnormal features.

[0147] The input is the multi-modal feature vector space obtained by feature uniform coding, which includes temperature modal feature parameters, current modal feature parameters and vibration modal feature parameters, as well as associated spatial position, structure parameter and working condition label coding information.

[0148] The maximum mean discrepancy minimization (MMD) feature alignment algorithm (parameters: Gaussian kernel width, batch size, maximum number of iterations) is used to realize the distribution alignment of different modal feature distributions under the mapping of spatial position, structure parameter and working condition label.

[0149] Further, by constructing the covariance matrix and first-order statistics between modal features, the MMD regularization term optimization objective is used to minimize the mean difference of source modal and target modal features, and the mathematical expression is as follows:

[0150] Where, and represent the feature vector set of the source modal and the target modal respectively, and is the number of samples, is the feature space mapping function.

[0151] Further, by iteratively optimizing the above MMD loss function, the Gaussian kernel function mapping is realized to align the distribution of multi-modal feature parameters under different models at all sampling positions, structure parameter states and working condition labels, and output the alignment transformation matrix.

[0152] Further, for each group of spatial position, structure parameter and working condition label condition, the distribution distance change before and after the modal feature alignment is calculated respectively to evaluate the convergence rate and distribution consistency of the alignment process, forming a distribution consistency evaluation index set.

[0153] Further, by aligning the distribution of the results, the key abnormal features of each modality are converted into a unified standard feature space representation under the constraints of spatial position, complex structure parameters, and multi-variable working condition labels, generating a set of multi-modal abnormal alignment feature vectors with high cross-modal comparability and consistency, providing standardized input support for subsequent feature migration and fusion.

[0154] Through the above feature alignment algorithm and chain processing method, the multi-modal key abnormal feature parameters obtained in the previous step are converted into a highly consistent alignment feature set under the constraints of structure parameters, spatial distribution, and working condition labels, achieving distribution consistency and mapping relationship standardization between multi-modal abnormal features, effectively improving the generalization ability of cross-modal abnormal detection and the reliability of feature migration.

[0155] For example, in the safety monitoring scene of a certain type of high-pressure special boiler, the input is 456 sets of uniformly encoded multi-modal feature vector samples, including temperature peak, current THD, vibration energy concentration, etc., from 6 spatial positions, 4 structure parameter configurations, and 3 working condition labels. The MMD alignment algorithm sets the Gaussian kernel width to 5.0, the batch size to 64, and the maximum number of iterations to 1000. The original temperature modality and current modality feature mean distance is 0.52, which decreases to 0.11 after alignment. The vibration modality and other modalities alignment distribution distance decreases from 0.47 to 0.09. The aligned multi-modal feature set is used in the downstream feature migration and fusion, and in the performance evaluation, it is found that the cross-modal abnormal detection discriminant F1 value is improved by about 16% compared to the unaligned scheme, and the feature consistency in each spatial position and structure scene is significantly enhanced. The final output is a high-consistency multi-modal abnormal alignment feature set that covers all structure parameters and working condition spaces, supporting efficient learning and generalization of dynamic adaptive safety event discrimination rules.

[0156] S6.3: For the distribution-aligned multi-modal feature parameters, apply a feature transfer learning algorithm (such as an adversarial domain adaptation network) to transfer the source modality key abnormal feature parameters to the target modality space, thereby enhancing the correlation expression ability of multi-modal features under the condition of structural differences and working condition changes.

[0157] S6.4: Based on feature migration, use a feature fusion operator (such as weighted concatenation and cross-attention mechanism) to fuse each modality abnormal feature parameter, generating a multi-modal collaborative abnormal candidate feature vector set covering multiple working conditions, structure attributes, and spatial distribution, preparing for the input of the subsequent discriminant model.

[0158] S6.5: For the multi-modal collaborative abnormal candidate feature vector set, use feature robustness screening and redundancy suppression strategies (such as principal component analysis and correlation threshold filtering) to extract a set of multi-modal collaborative abnormal features with high discriminant ability as the final output of this step, for use by the next step of dynamic safety event discrimination rule learning module.

[0159] The step S7: the multi-modal collaborative anomaly feature set is included in the active learning process of the safety event discrimination model, and a dynamic self-adaptive optimization algorithm is used to update and calibrate the discrimination rules regularly, and dynamic safety event discrimination rules suitable for various structures, working conditions and scenes are automatically generated. Specifically, it includes: S7.1: sample screening is performed on the multi-modal collaborative anomaly feature set obtained through cross-modal feature migration and fusion processing, and based on sample distribution sparsity and representativeness evaluation, a multi-modal collaborative anomaly feature set subset with high information gain is selected as the input of the safety event discrimination model, to ensure the diversity and representativeness of the input of the discrimination model.

[0160] S7.2: Based on the active learning framework, the selected multi-modal collaborative anomaly feature set is injected into the safety event discrimination model as an invisible sample, and a query strategy (such as uncertainty sampling, density sampling, etc.) is used to dynamically select the data segment with the lowest discrimination confidence, to improve the discrimination ability of the active learning model for unseen anomalies.

[0161] S7.3: Use a dynamic self-adaptive optimization algorithm (such as a discrimination rule optimization method based on genetic algorithm, particle swarm algorithm or reinforcement learning) to globally adjust and restructure the existing discrimination rule parameter set of the safety event discrimination model, to achieve optimal adaptation of the discrimination rules to new structures and working conditions.

[0162] S7.4: According to the real-time feedback of the model discrimination performance after the introduction of new samples by active learning, continuously evaluate the accuracy, sensitivity and generalization ability of the discrimination rules on the multi-modal collaborative anomaly feature set, and dynamically adjust the search space and hyperparameters of the optimization algorithm according to the feedback, to improve the convergence speed and robustness of the rule self-adaptive calibration.

[0163] S7.5: Automatically record the multi-modal collaborative anomaly feature set sample utilization rate, rule parameter change trajectory and model performance improvement value during each round of discrimination rule optimization update process, and archive it as a discrimination rule evolution log, to provide data support for subsequent discrimination model tracing, stability evaluation and discrimination rule retraining.

[0164] The step S8: after updating the safety event discrimination rule in each round, based on the combination of simulation and actual equipment backtesting, the discrimination sensitivity and generalization performance of the dynamic safety event discrimination rule under different working conditions, structures and new anomaly modes are evaluated. Specifically, it includes: S8.1: Based on the dynamic safety event discrimination rule, select a representative multi-modal collaborative anomaly feature set as the evaluation input object, use the historical working condition label and structure parameter information to construct a multi-dependent variable controlled simulation environment, use the system simulation platform to simulate the multi-modal data flow under different working conditions and structure states, and generate a set of simulation discrimination samples.

[0165] S8.2: Simulate the discriminant sample set of S8.1, rely on the dynamic safety event discriminant rule, implement automatic batch discriminant reasoning, use robust sensitivity analysis algorithm, quantify the detection rate, missed detection rate and false alarm rate of the discriminant rule under the conditions of changing working conditions, structural parameters and new abnormal mode, and obtain the simulation sensitivity evaluation results.

[0166] S8.3: Based on the actual equipment operating environment, cooperatively call the archived multi-modal collaborative abnormal feature set, respectively apply physical simulation failure or parameter disturbance to the key subsystems of the equipment, collect backtest multi-modal abnormal data under corresponding time series working conditions and structural parameters, and generate backtest discriminant test sample set.

[0167] S8.4: Input the backtest discriminant test sample set of S8.3 into the dynamic safety event discriminant rule, use online discriminant and real-time label comparison analysis method, calculate the immediate response rate, generalization discriminant performance and actual misjudgment rate of the discriminant rule for new abnormality and extreme disturbance scene, and obtain backtest discriminant performance data.

[0168] S8.5: Fusion of S8.2 simulation sensitivity evaluation results and S8.4 backtest discriminant performance data, using multi-modal performance normalization index system, constructing comprehensive performance evaluation model, quantitatively scoring the comprehensive sensitivity and generalization performance of the current dynamic safety event discriminant rule, and outputting performance evaluation report, providing data support for subsequent adaptive optimization.

[0169] The step S9: If the prediction accuracy of the dynamic safety event discriminant rule driven by the multi-modal collaborative abnormal feature set under rare, extreme or new abnormal scene does not reach the set standard after evaluation, the abnormal sample generation strategy, feature migration parameter or discriminant model structure is automatically adjusted according to the evaluation feedback results until the performance requirements are met. Specifically, it includes: S9.1: Based on the confusion matrix, AUC and other performance evaluation indexes, the discriminant accuracy and generalization ability are quantitatively analyzed to obtain discriminant performance evaluation data under different abnormal scenes.

[0170] S9.2: Based on the above discriminant performance evaluation data, using confidence interval analysis and sensitivity detection means, the scenes where the discriminant accuracy does not reach the threshold value are located and classified under different working condition labels and structural parameter subsets, and the performance defect type attribution report is output.

[0171] S9.3: With the performance defect type attribution report, automatically call the pseudo-exception sample generation module, adjust the sample distribution control parameters of the generative adversarial network (GAN) based on the source difference exception sample distribution table, and output an enhanced exception sample set for the performance weak scene to make up for the coverage deficiency of the original exception sample.

[0172] S9.4: Input the enhanced exception sample set into the representation learning model, dynamically adjust the feature extraction and feature mapping hyperparameters of the representation learning model according to the structural parameters and working condition label dimensions fed back by the attribution report, generate a more targeted modal feature mapping relationship, and optimize the cross-modal feature transfer performance.

[0173] S9.5: Jointly input the newly generated enhanced exception sample set and the optimized modal feature mapping relationship into the discriminant model, based on the active learning mechanism, perform structural fine-tuning and parameter adaptive optimization on the existing discriminant model structure (such as feature interaction layer, regularization term or loss function, etc.), and form a new round of dynamic safety event discrimination rule set.

[0174] S9.6: Repeat the simulation deduction and actual device backtest process in S8 for the updated dynamic safety event discrimination rule set, and perform performance evaluation and closed-loop feedback in a loop until the discrimination accuracy and generalization performance meet the preset standards in all specified abnormal scenarios. Finally, the optimized dynamic safety event discrimination rule is solidified.

[0175] Thus, the technical solutions of the present application have been described in combination with the preferred embodiments shown in the accompanying drawings, but it is easily understood by those skilled in the art that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to related technical features without departing from the principles of the present application, and the technical solutions after these changes or replacements will fall within the protection scope of the present application.

[0176] The above description is only the preferred embodiments of the present application and is not intended to limit the present application; for those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A special equipment safety monitoring method based on Internet of Things, characterized in that, The method comprises the following steps: S1: Collecting multi-modal sensor raw data from multiple physical locations in the special equipment, and synchronously recording the environmental labels and structural parameter information of each sampling point to form a multi-modal data set; S2: Removing noise, standardizing data format, and completing working condition label for different sensor data in the multi-modal data set to obtain a multi-modal data sequence; S3: Based on the multi-modal data sequence, screening out suspected rare abnormal sample segments, and distributing and classifying the abnormal samples according to the collection location, environmental label and structural parameter to form a difference abnormal sample distribution table; S4: For each type of abnormality in the difference abnormal sample distribution table, input the corresponding original multi-modal data into the generative adversarial network to perform virtual abnormal sample generation operation, and expand the abnormal data set to form a pseudo abnormal sample set; S5: Inputting the multi-modal abnormal samples in the expanded abnormal data set into the representation learning model to extract key abnormal feature parameters, and establishing a feature mapping relationship of each modal abnormal representation; S6: Based on the established modal feature mapping relationship, performing cross-modal feature migration and fusion operation to form a multi-modal collaborative abnormal feature set; S7: The multi-modal collaborative abnormal feature set is included in the active learning process of the safety event discrimination model, and a dynamic self-adaptive optimization algorithm is used to update and calibrate the discrimination rules regularly to automatically generate safety event discrimination rules; S8: After updating each round of safety event discrimination rules, the dynamic safety event discrimination rules are evaluated in different working conditions, structures and new abnormal modes based on the combination of simulation deduction and actual equipment backtest. 2.The special equipment safety monitoring method based on the Internet of Things according to claim 1, characterized in that, The step S8 further comprises: S9: If the evaluation shows that the prediction accuracy of the dynamic safety event discrimination rules driven by the multi-modal collaborative abnormal feature set in rare, extreme or new abnormal scenarios does not meet the set standard, the abnormal sample generation strategy, feature migration parameter or discrimination model structure is automatically adjusted according to the evaluation feedback until the performance requirement is met. 3.The special equipment safety monitoring method based on the Internet of Things according to claim 1, characterized in that, The step S1 specifically comprises: Deploying multi-modal sensor devices on each key monitoring unit of the special equipment, including temperature sensors, vibration sensors and current sensors, to obtain a complete spatial distribution multi-modal sensor configuration table; Based on the multi-modal sensor configuration table, periodically performing real-time acquisition of original signals of temperature sensors, vibration sensors and current sensors, and recording the corresponding spatial position label at each sampling time to obtain a multi-modal sensor raw signal sequence with time and space labels; Combining the equipment factory configuration archives and the local embedded sensor module, synchronously collecting and archiving the sampling point environmental label and structural parameter information to form a structural parameter label and working condition environmental label data stream; Using a unified data modeling framework to perform multi-level data fusion on the multi-modal sensor raw signal sequence with time and space labels, the structural parameter label and the working condition environmental label data stream to generate a preliminary structured multi-modal acquisition data set; Based on the preliminary structured multi-modal acquisition dataset, a data consistency verification algorithm is applied to compare the time series coverage, label integrity and abnormal value distribution of each physical location and sensor modality, and output a high-quality multi-modal dataset that has passed the integrity and consistency verification.

4. The special equipment safety monitoring method based on the Internet of Things according to claim 3, characterized in that, The multi-modal sensor deployment adopts an industrial bus communication protocol to realize unified identification and configuration of sensor nodes in each physical location, and periodically detects the online state to generate a spatial distribution multi-modal sensor configuration table.

5. The special equipment safety monitoring method based on the Internet of Things according to claim 1, characterized in that, The step S2 specifically comprises: The original data of the temperature sensor in the multi-modal dataset is processed by a median filter algorithm and an adaptive threshold filter algorithm to remove noise and obtain a denoised temperature time series signal; Based on the denoised temperature time series signal, independent noise removal processing is performed on the original data of the vibration sensor and the current sensor to output a multi-modal denoised signal stream; The multi-modal denoised signal stream is taken as input, and a data format standardization module is used to perform sampling rate resampling, dimension unification and data structure alignment on the temperature, vibration and current three modalities respectively to obtain a multi-modal standardized data stream that is time-synchronized and format-consistent; For the time-synchronized and format-consistent multi-modal standardized data stream, a label completion algorithm based on correlation analysis is used to automatically label the data without labeled or missing working condition labels using environmental labels and structural parameter information; The multi-modal data sequence with complete coverage labeling is input into a data integrity verification module, and sequence missing detection and redundant anomaly detection algorithms are used to comprehensively check the continuity, consistency and label accuracy of the data, and output a standardized multi-modal data sequence that meets high consistency requirements.

6. The special equipment safety monitoring method based on the Internet of Things according to claim 5, characterized in that, The independent noise removal processing specifically includes using a short-time Fourier transform and an autoregressive sliding mean method to remove independent noise from the original data of the vibration sensor and the current sensor.

7. The special equipment safety monitoring method based on the Internet of Things according to claim 1, characterized in that, The step S3 specifically comprises: The input standardized multi-modal data sequence is windowed and divided based on a time series segmentation algorithm to obtain structured time period multi-modal data segments; An unsupervised clustering algorithm is used to cluster and divide the high-dimensional features of the structured time period multi-modal data segments to discover the inherent working condition and structural state distribution clusters of the multi-modal data, and obtain multi-modal clustering labels for each time period; Based on the multi-modal clustering labels for each time period, combined with an anomaly detection algorithm, time period multi-modal data segments that do not belong to the dominant class and are significantly different from the normal distribution are determined and selected, and suspected rare abnormal sample groups are preliminarily locked; The suspected rare abnormal sample groups are classified and aggregated using acquisition location metadata, environmental labels and structural parameter metadata to obtain a multi-modal abnormal sample distribution subset with spatial layering, environmental label layering and structural type layering; The multi-modal abnormal sample distribution subset is classified according to the classification type, the distribution characteristics and quantity of abnormal samples in each layer are counted, and the abnormal occurrence conditions are encoded to finally form a difference abnormal sample distribution table with acquisition location, environmental label and structural parameter annotations. 8.The special equipment safety monitoring method based on the Internet of Things according to claim 1, wherein, The step S4 specifically comprises: For each type of abnormality in the differential abnormal sample distribution table, the associated original multi-modal data is extracted, and the working condition label, spatial position label and structural parameter information are identified to obtain a structured multi-modal abnormal data set; Based on the structured multi-modal abnormal data set, a conditional adversarial generation network is used to perform feature embedding and latent variable modeling on the input multi-modal abnormal sample; Using the multi-modal abnormal distribution parameters modeled by the latent feature variables, conditional sampling is performed and the multi-modal abnormal data is virtually synthesized through the generator of the adversarial generation network, to output pseudo abnormal samples with diversified distribution in working condition, spatial position and structural attributes; The generated multi-modal pseudo abnormal samples are subjected to authenticity discrimination and quality scoring by the discriminator, and through feature consistency comparison with the real abnormal sample distribution, high-quality pseudo abnormal samples with high discrimination scores and reasonable attributes are selected to form an expanded abnormal data set; The expanded abnormal data set and the structured multi-modal abnormal data set are archived and integrated to generate a final pseudo abnormal sample set with multi-dimensional attribute annotations such as working condition, spatial structure and environment label. 9.The special equipment safety monitoring method based on the Internet of Things according to claim 8, characterized in that, The structured multi-modal abnormal data set obtained in step S4 is specifically: For each type of abnormal sample, the temperature, vibration and current three modal original signals are assembled according to a unified time window and label by using multi-label mapping and feature normalization, and then standardized and packaged to integrate into a structured multi-modal abnormal data set. 10.The special equipment safety monitoring method based on the Internet of Things according to claim 1, characterized in that, In the evaluation stage of step S8, simulation and on-site physical backtesting are combined to comprehensively evaluate the generalization and sensitivity performance of the current discrimination rule according to the detection rate, missed detection rate and false alarm rate of the multi-modal feature set under different working conditions and structures.

Images