Industrial Internet of Things equipment operation state anomaly detection system and method oriented to dynamic time sequence data

Abstract

The invention relates to the technical field of Internet of Things equipment detection, in particular to an industrial Internet of Things equipment operation state anomaly detection system and method oriented to dynamic time sequence data. Comprising a data acquisition unit; a time series data preprocessing unit; an intelligent anomaly detection unit; and an abnormal result output and feedback unit. Synchronous time sequence stamps of the same reference clock source are added to multi-dimensional dynamic time sequence operation data and equipment key component images, and it is guaranteed that initial association of the two types of data is consistent; an optimal time corresponding relation is constructed through a dynamic time warping algorithm, nonlinear time correction and frequency adaptation are completed, and the problem of time asynchronization is solved; through a multi-modal feature drift correction module, a working condition adaptive drift correction factor is generated based on scale invariant feature transformation feature point matching, feature offset and time series data non-fault fluctuation are corrected, feature stability is improved, and the same operation node state of equipment is truly reflected.

Description

Technical Field

[0001] This invention relates to the field of IoT device detection technology, and more specifically, to an industrial IoT device anomaly detection system and method for dynamic time-series data. Background Technology

[0002] The development of Industrial Internet of Things (IIoT) technology has propelled industrial equipment monitoring into a data-driven phase, with multi-dimensional dynamic time-series operational data and images of key equipment components becoming core information for assessing equipment health. Due to the complexity of industrial equipment operating conditions, latent faults are easily masked by noise and fluctuations in operating conditions. Therefore, anomaly detection technology based on multi-modal data collaborative analysis has become crucial for ensuring stable system operation.

[0003] In the existing technology, relevant patents have explored the field of industrial IoT equipment monitoring. For example, invention patent CN202510976241.8 discloses a device monitoring method, system, device, and medium based on AI and industrial IoT. By acquiring key monitoring datasets, querying and parsing parameters and process tags, and calling analysis processes to obtain monitoring results, it achieves accurate monitoring of equipment operating status and improves monitoring efficiency and reliability. Another example is invention patent CN202411242105.8, which discloses an industrial IoT equipment monitoring method and system based on data analysis. Addressing the problems of single data processing and low monitoring accuracy, it collects data from multiple devices and fuses and processes them through a deep learning model to achieve anomaly detection and predictive maintenance, improving system performance and reducing energy consumption.

[0004] Despite the design advantages of the above technical solutions, they also have the following technical defects: First, the time coordination of multi-source data is insufficient and the feature stability is lacking. The data analysis of invention patent CN202510976241.8 only focuses on monitoring key datasets. Its solution design does not consider the complementary value of visual features and time-series parameters of equipment operation status, and lacks the logic for the correlation processing of the two types of data, resulting in the inability to comprehensively depict the real status of the same operating node of the equipment through multi-dimensional information. Although invention patent CN202411242105.8 mentions the fusion of multiple types of data, it does not design a time coordination solution for the objective problems of data acquisition frequency differences and transmission delays in industrial scenarios, and does not consider the impact of operating condition fluctuations on feature stability. Its fusion processing does not include the relevant design for feature offset correction, which makes the state representation after data fusion deviate from the actual operating status of the equipment, making it difficult to support accurate anomaly identification. Secondly, the feature verification logic is incomplete and the sample benchmark lacks dynamic adaptability: Invention patent CN202510976241.8 completes data parsing through a preset rule table and process tags. Its verification relies solely on preset rules and does not incorporate deep constraint logic based on the physical relationships between equipment components, power transmission, and other operating mechanisms. This may lead to feature matching results that contradict the actual operating rules of the equipment. Invention patent CN202411242105.8 has a relatively simple feature verification system, lacking multi-dimensional verification logic. Furthermore, its feature comparison benchmark relies on an initial sample library and does not design a dynamic update mechanism based on actual operating data. This makes it unable to adapt to the requirements of dynamic changes in operating conditions on the feature comparison benchmark, resulting in unreliable reliability of the system's anomaly identification across all operating conditions. In view of this, we propose an industrial IoT equipment operating status anomaly detection system and method oriented towards dynamic time-series data. Summary of the Invention

[0005] The purpose of this invention is to provide an industrial IoT device operation status anomaly detection system and method for dynamic time-series data, so as to solve the problems mentioned in the background art, such as insufficient time coordination of multi-source data, lack of feature stability, incomplete feature verification logic, and lack of dynamic adaptability of sample benchmarks.

[0006] To address the aforementioned technical problems, one objective of this invention is to provide an industrial IoT device operational status anomaly detection system for dynamic time-series data, comprising: The data acquisition unit collects multi-dimensional dynamic time-series operation data and real-time images of key components of industrial IoT devices in real time. It adds synchronization time stamps to the multi-dimensional dynamic time-series operation data and real-time images of key components of devices, performs format standardization processing on the collected multi-dimensional dynamic time-series operation data and real-time images of key components of devices, and then transmits them to the time-series data preprocessing unit. The time series data preprocessing unit performs noise filtering, missing value filling and time series feature enhancement processing on the received multi-dimensional dynamic time series running data, extracts the time correlation features in the multi-dimensional dynamic time series running data, and transmits the preprocessed multi-dimensional dynamic time series running data to the intelligent anomaly detection unit. The intelligent anomaly detection unit, based on preprocessed multi-dimensional dynamic time-series operating data and real-time images acquired by key equipment components, uses a dynamic time warping algorithm to construct the optimal time correspondence between the image frames of the multi-dimensional dynamic time-series operating data and the real-time images acquired by key equipment components, completing nonlinear time correction and frequency adaptation alignment; it calls a preset full-condition normal time-series-image feature sample library to generate an adaptive drift correction factor, corrects the image feature offset of the real-time images acquired by key equipment components through scale-invariant feature transformation feature point matching, and corrects the non-fault fluctuations of the multi-dimensional dynamic time-series operating data in conjunction with the operating condition benchmark; based on feature space mapping and mutual information verification, it adds local feature focusing verification and global correlation constraints to ensure the consistency of multi-modal feature alignment, completes the identification of abnormal operating status of industrial IoT equipment, and transmits the anomaly detection results to the anomaly result output and feedback unit; The abnormal result output and feedback unit pushes the abnormal detection results and the corresponding aligned and corrected time sequence nodes, real-time acquired images of key equipment components, and feature verification details to the industrial IoT monitoring platform and related operation and maintenance terminals. It records the abnormal correlation data and transmits the abnormal correlation data to the intelligent abnormal detection unit to update the full-condition normal time sequence-image feature sample library of the intelligent abnormal detection unit.

[0007] As a further improvement to this technical solution, the data acquisition unit includes a data image acquisition module, a synchronization time stamp marking module, and a format standardization processing module, wherein: The data and image acquisition module collects multi-dimensional dynamic time-series operating data of industrial IoT devices and real-time images of key components of the devices in real time. The synchronization timing stamp marking module adds synchronization timing stamp markings from the same reference clock source to the multi-dimensional dynamic timing operation data of the industrial IoT device and the real-time images of key components of the device, which are collected by the data image acquisition module. The format standardization processing module performs format standardization processing on the multi-dimensional dynamic time-series operation data and real-time images of key equipment components marked by the synchronous time-series stamp marking module, and then transmits them to the time-series data preprocessing unit.

[0008] As a further improvement to this technical solution, the time-series data preprocessing unit includes a noise filtering and imputation module and a time-series feature enhancement module, wherein: The noise filtering and missing value filling module performs noise filtering and missing value filling processing based on the received multi-dimensional dynamic time-series running data using sliding window technology. The time-series feature enhancement module uses wavelet transform technology to perform time-series feature enhancement processing on the multi-dimensional dynamic time-series running data processed by the noise filtering and filling module, extracts the time correlation features in the multi-dimensional dynamic time-series running data, and transmits the preprocessed multi-dimensional dynamic time-series running data to the intelligent anomaly detection unit.

[0009] As a further improvement to this technical solution, the intelligent anomaly detection unit includes a time-series-image alignment correction module, a multimodal feature drift correction module, a feature verification constraint module, and an anomaly recognition output module, wherein: The time-image alignment and correction module is based on preprocessed multi-dimensional dynamic time-series running data and real-time images acquired by key components of the equipment. It uses a dynamic time warping algorithm to construct the optimal time correspondence between the multi-dimensional dynamic time-series running data and the image frames of the real-time images acquired by key components of the equipment, and completes nonlinear time correction and frequency adaptation alignment. The multimodal feature drift correction module, based on the processing results of the time-image alignment correction module, calls the preset full-condition normal time-image feature sample library to generate an operating condition adaptive drift correction factor. It corrects the image feature offset of real-time acquired images of key components of the equipment by scale-invariant feature transformation feature point matching, and combines the operating condition benchmark to correct the non-fault fluctuations of multi-dimensional dynamic time-series operating data. The feature verification constraint module, based on the processing results of the multimodal feature drift correction module, adds local feature focusing verification and global correlation constraints on the basis of feature space mapping and mutual information verification. The anomaly identification output module, based on the processing results of the feature verification constraint module, completes the anomaly identification of the operating status of industrial IoT devices and transmits the anomaly detection results to the anomaly result output and feedback unit.

[0010] As a further improvement to this technical solution, the time-image alignment correction module uses a dynamic time warping algorithm to construct the optimal time correspondence and complete the nonlinear time correction and frequency adaptation alignment process, which includes the following steps: S31.1 Calculation of multi-dimensional dynamic time-series running data The data points and real-time images of key components of the equipment are collected in the first... The cumulative distance is calculated recursively based on the feature distance of the frame image. Construct a cumulative distance matrix covering all data points and image frames, through... Achieve dynamic time warping nonlinear matching adaptation; S31.2, Traverse the cumulative distance matrix and select the cumulative distance. Find the path with the smallest sum and accumulate the distance. The path with the minimum sum is determined as the optimal time correspondence between multi-dimensional dynamic time-series running data and image frames of real-time acquired images of key components of the equipment; S31.3 Based on the optimal time correspondence, nonlinear time correction is performed on the multi-dimensional dynamic time-series operation data and the real-time acquired images of key components of the equipment to synchronously complete frequency adaptation and alignment.

[0011] As a further improvement to this technical solution, the multimodal feature drift correction module includes a feature point extraction submodule, a feature matching submodule, and a drift correction submodule, wherein: The feature point extraction submodule performs Gaussian pyramid construction and scale-invariant feature transformation on real-time images of key components of the equipment to extract feature points, thereby obtaining the coordinates of feature points and corresponding feature vectors of the images. The feature matching submodule retrieves the standard feature point vector set of key components of the equipment under the corresponding working conditions from the full-condition normal time-series-image feature sample library, calculates the matching degree between the real-time feature vector and the standard feature vector using the nearest neighbor distance ratio method, and filters out effective feature point pairs with a matching degree higher than a preset threshold. The drift correction submodule generates an adaptive drift correction factor based on the coordinate offset of effective feature point pairs. The adaptive drift correction factor corrects the image feature offset of real-time acquired images of key components of the equipment. At the same time, it corrects the non-fault fluctuations of multi-dimensional dynamic time-series operating data by combining the operating condition benchmark.

[0012] As a further improvement to this technical solution, the process of the feature verification constraint module performing local feature focusing verification and global correlation constraint includes the following steps: S33.1. Based on the physical structure of key equipment components and the physical attributes of multi-dimensional dynamic time-series operational data, the multimodal features are divided into several groups of local features. Each group of local features corresponds to a key equipment component and its associated time-series data dimension. The matching degree is determined using cosine similarity. Quantify the matching accuracy of each set of local features; S33.2 Based on the operating mechanism of industrial IoT devices, a global correlation matrix between local features is constructed. The matrix elements are the physical correlation weights between different local features. These physical correlation weights are set based on the power transmission and signal transmission relationships between device components. S33.3, Combining cosine similarity matching degree The results verify whether each set of local feature pairs conforms to the weight constraints of the global association matrix, and eliminate false matching results that violate the device operation logic.

[0013] As a further improvement to this technical solution, the anomaly identification output module includes a feature deviation calculation submodule, an anomaly state determination submodule, and a result integration and transmission submodule, wherein: The feature deviation calculation submodule is based on the local feature matching results output by the feature verification constraint module, combined with the cumulative distance. Matching degree with cosine similarity The core data is used to calculate the overall deviation between real-time multimodal features and standard features in the normal time-series-image feature sample library under full working conditions; The abnormal state determination submodule compares the overall deviation value with the preset abnormal determination threshold. When the overall deviation value is higher than the abnormal determination threshold, the industrial IoT device is determined to be in an abnormal operating state. The result integration and transmission submodule integrates the anomaly detection results, which include anomaly judgment results, corresponding alignment and correction timing nodes, real-time acquired images of key components of the equipment, and feature verification details. The anomaly detection results are then transmitted to the anomaly result output and feedback unit.

[0014] As a further improvement to this technical solution, the abnormal result output and feedback unit includes an abnormal result push module, an abnormal data recording module, and a sample library update and transmission module, wherein: The abnormal result push module pushes the abnormal detection results transmitted by the intelligent abnormal detection unit to the industrial IoT monitoring platform and related operation and maintenance terminals. The abnormal data recording module records the abnormal detection results and the corresponding abnormal associated data. The abnormal associated data includes the abnormal judgment results, the alignment and correction time sequence nodes, and the real-time acquired images and feature verification details of the key components of the equipment. The sample library update and transmission module transmits the abnormal correlation data recorded by the abnormal data recording module to the intelligent anomaly detection unit to update the full-condition normal time-series-image feature sample library of the intelligent anomaly detection unit.

[0015] The second objective of this invention is to provide a method for detecting abnormal operating states of industrial IoT devices based on dynamic time-series data. The method, based on the aforementioned system for detecting abnormal operating states of industrial IoT devices based on dynamic time-series data, includes the following steps: Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention ensures the initial consistency of the two types of data by adding synchronous time stamps from the same reference clock source to multi-dimensional dynamic time-series operational data and real-time acquired images of key equipment components. Then, a dynamic time warping algorithm is used to construct the optimal time correspondence between the two types of data, completing nonlinear time correction and frequency adaptation alignment, thus solving the time asynchrony problem caused by differences in data acquisition frequency and transmission delay. Simultaneously, based on scale-invariant feature transformation feature point matching, an adaptive drift correction factor is generated to correct image feature offsets and, combined with the operating condition reference, corrects the non-fault fluctuations of the time-series data, improving feature stability and enabling multi-source data to more realistically reflect the actual state of the same operating node of the equipment. 2. This invention decomposes local features according to the physical structure and data physical attributes of key equipment components, quantifies matching accuracy using cosine similarity, and then constructs a global correlation matrix based on the equipment's operating mechanism for weight constraints, eliminating false matching results that violate the equipment's operating logic and improving the feature verification logic. At the same time, by recording abnormal correlation data and continuously updating the full-condition normal time-series-image feature sample library, the feature comparison benchmark can adapt to dynamic changes in operating conditions, improving the reliability and long-term adaptability of the system for anomaly identification in full-condition scenarios. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the system framework of the present invention; Figure 2 This is a schematic diagram of the macroscopic technical link of the intelligent anomaly detection unit in this invention; Figure 3 This is a schematic diagram of the method steps of the present invention; The meanings of the labels in the diagram are as follows: 1. Data acquisition unit; 11. Data image acquisition module; 12. Synchronization timing stamp module; 13. Format standardization processing module; 2. Time series data preprocessing unit; 21. Noise filtering and imputation module; 22. Time series feature enhancement module; 3. Intelligent anomaly detection unit; 31. Time-series-image alignment correction module; 32. Multimodal feature drift correction module; 33. Feature verification constraint module; 34. Anomaly recognition output module; 4. Abnormal result output and feedback unit; 41. Abnormal result push module; 42. Abnormal data recording module; 43. Sample library update and transmission module. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figures 1-2 As shown, this embodiment provides an industrial IoT device operating status anomaly detection system for dynamic time-series data, including: Data acquisition unit 1 collects multi-dimensional dynamic time-series operation data and real-time images of key components of industrial IoT devices in real time. It adds synchronization time stamps to the multi-dimensional dynamic time-series operation data and real-time images of key components of devices. After performing format standardization processing on the collected multi-dimensional dynamic time-series operation data and real-time images of key components of devices, it transmits them to time-series data preprocessing unit 2. In this embodiment, the data acquisition unit 1 includes a data image acquisition module 11, a synchronization timing stamp marking module 12, and a format standardization processing module 13, wherein: The data and image acquisition module 11 collects multi-dimensional dynamic time-series operating data of industrial IoT devices and real-time images of key components of the devices in real time. Specifically, the multi-dimensional dynamic time-series operation data includes core operating parameters such as temperature, pressure, speed, current, vibration amplitude, and flow rate during the operation of industrial IoT equipment. The data image acquisition module 11 collects the above time-series parameters through industrial-grade sensor groups (such as PT100 temperature sensor, piezoelectric vibration sensor, Hall current sensor, capacitive pressure sensor, etc.) deployed at the corresponding monitoring positions of the equipment. Real-time image acquisition of key components of the equipment: For the core moving parts of the equipment (such as bearings, gearboxes, motor shafts, transmission mechanisms, etc.), real-time image acquisition is performed through high-definition industrial cameras (resolution not less than 1920×1080 pixels, supporting continuous shooting mode) deployed on the front or side of the components.

[0019] Meanwhile, during the acquisition process, the sensor group and the industrial camera's acquisition trigger logic work together, and the acquisition frequency is set according to the equipment's operating speed and operating conditions: the acquisition frequency of multi-dimensional dynamic time-series operating data is 10-100Hz, and the acquisition frequency of images of key equipment components is 1-10fps, ensuring that the dynamic changes in the equipment's operating status can be fully captured.

[0020] The synchronization timing stamp marking module 12 adds synchronization timing stamp markings from the same reference clock source to the multi-dimensional dynamic timing operation data of the industrial IoT device and the real-time acquired images of key components of the device, based on the data image acquisition module 11. Specifically, the synchronization time stamp module 12 incorporates a GPS timing module or an industrial Ethernet PTP (Precision Time Protocol) synchronization module to acquire a high-precision unified reference clock signal (time accuracy down to the microsecond level). When the data image acquisition module 11 completes the acquisition of a single set of multi-dimensional dynamic time-series running data or a single frame of key component image, the synchronization time stamp module 12 immediately captures the trigger time of the acquisition action and generates a unique time stamp identifier containing "device ID - component number - UTC time - millisecond-level timestamp," which is embedded into the header field of the corresponding time-series data and the metadata of the image file. Through the time synchronization mechanism of the same reference clock source, it ensures that the timestamps of each set of time-series data and each key component image have a strict correlation, providing a foundation for the time alignment of subsequent multi-modal data.

[0021] The format standardization processing module 13 performs format standardization processing on the multi-dimensional dynamic time-series operation data and real-time acquired images of key equipment components marked by the synchronization time stamp marking module 12, and then transmits them to the time-series data preprocessing unit 2.

[0022] Specifically, the format standardization processing module 13 establishes unified format specifications for two types of data: For multi-dimensional dynamic time-series operational data, heterogeneous data output from different sensors (such as voltage signals, current signals, analog signals, etc.) are converted into digital data and encapsulated in JSON format. Fields include "device ID, component number, time stamp, parameter type, parameter value, and data validity identifier," where the data validity identifier is used to mark whether there are any acquisition anomalies in the data set (such as invalid values ​​caused by sensor offline). For real-time acquired images of key components of the equipment, they are uniformly converted into JPEG or PNG standard image formats. The image file naming follows the specification of "device ID-component number-time stamp." format. At the same time, the camera parameters (such as exposure time, focal length, and ISO) at the time of acquisition are supplemented in the image metadata.

[0023] After the format conversion is completed, the format standardization processing module 13 performs integrity verification on the data (verifies the integrity of the time stamp, missing data fields, and image file corruption status). The standardized data that passes the verification is transmitted to the time series data preprocessing unit 2 via industrial Ethernet or MQTT IoT communication protocol, ensuring that the subsequent preprocessing unit can directly perform data processing based on the unified format, avoiding low processing efficiency or errors caused by heterogeneous data formats.

[0024] The time series data preprocessing unit 2 performs noise filtering, missing value filling and time series feature enhancement processing on the received multi-dimensional dynamic time series running data, extracts the time correlation features in the multi-dimensional dynamic time series running data, and transmits the preprocessed multi-dimensional dynamic time series running data to the intelligent anomaly detection unit 3. In this embodiment, the time-series data preprocessing unit 2 includes a noise filtering and imputation module 21 and a time-series feature enhancement module 22, wherein: The noise filtering and missing value filling module 21 performs noise filtering and missing value filling processing based on the received multi-dimensional dynamic time-series running data using sliding window technology. Specifically, the size of the sliding window is adaptively set based on the acquisition frequency of the multi-dimensional dynamic time-series running data. The window length is 5-20 data points (20 data points when the acquisition frequency is 10Hz, and 5 data points when the acquisition frequency is 100Hz), ensuring that the window can cover the local fluctuation features of the data while avoiding feature lag caused by an excessively large window.

[0025] Meanwhile, in the noise filtering process, a sliding window median filtering algorithm is adopted: the window slides point by point on the time series data sequence, and the median of all data points in each window is used as the filtered value of the center data point of the current window, which effectively removes the pulse noise and random noise generated by electromagnetic interference and mechanical vibration of the sensor in the industrial environment.

[0026] In addition, for missing value imputation, the missing data segments are first screened out by data validity indicators. If it is a single missing point (≤1 consecutive invalid data indicators), the imputation value is calculated by linear interpolation of two adjacent valid data points. If it is a continuous missing point (>1 consecutive invalid data indicators), the segment with the highest feature matching degree with the valid data before and after the missing segment in the historical time series data of the same equipment and the same operating condition is retrieved as the imputation reference. The missing value is imputed by segmented fitting to ensure that the imputed data conforms to the time series pattern of equipment operation.

[0027] The time-series feature enhancement module 22 uses wavelet transform technology to perform time-series feature enhancement processing on the multi-dimensional dynamic time-series running data processed by the noise filtering and filling module 21, extracts the time correlation features in the multi-dimensional dynamic time-series running data, and transmits the preprocessed multi-dimensional dynamic time-series running data to the intelligent anomaly detection unit 3.

[0028] Specifically, in the process of enhancing time series features, the db4 wavelet basis is selected as the mother wavelet of the wavelet transform. Based on the length of the time series data (the length of a single segment of processed data is 1024-4096 data points), the number of wavelet decomposition layers is set to 3-5 layers. Multi-scale wavelet decomposition is performed on the denoised and filled time series data to obtain one low-frequency approximate component (reflecting the core trend features of the data) and several high-frequency detail components (including residual noise and local mutation features).

[0029] Meanwhile, a soft thresholding method is used for the high-frequency detail components. The threshold is set to 1.5 times the standard deviation of the component. Detail coefficients with absolute values ​​less than the threshold are set to zero, while detail coefficients with absolute values ​​greater than the threshold are corrected according to the rule of "coefficient value - threshold" to suppress residual noise and retain effective local features. Subsequently, wavelet inverse transform is performed based on the processed low-frequency approximation components and high-frequency detail components to reconstruct the signal with enhanced time-series features.

[0030] In addition, in the time-related feature extraction stage, for the reconstructed single-dimensional time-series data, statistical features such as mean, variance, peak factor, and kurtosis within the sliding window are calculated, as well as the first-order and second-order difference features of adjacent data points, to characterize the local time evolution of the data; for multi-dimensional time-series data (such as temperature and pressure, vibration and rotational speed), the cross-correlation coefficients and lag correlation coefficients between data sequences of different dimensions are calculated to mine the time-coordinated correlation features between various operating parameters, and finally form a comprehensive feature set containing single-dimensional time-series statistical features and multi-dimensional time-related features, providing accurate feature input for subsequent intelligent anomaly detection.

[0031] The intelligent anomaly detection unit 3, based on preprocessed multi-dimensional dynamic time-series operating data and real-time acquired images of key equipment components, uses a dynamic time warping algorithm to construct the optimal time correspondence between the image frames of the multi-dimensional dynamic time-series operating data and the real-time acquired images of key equipment components, completing nonlinear time correction and frequency adaptation alignment; it calls a preset full-condition normal time-series-image feature sample library to generate an adaptive drift correction factor, corrects the image feature offset of the real-time acquired images of key equipment components through scale-invariant feature transformation feature point matching, and combines the operating condition benchmark to correct the non-fault fluctuations of the multi-dimensional dynamic time-series operating data; based on feature space mapping and mutual information verification, it adds local feature focusing verification and global correlation constraints to ensure the consistency of multi-modal feature alignment, completes the identification of abnormal operating status of industrial IoT equipment, and transmits the anomaly detection results to the anomaly result output and feedback unit 4; the intelligent anomaly detection unit 3 includes a time-series-image alignment correction module 31, a multi-modal feature drift correction module 32, a feature verification constraint module 33, and an anomaly identification output module 34, wherein: In this embodiment, the time-image alignment correction module 31, based on preprocessed multi-dimensional dynamic time-series operation data and real-time images acquired by key components of the equipment, uses a dynamic time warping algorithm to construct the optimal time correspondence between the image frames of the multi-dimensional dynamic time-series operation data and the real-time images acquired by key components of the equipment, thereby completing nonlinear time correction and frequency adaptation alignment. The process by which the time-image alignment correction module 31 uses the dynamic time warping algorithm to construct the optimal time correspondence and complete nonlinear time correction and frequency adaptation alignment includes the following steps: S31.1 Calculation of multi-dimensional dynamic time-series running data The data points and real-time images of key components of the equipment are collected in the first... The cumulative distance is calculated recursively based on the feature distance of the frame image. Construct a cumulative distance matrix covering all data points and image frames, through... Achieve dynamic time warping nonlinear matching adaptation; Specifically, the purpose of setting up step S31.1 is to construct a cumulative distance matrix to achieve non-linear matching and adaptation between multi-dimensional dynamic time-series operational data and images of key equipment components. The specific implementation is as follows: First, clearly define the feature vectors involved in the matching (to ensure the consistency of feature dimensions in multimodal data): Let the total number of data points in the preprocessed multi-dimensional dynamic time-series running data be... , No. The comprehensive feature vector of each data point is , ;in The feature dimension consists of "single-dimensional statistical features (mean, variance) + multi-dimensional time correlation features (cross-correlation coefficients)" extracted by the time series data preprocessing unit 2. The specific extraction steps are as follows: the statistical feature calculation window calculates the mean and variance with a time window of 1 second (corresponding to 10 time series data points); the cross-correlation coefficient dimension combination selects "temperature-vibration amplitude" and "speed-load" as mandatory dimension groups to ensure that the features can characterize the dynamic changes in the operating status of the equipment.

[0032] Let the total number of frames of images acquired in real time for the key components of the equipment be... , No. The feature vector of the frame image is , ;in The feature dimension is consistent with the feature dimension of time series data, and the value selection rule is: basic value =5 (corresponding to the five basic statistical features of an image: grayscale mean, edge density, texture entropy, contrast, and gradient magnitude), for devices with complex structures that require finer-grained monitoring. It can be expanded to 8 (adding image-specific features, such as local binary pattern features, directional gradient histogram components, etc.), ensuring that feature distances are computable through dimensional unification, while ensuring that image features can characterize changes in the visual state of components.

[0033] Then, calculate the feature distance. : Feature distance Used to quantify the The time series data point and the first The degree of difference between frame images in the feature space is calculated using Euclidean distance, as shown in the formula: ; in: Represents the feature vector of time series data The Dimensional components; Represents image feature vectors The Dimensional components.

[0034] Next, the cumulative distance is calculated recursively. : Cumulative distance From the first time series data point, the first frame image to the... The time series data point, the first The cumulative feature distance of the frame image is calculated recursively to achieve non-linear time matching, as shown in the formula: ; Setting initial conditions ensures the rationality of recursion: when and hour: (The cumulative distance between the first time-series data point and the first frame image is equal to its feature distance); when and hour: (Distance is accumulated only along the image frame dimension); when and hour: (Distance is accumulated only along the time-series data dimension).

[0035] Finally, construct the cumulative distance matrix: Based on the above calculations, the generated dimension is: Cumulative distance matrix Each element in the matrix corresponds to a set of time-series data points and the cumulative distance between them and the image frame.

[0036] Understandably, traditional dynamic time warping is only used for matching single-type time series data. This step breaks through the type barrier of multi-source data by "unifying multimodal feature dimensions + recursive cumulative distance calculation", allowing one time series data point to correspond to multiple image frames (or vice versa), naturally adapting to the difference in acquisition frequency between the two types of data (such as 10Hz for time series data and 2fps for images), laying the foundation for time synchronization for subsequent multimodal feature fusion.

[0037] S31.2, Traverse the cumulative distance matrix and select the cumulative distance. Find the path with the smallest sum and accumulate the distance. The path with the minimum sum is determined as the optimal time correspondence between multi-dimensional dynamic time-series running data and image frames of real-time acquired images of key components of the equipment; Specifically, the purpose of setting step S31.2 is to select the optimal time correspondence to ensure that the multimodal data represents the state of the same operating node of the device. The specific implementation is as follows: Traversing the cumulative distance matrix The path with the smallest sum of cumulative distances is selected. The specific process is as follows: starting from the bottom right corner of the cumulative distance matrix (i.e., (Position) Traverse in reverse to the top left corner (i.e.) (Position), each step according to The calculation logic selects the direction with the smallest accumulated distance in the previous step (i.e., the corresponding direction). (direction), record all during the traversal Coordinate pairs, the paths formed by these coordinate pairs represent the optimal time correspondence between multi-dimensional dynamic time-series operational data and real-time acquired images of key components of the equipment.

[0038] It is understandable that the cumulative distance matrix... It is the total cumulative distance of all data. Reverse traversal can ensure that the path back from the final data point / image frame to the starting point is the path with the minimum cumulative feature distance. The time relationship corresponding to this path can maximize the guarantee that the two types of data represent the state of the same running node of the device.

[0039] S31.3 Based on the optimal time correspondence, nonlinear time correction is performed on the multi-dimensional dynamic time-series operation data and the real-time acquired images of key components of the equipment to synchronously complete frequency adaptation and alignment.

[0040] Specifically, the purpose of setting up step S31.3 is to complete the nonlinear time correction and frequency adaptation alignment of multimodal data, ensuring the temporal consistency of subsequent feature processing. The specific implementation is as follows: Based on the optimal time correspondence, the two types of data are synchronized. The specific operations are as follows: Nonlinear time correction: If there is one time series data point in the optimal path Corresponding to multiple image frames Then the synchronization time stamp of the time series data point is mapped to this... The timestamp interval of each image frame; If there is one image frame in the optimal path Corresponding to multiple time series data points Then the synchronization time stamp of the image frame is mapped to this The timestamp interval of each time series data point.

[0041] Frequency adaptation alignment: Adjust the "effective frequency" of the two types of data according to the corresponding quantity of data in the optimal path. If the time series data acquisition frequency is higher than the image acquisition frequency, the features of multiple time series data points corresponding to the same image frame are aggregated into a group of "equivalent time series features" (such as taking the feature mean). If the image acquisition frequency is higher than the time-series data acquisition frequency, the features of multiple image frames corresponding to the same time-series data point are aggregated into a group of "equivalent image features" (such as taking the feature mean).

[0042] After completing the above steps, the time-image alignment correction module 31 transmits the aligned multi-dimensional dynamic time-series operation data and real-time images of key components of the equipment to the multi-modal feature drift correction module 32 to continue the correction processing of feature offset and non-fault fluctuations.

[0043] Understandably, traditional time alignment often uses linear interpolation, which cannot adapt to the nonlinear time deviations caused by transmission delays and acquisition fluctuations in industrial scenarios. This step achieves time correction and frequency adaptation of multimodal data through "optimal path matching + feature aggregation", ensuring the time consistency of subsequent feature processing.

[0044] In this embodiment, the multimodal feature drift correction module 32, based on the processing results of the time-series-image alignment correction module 31, calls a preset full-condition normal time-series-image feature sample library to generate an adaptive drift correction factor. It corrects the image feature shift of real-time acquired images of key equipment components through scale-invariant feature transformation feature point matching, and combines this with the operating condition benchmark to correct non-fault fluctuations in multi-dimensional dynamic time-series operating data. The multimodal feature drift correction module 32 includes a feature point extraction submodule, a feature matching submodule, and a drift correction submodule, wherein: The feature point extraction submodule performs Gaussian pyramid construction and scale-invariant feature transformation on real-time images of key components of the equipment to extract feature points, thereby obtaining the coordinates of feature points and their corresponding feature vectors. Specifically, the feature point extraction submodule is designed to obtain stable feature points and corresponding feature vectors from images of key components of the device, providing a foundation for subsequent feature matching. The feature point extraction submodule extracts image feature points through Gaussian pyramid construction and the Scale Invariant Feature Transform (SIFT) algorithm. The process consists of four steps, detailed below: Constructing a Gaussian pyramid: Using real-time images of key equipment components as the initial images, a Gaussian pyramid is constructed according to a "group-layer" structure. The pyramids are 1000 Groups, each group contains The layers consist of an initial group (group 1) containing the original image, and subsequent groups containing the results of downsampling (reducing the size to 1 / 2) the previous group of images. Within each group layer( The image is composed of the previous layer image and a Gaussian kernel. The Gaussian kernel formula is obtained by convolution: ; in: , The initial scale parameter is set to 1.6 to ensure uniform scale differences between different image layers.

[0045] Constructing the Difference Gaussian Pyramid (DoG): By subtracting the images of adjacent layers in each group of the Gaussian pyramid, we obtain the difference Gaussian image: ; in Interlayer scaling factor (take) ), used to enhance the feature response corresponding to scale differences.

[0046] Detection scale space extreme points: For each pixel in the difference Gaussian image, compare it with 26 pixels: the 8 neighboring pixels of its own layer, the 9 neighboring pixels of the corresponding positions in the previous and next layers. If the pixel is a local extremum (maximum or minimum), mark it as a candidate feature point.

[0047] Precise feature point localization and feature vector generation: Taylor expansion fitting is performed on the candidate feature points to remove points with low contrast (response value less than 0.03) and edge response (principal curvature ratio greater than 10), thus obtaining the final feature points. Their coordinates in the image are recorded. ; Calculate the gradient magnitude of pixels in the neighborhood of a feature point. With direction : ; ; in: These are the image pixel values ​​for the corresponding layer in the Gaussian pyramid. The histogram of gradient directions in the neighborhood of a feature point is statistically analyzed. The direction corresponding to the peak of the histogram is taken as the main direction of the feature point, and a 128-dimensional SIFT feature vector is generated (the neighborhood is divided into 16 sub-regions, and the gradients in 8 directions are statistically analyzed in each sub-region, for a total of 16×8=128 dimensions), which serves as the representation vector of the feature point.

[0048] The feature matching submodule retrieves the standard feature point vector set of key components of the equipment under the corresponding working conditions from the full-condition normal time-series-image feature sample library, calculates the matching degree between the real-time feature vector and the standard feature vector using the nearest neighbor distance ratio method, and filters out valid feature point pairs with a matching degree higher than the preset threshold. Specifically, the purpose of the feature matching submodule is to match real-time image feature points with standard feature points corresponding to the working conditions, and to filter valid matching pairs. The specific operation is as follows: Core definitions of the full-condition normal time-series image feature sample library: Normal operating condition definition: The equipment can run continuously without fault for ≥72 hours, and the fluctuation of key indicators (temperature, vibration) is ≤±5%. Operating condition coverage logic: Covers 60%~120% of the rated load of the equipment, divides the operating condition range into 20% intervals, and synchronously matches the corresponding speed range; Data association format: Time series data and image data are associated through millisecond-level acquisition timestamps, with a timestamp error of ≤10ms.

[0049] Retrieve the standard characteristics for the corresponding operating condition: From the full-condition normal time-series image feature sample library, based on the operating condition parameters (such as equipment load and operating speed) in the multi-dimensional dynamic time-series operation data, retrieve the standard SIFT feature point vector set of the same equipment and the same key component under the same operating conditions. ( (Number of standard feature points).

[0050] Calculate feature matching degree and filter valid point pairs: The matching degree between the real-time feature vector and the standard feature vector is calculated using the nearest neighbor distance ratio method: For each feature vector of a real-time image In the standard feature vector set Find the nearest vector (Nearest neighbor distance) ) and the second closest vector (nearest neighbor distance) ); Calculate distance ratio Set a preset threshold (Take 0.8), if If the feature point pair is determined to be a valid feature point pair, the real-time feature point coordinates are recorded. Coordinates of corresponding standard feature points .

[0051] The drift correction submodule generates an adaptive drift correction factor based on the coordinate offset of effective feature point pairs. The adaptive drift correction factor corrects the image feature offset of real-time acquired images of key components of the equipment. At the same time, it combines the operating condition benchmark to correct the non-fault fluctuations of multi-dimensional dynamic time-series operating data.

[0052] Specifically, the purpose of setting up the drift correction submodule is to generate an adaptive correction factor for operating conditions, correcting image feature offsets and non-fault fluctuations in time-series data. The specific operation is as follows: Generate adaptive drift correction factor for operating conditions: For all valid feature point pairs, calculate the coordinate offset: ; ; in: This represents the offset of a single effective feature point pair in the horizontal direction of the image; Represents the x-coordinate pixel value of a valid feature point in a real-time image; This represents the x-coordinate pixel value of the corresponding standard feature point in the sample library; This represents the offset of a single effective feature point pair in the vertical direction of the image; This represents the ordinate pixel value of the effective feature point in the real-time image; This represents the ordinate pixel value of the corresponding standard feature point in the sample library.

[0053] The mean of the offsets of all valid points is taken as the adaptive drift correction factor for the operating condition. : ; in: The horizontal axis component represents the adaptive drift correction factor for operating conditions, characterizing the overall drift trend of image features in the horizontal axis direction; The vertical component of the adaptive drift correction factor represents the overall drift trend of image features in the vertical direction. The number of valid feature point pairs; valid feature point pairs When =0, the default drift correction factor for the current operating condition is used. Mark "Feature matching failed"; Indicates the first The offset of a group of valid feature point pairs in the horizontal direction; Indicates the first The offset of a group of valid feature points in the vertical axis direction.

[0054] Correcting feature offset in images of critical equipment components: The coordinates of all feature points in the real-time image are reverse-corrected according to the adaptive drift correction factor under the working condition: ; in, This indicates the final coordinates of feature points in the real-time acquired images of key equipment components after adaptive drift correction. The final coordinates of feature points in the real-time acquired images of key equipment components are obtained by subtracting the adaptive drift correction factor from the original coordinates of uncorrected feature points in the real-time image. The obtained precise coordinates are used to eliminate image feature offset caused by operating condition fluctuations, and to ensure the coordinate alignment consistency between real-time image features and standard features in the sample library. Align the corrected feature point coordinates with the standard feature point coordinates for the corresponding working conditions to eliminate working condition-related offsets in image features.

[0055] Correcting non-fault fluctuations in multi-dimensional dynamic time-series operational data: Retrieve the standard feature mean of multi-dimensional dynamic time-series operation data under the corresponding operating conditions from the full-condition normal time-series image feature sample library. ; Calculate the characteristics of real-time time series data Deviation from the standard mean If the deviation is within the normal fluctuation range of the operating condition in the sample library (determined by the standard deviation of the operating condition data in the sample library), then the real-time time series data is corrected. ; in: The correction factor (taken as 0.8) is used to make the corrected time series data characteristics conform to the normal baseline of the corresponding operating condition and eliminate the interference of non-fault fluctuations. After completing the above processing, the multimodal feature drift correction module 32 transmits the corrected multimodal features to the feature verification and constraint module of the intelligent anomaly detection unit 3 to continue to perform feature validity verification.

[0056] Understandably, traditional feature drift correction only applies to a single type of data and does not take into account differences in operating conditions. This module achieves operating condition adaptability correction for multimodal features by "retrieving standard features corresponding to the operating conditions and generating adaptive correction factors for the operating conditions". At the same time, it takes into account the processing of image feature offset and non-fault fluctuations in time series data, ensuring the consistency and authenticity of multimodal features.

[0057] In this embodiment, the feature verification constraint module 33, based on the processing results of the multimodal feature drift correction module 32, adds local feature focusing verification and global correlation constraints on the basis of feature space mapping and mutual information verification; the process of the feature verification constraint module 33 performing local feature focusing verification and global correlation constraints includes the following steps: S33.1. Based on the physical structure of key equipment components and the physical attributes of multi-dimensional dynamic time-series operational data, the multimodal features are divided into several groups of local features. Each group of local features corresponds to a key equipment component and its associated time-series data dimension. The matching degree is determined using cosine similarity. Quantify the matching accuracy of each set of local features; Specifically, the purpose of step S33.1 is to decompose the multimodal features into feature groups corresponding to the local structure of the device, and to accurately quantify the matching degree of each feature group. The specific operation is as follows: Local feature splitting rules: Based on the physical structure of key equipment components (such as bearings, gearboxes, and motor shafts) and the physical attributes of multi-dimensional dynamic time-series operating data (such as temperature, vibration amplitude, and rotational speed), the multimodal features are divided into several groups of local features: Each set of local features corresponds to "key components of a single device + associated time-series data dimensions", for example: Local feature group 1: bearing components + bearing temperature, bearing vibration amplitude time sequence features + bearing image features; Local feature group 2: Gearbox components + time-series features of gearbox temperature and gearbox vibration frequency + gearbox image features.

[0058] Cosine similarity matching accuracy calculation: The matching accuracy of each set of local features is quantified using a cosine similarity metric, as shown in the formula: ; in: This indicates the matching precision of local features, with a value range of [0,1]. The closer the value is to 1, the higher the degree of matching. This represents a real-time feature vector representing a set of local features. , The dimension of this set of local features. This represents the total number of local feature groups; This represents the standard feature vector of the local features of this group under the corresponding working condition in the sample library. ; Represents real-time feature vectors The Dimensional components; Represents the standard eigenvector The Dimensional components.

[0059] S33.2 Based on the operating mechanism of industrial IoT devices, a global correlation matrix between local features is constructed. The matrix elements are the physical correlation weights between different local features. These physical correlation weights are set based on the power transmission and signal transmission relationships between device components. Specifically, the purpose of setting step S33.2 is to clarify the physical correlation weights between different local features based on the device's operating mechanism, and the specific implementation is as follows: The basis for setting association weights: Based on the operating mechanism of industrial IoT devices (such as the power transmission and signal transmission relationships between components), physical association weights are set between different local feature groups: Direct power transmission refers to the power connection between components without any intermediate transmission components; indirect association refers to the power / signal association between components achieved through one intermediate transmission component. If the components corresponding to the two sets of local features have direct power transmission (such as motor shaft → gearbox), the association weight is set to 0.8~1.0; If the components corresponding to the two sets of local features are indirectly related (such as motor shaft → bearing, transmitted through gearbox), the association weight is set to 0.4~0.7; If the components corresponding to two sets of local features have no direct / indirect relationship, the association weight is set to 0~0.3.

[0060] The form of the global association matrix: Construction dimension Global correlation matrix ( (number of local feature groups), matrix elements Indicates the first Local features of group and the first The physical association weights of local features of a group, i.e.: ; in: This represents the global correlation matrix, used to characterize the strength of physical associations between different local feature groups; Indicates the first Local features of group and the first The physical association weight of the local features of the group, with a value range of [0,1].

[0061] S33.3, Combining cosine similarity matching degree The results verify whether each set of local feature pairs conforms to the weight constraints of the global association matrix, and eliminate false matching results that violate the device operation logic.

[0062] Specifically, the purpose of setting step S33.3 is to verify whether the local feature matching results conform to the device's operating logic and to eliminate false matches. The specific implementation is as follows: Constraint verification logic: For any two sets of local features (the first set) Group, No. (group), combined with its matching accuracy Weights of the global association matrix Verification required: like (Direct association) requires (The difference in matching accuracy is small); like (Indirect connection) requires ; like (Weak association) means there is no mandatory precision difference constraint.

[0063] False match removal rules: If a set of local features does not meet the constraints of the corresponding weights mentioned above, it is determined to be a "false matching result that violates the operating logic of the equipment", and the matching result of the set of local features is removed from the subsequent analysis.

[0064] After completing the above steps, the feature verification constraint module 33 transmits the filtered valid local feature matching results to the anomaly identification output module 34 to carry out the final equipment anomaly status determination; if all local feature groups are eliminated (no valid feature groups): it is determined as "detection failure", an early warning is output and the anomaly determination is cancelled.

[0065] Understandably, traditional feature verification only compares feature similarity in a single dimension without considering the physical operating logic of the device. This module ensures that the feature matching results are consistent with the actual operating mechanism of the device through "local feature focusing + global correlation constraints", reducing the interference of false matches on anomaly identification.

[0066] In this embodiment, the anomaly identification output module 34, based on the processing result of the feature verification constraint module 33, completes the anomaly identification of the industrial IoT device's operating status and transmits the anomaly detection result to the anomaly result output and feedback unit 4. The anomaly identification output module 34 includes a feature deviation calculation submodule, an anomaly state determination submodule, and a result integration and transmission submodule, wherein: The feature deviation calculation submodule calculates the local feature matching results output by the feature verification constraint module 33, combined with the cumulative distance. Matching degree with cosine similarity The core data is used to calculate the overall deviation between real-time multimodal features and standard features in the normal time-series-image feature sample library under full working conditions; Specifically, the feature deviation calculation submodule is designed to combine cumulative distance and cosine similarity to quantify the overall deviation of real-time multimodal features relative to standard features. The specific operation is as follows: The feature weighting allocation rule is determined based on the difference in importance of key components of the equipment, and weight coefficients are assigned to each group of verified local features. , (Number of local feature groups that passed the verification) The weighting coefficients of local feature groups corresponding to core components (such as motor shafts and spindle bearings) Take a value of 0.7~1.0; The weighting coefficients of the local feature groups corresponding to auxiliary components (such as cooling fans and lubrication systems) The weights are set to 0.3 to 0.6; the weights satisfy the normalization constraint. .

[0067] Calculate the deviation component pairs of a single set of local features: No. Group local features, combined with cumulative distance Similarity to cosine Calculate the deviation component The formula is: ; in: Indicates the first The deviation component of the local features of the group has a value range of [0,1]. The larger the value, the more significant the deviation of the features of the group. Indicates the first Weight coefficients of local features of a group; Indicates the first The cumulative distance matrix elements corresponding to the local features of a group represent the distance difference between the real-time features and the standard features; This represents the maximum cumulative distance corresponding to all local feature groups, used for... Normalization is performed to eliminate the influence of dimensions; Indicates the first The accuracy of cosine similarity matching of local features of a group.

[0068] The overall deviation value of the real-time multimodal features is obtained by summing the deviation components of all verified local feature groups. The formula is: ; in: This represents the overall deviation value of the real-time multimodal features, with a value range of [0,1]. The larger the value, the greater the deviation of the real-time features from the standard features.

[0069] The abnormal state determination submodule compares the overall deviation value with the preset abnormal determination threshold. When the overall deviation value is higher than the abnormal determination threshold, the industrial IoT device is determined to be in an abnormal operating state. Specifically, the purpose of the abnormal state determination submodule is to determine whether the equipment's operating state is abnormal based on a comparison of the overall deviation value with a preset threshold. The specific operation is as follows: The anomaly detection threshold is set based on the anomaly detection threshold. Based on historical data from a full-condition normal time-series image feature sample library, the following was determined: the overall deviation value under all normal operating conditions in the statistical sample library was used as the 95th percentile as the anomaly detection threshold. This ensures that the misjudgment rate under normal operating conditions is kept within a reasonable range.

[0070] The abnormal state determination rule will use the overall deviation value output by the characteristic deviation calculation submodule. With threshold Compare: like The system determines that the industrial IoT device is in an abnormal operating state. like This determines that the industrial IoT devices are in normal operating condition.

[0071] The result integration and transmission submodule integrates the anomaly detection results, which include the anomaly judgment result, the corresponding alignment and correction timing node, real-time acquired images of key components of the equipment, and feature verification details. The anomaly detection results are then transmitted to the anomaly result output and feedback unit 4.

[0072] Specifically, the purpose of the result integration and transmission submodule is to integrate all the anomaly detection information and transmit it to the anomaly result output and feedback unit 4. The specific implementation is as follows: Summary of anomaly detection results: Following the logic of "status determination + data traceability + verification details", the following four types of information are integrated: Anomaly Judgment Result: Clearly indicate the equipment's operating status as "normal" or "abnormal"; if determined to be abnormal, simultaneously indicate the anomaly level (based on...). The proportion exceeding the threshold is categorized as mild, moderate, or severe abnormality. Alignment-corrected timing nodes: Extract the synchronization timing stamp processed by the timing-image alignment correction module 31, and mark the device running time interval corresponding to the abnormal state; Real-time images of key equipment components: Includes corrected images of key equipment components, with component areas marked with abnormal features; Feature verification details: Record the number of local feature groups that pass the verification by feature verification constraint module 33, the false matching feature groups that are removed, and the reasons for removal.

[0073] Result transmission method: The integrated anomaly detection results are encapsulated into JSON format data and transmitted to the anomaly result output and feedback unit 4 via industrial Ethernet or MQTT IoT communication protocol. Data check bits are set during transmission to ensure the integrity and accuracy of the results transmission.

[0074] Understandably, traditional anomaly detection methods often use a single indicator (distance or similarity) for threshold comparison, which is easily affected by local feature bias. This module calculates the overall deviation value through "weight allocation + dual indicator fusion" and eliminates false matching interference by combining feature verification results, thereby improving the accuracy and reliability of anomaly detection. In addition, the integrated results contain complete traceability information, which makes it easy for operation and maintenance personnel to quickly locate the cause of the anomaly.

[0075] The abnormal result output and feedback unit 4 pushes the abnormal detection results and the corresponding aligned and corrected time sequence nodes, real-time acquired images of key equipment components, and feature verification details to the industrial IoT monitoring platform and related operation and maintenance terminals. It records the abnormal correlation data and transmits the abnormal correlation data to the intelligent abnormal detection unit 3 to update the full-condition normal time sequence-image feature sample library of the intelligent abnormal detection unit 3.

[0076] In this embodiment, the abnormal result output and feedback unit 4 includes an abnormal result push module 41, an abnormal data recording module 42, and a sample library update and transmission module 43, wherein: The abnormal result push module 41 pushes the abnormal detection results transmitted by the intelligent abnormal detection unit 3 to the industrial IoT monitoring platform and related operation and maintenance terminals; Specifically, the steps for pushing anomaly detection results to the industrial IoT monitoring platform and related operation and maintenance terminals are as follows: Push triggering conditions: When the anomaly identification output module 34 determines that the device is in an abnormal state, the push process is triggered immediately; if it is determined to be in a normal state, the result is only pushed when the monitoring platform actively queries.

[0077] Multi-terminal adaptation push strategy: Industrial IoT monitoring platform: It uses the HTTP protocol to push structured JSON data, including anomaly judgment results, time sequence nodes, image thumbnails and verification details, and supports the platform to display anomaly alarms and source tracing information in real time; Maintenance terminals (such as industrial tablets and mobile apps): use the MQTT protocol to push lightweight alarm information, including the anomaly level, corresponding device ID and component location, and also include a compressed preview of the anomaly image, which is convenient for maintenance personnel to handle on-site.

[0078] In addition, a three-stage retransmission mechanism is set up during the push process. If the first push fails, it will be retransmitted after 5 seconds, 10 seconds, and 20 seconds. After a retransmission fails, the push log is recorded and the local cache is triggered. The data will be automatically retransmitted after the network is restored to ensure that abnormal information is not lost.

[0079] The abnormal data recording module 42 records the abnormal detection results and the corresponding abnormal related data. The abnormal related data includes the abnormal judgment results, the alignment and correction time sequence nodes, and the real-time acquired images and feature verification details of the key components of the equipment. Specifically, the steps for recording anomaly detection results and corresponding anomaly-related data are as follows: A hybrid storage solution combining time-series databases (such as InfluxDB) and object storage (such as MinIO) is adopted, as detailed below: Structured data (anomaly detection results, time series nodes, verification details): stored in a time series database, with a secondary index built according to "device ID-timestamp", supporting fast query and correlation analysis; Unstructured data (images of key equipment components): stored in object storage, with file naming rules of "equipment ID-component number-time stamp.jpg", and the file storage path is recorded in the time series database to realize the correlation and traceability of structured and unstructured data.

[0080] Meanwhile, data redundancy and backup are implemented by setting up dual-copy storage for abnormal related data. The primary copy is stored on the local server, and the secondary copy is synchronized to the cloud object storage. At the same time, a data backup package is automatically generated every day to retain abnormal data from the most recent 30 days, meeting the data traceability needs of industrial scenarios.

[0081] The sample library update and transmission module 43 transmits the abnormal correlation data recorded by the abnormal data recording module 42 to the intelligent anomaly detection unit 3 to update the full-condition normal time-series-image feature sample library of the intelligent anomaly detection unit 3.

[0082] Specifically, the operation of feeding back abnormal correlation data to the intelligent anomaly detection unit 3 to achieve dynamic updating of the full-condition normal time-series image feature sample library is as follows: The mechanism of "incremental update + manual review" is adopted, as follows: Incremental update: Only the associated data that has been verified as valid anomalies by the feature verification constraint module 33 is transmitted as incremental samples to the intelligent anomaly detection unit 3 to avoid invalid data from polluting the sample library; Manual review: Before transmission, the operation and maintenance personnel will review the data. Only after the review is passed will the abnormal data be included in the "abnormal feature subset" of the sample library, and the abnormal type will be additionally marked (such as wear, looseness, overheating).

[0083] Simultaneously, when the number of newly added abnormal samples reaches 5% of the total sample library capacity, a dynamic update of the sample library is triggered. For the normal feature subset in the "Full-condition Normal Time Series-Image Feature Sample Library", add the operating condition benchmark features corresponding to the abnormal samples; For the subset of abnormal features, the multimodal features of newly added abnormal samples are integrated, the generation logic of the feature drift correction factor is optimized, and the system's ability to identify new anomalies is improved.

[0084] In addition, incremental sample data is transmitted using industrial Ethernet. Before transmission, the data is checked using MD5 to ensure data integrity. After transmission is completed, the intelligent anomaly detection unit 3 returns an acknowledgment receipt. If no receipt is received, retransmission is triggered to ensure the reliability of sample library updates.

[0085] like Figure 3 As shown, this embodiment also provides a method for detecting abnormal operating status of industrial IoT devices based on dynamic time-series data. The method, based on the aforementioned system for detecting abnormal operating status of industrial IoT devices based on dynamic time-series data, includes the following steps: S1. Real-time acquisition of multi-dimensional dynamic time-series operation data and images of key components of industrial IoT devices; adding synchronization time stamps with the same reference clock source to the multi-dimensional dynamic time-series operation data and images of key components; performing format standardization processing on the marked multi-dimensional dynamic time-series operation data and images of key components and transmitting them to S2. S2. Perform noise filtering and missing value imputation on the received multi-dimensional dynamic time-series running data, then perform time-series feature enhancement processing on the processed data, extract the time correlation features, and transmit the preprocessed multi-dimensional dynamic time-series running data to S3. S3. Based on the preprocessed multi-dimensional dynamic time-series operation data and images of key equipment components, a dynamic time warping algorithm is used to construct the optimal time correspondence between the multi-dimensional dynamic time-series operation data and the images of key equipment components, and to complete nonlinear time correction and frequency adaptation alignment. The aligned multi-dimensional dynamic time-series operation data and images of key equipment components are then transmitted to S4. S4. Call the preset full-condition normal time-series-image feature sample library, extract feature points from the images of key components of the equipment to obtain feature point coordinates and corresponding feature vectors, retrieve the standard feature point vector set of the corresponding working condition in the sample library and calculate the matching degree between the real-time feature vector and the standard feature vector, filter effective feature point pairs and generate working condition adaptive drift correction factor, correct the feature offset of the images of key components of the equipment through the correction factor, and at the same time, combine the working condition benchmark to correct the non-fault fluctuations of multi-dimensional dynamic time-series operation data, and transmit the corrected multi-modal features to S5. S5. Based on feature space mapping and mutual information verification, the multimodal features are split into several groups of local features according to the physical structure of the key components of the equipment and the physical attributes of the multi-dimensional dynamic time-series operation data. The matching accuracy of each group of local features is quantified. A global correlation matrix between local features is constructed based on the equipment operation mechanism. The weight constraints of each group of local features are verified to ensure that they meet the weight constraints of the global correlation matrix. False matching results are eliminated. The local feature matching results after feature verification constraints are transmitted to S6. S6. Based on the local feature matching results after feature verification constraints, and combined with relevant core data, calculate the overall deviation value between the real-time multimodal features and the standard features in the full-condition normal time-series-image feature sample library. Compare the overall deviation value with the preset anomaly judgment threshold to determine whether the industrial IoT device is in an abnormal operating state. Integrate the anomaly detection results and transmit them to S7. S7. Push the anomaly detection results to the industrial IoT monitoring platform and related operation and maintenance terminals, record the anomaly-related data, and transmit the anomaly-related data to the full-condition normal time series-image feature sample library for updating the sample library.

[0086] Those skilled in the art will understand that the process of implementing all or part of the steps of the above embodiments can be carried out by hardware or by a program instructing the relevant hardware.

[0087] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. An industrial IoT device operation status anomaly detection system based on dynamic time-series data, characterized in that, include: The data acquisition unit (1) collects multi-dimensional dynamic time-series operation data of industrial IoT devices and real-time images of key components of the devices in real time. It adds synchronous time-series stamps to the multi-dimensional dynamic time-series operation data and real-time images of key components of the devices. After performing format standardization processing on the collected multi-dimensional dynamic time-series operation data and real-time images of key components of the devices, it transmits them to the time-series data preprocessing unit (2). The time series data preprocessing unit (2) performs noise filtering, missing value filling and time series feature enhancement processing on the received multi-dimensional dynamic time series running data, extracts the time correlation features in the multi-dimensional dynamic time series running data, and transmits the preprocessed multi-dimensional dynamic time series running data to the intelligent anomaly detection unit (3). The intelligent anomaly detection unit (3) is based on the preprocessed multi-dimensional dynamic time-series operation data and real-time acquired images of key components of the equipment. It uses a dynamic time warping algorithm to construct the optimal time correspondence between the image frames of the multi-dimensional dynamic time-series operation data and the real-time acquired images of key components of the equipment, and completes nonlinear time correction and frequency adaptation alignment. It calls the preset full-condition normal time-series-image feature sample library to generate an adaptive drift correction factor for the operating condition. It corrects the image feature offset of the real-time acquired images of key components of the equipment by matching feature points through scale-invariant feature transformation. It combines the operating condition benchmark to correct the non-fault fluctuation of the multi-dimensional dynamic time-series operation data. On the basis of feature space mapping and mutual information verification, it adds local feature focusing verification and global correlation constraints to ensure the consistency of multi-modal feature alignment, completes the identification of abnormal operating status of industrial IoT equipment, and transmits the anomaly detection results to the anomaly result output and feedback unit (4). The abnormal result output and feedback unit (4) pushes the abnormal detection results and the corresponding aligned and corrected time sequence nodes, real-time acquired images of key components of the equipment, and feature verification details to the industrial Internet of Things monitoring platform and related operation and maintenance terminals, records the abnormal correlation data and transmits the abnormal correlation data to the intelligent abnormal detection unit (3) for updating the full-condition normal time sequence-image feature sample library of the intelligent abnormal detection unit (3).

2. The industrial IoT device operation status anomaly detection system based on dynamic time-series data as described in claim 1, characterized in that, The data acquisition unit (1) includes a data image acquisition module (11), a synchronization timing stamp marking module (12), and a format standardization processing module (13), wherein: The data image acquisition module (11) acquires multi-dimensional dynamic time-series operation data of industrial IoT devices and real-time images of key components of the devices in real time. The synchronization timing stamp marking module (12) adds a synchronization timing stamp mark with the same reference clock source to the multi-dimensional dynamic timing operation data of the industrial IoT device and the real-time acquisition images of key components of the device, based on the data image acquisition module (11) and the data image acquisition module (11). The format standardization processing module (13) performs format standardization processing on the multi-dimensional dynamic time-series running data and real-time acquired images of key equipment components after being marked by the synchronous time-series stamp marking module (12), and then transmits them to the time-series data preprocessing unit (2).

3. The industrial IoT device operation status anomaly detection system based on dynamic time-series data according to claim 2, characterized in that, The time-series data preprocessing unit (2) includes a noise filtering and filling module (21) and a time-series feature enhancement module (22), wherein: The noise filtering and filling module (21) performs noise filtering and missing value filling processing based on the received multi-dimensional dynamic time-series running data using sliding window technology. The time-series feature enhancement module (22) uses wavelet transform technology to perform time-series feature enhancement processing based on the multi-dimensional dynamic time-series running data processed by the noise filtering and filling module (21), extracts the time correlation features in the multi-dimensional dynamic time-series running data, and transmits the preprocessed multi-dimensional dynamic time-series running data to the intelligent anomaly detection unit (3).

4. The industrial IoT device operation status anomaly detection system based on dynamic time-series data according to claim 3, characterized in that, The intelligent anomaly detection unit (3) includes a time-series-image alignment correction module (31), a multimodal feature drift correction module (32), a feature verification constraint module (33), and an anomaly recognition output module (34), wherein: The time-image alignment correction module (31) is based on the preprocessed multi-dimensional dynamic time-series running data and the real-time acquired images of key components of the equipment. It uses the dynamic time warping algorithm to construct the optimal time correspondence between the image frames of the multi-dimensional dynamic time-series running data and the real-time acquired images of key components of the equipment, and completes nonlinear time correction and frequency adaptation alignment. The multimodal feature drift correction module (32) generates an adaptive drift correction factor based on the processing result of the time-image alignment correction module (31) by calling the preset full-condition normal time-image feature sample library. It corrects the image feature offset of the real-time acquired image of the key components of the equipment by scale-invariant feature transformation feature point matching, and corrects the non-fault fluctuation of multi-dimensional dynamic time-series operation data by combining the working condition benchmark. The feature verification constraint module (33) adds local feature focusing verification and global correlation constraint on the basis of feature space mapping and mutual information verification, based on the processing results of the multimodal feature drift correction module (32); The anomaly identification output module (34) completes the anomaly identification of the operating status of industrial IoT equipment based on the processing result of the feature verification constraint module (33) and transmits the anomaly detection result to the anomaly result output and feedback unit (4).

5. The industrial IoT device operation status anomaly detection system based on dynamic time-series data according to claim 4, characterized in that, The time-image alignment correction module (31) uses a dynamic time warping algorithm to construct the optimal time correspondence and complete the nonlinear time correction and frequency adaptation alignment process, which includes the following steps: S31.1 Calculation of multi-dimensional dynamic time-series running data Real-time image acquisition of data points and key components of the equipment. The cumulative distance is calculated recursively based on the feature distance of the frame image. Construct a cumulative distance matrix covering all data points and image frames, through Achieve dynamic time warping nonlinear matching adaptation; S31.2, Traverse the cumulative distance matrix and select the cumulative distance. Find the path with the smallest sum and accumulate the distance. The path with the minimum sum is determined as the optimal time correspondence between multi-dimensional dynamic time-series running data and image frames of real-time acquired images of key components of the equipment; S31.3 Based on the optimal time correspondence, nonlinear time correction is performed on the multi-dimensional dynamic time-series operation data and the real-time acquired images of key components of the equipment to synchronously complete frequency adaptation and alignment.

6. The industrial IoT device operation status anomaly detection system based on dynamic time-series data according to claim 5, characterized in that, The multimodal feature drift correction module (32) includes a feature point extraction submodule, a feature matching submodule, and a drift correction submodule, wherein: The feature point extraction submodule performs Gaussian pyramid construction and scale-invariant feature transformation on real-time images of key components of the equipment to extract feature points, thereby obtaining the coordinates of feature points and corresponding feature vectors of the images. The feature matching submodule retrieves the standard feature point vector set of key components of the equipment under the corresponding working conditions from the full-condition normal time-series-image feature sample library, calculates the matching degree between the real-time feature vector and the standard feature vector using the nearest neighbor distance ratio method, and filters out valid feature point pairs with a matching degree higher than a preset threshold. The drift correction submodule generates an adaptive drift correction factor based on the coordinate offset of effective feature point pairs. The adaptive drift correction factor corrects the image feature offset of real-time acquired images of key components of the equipment. At the same time, it corrects the non-fault fluctuations of multi-dimensional dynamic time-series operating data by combining the operating condition benchmark.

7. The industrial IoT device operation status anomaly detection system based on dynamic time-series data according to claim 6, characterized in that, The process of the feature verification constraint module (33) performing local feature focusing verification and global correlation constraint includes the following steps: S33.

1. Based on the physical structure of key equipment components and the physical attributes of multi-dimensional dynamic time-series operational data, the multimodal features are divided into several groups of local features. Each group of local features corresponds to a key equipment component and its associated time-series data dimension. The matching degree is determined using cosine similarity. Quantify the matching accuracy of each set of local features; S33.2 Based on the operating mechanism of industrial IoT devices, a global correlation matrix between local features is constructed. The matrix elements are the physical correlation weights between different local features. These physical correlation weights are set based on the power transmission and signal transmission relationships between device components. S33.3, Combining cosine similarity matching degree The results verify whether each set of local feature pairs conforms to the weight constraints of the global association matrix, and eliminate false matching results that violate the device operation logic.

8. The industrial IoT device operation status anomaly detection system based on dynamic time-series data according to claim 7, characterized in that, The anomaly identification output module (34) includes a feature deviation calculation submodule, an anomaly state determination submodule, and a result integration and transmission submodule, wherein: The feature deviation calculation submodule is based on the local feature matching results output by the feature verification constraint module (33), combined with the cumulative distance. Matching degree with cosine similarity The core data is used to calculate the overall deviation between real-time multimodal features and standard features in the normal time-series-image feature sample library under full working conditions; The abnormal state determination submodule compares the overall deviation value with the preset abnormal determination threshold. When the overall deviation value is higher than the abnormal determination threshold, the industrial IoT device is determined to be in an abnormal operating state. The result integration and transmission submodule integrates the anomaly detection results, which include anomaly judgment results, corresponding alignment and correction timing nodes, real-time acquired images of key components of the equipment, and feature verification details. The anomaly detection results are then transmitted to the anomaly result output and feedback unit (4).

9. The industrial IoT device operation status anomaly detection system based on dynamic time-series data according to claim 8, characterized in that, The abnormal result output and feedback unit (4) includes an abnormal result push module (41), an abnormal data recording module (42), and a sample library update and transmission module (43), wherein: The abnormal result push module (41) pushes the abnormal detection results transmitted by the intelligent abnormal detection unit (3) to the industrial Internet of Things monitoring platform and related operation and maintenance terminals. The abnormal data recording module (42) records the abnormal detection results and the corresponding abnormal associated data. The abnormal associated data includes the abnormal judgment results, the alignment and correction time sequence nodes, and the real-time acquisition images and feature verification details of the key components of the equipment. The sample library update transmission module (43) transmits the abnormal correlation data recorded by the abnormal data recording module (42) to the intelligent anomaly detection unit (3) to update the full-condition normal time-series-image feature sample library of the intelligent anomaly detection unit (3).

10. A method for detecting abnormal operating status of industrial IoT devices based on dynamic time-series data, wherein the method is based on the industrial IoT device operating status anomaly detection system based on dynamic time-series data as described in any one of claims 1-9, characterized in that... Includes the following steps: S1. Real-time acquisition of multi-dimensional dynamic time-series operation data and images of key components of industrial IoT devices; adding synchronization time stamps with the same reference clock source to the multi-dimensional dynamic time-series operation data and images of key components; performing format standardization processing on the marked multi-dimensional dynamic time-series operation data and images of key components and transmitting them to S2. S2. Perform noise filtering and missing value imputation on the received multi-dimensional dynamic time-series running data, then perform time-series feature enhancement processing on the processed data, extract the time correlation features, and transmit the preprocessed multi-dimensional dynamic time-series running data to S3. S3. Based on the preprocessed multi-dimensional dynamic time-series operation data and images of key equipment components, a dynamic time warping algorithm is used to construct the optimal time correspondence between the multi-dimensional dynamic time-series operation data and the images of key equipment components, and to complete nonlinear time correction and frequency adaptation alignment. The aligned multi-dimensional dynamic time-series operation data and images of key equipment components are then transmitted to S4. S4. Call the preset full-condition normal time-series-image feature sample library, extract feature points from the images of key components of the equipment to obtain feature point coordinates and corresponding feature vectors, retrieve the standard feature point vector set of the corresponding working condition in the sample library and calculate the matching degree between the real-time feature vector and the standard feature vector, filter effective feature point pairs and generate working condition adaptive drift correction factor, correct the feature offset of the images of key components of the equipment through the correction factor, and at the same time, combine the working condition benchmark to correct the non-fault fluctuations of multi-dimensional dynamic time-series operation data, and transmit the corrected multi-modal features to S5. S5. Based on feature space mapping and mutual information verification, the multimodal features are split into several groups of local features according to the physical structure of the key components of the equipment and the physical attributes of the multi-dimensional dynamic time-series operation data. The matching accuracy of each group of local features is quantified. A global correlation matrix between local features is constructed based on the equipment operation mechanism. The weight constraints of each group of local features are verified to ensure that they meet the weight constraints of the global correlation matrix. False matching results are eliminated. The local feature matching results after feature verification constraints are transmitted to S6. S6. Based on the local feature matching results after feature verification constraints, and combined with relevant core data, calculate the overall deviation value between the real-time multimodal features and the standard features in the full-condition normal time-series-image feature sample library. Compare the overall deviation value with the preset anomaly judgment threshold to determine whether the industrial IoT device is in an abnormal operating state. Integrate the anomaly detection results and transmit them to S7. S7. Push the anomaly detection results to the industrial IoT monitoring platform and related operation and maintenance terminals, record the anomaly-related data, and transmit the anomaly-related data to the full-condition normal time series-image feature sample library for updating the sample library.

Images