Method, equipment, and apparatus for continuous prediction of device status based on heterogeneous sensing data
By preprocessing sensor data and using a feature extraction network, the accuracy problem caused by individual differences in equipment status prediction in deep neural networks is solved, and accurate prediction of equipment status is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NAT UNIV OF DEFENSE TECH
- Filing Date
- 2024-02-23
- Publication Date
- 2026-06-30
AI Technical Summary
Existing deep neural network device state prediction methods lack a global perspective and struggle to handle individual differences between different devices, resulting in reduced prediction accuracy.
By acquiring temporal data from multiple sensors, preprocessing it, extracting state data and global compressed data, using attention feature extraction network and feature extraction network for feature selection and weighting, employing multi-layer threshold temporal convolutional units for feature extraction, and finally performing continuous prediction of device state.
It improves the model's accuracy in predicting device states with individual differences, guides the effective understanding of the current state through macroscopic state changes, and enhances feature extraction performance.
Smart Images
Figure CN118035862B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of industrial equipment management technology, and in particular to a method, equipment and apparatus for continuous prediction of equipment status based on heterogeneous sensing data. Background Technology
[0002] Thanks to the widespread adoption of sensor networks, a massive amount of data characterizing equipment status has been collected and stored over the past decade. This acquisition of massive amounts of data has greatly facilitated the development of data-driven equipment status prediction methods. These methods can establish a reliable mapping from data to labels using sufficient historical data, even in the absence of specialized expertise, thereby achieving effective equipment status prediction. In recent years, research on deep neural networks has progressed rapidly, and related findings have also been applied to equipment status prediction tasks. Compared to traditional statistical analysis models and traditional machine learning models represented by shallow neural networks and support vector machines, deep neural network-based methods can leverage their deep network structure to extract deep features from complex objects, making them suitable for today's increasingly complex equipment status prediction tasks.
[0003] In applications, prediction methods based on deep neural networks often rely on training data collected from numerous device entities to discover effective state patterns. However, in real-world environments, due to complexity and diverse initial settings, the distribution of monitoring data varies across different devices, while the local data describing the state inevitably overlaps. This leads to the same monitoring signal pointing to different states on different devices. It's important to note that the differences between individual devices are often characterized by their macroscopic state distribution and trends. However, for most current deep learning models, constrained by model complexity, existing methods often employ a sliding window strategy, using local data within a fixed window to describe the system state at a given moment. This model infers device state based solely on current data, thus requiring uniformity in device data patterns. Due to the lack of a global perspective, the model is prone to misjudging device states based on similar local data, leading to reduced prediction accuracy. This problem is a major limiting factor for model performance and requires further resolution. Summary of the Invention
[0004] Therefore, it is necessary to provide a method, device, and apparatus for continuous prediction of device status based on heterogeneous sensing data that can effectively improve the accuracy of neural network prediction of device status, addressing the aforementioned technical problems.
[0005] A method for continuous prediction of device status based on heterogeneous sensing data, the method comprising:
[0006] Acquire observation data related to the device to be predicted in state, wherein the observation data is time-domain data obtained from multiple sensors;
[0007] The observation data is preprocessed, and state data at a certain moment is extracted from the preprocessed observation data. The observation data is then downsampled to obtain globally compressed data with the same length as the state data.
[0008] The globally compressed data is weighted and then input into an attention feature extraction network. In the attention feature extraction network, shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain feature selection vectors and element attention matrices based on the shared features.
[0009] The state data is input into a feature extraction network. In the feature extraction network, the fusion features of the state data are extracted. The fusion features are multiplied by the element attention matrix to obtain weighted features. Multi-layer threshold temporal convolutional units are used to obtain temporal features based on the weighted features. Multi-layer fully connected units are used to obtain output features based on the temporal features.
[0010] The device's state is continuously predicted based on the feature selection vector and output features obtained from multiple consecutive time points.
[0011] In one embodiment, the preprocessing of the observation data includes: sequentially performing data cleaning, data standardization, and data filtering on the observation data.
[0012] In one embodiment, a sliding window of a preset size is used to extract local data from the preprocessed observation data as the device's state data at a certain moment;
[0013] The preprocessed observation data is compressed using an equal-interval sampling strategy to obtain globally compressed data with the same length as the state data.
[0014] In one embodiment, the attention feature extraction network includes a first convolutional unit, a first attention sub-branch unit, and a second attention sub-branch unit;
[0015] The first convolutional unit includes two sequentially connected one-dimensional convolutional layers, which extract features from the globally compressed data to obtain the shared features;
[0016] The first attention sub-branch unit first adaptively scales the shared features, and then obtains the feature selection vector through a convolutional layer, an average pooling layer, and a fully connected layer;
[0017] The second attention branch unit obtains channel attention vectors and temporal attention vectors through convolutional layers, average pooling layers, and fully connected layers respectively based on the shared features. The element attention matrix with the same size as the fused feature is obtained by vector multiplying the channel attention vectors and temporal attention vectors.
[0018] In one embodiment, the feature extraction network includes: a second convolutional unit and a threshold temporal convolutional unit;
[0019] The second convolutional unit includes two convolutional layers with a kernel size of one connected in sequence, which extract features from the state data to obtain the fused features;
[0020] The fused feature is multiplied by the element attention matrix to obtain a weighted feature, and the weighted feature is input into the threshold temporal convolutional unit;
[0021] The threshold temporal convolutional unit includes three threshold temporal convolutional layers. Each threshold temporal convolutional layer is equipped with a feature selection mechanism. The weighted features are extracted sequentially through the three threshold temporal convolutional layers to obtain the output features.
[0022] In one embodiment, the input data of each threshold temporal convolutional layer is a feature sequence corresponding to multiple consecutive time points, and in each threshold temporal convolutional layer:
[0023] The first temporary feature is extracted based on the feature data corresponding to the latest time in the feature sequence;
[0024] The second temporary feature is obtained by extracting all feature data in the feature sequence except for the feature data corresponding to the latest time.
[0025] The aforementioned feature selection mechanism is used to achieve adaptive feature selection of the first temporary feature and the second temporary feature, thereby obtaining the output data of the current threshold temporal convolutional layer.
[0026] In one embodiment, the feature selection mechanism is represented as:
[0027]
[0028] In the above formula, This represents the output data of the i-th threshold temporal convolutional layer. This represents the attention vector calculated based on the first temporary feature. This indicates the first temporary feature. This indicates the second temporary feature.
[0029] In one embodiment, a device state prediction network is constructed based on the feature extraction network and the attention feature extraction network. When training the device state prediction network:
[0030] The training data in the training dataset are sorted from simple to complex according to prediction difficulty, and then the training data in the training dataset are sequentially input into the device state prediction network for training.
[0031] This application also provides a device for continuous prediction of device status based on heterogeneous sensing data, the device comprising:
[0032] The observation data acquisition module is used to acquire continuous observation data related to the device to be predicted in the state, wherein the observation data is time-domain data obtained from multiple sensors.
[0033] The state data and global compressed data acquisition module is used to preprocess the observation data, extract state data at a certain moment based on the preprocessed observation data, and downsample the observation data to obtain global compressed data with the same length as the state data.
[0034] The attention feature extraction module is used to input the globally compressed data into the attention feature extraction network after data weighting. In the attention feature extraction network, the shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain the feature selection vector and the element attention matrix based on the shared features.
[0035] The feature extraction module is used to input the state data into the feature extraction network, extract the fusion features of the state data in the feature extraction network, multiply the fusion features with the element attention matrix to obtain weighted features, use multi-layer threshold temporal convolutional units to obtain temporal features based on the weighted features, and use multi-layer fully connected units to obtain output features based on the temporal features.
[0036] The state prediction module is used to continuously predict the state of the device based on the feature selection vector and output features obtained from multiple consecutive time points.
[0037] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:
[0038] Acquire observation data related to the device to be predicted in state, wherein the observation data is time-domain data obtained from multiple sensors;
[0039] The observation data is preprocessed, and state data at a certain moment is extracted from the preprocessed observation data. The observation data is then downsampled to obtain globally compressed data with the same length as the state data.
[0040] The globally compressed data is weighted and then input into an attention feature extraction network. In the attention feature extraction network, shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain feature selection vectors and element attention matrices based on the shared features.
[0041] The state data is input into a feature extraction network. In the feature extraction network, the fusion features of the state data are extracted. The fusion features are multiplied by the element attention matrix to obtain weighted features. Multi-layer threshold temporal convolutional units are used to obtain temporal features based on the weighted features. Multi-layer fully connected units are used to obtain output features based on the temporal features.
[0042] The device's state is continuously predicted based on the feature selection vector and output features obtained from multiple consecutive time points.
[0043] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0044] Acquire observation data related to the device to be predicted in state, wherein the observation data is time-domain data obtained from multiple sensors;
[0045] The observation data is preprocessed, and state data at a certain moment is extracted from the preprocessed observation data. The observation data is then downsampled to obtain globally compressed data with the same length as the state data.
[0046] The globally compressed data is weighted and then input into an attention feature extraction network. In the attention feature extraction network, shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain feature selection vectors and element attention matrices based on the shared features.
[0047] The state data is input into a feature extraction network. In the feature extraction network, the fusion features of the state data are extracted. The fusion features are multiplied by the element attention matrix to obtain weighted features. Multi-layer threshold temporal convolutional units are used to obtain temporal features based on the weighted features. Multi-layer fully connected units are used to obtain output features based on the temporal features.
[0048] The device's state is continuously predicted based on the feature selection vector and output features obtained from multiple consecutive time points.
[0049] The aforementioned method, device, and apparatus for continuous prediction of device status based on heterogeneous sensing data preprocesses observation data received from multiple sensors about the same device, then extracts state data and globally compressed data from these data. Attention feature extraction networks and feature extraction networks are then used to extract feature selection vectors and output features, respectively. The globally compressed data represents the macroscopic state changes of the observation data, while the state data represents the specific observation data at a given moment. Finally, the device status is predicted based on the feature selection vectors and output features. This method uses the macroscopic state changes of instances to guide the model's effective understanding of the current state, thereby improving the model's feature extraction performance for entities with individual differences, and achieving accurate prediction of device status. Attached Figure Description
[0050] Figure 1 This is a flowchart illustrating a method for continuous prediction of device status based on heterogeneous sensing data in one embodiment.
[0051] Figure 2 This is a schematic diagram of a nearest neighbor-based data compensation strategy in one embodiment;
[0052] Figure 3 This is a schematic diagram of the structure of the attention feature extraction network and the feature extraction network in one embodiment;
[0053] Figure 4 This is a schematic diagram of the structure of an improved temporal convolutional network in one embodiment;
[0054] Figure 5 This is a structural block diagram of a device state continuous prediction device based on heterogeneous sensing data in one embodiment;
[0055] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0057] In existing technologies, when using deep neural networks to predict the state of devices, the data distribution of different devices of the same category varies due to differences in individual characteristics and environment. Similar samples may point to different labels, causing ambiguity in the training data labels. This problem makes it difficult for the model to establish a reliable mapping from data to labels, limiting the model's feature extraction performance. Figure 1 As shown, a method for continuous prediction of device status based on heterogeneous sensing data is provided, which specifically includes the following steps:
[0058] Step S100: Obtain observation data related to the device to be predicted in the state. The observation data consists of time-domain data obtained from multiple sensors.
[0059] Step S110: Preprocess the observation data, extract the state data at a certain moment based on the preprocessed observation data, and downsample the observation data to obtain global compressed data with the same length as the state data.
[0060] Step S120: The globally compressed data is weighted and then input into the attention feature extraction network. In the attention feature extraction network, the shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain the feature selection vector and the element attention matrix based on the shared features.
[0061] Step S130: Input the state data into the feature extraction network. In the feature extraction network, extract the fusion features of the state data. Multiply the fusion features with the element attention matrix to obtain the weighted features. Use multi-layer threshold temporal convolutional units to obtain temporal features based on the weighted features. Use multi-layer fully connected units to obtain the output features based on the temporal sequence.
[0062] Step S140: Based on the feature selection vector and output features obtained from multiple consecutive time points, the device's state is continuously predicted.
[0063] In this embodiment, a neural network model based on a global attention mechanism is proposed. In this model, the macroscopic state changes of the instance guide the model to effectively recognize the current state, thereby improving the model's feature extraction performance for target devices with individual differences, so as to achieve the goal of accurately predicting the state of the device.
[0064] In this embodiment, the method described herein can be applied to mechanical equipment, such as for continuous monitoring of engine status, and more specifically, for aircraft engines. When continuously monitoring the status of an aircraft engine, the observation data in different dimensions can be real-time continuous data received by temperature sensors and vibration frequency sensors located at different positions.
[0065] In step S100, multiple sensors of different types and functions, each receiving different signals, are used to monitor the target device, thereby obtaining observation data in different dimensions. All observation data are time-domain data and may include temperature, pressure, vibration, etc. It can be understood that one or more sensors of various types can be installed in different spatial locations or in different parts of the target device.
[0066] In this embodiment, the observation data is matrix data, consisting of multi-dimensional data corresponding to each moment within an observation period. Since the sampling intervals of different sensors often differ, this embodiment uses the average value of the signals from each sensor within a fixed time interval [tk,t] as the state description of the device at time t. Therefore, the observation data R... t m Can be derived from [x1] m x2 m ,…,x t m ] indicates that x i =[x t0 m x t1 m ,…,x ti m ] represents the descriptive variable of the i-th sensor at time t.
[0067] In step S110, the observation data first needs to be preprocessed, including data cleaning, data standardization, and data filtering.
[0068] Specifically, during the cleaning of the observation data, the following steps are performed sequentially: singular value limiting, missing data completion, and data alignment. The singular value limiting is performed using the following formula:
[0069]
[0070] In formula (1), x ij Represents the j-th component in the i-th mode. The corresponding normalized variable, and They represent what is determined by expert knowledge. Upper and lower limits of normal data distribution
[0071] Specifically, when standardizing the cleaned observation data, features with different working ranges are compressed into the range [0,1] to achieve uniformity and simplification of data distribution. The data standardization strategy is expressed as follows:
[0072]
[0073] In formula (2), x ij_max With x ij_min Then they represent The maximum and minimum values of the actual distribution.
[0074] Specifically, considering that there are various forms of noise in the actual working environment, it is necessary to select an appropriate filter based on the characteristics of the actual data to filter the data and eliminate noise of each variable in a specific frequency band.
[0075] Due to environmental and task-related factors, equipment often operates under multiple conditions. Under different operating modes, observed data will exhibit different shifts, resulting in different data distributions. These differently distributed data often overlap, causing confusion in the data distribution across different states. To eliminate this problem, this embodiment also incorporates measures such as... Figure 2 The data compensation strategy shown is used to eliminate data bias caused by mode switching in the observed data, thereby simplifying the data distribution. This step is implemented after data standardization, that is, before data filtering.
[0076] Specifically, such as Figure 2 As shown, in this data compensation strategy, the observed variables are further divided into control variables x. c and observed variable x o These two types of variables represent the machine's operating mode and working state, respectively. Since the normal working state of the machine is always stable within a single operating mode i, its control variables often do not experience data shifts due to equipment degradation. Therefore, formula (3) can be used to calculate the statistical center of the control variables corresponding to each operating mode, which serves as the representation R of this class. i o Then, the nearest neighbor strategy shown in formula (4) is used to determine the current working mode O of the device.
[0077]
[0078]
[0079] Similarly, the data bias corresponding to mode i It can also be calculated using formula (5). The monitoring data d after modal compensation t It can be finally expressed as formula (6).
[0080]
[0081]
[0082] Next, in step S110, a sliding window of a preset size is used to extract local data from the preprocessed observation data, which is used as the state data S at a certain moment. t =[d t-n+1 d t-n+2 , ..., d tSimultaneously, an equal-interval sampling strategy is used to compress the preprocessed observation data, resulting in globally compressed data S with the same length as the state data. t g When subsequently using neural networks to predict device status, feature extraction of the status data is guided by globally compressed data describing the macroscopic changing trends of the device.
[0083] In steps S120 and S130, the attention feature extraction network and the feature extraction network in the neural network model extract features from the global compressed data and the state data, respectively. The structures of the attention feature extraction network and the feature extraction network are referenced below. Figure 3 As shown.
[0084] In this embodiment, the attention feature extraction network includes a first convolutional unit, a first attention sub-branch unit, and a second attention sub-branch unit. The first convolutional unit includes two sequentially connected one-dimensional convolutional layers to extract shared features from the globally compressed data. Then, the first attention sub-branch unit adaptively scales the shared features and then passes them through a convolutional layer, an average pooling layer, and a fully connected layer to obtain a feature selection vector. Simultaneously, the second attention sub-branch unit obtains channel attention vectors and temporal attention vectors based on the shared features through a convolutional layer, an average pooling layer, and a fully connected layer, respectively. By multiplying the channel attention vectors and temporal attention vectors, an element-matrix with the same size as the fused feature is obtained.
[0085] Specifically, because in globally compressed data, the variable d t The closer the data collection time is to t, the higher its correlation with the current state of the target device. To improve the model's focus on recent trends in descriptive data, we first... t g Manual weighting is performed using the following formula:
[0086]
[0087]
[0088] Among them, f pe (i,p) represents the time code f pe The element at time position i and dimension p. pe (t) represents the d at time t. m Dimensional encoding features. f w Represents the global sample
[0089] As shown in equations (7) and (8), a temporal encoding f is added to the attention feature extraction network. peThen, the data encoding f is described by calculating the current time. pe (t) and S t g The encoding of each variable f in the middle pe (i) Calculate the current S using the cosine distance method. t With S t g The degree of correlation between the variables in the S-value is used as the basis for determining the S-value. t g The weights of each variable in the equation.
[0090] Specifically, in the first convolutional unit, two one-dimensional convolutional layers, one with a kernel size of 5 and the other with a kernel size of 3, sequentially obtain the feature F1. g and feature F2 g At this time, feature F2 g For shared features.
[0091] Specifically, the first attention sub-branch unit first adaptively scales the shared features using the following formula:
[0092]
[0093] Furthermore, shared features will be adaptively scaled. The feature selection vector F is obtained through a convolutional layer with a kernel size of 3, an average pooling layer, and two fully connected layers. s .
[0094] Specifically, when the second attention sub-branch unit extracts the channel attention vector, it sequentially passes the shared feature F2 through a convolutional layer with a kernel of 3, an average pooling layer, and two fully connected layers. g Processing is performed. When extracting the temporal attention vector, the shared feature F2 is sequentially processed through a convolutional layer with a kernel of 3, an average pooling layer, and two fully connected layers. g The process is then performed. Finally, the extracted channel attention vector and temporal attention vector are multiplied together to obtain the element-wise attention matrix F. a .
[0095] In this embodiment, the feature extraction network includes a second convolutional unit and a threshold temporal convolutional unit. The second convolutional unit includes two convolutional layers with a kernel size of one connected in sequence. It extracts features from the state data to obtain a fused feature F2. The fused feature is multiplied by the element attention matrix to obtain a weighted feature. The weighted feature is then input into the threshold temporal convolutional unit. The threshold temporal convolutional unit includes three threshold temporal convolutional layers. A feature selection mechanism is set in each threshold temporal convolutional layer. The weighted feature is extracted sequentially through the three threshold temporal convolutional layers to obtain the output feature.
[0096] In this embodiment, the kernel sizes of the three threshold temporal convolutional layers are 7, 5, and 5. In fact, the output of the threshold temporal convolutional unit is matrix data. Therefore, the data corresponding to the latest time step, i.e., the representation of the target device's state at time t, is extracted as the output feature.
[0097] In this embodiment, to improve the feature extraction capability of the temporal convolutional network, further improvements were made to the temporal convolutional network. The structure of the improved threshold temporal convolutional unit is as follows: Figure 4 As shown.
[0098] Specifically, the input data for each threshold temporal convolutional layer is a feature sequence corresponding to multiple consecutive time points, where... Figure 4 The parameter x in the equation represents the fused feature F2. In each thresholded temporal convolutional layer, the first temporary feature is extracted based on the feature data corresponding to the last time step in the feature sequence, and the second temporary feature is extracted based on all feature data in the feature sequence except for the feature data corresponding to the last time step. A feature selection mechanism is adopted to achieve adaptive feature selection of the first and second temporary features, and the output data of the current thresholded temporal convolutional layer is obtained.
[0099] Furthermore, in each layer of features f t i In the calculation process, the model first uses a size of n k -1 and 1's 1-D convolution kernels respectively affect f t-nk+1 i-1 ~f t-1 i-1 with f t i-1 Feature extraction is performed to obtain temporary features f. ht i with f tt i Then a fully connected layer is used, based on feature f t i Calculate the attention vector f a i To achieve f ht i with f tt i The adaptive feature selection employs a feature selection mechanism. Finally, the model uses the representation f of output layer i at time t. t 1 As the final representation of the equipment state, it is used in subsequent state prediction.
[0100] In this embodiment, the feature selection mechanism is represented as follows:
[0101]
[0102] In formula (10), This represents the output data of the i-th threshold temporal convolutional layer. This represents the attention vector calculated based on the first temporary feature. This indicates the first temporary feature. This indicates the second temporary feature.
[0103] In this embodiment, a device state prediction network is constructed based on the feature extraction network and the attention feature extraction network. When training the device state prediction network, the training data in the training dataset is sorted from simple to complex according to the prediction difficulty, and the training data in the training dataset is input into the device state prediction network for training in sequence.
[0104] Specifically, when training the entire equipment status prediction network, a corresponding loss function is formulated based on the actual task. For the prediction task, the mean squared error of the prediction results can be used as the regression loss to guide model training.
[0105] During training, the training samples are typically of varying difficulty. To accelerate model training, the patent employs a curriculum learning technique, gradually increasing the learning difficulty according to the order of samples from simple to complex, thereby enabling the model to effectively learn from simple to complex patterns.
[0106] In one embodiment, the training process algorithm is shown in Algorithm 1.
[0107]
[0108] First, it is necessary to develop a metric to measure the learning difficulty of the samples, and assign samples from 0 to L in order of increasing difficulty. max The difficulty label. Secondly, select training samples with relatively simple patterns to construct a temporary training set D. s The model is trained to extract typical features of the monitored objects. Then, the training samples are ranked according to their training difficulty, from easiest to hardest, towards D. s Add samples to gradually improve the model's feature extraction capabilities.
[0109]
[0110] In formula (11), L upper This is a preset value, slightly larger than the largest difficulty label L in the training data. max (L) upper -L range η defines the range of training data in the first training epoch. η controls D. s The expansion rate, its setting needs to satisfy: Lupper -e -η L range >L max .
[0111] In step S140, the feature selection vector and output features are input into the fully connected layer to predict the device state. The output is a prediction score, which should be within a preset range and the range extreme value is set. Different prediction scores represent different degrees of device state.
[0112] In the aforementioned method for continuous device state prediction based on heterogeneous sensing data, observation data received from multiple sensors regarding the same device is preprocessed. State data and globally compressed data are then extracted from these data. Attention feature extraction networks and feature extraction networks are used to extract feature selection vectors and output features, respectively. The globally compressed data represents the macroscopic state changes of the observation data, while the state data represents the specific observation data at a given moment. Finally, the device state is predicted based on the feature selection vectors and output features. This method uses the macroscopic state changes of instances to guide the model's effective understanding of the current state, thereby improving the model's feature extraction performance for entities with individual differences, and ultimately achieving accurate device state prediction.
[0113] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
[0114] In one embodiment, such as Figure 5 As shown, a device state prediction apparatus based on multi-dimensional sensor data is provided, comprising: an observation data acquisition module 200, a state data and global compressed data acquisition module 210, an attention feature extraction module 220, a feature extraction module 230, and a state prediction module 240, wherein:
[0115] The observation data acquisition module 200 is used to acquire observation data related to the device to be predicted in the state, wherein the observation data is time-domain data obtained from multiple sensors.
[0116] The state data and global compressed data acquisition module 210 is used to preprocess the observation data, extract state data at a certain moment based on the preprocessed observation data, and downsample the observation data to obtain global compressed data with the same length as the state data.
[0117] The attention feature extraction module 220 is used to input the globally compressed data into the attention feature extraction network after data weighting. In the attention feature extraction network, the shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain the feature selection vector and the element attention matrix based on the shared features.
[0118] Feature extraction module 230 is used to input the state data into the feature extraction network, extract the fusion features of the state data in the feature extraction network, multiply the fusion features with the element attention matrix to obtain weighted features, use multi-layer threshold temporal convolutional units to obtain temporal features based on the weighted features, and use multi-layer fully connected units to obtain output features based on the temporal features.
[0119] The state prediction module 240 is used to continuously predict the state of the device based on the feature selection vector and output features obtained from multiple consecutive time points.
[0120] Specific limitations regarding the device for continuous prediction of device status based on heterogeneous sensing data can be found in the limitations of the method for continuous prediction of device status based on heterogeneous sensing data described above, and will not be repeated here. Each module in the aforementioned device for continuous prediction of device status based on heterogeneous sensing data can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0121] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a method for continuous prediction of device state based on heterogeneous sensing data. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.
[0122] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0123] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0124] Acquire observation data related to the device to be predicted in state, wherein the observation data is time-domain data obtained from multiple sensors;
[0125] The observation data is preprocessed, and state data at a certain moment is extracted from the preprocessed observation data. The observation data is then downsampled to obtain globally compressed data with the same length as the state data.
[0126] The globally compressed data is weighted and then input into an attention feature extraction network. In the attention feature extraction network, shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain feature selection vectors and element attention matrices based on the shared features.
[0127] The state data is input into a feature extraction network. In the feature extraction network, the fusion features of the state data are extracted. The fusion features are multiplied by the element attention matrix to obtain weighted features. Multi-layer threshold temporal convolutional units are used to obtain temporal features based on the weighted features. Multi-layer fully connected units are used to obtain output features based on the temporal features.
[0128] The device's state is continuously predicted based on the feature selection vector and output features obtained from multiple consecutive time points.
[0129] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0130] Acquire observation data related to the device to be predicted in state, wherein the observation data is time-domain data obtained from multiple sensors;
[0131] The observation data is preprocessed, and state data at a certain moment is extracted from the preprocessed observation data. The observation data is then downsampled to obtain globally compressed data with the same length as the state data.
[0132] The globally compressed data is weighted and then input into an attention feature extraction network. In the attention feature extraction network, shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain feature selection vectors and element attention matrices based on the shared features.
[0133] The state data is input into a feature extraction network. In the feature extraction network, the fusion features of the state data are extracted. The fusion features are multiplied by the element attention matrix to obtain weighted features. Multi-layer threshold temporal convolutional units are used to obtain temporal features based on the weighted features. Multi-layer fully connected units are used to obtain output features based on the temporal features.
[0134] The device's state is continuously predicted based on the feature selection vector and output features obtained from multiple consecutive time points.
[0135] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0136] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0137] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A method for continuous prediction of device status based on heterogeneous sensing data, characterized in that, The method includes: Acquire observational data related to the device to be predicted in its state. The observational data consists of time-domain data obtained from multiple sensors, including temperature, pressure, and vibration. The observation data is preprocessed, and state data at a certain moment is extracted from the preprocessed observation data. The observation data is then downsampled to obtain globally compressed data with the same length as the state data. The globally compressed data is weighted and then input into an attention feature extraction network. In this network, shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain a feature selection vector and an element-based attention matrix based on these shared features. The attention feature extraction network includes a first convolutional unit, a first attention sub-branch unit, and a second attention sub-branch unit. The first convolutional unit includes two sequentially connected one-dimensional convolutional layers to extract the shared features from the globally compressed data. The first attention sub-branch unit adaptively scales the shared features and then passes them through a convolutional layer, an average pooling layer, and a fully connected layer to obtain the feature selection vector. The second attention sub-branch unit obtains a channel attention vector and a temporal attention vector based on the shared features through a convolutional layer, an average pooling layer, and a fully connected layer. The channel attention vector and the temporal attention vector are then multiplied together to obtain the element-based attention matrix, which has the same size as the fused feature. The state data is input into a feature extraction network. In this network, fused features of the state data are extracted. These fused features are multiplied by the element-wise attention matrix to obtain weighted features. Multi-layer thresholded temporal convolutional units are used to obtain temporal features based on these weighted features. Finally, multi-layer fully connected units are used to obtain output features based on these temporal features. The feature extraction network includes a second convolutional unit and a thresholded temporal convolutional unit. The second convolutional unit comprises two sequentially connected convolutional layers with a kernel size of one. It extracts features from the state data to obtain the fused features. These fused features are multiplied by the element-wise attention matrix to obtain weighted features. The weighted features are then input into the thresholded temporal convolutional unit, which comprises three thresholded temporal convolutional layers. Each thresholded temporal convolutional layer has a feature selection mechanism. The three thresholded temporal convolutional layers sequentially extract features from the weighted features to obtain the output features. The device's state is continuously predicted based on the feature selection vector and output features obtained from multiple consecutive time points.
2. The method for continuous prediction of equipment status according to claim 1, characterized in that, The preprocessing of the observation data includes: sequentially performing data cleaning, data standardization, and data filtering on the observation data.
3. The method for continuous prediction of equipment status according to claim 1, characterized in that, A sliding window of a preset size is used to extract local data from the preprocessed observation data, which is then used as the device's status data at a certain moment. The preprocessed observation data is compressed using an equal-interval sampling strategy to obtain globally compressed data with the same length as the state data.
4. The method for continuous prediction of equipment status according to claim 1, characterized in that, The input data for each threshold temporal convolutional layer is a feature sequence corresponding to multiple consecutive time points. In each threshold temporal convolutional layer: The first temporary feature is extracted based on the feature data corresponding to the latest time in the feature sequence; The second temporary feature is obtained by extracting all feature data in the feature sequence except for the feature data corresponding to the latest time. The aforementioned feature selection mechanism is used to achieve adaptive feature selection of the first temporary feature and the second temporary feature, thereby obtaining the output data of the current threshold temporal convolutional layer.
5. The method for continuous prediction of equipment status according to claim 4, characterized in that, The feature selection mechanism is expressed as follows: In the above formula, Indicates the first Output data of a threshold-based temporal convolutional layer. This represents the attention vector calculated based on the first temporary feature. This indicates the first temporary feature. This indicates the second temporary feature.
6. The method for continuous prediction of equipment status according to any one of claims 1-5, characterized in that, A device state prediction network is constructed based on the feature extraction network and the attention feature extraction network. During the training of the device state prediction network: The training data in the training dataset are sorted from simple to complex according to prediction difficulty, and then the training data in the training dataset are sequentially input into the device state prediction network for training.
7. A device for continuous prediction of equipment status based on heterogeneous sensing data, characterized in that, The apparatus implements the continuous equipment state prediction method according to any one of claims 1-6, and the apparatus comprises: The observation data acquisition module is used to acquire continuous observation data related to the device to be predicted in the state, wherein the observation data is time-domain data obtained from multiple sensors. The state data and global compressed data acquisition module is used to preprocess the observation data, extract state data at a certain moment based on the preprocessed observation data, and downsample the observation data to obtain global compressed data with the same length as the state data. The attention feature extraction module is used to input the globally compressed data into the attention feature extraction network after data weighting. In the attention feature extraction network, the shared features of the globally compressed data are extracted. Two attention sub-branch units are used to obtain the feature selection vector and the element attention matrix based on the shared features. The feature extraction module is used to input the state data into the feature extraction network, extract the fusion features of the state data in the feature extraction network, multiply the fusion features with the element attention matrix to obtain weighted features, use multi-layer threshold temporal convolutional units to obtain temporal features based on the weighted features, and use multi-layer fully connected units to obtain output features based on the temporal features. The state prediction module is used to continuously predict the state of the device based on the feature selection vector and output features obtained from multiple consecutive time points.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-6.