An intelligent health management system and display console that introduces attention mechanism

Abstract

The present invention provides an intelligent health management system and a display and control console that introduce an attention mechanism, and constructs a health assessment and fault prediction model based on CNN, LSTM and attention mechanism to achieve real-time assessment of the health status of the display and control console, prediction of common fault risks and trend prediction of key parameters. The model is an end-to-end multi-task model based on deep learning, which uses CNN convolutional neural network, bidirectional LSTM and multi-head attention mechanism to extract shared features, and then creates independent parallel branches for health assessment, fault risk prediction and key parameter trend prediction, and outputs the health status of the display and control console, the predicted probability of fault occurrence and the trend of key parameters in the future.

Description

Technical Field

[0001] The present invention relates to the field of health management of display and control consoles, and specifically proposes an intelligent health management system for display and control consoles and a display and control console, aiming to improve the intelligence level and efficiency of health management of display and control consoles. Background Art

[0002] The display and control console integrates the display terminal and the control equipment, and is one of the important ship combat equipment. Its operating status is a key factor affecting the normal operation of the entire ship combat system. However, the display and control console may sometimes experience system overheating or communication interruption during operation, affecting the stable operation of the entire system. At the same time, most users are non-professional technicians, who do not know much about the display and control console and cannot intuitively understand the health status of the display and control console. Professional maintenance personnel are also required to conduct on-site troubleshooting and positioning, which is time-consuming and labor-intensive and seriously affects maintenance efficiency. Therefore, the health management of the operating status of the display and control console is an important technical problem to be solved by technical personnel in this field.

[0003] Traditional health management methods for display and control consoles mainly rely on regular inspections and passive responses, which have problems such as failure to detect potential faults in a timely manner, increased costs due to excessive maintenance, lack of real-time comprehensive evaluation, and inability to predict future performance trends. In order to solve these problems, fault prognosis and health management (PHM) technology has received widespread attention. However, traditional PHM methods still face challenges such as high data dimensionality, complex physical modeling, scarce fault samples, and difficulty in unified modeling for multi-task prediction when dealing with complex electronic devices.

[0004] With the rapid development of artificial intelligence technology, deep learning provides a new way to solve the above problems. Deep learning models such as convolutional neural networks (CNN) and long short-term memory networks (LSTM) have shown great potential in equipment status monitoring, fault diagnosis and performance prediction due to their powerful feature learning and pattern recognition capabilities. Summary of the invention

[0005] The purpose of the present invention is to provide an intelligent health management system for a display and control console that introduces an attention mechanism, construct a health assessment and fault prediction model based on CNN, LSTM and attention mechanism, and realize real-time assessment of the health status of the display and control console, prediction of common fault risks and trend prediction of key parameters.

[0006] The specific technical solutions are as follows:

[0007] An intelligent health management system that introduces an attention mechanism includes a health data collection module, a real-time status monitoring module, an information storage module, a system configuration and feedback module, and also includes a real-time health assessment, a fault risk prediction, and a key parameter trend prediction module. The module is an end-to-end multi-task model based on deep learning. The model takes the preprocessed data collected by the health data collection module as input, and simultaneously outputs the health status of the display console, the predicted probability of fault occurrence, and the change trend of key parameters in the future time; the model uses CNN convolutional neural network, bidirectional LSTM and multi-head attention mechanism to extract shared features in turn, and then creates independent parallel branches for health assessment, fault risk prediction, and key parameter trend prediction, and each branch selects a different number of neurons and activation function according to the characteristics of the task.

[0008] Furthermore, the model working process includes:

[0009] The input layer receives multidimensional time series data X;

[0010] The 1D CNN layer uses a one-dimensional convolutional neural network to capture local temporal patterns;

[0011] The bidirectional LSTM layer processes the output H1 of the CNN layer and learns long-term temporal dependencies. The output of the bidirectional LSTM is the concatenation of the forward and reverse LSTMs H2.

[0012] The attention mechanism first maps the input H2 into three different spaces, namely the query space Q, the key space K, and the value space V, through a linear transformation; then the attention weights are calculated: Among them, d k is the dimension of the key vector; finally, residual connection and layer normalization are performed, and the formula is expressed as follows,

[0013] H3 = LayerNorm(H2 + A);

[0014] Global average pooling, summing the contents from 1 to T in H3, compressing the features in the time dimension into a fixed-length vector H4;

[0015] Vector H4 is input into each task branch to complete the real-time health assessment, fault risk prediction, and key parameter trend prediction respectively.

[0016] Furthermore, the health score branch uses a fully connected layer to map the global features to a single health score;

[0017] The fault risk prediction branch uses the sigmoid activation function to output the risk probabilities of multiple fault types;

[0018] The key parameter trend prediction branch uses another bidirectional LSTM to process the key parameter trend prediction task and generate a key parameter trend prediction sequence within the future Z time.

[0019] Furthermore, the total loss function is the weighted sum of the three task losses.

[0020] Furthermore, the health data collection module is used to collect health data of each component device of the display and control console; the real-time status monitoring module is responsible for the real-time visualization of the original health data of each device collected by the health data collection module, and performs real-time display of health event data according to each device; the information storage module is used to store the pre-processed data, model parameters, model prediction results and actual observation results data collected by the health data collection module in a recent period of time, as well as user replacement and maintenance operation records; the system configuration and feedback module is used to configure the system sampling time, sampling frequency, health data events, fault risk prediction type, key parameters, model update conditions, and human-computer collaborative feedback, allowing operators to provide feedback on the model prediction results for iterative model updates.

[0021] A display and control console that introduces an attention mechanism, which includes a computer unit, a display unit and a control unit; the computer unit is composed of one or more reinforced computers or all-in-one machines, one of which is used for main processing and is the core processing unit of the display and control console, used for health data processing, analysis and storage, and the computers in the unit are electrically connected to each other and can communicate with each other. In addition, this unit is also electrically connected to the devices of the display unit and the control unit to ensure the operation of the display and control console functions; the display unit is composed of a separate display or a display on an all-in-one machine, which is used for real-time display of health status; the control unit is composed of a series of external devices, which are used to operate and control the display and control console; the intelligent health management system is deployed on the computer responsible for main processing to realize the communication status monitoring between the various devices of the display and control console, the health information monitoring of each device itself, and the online status monitoring of peripherals.

[0022] Beneficial Effects

[0023] 1. Multi-task learning improves efficiency: The system uses an end-to-end multi-task model based on deep learning to simultaneously output health status scores, fault risk predictions, and key parameter trend predictions. This multi-task learning method not only improves computational efficiency, but also prevents overfitting in the case of limited data, thereby improving overall performance.

[0024] 2. Attention mechanism enhances feature extraction: The introduction of the attention mechanism enables the model to focus on the most relevant parts of the sequence, improving the ability to capture important information. This helps to more accurately assess health status, predict failure risks, and predict trends in key parameters, especially when processing long time series data.

[0025] 3. Real-time monitoring and prediction: The system can collect and process the health data of each device on the display console in real time and display it in real time. This enables operators to understand the system status in a timely manner and respond quickly to potential problems, thus improving the reliability and safety of the display console.

[0026] 4. Adaptive model update: By regularly evaluating model performance and updating as needed, the system can adapt to changes in the console's operating environment and emerging failure modes. This adaptive capability ensures that the system remains efficient and accurate over the long term.

[0027] 5. Human-machine collaborative optimization: The system allows operators to provide feedback on model prediction results and incorporate these feedbacks into subsequent model updates. This human-machine collaborative mechanism can not only continuously improve model performance, but also combine the experience and knowledge of human experts to improve the credibility and practicality of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is the model block diagram of the real-time health assessment, fault risk prediction, and key parameter trend prediction modules.

[0029] Figure 2 This is a flow chart of an intelligent health management system for a display console that introduces an attention mechanism. DETAILED DESCRIPTION

[0030] The present invention is achieved through the following technical solutions.

[0031] Wherein, the display and control console at least includes a computer unit, a display unit and a control unit;

[0032] The computer unit is composed of one or more reinforced computers or all-in-one computers, one of which is used for main processing and is the core processing unit of the display and control console, used for health data processing, analysis and storage. The computers in the unit are electrically connected to each other and can communicate with each other. In addition, this unit is also electrically connected to the equipment of the display unit and the control unit to ensure the operation of the display and control console functions;

[0033] The display unit consists of a separate display or an all-in-one display, which is used to display the health status in real time; the control unit consists of a series of external devices, such as a mouse and keyboard, which are used to operate and control the display console.

[0034] An intelligent health management system for a display and control console that introduces an attention mechanism is deployed on the entire display and control console, involving all the devices included in the display and control console. Its core module is deployed on the computer responsible for the main processing in the computer unit of the display and control console. The health management system for the display and control console can realize the communication status monitoring between the devices of the display and control console, the health information monitoring of each device itself, and the online status monitoring of peripherals. The system includes multiple modules such as health data collection, real-time status monitoring, real-time health assessment, fault risk prediction, key parameter trend prediction, information storage, and system configuration and feedback.

[0035] The health data collection module collects the health data of each component device of the display and control console. It is the source of basic data for the entire health management system. Its collection method is divided into two methods according to the type of communication between devices: each device actively reports at regular intervals and the main processor obtains at regular intervals. If there is an Ethernet port, serial port, or CAN port communication between the device and the electronic chassis unit, the device will actively report its own health information at regular intervals. If the device is a USB slave device of the electronic chassis unit, the health information of the device will be obtained at regular intervals through the main processor. The health information data table of each device consists of two parts: the device ID and the health information event table. The device ID is used to distinguish the device that reports information. The health information event table is used to store information on the device's operating status, which is generally the key parameter information for the device's operation. Each device can customize its own health information events according to the category. For example, the health information events of computer devices may include CPU temperature, CPU occupancy, memory occupancy, disk occupancy, etc.; the health information events of simple peripherals (mouse, keyboard) may include online status; the health information events of complex peripherals may include voltage, current, operating temperature, etc.; in order to monitor the communication status of the display and control console's external interface, additional health information events of the communication interface can be set, such as Ethernet link status, CAN link status, serial port communication status, etc.

[0036] The real-time status monitoring module is responsible for the real-time visualization of the original health data of each device collected by the health data collection module, and for the real-time display of health event data according to each module.

[0037] The real-time health assessment, fault risk prediction, and key parameter trend prediction modules are the core units of the entire intelligent health management system. They mainly include a health status assessment and fault prediction model that introduces an attention mechanism. The model is an end-to-end multi-task model based on deep learning. The model takes the preprocessed health data acquisition module data as input, and outputs the health status of the display console (float type data between 0-1, 0 indicates complete failure, 1 indicates health, and the closer the value is to 1, the healthier the status), fault risk prediction (the probability of each custom fault type), and key parameter trend prediction (the trend of key parameters in the future Z time). The data preprocessing process of the health data acquisition module is as follows: (1) Missing value processing: In order to maintain the temporal continuity of the data, the forward filling method is used to process the missing values ​​of all features; (2) Outlier processing: The IQR method is used to detect outliers and limit them to a reasonable range. Values ​​outside the range are regarded as outliers and deleted; (3) Data standardization: Z-score standardization is applied to all features so that all features have the same scale (mean is 0 and standard deviation is 1). The model architecture is as follows: Figure 1 As shown in the figure. First, CNN convolutional neural network, bidirectional LSTM and multi-head attention mechanism are used to extract shared features. Then, independent parallel branches are created for each task (health status scoring, fault risk prediction, key parameter trend prediction). Each branch selects a different number of neurons and activation functions according to the characteristics of the task. The shared feature extraction layer reduces the overall number of parameters and improves computational efficiency. At the same time, this architecture also helps prevent overfitting, especially when data is limited.

[0038] Specifically, the main components of a model for health status assessment and fault prediction that introduces an attention mechanism are as follows:

[0039] (1) Input layer: Receive multidimensional time series data X∈R^(T×D), where T is the sequence length, T=sampling frequency×sampling time, and D is the feature dimension, including but not limited to CPU temperature, display console operating temperature and humidity, voltage and current of each module, CPU occupancy rate, etc.

[0040] (2) 1D CNN layer: A one-dimensional convolutional neural network is used to capture local temporal patterns and effectively extract short-term features. Its operation can be expressed as: H1 = ReLU(W1*X+b1), where * represents the convolution operation, W1 and b1 are the convolution kernel and bias, respectively.

[0041] (3) Bidirectional LSTM layer: processes the output of the CNN layer and learns long-term temporal dependencies. For time steps t = 1 to T (forward LSTM), it calculates:

[0042] For time steps t=T to 1 (reverse LSTM) calculation:

[0043] The output of the bidirectional LSTM is the concatenation of the forward and backward LSTMs. Where ⊕ represents a vector concatenation operation, which provides richer feature representations for subsequent attention layers and task-specific branches.

[0044] (4) Attention layer: The attention mechanism allows the model to focus on the most relevant parts of the sequence. First, the model maps the input H2 into three different spaces through linear transformation: query space (Q = W Q H2), key space (K = W K H2) and value space (V=W V H2), this mapping allows the model to understand the input data from different perspectives; then calculate the attention weights: Among them, d k Is the dimension of the key vector, and finally performs residual connection and layer normalization: H3 = LayerNorm(H2+A). The output A of the attention is added to the original input H2 to form a residual connection, which helps the lossless transmission of information and the reflux of gradients. LayerNorm is applied to the residual sum, which stabilizes the training process by normalizing the features of each sample and reduces internal covariate shift.

[0045] (5) Global average pooling: Among them, H 3,t Represents a slice of h3 at time step t, sums the contents of t from 1 to T in H3, and compresses the features in the time dimension into a fixed-length vector H4. This provides a compact global feature representation for subsequent task-specific layers, which helps the model make overall judgments.

[0046] (6) Task-specific branches: a. Health score branch: y health =W health H4+b health , this branch uses a fully connected layer to map global features to a single health score, which provides a quantitative assessment of the overall health status of the console. b. Fault risk prediction branch: y fault =σ(W fault H4+b fault ), the fault risk prediction branch uses the sigmoid activation function to output the risk probabilities of multiple fault types, which helps to identify potential fault risks in a timely manner. c. Key parameter trend prediction branch: This branch uses another bidirectional LSTM to specifically handle the key parameter trend prediction task. y kp,t =W kp h kp,t +b kp It uses the features extracted in the previous layer to generate the key parameter trend prediction sequence y within the future Z time kp =[y kp,1 ,y kp,2 ,...,y kp,z ].

[0047] (7) The total loss function is the weighted sum of the three task losses:

[0048]

[0049] The information storage module is used to store the pre-processed data, model parameters, model prediction results and actual observation results data collected by the health data collection module in the recent period, as well as the user's replacement and maintenance operation records.

[0050] The system configuration module and feedback module are used to configure the system sampling time, sampling frequency, health data events, fault risk prediction type, key parameters, model update conditions (such as setting a model update cycle (such as weekly or monthly), or triggering an update when the model performance drops below a preset threshold), and human-computer collaborative feedback, allowing operators to provide feedback on the model prediction results (such as marking false positives or false negatives) to facilitate iterative model updates.

[0051] The operation process of the intelligent health management system for the display and control console that introduces the attention mechanism is as follows:

[0052] (1) The user sets the system sampling time, sampling frequency, health data events, fault risk prediction type, and key parameters through the system configuration module;

[0053] (2) Health data collection: The module that actively reports health data reports its own health information to the main processing computer through the network, CAN and other communication methods according to the sampling frequency and health data events configured by the system; the main processing computer also obtains the health data of other modules at a regular interval.

[0054] (3) visualizing the raw data collected in step (2) in real time through a user interface;

[0055] (4) Preprocessing the data in step (2): missing value processing, outlier processing, and data standardization;

[0056] (5) When the health data collection time reaches the configured sampling time, the data preprocessed in step (4) is input into the health status assessment and fault prediction model. The model simultaneously outputs the health status score, the probability of predicted fault risk type occurrence, and the trend prediction of key parameters, and visualizes them in the form of numerical values ​​and charts through the user interface.

[0057] (6) Model performance evaluation and data accumulation:

[0058] a. Compare the model’s predictions (health status score, failure risk probability, key parameter trends) with the actual observed system performance.

[0059] b. Calculate the performance indicators of the model on each task (such as mean square error, accuracy, F1 score, etc.).

[0060] c. The preprocessed data, model prediction results and actual observation results of step (4) are stored in the historical database through the information storage module.

[0061] (7) Regular model updates:

[0062] a. Set a model update cycle (such as weekly or monthly), or trigger an update when the model performance drops below a preset threshold.

[0063] b. Extract the latest data set from the historical database, including data from the most recent period and some previous historical data.

[0064] c. Retrain the model using this updated dataset:

[0065] d. Evaluate the updated model performance on an independent validation set.

[0066] e. If performance improves, deploy the updated model; otherwise, keep the original model and log the update attempt.

[0067] (8) Human-machine collaboration feedback loop:

[0068] a. Allow operators to provide feedback on model predictions (e.g. flag false positives or false negatives).

[0069] b. Incorporate these manual feedback into the next model update to improve the model's prediction accuracy.

[0070] The following is combined with reference Figure 2 It should be understood that the embodiments shown and described in the drawings are only exemplary and are intended to explain the principles and spirit of the present invention, rather than to limit the scope of the present invention.

[0071] Taking the intelligent health management process of a display and control console as an example, the intelligent health management system of the display and control console that introduces the attention mechanism of this embodiment is described in detail, where the display and control console is composed of: computer unit: 3 reinforced computers (1 main processing computer and 2 auxiliary computers); display unit: 2 24-inch high-resolution displays; control unit: 2 sets of keyboard and mouse combinations, 1 touch pad.

[0072] (1) System initialization

[0073] Set the following sampling frequency, sampling time, fault type, key parameters, and model update conditions through the system configuration module:

[0074] a. Sampling frequency: collect data every 5 seconds;

[0075] b. Sampling time: 30 minutes (i.e. 360 data points);

[0076] c. Fault type: CPU overheating, memory overflow, hard disk failure, network connection interruption, display failure, input device unresponsiveness;

[0077] d. Key parameter settings: CPU usage (%), memory usage (%), network throughput (Mbps), system response time (ms)

[0078] e. Model update method: Regular update: The model update process is automatically triggered at 2:00 a.m. every Sunday; Performance-triggered update: When the average F1 score of the model on the validation set is lower than 0.85, the update is triggered;

[0079] Update the dataset: use data from the last 4 weeks and randomly sample 20% of historical data; Verification standard: the average F1 score of the new model on the independent validation set must be improved by at least 1% before it is deployed.

[0080] (2) Data collection:

[0081] The main processing computer obtains the online status of the input device of the control unit through the USB interface every 5 seconds;

[0082] The display and the other two computers actively report their health data through the network interface every 5 seconds;

[0083] Environmental sensors (temperature, humidity) send data every 5 seconds via the CAN bus.

[0084] (3) Data preprocessing:

[0085] A total of 360 data points collected within 30 minutes were processed, missing values ​​were handled using the forward filling method, outliers were detected and handled using the IQR method, and Z-score normalization was applied.

[0086] (4) Model input:

[0087] The preprocessed data is input into the health status assessment and fault prediction model. The model architecture is as follows: 1DCNN layer: 32 convolution kernels of size 3 are used; bidirectional LSTM layer: 64 hidden units; multi-head attention layer: 4 attention heads; global average pooling layer; task-specific branches: a. Health score branch: 2 layers of fully connected layers (64 nodes and 1 node), b. Fault risk prediction branch: 2 layers of fully connected layers (64 nodes and 6 nodes, corresponding to 6 types of faults), c. Key parameter trend prediction branch: 1 layer of bidirectional LSTM (32 hidden units) and 1 layer of fully connected layer.

[0088] (5) Model output and visualization:

[0089] Health status score: a floating point number between 0 and 1, displayed in the form of a dashboard;

[0090] Failure risk prediction: The probability of 6 types of failures is displayed in a bar chart;

[0091] Key parameter trend: predict the trend in the next hour, displayed in a line chart;

[0092] (6) Model update:

[0093] The update process is automatically triggered at 2:00 am every Sunday. The data of the last 4 weeks and 20% of random historical data are extracted from the database. The Adam optimizer is used with a learning rate of 0.001 and a batch size of 64 for 50 epochs. The model performance is evaluated on the validation set, and a new model is deployed if the average F1 score improves by more than 1%.

[0094] (7) Human-machine collaboration feedback:

[0095] Operators can flag false positives or false negatives through the interface, and this feedback will be recorded and used to adjust the weights of the training data during the next model update.

Claims

1. An intelligent health management system that introduces an attention mechanism, including a health data collection module, a real-time status monitoring module, an information storage module, and a system configuration and feedback module, characterized in that: It also includes real-time health assessment, fault risk prediction, and key parameter trend prediction modules. This module is an end-to-end multi-task model based on deep learning. The model takes the pre-processed data collected by the health data collection module as input, and outputs the health status of the display console, the predicted probability of fault occurrence, and the change trend of key parameters in the future. The model uses CNN convolutional neural network, bidirectional LSTM and multi-head attention mechanism to extract shared features in sequence, and then creates independent parallel branches for health assessment, fault risk prediction and key parameter trend prediction. Each branch selects a different number of neurons and activation functions according to the characteristics of the task.

2. According to claim 1, an intelligent health management system that introduces an attention mechanism is characterized in that: The model working process includes: The input layer receives multidimensional time series data X; The 1D CNN layer uses a one-dimensional convolutional neural network to capture local temporal patterns; The bidirectional LSTM layer processes the output H1 of the CNN layer and learns long-term temporal dependencies. The output of the bidirectional LSTM is the concatenation of the forward and reverse LSTMs H2. The attention mechanism first maps the input H2 into three different spaces, namely the query space Q, the key space K, and the value space V, through a linear transformation; then the attention weights are calculated: Among them, d k is the dimension of the key vector; finally, residual connection and layer normalization are performed, and the formula is expressed as follows, H3 = LayerNorm(H2 + A); Global average pooling, summing the contents from 1 to T in H3, compressing the features in the time dimension into a fixed-length vector H4; Vector H4 is input into each task branch to complete the real-time health assessment, fault risk prediction, and key parameter trend prediction respectively.

3. According to claim 1, the intelligent health management system introducing the attention mechanism is characterized in that: The health score branch uses a fully connected layer to map global features to a single health score; The fault risk prediction branch uses the sigmoid activation function to output the risk probabilities of multiple fault types; The key parameter trend prediction branch uses another bidirectional LSTM to process the key parameter trend prediction task and generate a key parameter trend prediction sequence within the future Z time.

4. The intelligent health management system for display and control consoles that introduces an attention mechanism according to claim 1 is characterized in that: The total loss function is the weighted sum of the three task losses.

5. An intelligent health management system introducing an attention mechanism according to any one of claims 1 to 4, characterized in that: The health data collection module is used to collect the health data of each component device of the display and control console; the real-time status monitoring module is responsible for the real-time visualization of the original health data of each device collected by the health data collection module, and performs real-time display of health event data according to each device; The information storage module is used to store the pre-processed data, model parameters, model prediction results and actual observation results data collected by the health data acquisition module in the recent period, as well as the user's replacement and maintenance operation records; the system configuration and feedback module is used to configure the system sampling time, sampling frequency, health data events, fault risk prediction type, key parameters, model update conditions, and human-computer collaboration feedback, allowing operators to provide feedback on the model prediction results for iterative model updates.

6. A display console that introduces an attention mechanism, characterized in that: in, It includes a computer unit, a display unit and a control unit; the computer unit is composed of one or more reinforced computers or all-in-one machines, one of which is used for main processing and is the core processing unit of the display and control console, used for health data processing, analysis and storage. The computers in the unit are electrically connected to each other and can communicate with each other. In addition, this unit is also electrically connected to the equipment of the display unit and the control unit to ensure the operation of the display and control console functions; the display unit is composed of a separate display or a display on an all-in-one machine, which is used for real-time display of health status; the control unit is composed of a series of external devices, which are used to operate and control the display and control console; the intelligent health management system is deployed on the computer responsible for main processing to realize the communication status monitoring between the various devices on the display and control console, the health information monitoring of each device itself, and the online status monitoring of peripherals.

Images