Sensor fault identification method and system based on SAC reinforcement learning, and medium

Abstract

The invention provides a sensor fault identification method and system based on SAC reinforcement learning, and a medium. The method comprises the following steps: preprocessing a collected sensor historical data set of an aero-engine complete machine test to obtain a sample set; analyzing the sensor fault modes to obtain a sensor data change rule of each sensor fault mode; optimizing the SAC deep reinforcement learning initial model by taking the sample set as input and a sensor data change rule as a constraint condition, and performing back propagation training on the optimized model by taking accuracy, a recall rate and F-Score as evaluation indexes to obtain a trained SAC deep reinforcement learning model; and preprocessing acquired to-be-tested data of a to-be-tested sensor in a continuous time period, and inputting the preprocessed to-be-tested data into the trained SAC deep reinforcement learning model for fault identification to obtain a fault result. According to the method, the fault state of the sensor can be identified more accurately in a complex and changeable practical application scene.

Description

Technical Field

[0001] The present invention relates to the field of fault diagnosis of aviation engine aerial simulation test equipment, to sensor fault state recognition technology, and in particular to a sensor fault recognition method, system and medium based on SAC reinforcement learning. Background Art

[0002] During the operation of aircraft engine aerial simulation test equipment, multiple sensors are required to monitor various key parameters of the equipment in real time, such as temperature, pressure, speed, vibration, etc. However, due to the extremely harsh working environment of aircraft engine test equipment, it faces complex conditions such as high temperature, high pressure, high vibration, and strong electromagnetic interference, which makes the sensors very prone to failure. When a sensor fails, on the one hand, it will affect the accuracy of the measurement data, and then seriously affect the normal operation of the control and monitoring system of the equipment; on the other hand, if the fault state of the sensor cannot be identified in a timely and accurate manner, it may also cause a series of serious consequences.

[0003] At present, the fault diagnosis technology of aircraft engine test equipment usually adopts model-based and deep learning methods, but the model-based method is difficult to accurately model complex test equipment, and the model reliability is low. The deep learning-based method requires a large amount of labeled data, is prone to overfitting and has poor interpretability.

[0004] In view of this, the present invention intends to propose a sensor fault diagnosis method with deep reinforcement learning, deep learning perception and reinforcement learning decision-making capabilities. This method can adapt to dynamic changes, does not require a large amount of labeled data, has relatively good interpretability and is computationally efficient, providing a new means to improve the reliability and safety of aviation engine test equipment. Summary of the invention

[0005] In order to solve the technical problems of existing fault diagnosis methods based on models and deep learning, such as high modeling difficulty, low model reliability, large amount of labeled data required, easy overfitting and poor interpretability, the present invention discloses a sensor fault identification method based on SAC reinforcement learning, which comprises the following steps:

[0006] S101, collecting sensor historical data sets of aircraft engine whole machine tests, wherein the sensor historical data sets include sensor historical data sets of engines with different configurations under different test conditions, and preprocessing all the sensor historical data sets to obtain sample sets;

[0007] S102, analyzing the sensor failure mode to obtain the sensor data change rule of each sensor failure mode;

[0008] S104, taking the sample set as input and the sensor data change rule as a constraint condition, optimizing the SAC deep reinforcement learning initial model, performing back propagation training on the optimized SAC deep reinforcement learning model by using accuracy, recall rate and F-Score as evaluation indicators, and obtaining a trained SAC deep reinforcement learning model;

[0009] S105, collecting the test data of the sensor to be tested in a continuous time period, and inputting the pre-processed test data into the trained SAC deep reinforcement learning model to perform fault identification to obtain a fault result.

[0010] Furthermore, in step S101, the preprocessing of all the sensor historical data groups to obtain a sample set includes:

[0011] S1011, identifying abnormal data in each sensor historical data group by a statistical method or an outlier detection method, and removing the abnormal data;

[0012] S1012, using the mean of the remaining data in each of the sensor historical data groups to fill the missing values ​​in the corresponding sensor historical data group, to obtain a first preprocessed data group for each of the sensor historical data groups;

[0013] S1013. Perform feature dimensionality reduction processing on all the first preprocessed data groups respectively by using a kernel principal component analysis method to obtain the sample set.

[0014] Furthermore, the sensor fault identification method based on SAC reinforcement learning further includes:

[0015] S103, connecting the gated recurrent neural initial network to the output end of the deep convolutional neural initial network to obtain an initial policy network, connecting the critic network to the initial policy network, and constructing the SAC deep reinforcement learning initial model, wherein the critic network is used to perform back-propagation training on the policy network obtained after optimization by the Bayesian optimization algorithm, and obtain a trained SAC deep reinforcement learning model based on the policy network and the critic network.

[0016] Furthermore, the above-mentioned failure modes include drift failure, bias failure, spike failure, interference failure, noise failure, impact failure, output constant value failure and performance degradation failure.

[0017] Furthermore, the above-mentioned sensor includes a temperature sensor and a pressure sensor.

[0018] Furthermore, the sensor fault identification method based on SAC reinforcement learning further includes:

[0019] S106. Through the established fault implementation list, the fault result of the sensor to be tested is back-queried to obtain the fault cause, and the fault removal operation is performed on the sensor to be tested according to the fault cause, wherein the fault implementation list includes the fault cause and fault simulation operation corresponding to each sensor failure mode.

[0020] The embodiment of the present invention also provides a sensor fault identification method system based on SAC reinforcement learning, wherein the system includes a sensor data acquisition module, a fault analysis module, a network construction and training module, and an identification module.

[0021] The sensor data acquisition module is used to collect sensor historical data sets of aircraft engine whole machine tests, wherein the sensor historical data sets include sensor historical data groups of engines with different configurations under different test conditions, and all the sensor historical data groups are preprocessed to obtain sample sets, and is also used to collect the test data of the sensor to be tested in a continuous time period and preprocess the test data;

[0022] The fault analysis module is used to analyze the sensor fault mode and obtain the sensor data change rule of each sensor fault mode;

[0023] The network construction and training module is used to connect the gated recurrent neural initial network to the output end of the deep convolution neural initial network to obtain an initial policy network, connect the critic network to the initial policy network, and build a SAC deep reinforcement learning initial model; it is also used to use the sample set as input and the sensor data change law as a constraint condition to optimize the SAC deep reinforcement learning initial model, and perform back propagation training on the optimized SAC deep reinforcement learning model by using precision, recall rate and F-Score as evaluation indicators to obtain a trained SAC deep reinforcement learning model;

[0024] The identification module is used to input the pre-processed test data into the SAC deep reinforcement learning model to perform fault identification to obtain a fault result.

[0025] Furthermore, the above-mentioned sensor fault identification system based on SAC reinforcement learning also includes a fault backtracking query module, which is used to backtrack the fault result of the sensor to be tested to obtain the fault cause through an established fault implementation list, and perform a fault elimination operation on the sensor to be tested according to the fault cause, wherein the fault implementation list includes the fault cause and fault simulation operation corresponding to each of the sensor failure modes.

[0026] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned sensor fault identification methods based on SAC reinforcement learning is implemented to solve the technical problems existing in the existing model-based and deep learning-based fault diagnosis methods, such as high modeling difficulty, low model reliability, requirement for a large amount of labeled data, easy overfitting, and poor interpretability.

[0027] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned sensor fault identification methods based on SAC reinforcement learning, so as to solve the technical problems existing in the existing model-based and deep learning-based fault diagnosis methods, such as high modeling difficulty, low model reliability, requirement for a large amount of labeled data, easy overfitting and poor interpretability.

[0028] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above technical solutions adopted in the embodiments of this specification include at least the following: The sensor fault identification method based on SAC reinforcement learning of the present invention can more accurately identify the fault status of the sensor in complex and changeable practical application scenarios compared with the traditional sensor fault identification technology based on models and deep learning. The method of the present invention does not require a large amount of labeled data. It continuously learns and adjusts strategies through interaction with the environment to adapt to the dynamic changes in the operation of aircraft engine test equipment. It can dynamically adjust the fault identification strategy according to real-time sensor data and feedback information to improve the adaptability to different fault modes. At the same time, there will be no problem of error accumulation caused by multiple predictions, and it also has excellent performance in predicting fault states for a long time in the future, providing a more reliable and efficient technical means for sensor fault state identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 A flow chart of a sensor fault identification method based on SAC reinforcement learning disclosed in an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of sensor data preprocessing disclosed in an embodiment of the present invention;

[0032] Figure 3 Another flow chart of the sensor fault identification method based on SAC reinforcement learning disclosed in an embodiment of the present invention;

[0033] Figure 4 The architecture diagram of the SAC deep reinforcement learning model disclosed in the embodiment of the present invention;

[0034] Figure 5 This is an architecture diagram of a sensor fault identification system based on SAC reinforcement learning disclosed in an embodiment of the present invention;

[0035] Figure 6 A schematic diagram of a computer device disclosed in an embodiment of the present invention;

[0036] Among them, 501 is a sensor data acquisition module; 502 is a fault analysis module; 503 is a network construction and training module; 504 is an identification module; 505 is a fault backtracking query module; 601 is a memory; and 602 is a processor. DETAILED DESCRIPTION

[0037] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0038] The following describes the implementation methods of the present application through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and the features of the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0039] The embodiment of the present invention discloses a sensor fault identification method based on SAC reinforcement learning, see Figure 1 As shown, the method comprises the following steps:

[0040] S101, collecting sensor historical data sets of aircraft engine whole machine tests, wherein the sensor historical data sets include sensor historical data sets of engines with different configurations under different test conditions, and preprocessing all the sensor historical data sets to obtain sample sets;

[0041] S102, analyzing the sensor failure mode to obtain the sensor data change rule of each sensor failure mode;

[0042] S104, taking the sample set as input and the sensor data change rule as a constraint condition, optimizing the SAC deep reinforcement learning initial model, performing back propagation training on the optimized SAC deep reinforcement learning model by using accuracy, recall rate and F-Score as evaluation indicators, and obtaining a trained SAC deep reinforcement learning model;

[0043] S105, collecting the test data of the sensor to be tested in a continuous time period, and inputting the pre-processed test data into the trained SAC deep reinforcement learning model to perform fault identification to obtain a fault result.

[0044] Furthermore, the above-mentioned failure modes include drift failure, bias failure, spike failure, interference failure, noise failure, impact failure, output constant value failure and performance degradation failure.

[0045] Furthermore, the above-mentioned sensor includes a temperature sensor and a pressure sensor.

[0046] Furthermore, in step S101, the preprocessing of all the sensor historical data groups to obtain a sample set includes:

[0047] S1011, identifying abnormal data in each sensor historical data group by a statistical method or an outlier detection method, and removing the abnormal data;

[0048] S1012, using the mean of the remaining data in each of the sensor historical data groups to fill the missing values ​​in the corresponding sensor historical data group, to obtain a first preprocessed data group for each of the sensor historical data groups;

[0049] S1013. Perform feature dimensionality reduction processing on all the first preprocessed data groups respectively by using a kernel principal component analysis method to obtain the sample set.

[0050] When conducting aircraft engine whole machine tests, the sensor data of each engine configuration and each test condition (including sensor historical data and sensor data to be tested) can be obtained through real-time acquisition of sensor readings, databases, log files, API interfaces, etc. Complete, accurate and timely data can be obtained. The quality and reliability of the data can be ensured by preprocessing the collected data. For specific implementation, please refer to Figure 2 As shown in the figure, the sensor data obtained from different sources and in different formats are tabulated to identify and correct errors, duplications and inconsistencies in the data. First, identify the outliers in the sensor historical data set. The Z of each data point can be calculated by statistical methods using the following formula (1): score :

[0051]

[0052] In formula (1), x i is the i-th data point in the sensor historical data set, μ is the mean of the sensor historical data set, σ is the standard deviation, and usually, Z with an absolute value exceeding 3 score are considered outliers.

[0053] The outlier detection method can also be used to calculate the outlier IQR of each data point using the following formula (2):

[0054] IQR=Q3-Q1 (Formula 2);

[0055] In formula (2), Q1 and Q3 are the first and third quartiles of the data points in the sensor historical data set, respectively. Usually, data points lower than Q1-1.5×IQR or higher than Q3+1.5×IQR are regarded as outliers.

[0056] Secondly, the missing values ​​in the data are filled with the mean value. The size of the missing values ​​can be calculated by the following formula (3):

[0057]

[0058] In formula (3), x i is a missing value, x j For other non-missing values, n is the number of data points in the sensor historical data set.

[0059] Finally, the kernel principal component analysis (KPCA) method performs feature dimensionality reduction on the data group after outliers are removed and missing values ​​are filled to obtain preprocessed samples. Preprocessing can improve the accuracy, consistency, integrity and availability of the data to ensure that it can effectively support data analysis and decision-making processes and achieve data quality enhancement. KPCA realizes the conversion from input space to feature space through nonlinear mapping Φ, and then performs linear PCA on the mapped data, so it has strong nonlinear processing capabilities. The kernel principal component analysis (KPCA) method is specifically as follows:

[0060] For M samples x in the input space k (k=1,2,…M), x k ∈R N , the covariance matrix is ​​calculated by the following formula (4):

[0061]

[0062] Among them, C is the covariance matrix, and cov is the covariance calculation method in the block matrix:

[0063]

[0064] For the general PCA method, the characteristic equation is solved by the following formula (6) to obtain the eigenvalue with a large contribution rate (corresponding to a larger eigenvalue) and the corresponding eigenvector:

[0065] λv=Cv (Formula 6);

[0066] At the same time, a nonlinear mapping function Φ is introduced to make the sample points x1, x2…x M , transformed into sample points φ(x1), φ(x2),…φ(x M ). )

[0067] Then the covariance matrix in the feature space F is expressed as formula (7):

[0068] It can be obtained that the PCA in the feature space is solved by the following formula (8):

[0069] Among them, λ is The eigenvalue of , v is the eigenvector corresponding to the eigenvalue.

[0070] Furthermore, the sensor fault identification method based on SAC reinforcement learning is described in Figure 3 As shown, it also includes:

[0071] S103, connecting the gated recurrent neural initial network to the output end of the deep convolutional neural initial network to obtain an initial policy network, connecting the critic network to the initial policy network, and constructing the SAC deep reinforcement learning initial model, wherein the critic network is used to perform back-propagation training on the policy network obtained after optimization by the Bayesian optimization algorithm, and obtain a trained SAC deep reinforcement learning model based on the policy network and the critic network.

[0072] Specifically, the architecture of the SAC deep reinforcement learning model is as follows Figure 4 As shown in the figure, it consists of a policy network and a critic network. The policy network uses a combination of a deep convolutional neural network and a gated recurrent neural network for fault recognition to improve recognition accuracy. Its construction and training process is as follows:

[0073] First, a deep convolutional neural network is constructed. The convolutional neural network gradually mines the deep features of the data through convolution templates and pooling operations, enriching the expression of the data. It will also directly affect the fitting effect of the network. The core elements that determine the model include convolution kernel size, number of convolution kernels, convolution step, pooling kernel size, pooling step, pooling method, learning rate, batch size and optimization method (optimizer).

[0074] Secondly, a gated recurrent neural network is constructed. The gated recurrent neural network has the advantages of capturing long-term information, alleviating gradient vanishing, reducing noise impact, adapting to dynamic transformation, and multi-time scale measurement. A large number of information superposition, replication, bias and other operations are performed inside each unit of the recurrent neural network. Its main function is to be able to realize the gradual encoding of time series signals through hidden layers. At this time, the encoded information saves the information in the time series signal for a long time, and finally directly outputs the recognition result through a fully connected layer.

[0075] Thirdly, Bayesian optimization is used to adjust the optimal structure of the network to improve the generalization and recognition capabilities of the model. Given the optimized objective function, the algorithm updates the posterior distribution of the objective function by continuously adding sample points until the posterior distribution basically fits the true distribution. This optimization method can take into account previous parameter information, continuously update the prior, has a fast optimization speed, and can solve non-convex problems, ultimately obtaining the global optimal solution.

[0076] The parameter structure range of the deep convolutional neural network in the initial strategy network is shown in Table 1 below, and the optimal structure will be generated based on Table 1.

[0077] Table 1. Parameter structure range of deep convolutional neural network

[0078]

[0079]

[0080] After Bayesian optimization, the parameters of the optimized convolutional neural network are shown in Table 2 below.

[0081] Table 2. Optimized convolutional neural network structure

[0082] Convolutional layer 1 The convolution kernel size is 1×2, the number of convolution kernels is 16, and the convolution step size is 2. Convolutional Layer 2 The convolution kernel size is 1×3, the number of convolution kernels is 8, and the convolution step size is 2. Convolutional layer 3 The convolution kernel size is 1×2, the number of convolution kernels is 8, and the convolution step size is 1. Pooling layer 1 Pooling kernel size 1×3, pooling step size 3, pooling method MaxPooling Pooling layer 2 Pooling kernel size 1×4, pooling step size 3, pooling method MaxPooling Pooling layer 3 Pooling kernel size 1×4, pooling step size 3, pooling method MaxPooling Pooling layer 4 Pooling kernel size 1×3, pooling step size 3, pooling method MaxPooling Learning Rate 0.0001 Batch processing capacity 32 Optimizer Adam

[0083] Finally, the optimized SAC deep reinforcement learning model is trained by back propagation. First, the policy network inputs the timing fault signal and outputs the corresponding category action. The current state, action, state after transfer, and reward are recorded in the experience pool. When the data in the experience pool is greater than the batch size, samples are extracted according to the batch size for network training. At this time, the critic model is used to score the policy network to reflect the quality of the strategy. The critic network is mainly composed of two parts: the action value network and the value network. The action value network inputs the current state and the current action, and the output is the value of the action. The input of the value network is the next state after the state transfer, and the output is the value under the state. Then, the TD-error is constructed to perform back propagation training on the weight coefficients in the network, and the trained SAC deep reinforcement learning model can be obtained.

[0084] Among them, reinforcement learning (RL) is a machine learning method that aims to learn the optimal strategy through interaction with the environment. The core of reinforcement learning is the four-tuple (S, A, R, P), namely state, action, reward and transition probability. The four-tuple components are designed as follows:

[0085] State S: The sensor signal data with a fixed length time series is used as the state.

[0086] S={s t0 ,s t1 ,s t2 ,s t3 ,s t4 ,s t5}

[0087] Among them, s t[…] Represents the signal data in continuous time, which can be specifically used Figure 1 and Figure 2 A segment in any subgraph in is input into the policy network as a state.

[0088] Action A: The action set is the policy output in reinforcement learning, and the action can be performed to interact with the environment.

[0089] A = {a1, a2, a3, a4, a5, a6}, specifically, the probabilities of six fault identification results (drift, bias, noise, impact, constant value, and performance degradation).

[0090] Reward R: The reward is used to evaluate the quality of the strategy and to measure the quality of the action, so as to guide the agent to learn the best strategy. The specific setting is that the reward for a correct recognition result is 1, and the reward for an incorrect recognition result is -1.

[0091] Transition probability P: In the sequence data sample targeted by the present invention, the state is acquired from the experience pool in sequence, and the transition probability is determined, with a value of 1. Specifically, it refers to data that are sequentially connected at equal intervals.

[0092] After back-propagation training of the optimized SAC deep reinforcement learning model, in order to evaluate the recognition ability of the model, the model's precision, recall, and F-Score indicators are weighed and compared to evaluate the recognition ability. Specifically, the calculation methods of the three indicators are as follows:

[0093]

[0094]

[0095]

[0096] In the above formulas (9) to (11), TP represents the proportion of samples predicted as positive, FN represents the proportion of positive samples predicted as negative (other states), and FP is the proportion of negative samples predicted as positive. This is used to measure the performance of the designed deep learning model. The three index results of the trained SAC deep reinforcement learning model for identifying the data of the temperature sensor and the pressure sensor are shown in Table 3 and Table 4 respectively. The results show that the trained SAC deep reinforcement learning model can achieve a high recognition rate for the data of the temperature sensor and the pressure sensor under all working conditions.

[0097] Table 3. Calculation results of the accuracy, recall rate, and F-Score indicators of the temperature sensor by the trained SAC deep reinforcement learning model.

[0098] Sample category Accuracy (%) Recall rate (%) F-measure (%) Normal state 100 96 97.96 Bias 100 100 100 drift 100 100 100 Shock 98 98 98 peak 96.15 100 98.04 interference 98 98 98 Output value is constant 100 100 100

[0099] Table 4. Calculation results of the accuracy, recall rate, and F-Score indicators of the pressure sensor by the trained SAC deep reinforcement learning model;

[0100] Sample category Accuracy (%) Recall rate (%) F-measure (%) Normal state 100 90 94.74 Bias 98.04 100 99.01 drift 100 100 100 Shock 94.23 98 96.08 peak 90.91 100 95.24 interference 100 94 96.91 Output value is constant 100 100 100

[0101] Furthermore, when the above step S105 is implemented, the test data of the sensor to be tested in a continuous time period is collected, and the pre-processed test data is input into the trained SAC deep reinforcement learning model for fault identification to obtain a fault result, which specifically includes: firstly, the cleaned data is input into the convolutional model of the policy network, and the convolutional model outputs feature data with time series, and the feature data is sorted according to the time series and the same dimension, and then sent to the gate control recurrent network, through which the time series data of a certain feature dimension is output, and the data in the last column is taken as the output result, and then the result is used as input to output the identification category through a fully connected layer, that is, the output sensor fault mode.

[0102] Furthermore, the sensor fault identification method based on SAC reinforcement learning is described in Figure 3 As shown, it also includes:

[0103] S106. Through the established fault implementation list, the fault result of the sensor to be tested is back-queried to obtain the fault cause, and the fault removal operation is performed on the sensor to be tested according to the fault cause, wherein the fault implementation list includes the fault cause and fault simulation operation corresponding to each sensor failure mode.

[0104] Specifically, the fault implementation lists of the temperature sensor and the pressure sensor are shown in Table 5 and Table 6 respectively.

[0105] Table 5. Fault implementation list of temperature sensor

[0106]

[0107]

[0108] Table 6. Fault implementation list of pressure sensor

[0109]

[0110] Compared with traditional sensor fault identification technologies based on models and deep learning, the sensor fault identification method based on SAC reinforcement learning of the present invention can more accurately identify the fault status of sensors in complex and changeable practical application scenarios. The method of the present invention does not require a large amount of labeled data. It continuously learns and adjusts strategies through interaction with the environment to adapt to dynamic changes in the operation of aircraft engine test equipment. It can dynamically adjust fault identification strategies according to real-time sensor data and feedback information to improve adaptability to different fault modes. At the same time, there will be no problem of error accumulation caused by multiple predictions, and it also has excellent performance in predicting fault states for a long time in the future, providing a more reliable and efficient technical means for sensor fault state identification.

[0111] Based on the same inventive concept, a sensor fault identification system based on SAC reinforcement learning is also provided in an embodiment of the present invention, as described in the following embodiments. Since the principle of solving the problem by the sensor fault identification system is similar to the sensor fault identification method disclosed in the above embodiments, the implementation of the sensor fault identification system can refer to the implementation of the sensor fault identification method, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0112] Figure 5 is a structural block diagram of a sensor fault identification system based on SAC reinforcement learning disclosed in an embodiment of the present invention, such as Figure 5 As shown, the sensor fault identification system includes a sensor data acquisition module 501, a fault analysis module 502, a network construction and training module 503 and an identification module 504, and the structure is described below.

[0113] The sensor data acquisition module 501 is used to collect sensor historical data sets of aircraft engine whole machine tests, the sensor historical data sets include sensor historical data groups of engines with different configurations under different test conditions, all the sensor historical data groups are preprocessed to obtain sample sets, and are also used to collect the test data of the sensor to be tested in a continuous time period and preprocess the test data;

[0114] The fault analysis module 502 is used to analyze the sensor fault mode and obtain the sensor data change rule of each sensor fault mode;

[0115] The network construction and training module 503 is used to connect the gated recurrent neural initial network to the output end of the deep convolution neural initial network to obtain an initial policy network, connect the critic network to the initial policy network, and build a SAC deep reinforcement learning initial model; it is also used to use the sample set as input and the sensor data change law as a constraint condition to optimize the SAC deep reinforcement learning initial model, and perform back propagation training on the optimized SAC deep reinforcement learning model by using precision, recall rate and F-Score as evaluation indicators to obtain a trained SAC deep reinforcement learning model;

[0116] The identification module 504 is used to input the pre-processed test data into the SAC deep reinforcement learning model to perform fault identification to obtain a fault result.

[0117] Furthermore, the sensor fault identification system based on SAC reinforcement learning is shown in Figure 5 As shown, it also includes a fault backtracking query module 505, which is used to backtrack the fault result of the sensor to be tested to obtain the fault cause through the established fault implementation list, and perform a fault elimination operation on the sensor to be tested according to the fault cause, wherein the fault implementation list includes the fault cause and fault simulation operation corresponding to each sensor failure mode.

[0118] In this embodiment, a computer device is provided, such as Figure 6 As shown, it includes a memory 601, a processor 602, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned sensor fault identification methods based on SAC reinforcement learning is implemented.

[0119] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0120] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned sensor fault identification methods based on SAC reinforcement learning.

[0121] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0122] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A sensor fault identification method based on SAC reinforcement learning, characterized in that: include: Collecting a sensor historical data set of an aircraft engine whole machine test, wherein the sensor historical data set includes sensor historical data groups of engines with different configurations under different test conditions, and preprocessing all the sensor historical data groups to obtain a sample set; Analyze the sensor failure mode to obtain the sensor data change rule of each sensor failure mode; Taking the sample set as input and the sensor data change law as a constraint condition, the SAC deep reinforcement learning initial model is optimized, and the optimized SAC deep reinforcement learning model is back-propagated and trained by using precision, recall rate and F-Score as evaluation indicators to obtain a trained SAC deep reinforcement learning model; Collect the test data of the sensor to be tested in a continuous time period, and input the pre-processed test data into the trained SAC deep reinforcement learning model to perform fault identification to obtain the fault result.

2. The sensor fault identification method based on SAC reinforcement learning according to claim 1 is characterized in that: The preprocessing of all the sensor historical data groups to obtain a sample set includes: Identify abnormal data in each sensor historical data group by using a statistical method or an outlier detection method, and remove the abnormal data; Fill the missing values ​​in the corresponding sensor historical data group with the mean of the remaining data in each of the sensor historical data groups to obtain a first preprocessed data group for each of the sensor historical data groups; The sample set is obtained by performing feature dimension reduction processing on all the first preprocessed data groups respectively using a kernel principal component analysis method.

3. The sensor fault identification method based on SAC reinforcement learning according to claim 1 is characterized in that: Also includes: The gated recurrent neural initial network is connected to the output end of the deep convolutional neural initial network to obtain an initial policy network, and the critic network is connected to the initial policy network to construct the SAC deep reinforcement learning initial model, wherein the critic network is used to perform back-propagation training on the policy network obtained after optimization by the Bayesian optimization algorithm to obtain a trained SAC deep reinforcement learning model based on the policy network and the critic network.

4. The sensor fault identification method based on SAC reinforcement learning according to claim 1 is characterized in that: Failure modes include drift failure, offset failure, spike failure, interference failure, noise failure, impact failure, output constant value failure and performance degradation failure.

5. The sensor fault identification method based on SAC reinforcement learning according to claim 1 is characterized in that: The sensors include a temperature sensor and a pressure sensor.

6. The sensor fault identification method based on SAC reinforcement learning according to claim 1 is characterized in that: Also includes: By establishing a fault implementation list, the fault result of the sensor to be tested is back-tracked to obtain the fault cause, and the fault elimination operation is performed on the sensor to be tested according to the fault cause, wherein the fault implementation list includes the fault cause and fault simulation operation corresponding to each sensor failure mode.

7. A sensor fault identification system based on SAC reinforcement learning, characterized in that: include: A sensor data acquisition module, the sensor data acquisition module is used to collect sensor historical data sets of aircraft engine whole machine tests, the sensor historical data sets include sensor historical data groups of engines with different configurations under different test conditions, all the sensor historical data groups are preprocessed to obtain sample sets, and the sensor data acquisition module is also used to collect the test data of the sensor to be tested in a continuous time period and preprocess the test data; A fault analysis module, the fault analysis module is used to analyze the sensor fault mode and obtain the sensor data change rule of each sensor fault mode; A network construction and training module, wherein the network construction and training module is used to connect the gated recurrent neural initial network to the output end of the deep convolution neural initial network to obtain an initial policy network, and to connect the critic network to the initial policy network to construct a SAC deep reinforcement learning initial model; and is also used to optimize the SAC deep reinforcement learning initial model using the sample set as input and the sensor data change law as a constraint condition, and to perform back propagation training on the optimized SAC deep reinforcement learning model using precision, recall rate and F-Score as evaluation indicators to obtain a SAC deep reinforcement learning model; An identification module is used to input the pre-processed test data into the SAC deep reinforcement learning model to perform fault identification to obtain a fault result.

8. The sensor fault identification system based on SAC reinforcement learning according to claim 7 is characterized in that: Also includes: A fault backtracking query module is used to backtrack the fault result of the sensor to be tested to obtain the fault cause through the established fault implementation list, and perform a fault elimination operation on the sensor to be tested according to the fault cause, wherein the fault implementation list includes the fault cause and fault simulation operation corresponding to each sensor fault mode.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the sensor fault identification method based on SAC reinforcement learning according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the sensor fault identification method based on SAC reinforcement learning according to any one of claims 1 to 6.

Images