Angular velocity sensor fault diagnosis method and device and electronic equipment

Abstract

The invention provides an angular velocity sensor fault diagnosis method and device and electronic equipment, which can be applied to the technical field of computers and the technical field of data processing. The method comprises the following steps: continuously collecting data output by an angular velocity sensor within a preset duration to obtain a time domain signal; performing wavelet packet decomposition on the time domain signal to obtain a plurality of frequency band signals; respectively calculating energy of the plurality of frequency band signals to obtain a first energy feature; performing image processing on the time domain signal to generate a two-dimensional grayscale image; performing feature extraction on the first energy feature by using an energy feature processing sub-network to obtain a second energy feature; performing feature extraction on the two-dimensional grayscale image by using the grayscale image processing sub-network to obtain image features; and processing the second energy feature and the image feature by using the fusion output sub-network to obtain the fault type of the angular velocity sensor.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and data processing technology, and more specifically, to a method, device and electronic equipment for angular velocity sensor fault diagnosis. Background Art

[0002] Electro-Optical Targeting System (EOTS) is widely used in the fields of long-distance remote sensing imaging, laser communications, laser weapons, etc. In EOTS, angular velocity sensors, as its key measuring components, provide accurate angular velocity information for the closed-loop control loop, ensuring that EOTS can achieve high-precision target tracking and aiming. However, EOTS deployed on a moving platform is prone to aging, wear and damage due to long-term vibration environment, resulting in a significant decrease in EOTS tracking accuracy and even system failure. Therefore, in order to ensure the stable operation of EOTS, it is necessary to study the fault diagnosis technology of angular velocity sensors.

[0003] In the related art, the fault diagnosis methods of angular velocity sensors mainly include model-based methods and data-driven methods. The physical model-based method requires the establishment of a comprehensive mathematical model to describe the system behavior, predict the expected output of the system, and diagnose the fault by comparing the difference between the expected output and the actual output. This method is highly dependent on the accuracy of the model, and EOTS is a complex system that integrates optoelectronic imaging, motor control, and inertial sensing, so it is difficult to establish an accurate fault diagnosis model. The data-driven method does not rely on an accurate system model. This method extracts effective features from historical data and then uses machine learning algorithms to perform fault diagnosis. However, this method cannot distinguish some fault types whose frequency domain characteristics do not change much, and the generalization ability of the model is limited. Summary of the invention

[0004] In view of this, the present invention provides an angular velocity sensor fault diagnosis method, device and electronic equipment.

[0005] One aspect of the present invention provides a method for diagnosing angular velocity sensor faults, comprising: continuously collecting data output by the angular velocity sensor within a preset time length to obtain a time domain signal; performing wavelet packet decomposition on the time domain signal to obtain multiple frequency band signals; respectively calculating the energy of the multiple frequency band signals to obtain a first energy feature; performing imaging processing on the time domain signal to generate a two-dimensional grayscale image; using an energy feature processing subnetwork to perform feature extraction on the first energy feature to obtain a second energy feature; using a grayscale image processing subnetwork to perform feature extraction on the two-dimensional grayscale image to obtain an image feature; using a fusion output subnetwork to process the second energy feature and the image feature to obtain the angular velocity sensor fault type.

[0006] Another aspect of the present invention provides an angular velocity sensor fault diagnosis device, comprising: a first obtaining module, used to continuously collect data output by the angular velocity sensor within a preset time length to obtain a time domain signal; a second obtaining module, used to perform wavelet packet decomposition on the above-mentioned time domain signal to obtain multiple frequency band signals; a third obtaining module, used to respectively calculate the energy of the multiple frequency band signals to obtain a first energy feature; a generating module, used to perform imaging processing on the above-mentioned time domain signal to generate a two-dimensional grayscale image; a fourth obtaining module, used to use an energy feature processing subnetwork to perform feature extraction on the above-mentioned first energy feature to obtain a second energy feature; a fifth obtaining module, used to use a grayscale image processing subnetwork to perform feature extraction on the above-mentioned two-dimensional grayscale image to obtain an image feature; a sixth obtaining module, used to use a fusion output subnetwork to process the above-mentioned second energy feature and the above-mentioned image feature to obtain the above-mentioned angular velocity sensor fault type.

[0007] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0008] According to an embodiment of the present invention, by continuously collecting data output by an angular velocity sensor within a preset time length, a time domain signal is obtained, and the time domain signal is decomposed by wavelet packets to obtain multiple frequency band signals; the energy of the multiple frequency band signals is respectively calculated to obtain a first energy feature, and the time domain signal is imaged to generate a two-dimensional grayscale image, and then the energy feature processing sub-network is used to extract the first energy feature to obtain a second energy feature, which can capture the feature reflecting the energy change of the time domain signal in the sub-frequency band. By using the grayscale image processing sub-network to extract the feature of the two-dimensional grayscale image to obtain the image feature, the spatial structure feature of the time domain signal in the time domain can be captured. By using the fusion output sub-network to process the second energy feature and the image feature, the angular velocity sensor fault type is obtained, and the multimodal features based on the time domain signal, that is, the energy change features in the sub-frequency band and the spatial structure features in the time domain, are realized to predict the angular velocity sensor fault type, effectively improve the accuracy of sensor fault diagnosis, and enhance the generalization ability of the model for different fault types. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0010] Figure 1 An exemplary system architecture to which an angular velocity sensor fault diagnosis method according to an embodiment of the present invention can be applied is shown;

[0011] Figure 2 A flow chart of a method for diagnosing a fault of an angular velocity sensor according to an embodiment of the present invention is shown;

[0012] Figure 3 A schematic diagram showing wavelet packet decomposition of a time domain signal according to an angular velocity sensor fault diagnosis method according to an embodiment of the present invention;

[0013] Figure 4A A schematic diagram showing a time domain signal output by the MHD angular velocity sensor in a normal state according to an embodiment of the present invention;

[0014] Figure 4B A schematic diagram showing a time domain signal output by the MHD angular velocity sensor when there is an AC interference fault in the MHD angular velocity sensor according to an embodiment of the present invention;

[0015] Figure 4C A schematic diagram showing a time domain signal output by the MHD angular velocity sensor when there is a power failure in the MHD angular velocity sensor according to an embodiment of the present invention;

[0016] Figure 5AA schematic diagram showing a time domain signal output by a MEMS angular velocity sensor in a normal state according to an embodiment of the present invention;

[0017] Figure 5B A schematic diagram showing a time domain signal output by a MEMS angular velocity sensor when a mounting screw of the MEMS angular velocity sensor according to an embodiment of the present invention is loosened is shown;

[0018] Figure 5C A schematic diagram showing a time domain signal output by a MHD angular velocity sensor when a short circuit fault occurs in a MEMS angular velocity sensor according to an embodiment of the present invention;

[0019] Fig. 6A A schematic diagram showing a first energy feature obtained from a time domain signal of an MHD angular velocity sensor in a normal state according to an embodiment of the present invention;

[0020] Figure 6B A schematic diagram showing a first energy feature obtained by outputting a time domain signal of the MHD angular velocity sensor when there is an AC interference fault in the MHD angular velocity sensor according to an embodiment of the present invention;

[0021] Figure 6C A schematic diagram showing a first energy feature obtained by outputting a time domain signal of the MHD angular velocity sensor when there is a power failure in the MHD angular velocity sensor according to an embodiment of the present invention;

[0022] Fig. 7A A schematic diagram showing a first energy characteristic obtained by outputting a time domain signal of a MEMS angular velocity sensor in a normal state according to an embodiment of the present invention;

[0023] Figure 7B A schematic diagram showing a first energy characteristic obtained by outputting a time domain signal of the MEMS angular velocity sensor when a mounting screw of the MEMS angular velocity sensor according to an embodiment of the present invention is loosened is shown;

[0024] Figure 7C A schematic diagram showing a first energy characteristic obtained by a time domain signal output by a MEMS angular velocity sensor when a short circuit fault occurs in the MEMS angular velocity sensor according to an embodiment of the present invention;

[0025] Fig. 8A shows a two-dimensional grayscale image obtained by using a time domain signal of an MHD angular velocity sensor in a normal state according to an embodiment of the present invention;

[0026] Figure 8BThe figure shows a two-dimensional grayscale image obtained by outputting a time domain signal of the MHD angular velocity sensor when there is an AC interference fault in the MHD angular velocity sensor according to an embodiment of the present invention;

[0027] Figure 8C The figure shows a two-dimensional grayscale image obtained by outputting a time domain signal of a MEMS angular velocity sensor when a power failure occurs in an MHD angular velocity sensor according to an embodiment of the present invention;

[0028] Fig. 9A The figure shows a two-dimensional grayscale image obtained by outputting a time domain signal of a MEMS angular velocity sensor in a normal state according to an embodiment of the present invention;

[0029] Fig. 9B The figure shows a two-dimensional grayscale image obtained by a time domain signal output by a MEMS angular velocity sensor when a mounting screw of the MEMS angular velocity sensor according to an embodiment of the present invention is loosened;

[0030] Fig. 9C The figure shows a two-dimensional grayscale image obtained by outputting a time domain signal of the MEMS angular velocity sensor when there is a short circuit fault in the MEMS angular velocity sensor according to an embodiment of the present invention;

[0031] Fig.10 The figure shows the result of classifying the fault of the MHD angular velocity sensor based on the first energy feature corresponding to the MHD angular velocity sensor by the trained fully connected neural network model according to an embodiment of the present invention;

[0032] Fig.11 The figure shows the result of classifying the faults of the MEMS angular velocity sensor based on the first energy feature corresponding to the MEMS angular velocity sensor by the trained fully connected neural network model according to an embodiment of the present invention;

[0033] Fig.12 The figure shows the result of classifying the faults of the MHD angular velocity sensor based on the two-dimensional grayscale image corresponding to the MHD angular velocity sensor according to the trained convolutional neural network model of the embodiment of the present invention;

[0034] Fig.13 The figure shows the result of classifying the faults of the MEMS angular velocity sensor based on the two-dimensional grayscale image corresponding to the MEMS angular velocity sensor according to the trained convolutional neural network model of the embodiment of the present invention;

[0035] Fig.14 The figure shows the result of classifying the fault of the MHD angular velocity sensor based on the first energy feature and the two-dimensional grayscale image corresponding to the MHD angular velocity sensor by the trained fusion output model according to an embodiment of the present invention;

[0036] Fig.15 The figure shows the result of classifying the faults of the MEMS angular velocity sensor based on the first energy feature and the two-dimensional grayscale image corresponding to the MEMS angular velocity sensor by the trained fusion output model according to an embodiment of the present invention;

[0037] Fig.16 A structural block diagram of an angular velocity sensor fault diagnosis device according to an embodiment of the present invention is shown;

[0038] Fig.17 A block diagram of an electronic device suitable for implementing a method for diagnosing a fault of an angular velocity sensor according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0039] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0040] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0041] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0042] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0043] Since EOTS is a complex system that integrates optoelectronic imaging, motor control and inertial sensing, it is difficult to establish an accurate fault diagnosis model. Therefore, when using model-based methods to predict the fault type of angular velocity sensors, the prediction accuracy is low. When extracting effective features from historical data, data-driven methods cannot capture the time domain information and spatial structure of the data to be tested, resulting in the model being unable to distinguish some fault types whose frequency domain features do not change much, and the generalization ability of the model is limited.

[0044] In view of this, embodiments of the present invention provide an angular velocity sensor fault diagnosis method, device, and electronic device, which can be applied to the fields of computer technology and data processing technology.

[0045] Figure 1 An exemplary system architecture to which the angular velocity sensor fault diagnosis method according to an embodiment of the present invention can be applied is shown. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present invention may be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention may not be used in other devices, systems, environments or scenarios.

[0046] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0047] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0048] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0049] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0050] It should be noted that the angular velocity sensor fault diagnosis method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the angular velocity sensor fault diagnosis device provided in the embodiment of the present invention can generally be set in the server 105. The angular velocity sensor fault diagnosis method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the angular velocity sensor fault diagnosis device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the angular velocity sensor fault diagnosis method provided in the embodiment of the present invention can also be executed by the first terminal device 101, the second terminal device 102, and the third terminal device 103, or can also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102, and the third terminal device 103. Correspondingly, the angular velocity sensor fault diagnosis device provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102, the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102, the third terminal device 103.

[0051] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0052] Figure 2 A flow chart of a method for diagnosing a fault of an angular velocity sensor according to an embodiment of the present invention is shown.

[0053] like Figure 2 As shown, the angular velocity sensor fault diagnosis method includes operations S201-S207.

[0054] In operation S201, data output by the angular velocity sensor within a preset time period is continuously collected to obtain a time domain signal.

[0055] For example, the angular velocity sensor may be a MHD (magneto hydrodynamic) angular velocity sensor or a MEMS (Micro-Electro-Mechanical System) angular velocity sensor in EOTS.

[0056] For example, the data acquisition module may be used to read the output signals of the MHD angular velocity sensor and the MEMS angular velocity sensor, wherein the sampling rate of the sampling module may be 10 kHz.

[0057] According to an embodiment of the present invention, the preset duration can be selected according to actual conditions and is not limited here. For example, the preset duration can be 5 minutes, 45 minutes or 1 hour.

[0058] In operation S202, wavelet packet decomposition is performed on the time domain signal to obtain multiple frequency band signals.

[0059] For example, the output data sequences (ie, time domain signals) of the MHD angular velocity sensor and the MEMS angular velocity sensor may be subjected to wavelet packet decomposition to obtain multiple frequency band signals.

[0060] According to an embodiment of the present invention, wavelet packet decomposition is performed by recursively using low-pass and high-pass filters to decompose the signal to obtain signal components in different frequency bands.

[0061] For example, when the sampling rate of the time domain signal is 10kHz, the frequency band of interest is 0~1000Hz. Therefore, when performing wavelet packet decomposition on the time domain signal, the time domain signal can be downsampled first to obtain a discrete time signal. Among them, when downsampling the time domain signal, the downsampling factor can be set to 2. Then the discrete time signal is iteratively decomposed 5 times to obtain 32 node signals of the 5th layer. The 32 node signals of the 5th layer are sorted from low to high based on frequency, and the first 10 node signals in the sorting order are extracted as multiple frequency band signals.

[0062] For example, the time domain signal x(t) can be downsampled to obtain a discrete time signal x(n). The process of decomposing the first layer of x(n) can be described as formula (1) and formula (2).

[0063] (1)

[0064] (2)

[0065] Among them, a 1 is the low-frequency component of the first layer of the time domain signal, h is the coefficient of the low-pass filter, d 1 is the high-frequency component of the first layer of the time domain signal, is the coefficient of the high-pass filter, k is the sequence index of the component components obtained by wavelet packet analysis, and n is the time series index of the discrete-time signal.

[0066] According to an embodiment of the present invention, the coefficient h of the low-pass filter and the coefficient h of the high-pass filter It is determined based on the selected wavelet basis function.

[0067] Furthermore, the low-frequency components of the first layer can be further decomposed into the low-frequency components and high-frequency components of the next layer, and the high-frequency components of the first layer can be further decomposed into the low-frequency components and high-frequency components of the next layer. By analogy, each low-frequency component of each layer can be further decomposed into the low-frequency components and high-frequency components of the next level, and each high-frequency component of each layer can be further decomposed into the low-frequency components and high-frequency components of the next level, until the number of decompositions is equal to the number of decomposition layers.

[0068] For example, for the low-frequency component a of the jth layer of the time domain signal j , can be further decomposed into the low-frequency component a of the j+1th layer of the time domain signal according to formula (3) and formula (4): j+1 and the high frequency component d of the j+1th layer j+1 . Wherein, j is an integer greater than or equal to 1 and less than the number of decomposition levels.

[0069] (3)

[0070] (4)

[0071] Similarly, for the high-frequency component d of the jth layer of the time domain signal j , it can also be further decomposed into low-frequency components and high-frequency components of the next layer.

[0072] According to an embodiment of the present invention, the decomposition method of iteratively performing wavelet packet decomposition on the time domain signal x(t) can form a binary tree structure, such as Figure 3 shown.

[0073] Figure 3 A schematic diagram showing wavelet packet decomposition of a time domain signal in a method for diagnosing a fault of an angular velocity sensor according to an embodiment of the present invention is shown.

[0074] like Figure 3 The binary tree shown represents the process of iteratively performing three-layer wavelet packet decomposition on the time domain signal x(t). Each node of the tree represents a frequency band signal component, and each layer of the tree corresponds to a decomposition of the time domain signal. Each decomposition generates a more detailed frequency band signal component.

[0075] exist Figure 3 In the above example, after performing the first layer of packet decomposition on the time domain signal x(t), we can obtain the first layer of low-frequency component a1 and the first layer high frequency component d 1 . In the first layer of low-frequency components a 1 and the first layer high frequency component d 1 After decomposing the small packet wave separately, we get 1 The corresponding second layer low frequency component aa and the second layer high frequency component ad, as well as d 1 The corresponding second-layer low-frequency component da and the second-layer high-frequency component dd. After performing small packet wavelet decomposition on aa, ad, da and dd respectively, we obtain the third-layer low-frequency component aaa and the third-layer high-frequency component aad corresponding to aa, the third-layer low-frequency component ada and the third-layer high-frequency component add corresponding to ad, the third-layer low-frequency component daa and the third-layer high-frequency component dad corresponding to da, and the third-layer low-frequency component dda and the third-layer high-frequency component ddd corresponding to dd.

[0076] After sorting aaa, aad, ada, add, daa, dad, dda and ddd from low to high based on frequency, the first 3 or 4 node signals in the sorting order can be extracted as multiple frequency band signals.

[0077] In operation S203, the energies of the signals in the plurality of frequency bands are calculated respectively to obtain a first energy feature.

[0078] For example, the energy of multiple frequency band signals can be calculated respectively using the following formula (5).

[0079] (5)

[0080] Among them, e m is the energy of the mth frequency band signal, is the amplitude of the mth frequency band signal when the index is k.

[0081] According to an embodiment of the present invention, in order to enhance the discrimination of energy features, each energy calculated by formula (5) can be converted into a logarithmic form and normalized to obtain a normalized energy. For example, each energy calculated by formula (5) can be normalized according to formula (6) to obtain a normalized energy E m .

[0082] (6)

[0083] Then based on the normalized energy E m (m=1, 2…10), the first energy feature V shown in formula (7) is obtained energy .

[0084] (7)

[0085] In operation S204, the time domain signal is image processed to generate a two-dimensional grayscale image.

[0086] In operation S205, the energy feature processing sub-network is used to perform feature extraction on the first energy feature to obtain a second energy feature.

[0087] For example, the energy feature processing subnetwork may be a fully connected neural subnetwork. The energy feature processing subnetwork may include an input layer and a hidden layer connected in sequence. The input layer and the hidden layer in the energy feature processing subnetwork may be used to iteratively extract the first energy feature in sequence to obtain the second energy feature.

[0088] In operation S206, the grayscale image processing subnetwork is used to perform feature extraction on the two-dimensional grayscale image to obtain image features.

[0089] For example, the grayscale image processing subnetwork may be a convolutional neural subnetwork, which can be used to extract features from a two-dimensional grayscale image to obtain image features.

[0090] In operation S207, the second energy feature and the image feature are processed by using the fusion output sub-network to obtain the angular velocity sensor fault type.

[0091] For example, the fusion output subnetwork may include a second fully connected layer and a third output layer connected in sequence. The second fully connected layer and the third output layer in the fusion output subnetwork may be used to iteratively process the fusion features formed by the second energy feature and the image feature in sequence to obtain the angular velocity sensor fault type.

[0092] According to an embodiment of the present invention, the energy feature processing subnetwork and the grayscale image processing subnetwork are independent parallel subnetworks, which are used to extract the input first energy feature and the high-level features of the two-dimensional grayscale image, respectively. The fusion output subnetwork can effectively fuse the second energy feature output by the energy feature processing subnetwork and the high-level features output by the grayscale image processing subnetwork, i.e., the image features, and convert the fused features into the final fault classification results.

[0093] According to an embodiment of the present invention, by continuously collecting data output by an angular velocity sensor within a preset time length, a time domain signal is obtained, and the time domain signal is decomposed by wavelet packets to obtain multiple frequency band signals; the energy of the multiple frequency band signals is respectively calculated to obtain a first energy feature, and the time domain signal is imaged to generate a two-dimensional grayscale image, and then the energy feature processing sub-network is used to extract the first energy feature to obtain a second energy feature, which can capture the feature reflecting the energy change of the time domain signal in the sub-frequency band. By using the grayscale image processing sub-network to extract the feature of the two-dimensional grayscale image to obtain the image feature, the spatial structure feature of the time domain signal in the time domain can be captured. By using the fusion output sub-network to process the second energy feature and the image feature, the angular velocity sensor fault type is obtained, and the multimodal features based on the time domain signal, that is, the energy change features in the sub-frequency band and the spatial structure features in the time domain, are realized to predict the angular velocity sensor fault type, effectively improve the accuracy of sensor fault diagnosis, and enhance the generalization ability of the model for different fault types.

[0094] According to an embodiment of the present invention, Figure 2 The operation S204 shown, which performs imaging processing on the time domain signal to generate a two-dimensional grayscale image, may include the following operations: normalizing the time domain signal to obtain a standardized time series; performing grayscale mapping on the standardized time series to obtain a one-dimensional grayscale sequence; and performing dimensionality upgrading on the one-dimensional grayscale sequence to generate a two-dimensional grayscale image.

[0095] For example, the time domain signal x(t) can be normalized according to the following formula (8) to obtain the standardized time series x norm (t).

[0096] (8)

[0097] Among them, min() is the operation to find the minimum value, and max() is the operation to find the maximum value.

[0098] The standardized time series x can be realized according to the following formula (9): norm (t) Perform grayscale mapping to obtain a one-dimensional grayscale sequence x grey (t).

[0099] (9)

[0100] The one-dimensional grayscale sequence x of length N can be grey (t) Reshape into an L×L matrix with L rows and columns, so as to increase the dimension of the one-dimensional grayscale sequence and generate a two-dimensional grayscale image. 2 , then we need to have a one-dimensional grayscale sequence x grey (t) is padded with 0 to ensure that its length is equal to L2 .

[0101] For example, N may be 20000, and thus the reshaped two-dimensional grayscale image may be a 142×142 matrix.

[0102] According to an embodiment of the present invention, Figure 2 The operation S202 shown, performing wavelet packet decomposition on the time domain signal to obtain multiple frequency band signals, may include the following operations: determining the wavelet basis function according to the amplitude characteristics of the time domain signal; determining the number of decomposition layers according to the sampling frequency and signal frequency band range of the time domain signal; performing wavelet packet decomposition on the time domain signal according to the wavelet basis function and the number of decomposition layers to obtain multiple frequency band signals.

[0103] For example, if the time domain signal is determined to be a stationary signal based on the amplitude characteristics of the time domain signal, a wavelet basis function with good orthogonality can be selected, such as Daubechies wavelet and Symlet wavelet. If the time domain signal is determined to be a non-stationary signal based on the amplitude characteristics of the time domain signal, a wavelet basis function with better time-frequency localization properties can be selected, such as Coiflets wavelet, Morlet wavelet, Gabor wavelet and Mexican Hat wavelet.

[0104] For example, according to the amplitude characteristics of the time domain signal, the selected wavelet basis function can be determined to be the Daubechies 4 wavelet basis. When the sampling frequency of the time domain signal is 10kHz and the main frequency band range is 0~1000Hz, the number of decomposition layers can be determined to be 5. After downsampling the time domain signal, a discrete time signal can be obtained. Among them, when downsampling the time domain signal, the downsampling factor can be set to 2. Then, based on the Daubechies 4 wavelet basis, the discrete time signal is iteratively decomposed 5 times to obtain 32 node signals of the 5th layer. The 32 node signals of the 5th layer are sorted from low to high based on frequency, and the first 10 node signals in the sorting order are extracted as multiple frequency band signals.

[0105] According to an embodiment of the present invention, the Daubechies 4 wavelet basis considers the balance between efficiency and performance when performing multi-scale analysis and processing on a signal. The Daubechies 4 wavelet basis can provide multi-resolution signal analysis, and can accurately decompose and reconstruct different frequency components of a signal through wavelet functions of different scales. The Daubechies 4 wavelet basis has high computational efficiency because its support length is moderate, usually selected between 5 and 9, which can avoid excessive computational time and boundary problems.

[0106] According to an embodiment of the present invention, the grayscale image processing subnetwork includes a convolutional layer, a maximum pooling layer, a flattening layer and a first fully connected layer connected in sequence.

[0107] According to an embodiment of the present invention, Figure 2 The operation S206 shown uses a grayscale image processing subnetwork to perform feature extraction on a two-dimensional grayscale image to obtain image features, which may include the following operations: using a convolutional layer to extract features from a two-dimensional grayscale image to obtain shallow image features; using a maximum pooling layer to downsample the shallow image features to obtain deep image features; using a flattening layer to perform dimension compression on the deep image features to obtain one-dimensional image features; using a first fully connected layer to fully connect the one-dimensional image features to obtain image features.

[0108] According to an embodiment of the present invention, after obtaining a two-dimensional grayscale image corresponding to a time domain signal, a convolution layer is used to perform feature extraction on the two-dimensional grayscale image to obtain shallow image features; a maximum pooling layer is used to downsample the shallow image features to obtain deep image features; a flattening layer is used to perform dimension compression on the deep image features to obtain one-dimensional image features; and a first fully connected layer is used to fully connect the one-dimensional image features to obtain image features. This technical approach can capture the spatial structural features of the time domain signal in the time domain, and prepare for the subsequent judgment of the angular velocity sensor fault type.

[0109] According to an embodiment of the present invention, Figure 2 The operation S207 shown, using the fusion output sub-network to process the second energy feature and the image feature to obtain the angular velocity sensor fault type, may include the following operations: performing feature splicing on the second energy feature and the image feature to obtain a fusion feature; using the fusion output sub-network to fully connect and classify the fusion feature to obtain the angular velocity sensor fault type.

[0110] For example, the second energy feature and the image feature can be concatenated in a feature cascade manner to obtain a fusion feature. The fusion feature is fully connected using the fully connected layer in the fusion output subnetwork, and then the output layer in the fusion output subnetwork is used to classify the fully connected features to obtain the fault type of the angular velocity sensor.

[0111] According to an embodiment of the present invention, a fusion feature is obtained by performing feature splicing on the second energy feature and the image feature corresponding to the time domain signal; the fusion feature is fully connected and classified using a fusion output sub-network to obtain a technical means for obtaining the fault type of the angular velocity sensor, thereby realizing multimodal features based on the time domain signal, namely, the features of energy changes in sub-frequency bands and the spatial structure features in the time domain, and predicting the fault type of the angular velocity sensor, thereby effectively improving the accuracy of sensor fault diagnosis and enhancing the generalization ability of the model for different fault types.

[0112] According to an embodiment of the present invention, the energy feature processing subnetwork is obtained according to the following operations:

[0113] Constructing an energy feature training set and an energy feature test set, wherein each energy feature included in the energy feature training set and the energy feature test set is obtained by extracting energy features from a data sequence output by the angular velocity sensor in a predetermined state, and the energy features included in the energy feature training set and the energy feature test set are different;

[0114] Iteratively training the initial fully connected neural network model using multiple energy features in the energy feature training set to obtain a trained fully connected neural network model, wherein the trained fully connected neural network model includes an input layer, a hidden layer, and a first output layer that are connected in sequence;

[0115] Using the trained fully connected neural network model, respectively predict the fault types of multiple energy features in the energy feature test set to obtain multiple first fault type prediction results, and calculate the first mean square error based on the multiple first fault type prediction results and the first labels corresponding to the multiple energy features in the energy feature test set;

[0116] When the first mean square error is greater than the first preset threshold, the trained fully connected neural network model is used as the fully connected neural network model for the next round, and the operation of iteratively training the initial fully connected neural network model using multiple energy features in the energy feature training set is returned;

[0117] When the first mean square error is less than or equal to the first preset threshold, the input layer and the hidden layer in the trained fully connected neural network model are respectively determined as the input layer and the hidden layer in the energy feature processing subnetwork.

[0118] According to an embodiment of the present invention, the input layer in the fully connected neural network model may adopt a fully connected layer. Since the length of the first energy feature vector may be 10, the dimension of the input data of the trained fully connected neural network model may be set to 10. The fully connected neural network model includes an initial fully connected neural network model and a trained fully connected neural network model.

[0119] The hidden layer in the fully connected neural network model may be a fully connected layer including 64 neurons and using a relu (rectified linear unit) function as an activation function, which is used to increase the nonlinear processing capability of the model and learn complex patterns.

[0120] The first output layer in the fully connected neural network model can also be a fully connected layer, the number of its neurons can be equal to the number of categories of the target classification, and the activation function can be a normalized exponential function softmax. Among them, the softmax function ensures that the output is a probability distribution, that is, each category has a probability.

[0121] When compiling a fully connected neural network model, you can use the Adam (Adaptive Moment Estimation, a first-order gradient optimization algorithm) optimizer, and the initial learning rate can be set to 0.001. The cross entropy function is used as the loss function, which can compare the probability distribution of the softmax function output and the distribution of the true label, and is suitable for multi-classification problems.

[0122] According to an embodiment of the present invention, the predetermined state may include a normal state and a state where a fault exists.

[0123] Figure 4A FIG. 4 is a schematic diagram showing a time domain signal output by an MHD angular velocity sensor in a normal state according to an embodiment of the present invention. Figure 4B A schematic diagram showing a time domain signal output by the MHD angular velocity sensor when there is an AC interference fault in the MHD angular velocity sensor according to an embodiment of the present invention is shown. Figure 4C FIG. 1 is a schematic diagram showing a time domain signal output by the MHD angular velocity sensor when there is a power failure in the MHD angular velocity sensor according to an embodiment of the present invention.

[0124] according to Figure 4A , Figure 4B and Figure 4C From the waveform of the time domain signal shown, it can be seen that the typical faults of the MHD angular velocity sensor are AC interference fault and power failure fault. Among them, the AC interference fault is mainly caused by the MHD angular velocity sensor being interfered by the alternating electromagnetic field during operation, resulting in the presence of high-frequency signal components in the output signal and the increase of sensor noise. The power failure fault is mainly caused by the internal components of the MHD angular velocity sensor being broken or faulty during operation, causing the sensor to suddenly lose the output signal.

[0125] Figure 5A FIG. 4 is a schematic diagram showing a time domain signal output by a MEMS angular velocity sensor in a normal state according to an embodiment of the present invention. Figure 5B A schematic diagram showing a time domain signal output by a MEMS angular velocity sensor when a mounting screw of the MEMS angular velocity sensor according to an embodiment of the present invention is loosened is shown. Figure 5C FIG. 1 is a schematic diagram showing a time domain signal output by a MHD angular velocity sensor when a short circuit fault occurs in the MEMS angular velocity sensor according to an embodiment of the present invention.

[0126] according to Figure 5A , Figure 5B and Figure 5CFrom the waveform of the time domain signal shown, it can be seen that the typical faults of the MEMS angular velocity sensor are mainly loose mounting screw faults and short circuit faults. Among them, the loose mounting screw fault is mainly due to the loose connection between the mounting screw and the fixed carrier during the operation of the MEMS angular velocity sensor, which causes relative movement between the MEMS angular velocity sensor and the mounting carrier, resulting in distortion of the sensor output waveform.

[0127] For example, the MHD angular velocity sensor and the MEMS angular velocity sensor can be installed on an angular vibration table, and the angular vibration table vibrates at a certain frequency to simulate actual working conditions. The output signals of the MHD angular velocity sensor and the MEMS angular velocity sensor are read through the data acquisition module, wherein the sampling module can use a rate of 10kHz. Experiments are carried out using normal and faulty MHD angular velocity sensors and MEMS angular velocity sensors to obtain the time domain signals of the angular velocity sensor in normal state and typical fault state (i.e., the data sequence output by the angular velocity sensor in a predetermined state).

[0128] After obtaining the time domain signals (i.e. data sequences) of the angular velocity sensor in normal state and typical fault state, we can Figure 2 Operations S202 and S203 in the above method are iteratively processed to process each time domain signal in turn, to obtain a first energy feature corresponding to each time domain signal, and to extract energy features from a data sequence output by the angular velocity sensor in a predetermined state.

[0129] Fig. 6A A schematic diagram showing a first energy feature obtained from a time domain signal of an MHD angular velocity sensor in a normal state according to an embodiment of the present invention. Figure 6B A schematic diagram showing a first energy feature obtained from a time domain signal output by the MHD angular velocity sensor when there is an AC interference fault in the MHD angular velocity sensor according to an embodiment of the present invention. Figure 6C A schematic diagram showing a first energy feature obtained from a time domain signal output by the MHD angular velocity sensor when there is a power failure in the MHD angular velocity sensor according to an embodiment of the present invention.

[0130] according to Fig. 6A , Figure 6B and Figure 6C It can be seen from the first energy characteristics corresponding to the time domain signal that when the MHD angular velocity sensor is in a normal state and in different fault states, the first energy characteristics corresponding to the MHD angular velocity sensor are different.

[0131] Fig. 7A A schematic diagram of a first energy characteristic obtained by outputting a time domain signal of a MEMS angular velocity sensor in a normal state according to an embodiment of the present invention is shown. Figure 7BA schematic diagram of a first energy feature obtained by outputting a time domain signal of a MEMS angular velocity sensor when a mounting screw of the MEMS angular velocity sensor according to an embodiment of the present invention is loose is shown. Figure 7C A schematic diagram showing a first energy characteristic obtained from a time domain signal output by a MEMS angular velocity sensor when a short circuit fault occurs in the MEMS angular velocity sensor according to an embodiment of the present invention.

[0132] according to Fig. 7A , Figure 7B and Figure 7C It can be seen from the first energy characteristics corresponding to the time domain signal that when the MEMS angular velocity sensor is in a normal state and in different fault states, the first energy characteristics corresponding to the MEMS angular velocity sensor are different.

[0133] According to an embodiment of the present invention, the energy characteristic data sets of the MHD angular velocity sensor and the MEMS angular velocity sensor can be segmented, with 70% of the data used as a training set and 30% of the data used as a test set. The training set is used as the input of a fully connected neural network model to train the fully connected neural network model. The number of iterations can be set to 50 to obtain a trained fully connected neural network model.

[0134] According to an embodiment of the present invention, the grayscale image processing subnetwork is obtained according to the following operations:

[0135] Constructing a grayscale image training set and a grayscale image test set, wherein each grayscale image included in the grayscale image training set and the grayscale image test set is obtained by image processing of a data sequence output by the angular velocity sensor in a predetermined state, and the grayscale images included in the grayscale image training set and the grayscale image test set are different;

[0136] Iteratively training the initial convolutional neural network model using multiple grayscale images in the grayscale image training set to obtain a trained convolutional neural network model, wherein the trained convolutional neural network model includes a convolutional layer, a maximum pooling layer, a flattening layer, a first fully connected layer, and a second output layer connected in sequence;

[0137] Using the trained convolutional neural network model, respectively predict the fault types of multiple grayscale images in the grayscale image test set to obtain multiple second fault type prediction results, and calculate the second mean square error based on the multiple second fault type prediction results and the second labels corresponding to each of the multiple grayscale images in the grayscale image test set;

[0138] When the second mean square error is greater than the second preset threshold, the trained convolutional neural network model is used as the convolutional neural network model for the next round, and the operation of using the trained convolutional neural network model to respectively perform fault type prediction on multiple grayscale images in the grayscale image test set is returned;

[0139] When the second mean square error is less than or equal to the second preset threshold, the convolution layer, maximum pooling layer, flattening layer and first fully connected layer in the trained convolutional neural network model are respectively determined as the convolution layer, maximum pooling layer, flattening layer and first fully connected layer in the grayscale image processing subnetwork.

[0140] According to an embodiment of the present invention, the convolution layer in the convolutional neural network model is used to process the data in the input two-dimensional grayscale image, and the layer has 32 filters, each filter size is 3x3, and the relu function is used as the activation function. The dimension of the input data can be set to (142, 142, 1), corresponding to the input size and color channel of the two-dimensional grayscale image. The maximum pooling layer in the convolutional neural network model uses a 2x2 window for pooling to reduce the dimension of the feature map, which helps to reduce the amount of calculation and extract important features. The flattening layer in the convolutional neural network model flattens the previous two-dimensional features into one dimension so that they can be used in the fully connected layer. The first fully connected layer in the convolutional neural network model is used to further process the features, contains 64 neurons and uses the relu function as the activation function. The second output layer in the convolutional neural network model is also defined as a fully connected layer, and the number of its output neurons is equal to the number of categories of classification, for example, it can be 3, and the predicted probability of each category is output using the softmax activation function. Among them, the convolutional neural network model includes an initial convolutional neural network model and a trained convolutional neural network model.

[0141] According to an embodiment of the present invention, after obtaining the time domain signal (or data sequence) of the angular velocity sensor in a normal state and a typical fault state, it is possible to Figure 2 Operation S204 in the process processes each time domain signal (or data sequence) to obtain a two-dimensional grayscale image (or grayscale image) corresponding to each time domain signal (or data sequence), thereby realizing imaging processing of the data sequence output by the angular velocity sensor in a predetermined state.

[0142] Fig. 8A The figure shows a two-dimensional grayscale image obtained by a time domain signal of an MHD angular velocity sensor in a normal state according to an embodiment of the present invention. Figure 8B The figure shows a two-dimensional grayscale image obtained by outputting a time domain signal of the MHD angular velocity sensor when there is an AC interference fault in the MHD angular velocity sensor according to an embodiment of the present invention. Figure 8CThe figure shows a two-dimensional grayscale image obtained by outputting a time domain signal of a MEMS angular velocity sensor when there is a power failure in the MHD angular velocity sensor according to an embodiment of the present invention.

[0143] according to Fig. 8A , Figure 8B and Figure 8C It can be seen from the two-dimensional grayscale image corresponding to the time domain signal that when the MHD angular velocity sensor is in a normal state and in different fault states, the two-dimensional grayscale images corresponding to the MHD angular velocity sensor are different.

[0144] Fig. 9A The figure shows a two-dimensional grayscale image obtained by outputting a time domain signal of a MEMS angular velocity sensor according to an embodiment of the present invention in a normal state. Fig. 9B The figure shows a two-dimensional grayscale image obtained by outputting a time domain signal of the MEMS angular velocity sensor when a mounting screw of the MEMS angular velocity sensor according to an embodiment of the present invention is loose. Fig. 9C The figure shows a two-dimensional grayscale image obtained by a time domain signal output by the MEMS angular velocity sensor when there is a short circuit fault in the MEMS angular velocity sensor according to an embodiment of the present invention.

[0145] according to Fig. 9A , Fig. 9B and Fig. 9C It can be seen from the two-dimensional grayscale image corresponding to the time domain signal that when the MEMS angular velocity sensor is in a normal state and in different fault states, the two-dimensional grayscale images corresponding to the MEMS angular velocity sensor are different.

[0146] According to an embodiment of the present invention, the grayscale image data sets of the MHD angular velocity sensor and the MEMS angular velocity sensor can be segmented, 70% of the data are used as a training set, and 30% of the data are used as a test set. The training set is used as the input of the convolutional neural network model to train the convolutional neural network model. The number of iterations can be set to 50 times to obtain a trained convolutional neural network model.

[0147] According to an embodiment of the present invention, the fused output sub-network is obtained according to the following operations:

[0148] Constructing a fusion feature training set and a fusion feature test set, wherein each fusion feature included in the fusion feature training set and the fusion feature test set is obtained by performing feature extraction on energy features corresponding to a target data sequence using an energy feature processing subnetwork, performing feature extraction on a two-dimensional grayscale image corresponding to the target data sequence using a grayscale image processing subnetwork, and then concatenating the extracted features, wherein the target data sequence is a data sequence output by the angular velocity sensor under a predetermined state;

[0149] Iteratively training the initial fusion output model using multiple fusion features in the fusion feature training set to obtain a trained fusion output model, wherein the trained fusion output model includes a second fully connected layer and a third output layer connected in sequence;

[0150] Using the trained fusion output model, respectively predict the fault type of multiple fusion features in the fusion feature test set to obtain multiple third fault type prediction results, and calculate the third mean square error based on the multiple third fault type prediction results and the third labels corresponding to the multiple fusion features in the fusion feature test set;

[0151] When the third mean square error is greater than the third preset threshold, the trained fusion output model is used as the initial fusion output model for the next round, and the operation of iteratively training the initial fusion output model using multiple fusion features in the fusion feature training set is returned;

[0152] When the third mean square error is less than or equal to the third preset threshold, the second fully connected layer and the third output layer in the trained fusion output model are respectively determined as the second fully connected layer and the third output layer in the fusion output subnetwork.

[0153] According to an embodiment of the present invention, an energy feature processing subnetwork is used to perform feature extraction on an energy feature corresponding to a target data sequence to obtain a second energy feature, and a grayscale image processing subnetwork is used to perform feature extraction on a two-dimensional grayscale image corresponding to the target data sequence to obtain image features. Feature concatenation is then performed on the second energy feature and the image feature to obtain a fused feature.

[0154] According to the embodiment of the present invention, the first preset threshold, the second preset threshold and the third preset threshold can all be selected according to actual conditions and are not limited here.

[0155] According to an embodiment of the present invention, the second fully connected layer of the fusion output model may include 128 neurons and use the relu function as the activation function, aiming to further integrate the features from the fully connected neural network model and the convolutional neural network model. The number of neurons in the third output layer of the fusion output model is equal to the number of categories of the output label, and the softmax activation function is used for multi-classification. The fusion output model includes an initial fusion output model and a trained fusion output model.

[0156] In the process of training the fusion output model, the classification cross entropy function can be used as the loss function, and the back propagation algorithm and gradient descent method can be used to optimize the network weights. The model performance can be evaluated using an independent training set, and the model can be optimized by adjusting the network parameters, and finally a trained multi-modal fusion neural network model can be obtained.

[0157] According to an embodiment of the present invention, the outputs of the MHD angular velocity sensor and the MEMS angular velocity sensor in normal and faulty states may be used as test data, and the number of outputs in each state may be 24, to verify the accuracy of the trained model.

[0158] The test data may be preprocessed to obtain a first energy feature and a two-dimensional grayscale image corresponding to each data in the test data.

[0159] The first energy feature corresponding to each data in the test data can be input into the trained fully connected neural network model to obtain the result of the trained fully connected neural network model classifying the fault of the angular velocity sensor. The two-dimensional grayscale image corresponding to each data in the test data can be input into the trained convolutional neural network model to obtain the result of the trained convolutional neural network model classifying the fault of the angular velocity sensor.

[0160] The first energy feature and the two-dimensional grayscale image corresponding to each data in the test data can be simultaneously input into the trained fusion output model to obtain a result of the trained fusion output model classifying the fault of the angular velocity sensor.

[0161] Fig.10 The figure shows the result of classifying the faults of the MHD angular velocity sensor based on the first energy feature corresponding to the MHD angular velocity sensor by the trained fully connected neural network model according to an embodiment of the present invention. Fig.11 The figure shows the result of classifying the faults of the MEMS angular velocity sensor based on the first energy feature corresponding to the MEMS angular velocity sensor by the trained fully connected neural network model according to an embodiment of the present invention.

[0162] Fig.12 The figure shows the result of classifying the faults of the MHD angular velocity sensor based on the two-dimensional grayscale image corresponding to the MHD angular velocity sensor by using the trained convolutional neural network model according to an embodiment of the present invention. Fig.13 The figure shows the result of classifying the faults of the MEMS angular velocity sensor based on the two-dimensional grayscale image corresponding to the MEMS angular velocity sensor by using the trained convolutional neural network model according to an embodiment of the present invention.

[0163] Fig.14 The figure shows the result of classifying the faults of the MHD angular velocity sensor based on the first energy feature and the two-dimensional grayscale image corresponding to the MHD angular velocity sensor according to the trained fusion output model of the embodiment of the present invention. Fig.15The figure shows the result of classifying the faults of the MEMS angular velocity sensor based on the first energy feature and the two-dimensional grayscale image corresponding to the MEMS angular velocity sensor according to the trained fusion output model of an embodiment of the present invention.

[0164] exist Fig.10 , Fig.11 , Fig.12 , Fig.13 , Fig.14 , Fig.15 In the figure, the horizontal axis is the predicted label, the vertical axis is the true label, and each figure is a classification confusion matrix.

[0165] according to Fig.10 , Fig.11 , Fig.12 , Fig.13 , Fig.14 , Fig.15 The classification results shown in Table 1 show the fault diagnosis accuracy of each model.

[0166] Table 1

[0167]

[0168] As can be seen from Table 1, for typical faults of MEMS angular velocity sensors, the method combining two-dimensional grayscale images with trained convolutional neural networks is better than the method combining the first energy feature with trained fully connected neural networks. The opposite is true for typical faults of MHD angular velocity sensors. This shows that there are certain limitations in using single modal features for angular velocity sensor fault diagnosis. Different data processing and classification methods are required for different sensors and fault types, and it is difficult to achieve a high fault recognition accuracy for MEMS angular velocity sensors using only single modal features. By comparing the results of the multi-modal fusion neural network model with the single-modal neural network model, it can be seen that the recognition accuracy has been effectively improved. This shows that this fusion strategy can more comprehensively capture the complex characteristics of the fault by integrating features from different data processing methods, thereby improving the fault diagnosis accuracy.

[0169] Based on the above angular velocity sensor fault diagnosis method, the present invention also provides an angular velocity sensor fault diagnosis device. Fig.16 The device is described in detail.

[0170] Fig.16 A structural block diagram of an angular velocity sensor fault diagnosis device according to an embodiment of the present invention is shown.

[0171] like Fig.16As shown, the angular velocity sensor fault diagnosis device 1600 of this embodiment may include a first obtaining module 1610, a second obtaining module 1620, a third obtaining module 1630, a generating module 1640, a fourth obtaining module 1650, a fifth obtaining module 1660 and a sixth obtaining module 1670.

[0172] The first obtaining module 1610 is used to continuously collect data output by the angular velocity sensor within a preset time length to obtain a time domain signal. In one embodiment, the first obtaining module 1610 can be used to perform the operation S201 described above, which will not be described in detail here.

[0173] The second obtaining module 1620 is used to perform wavelet packet decomposition on the time domain signal to obtain multiple frequency band signals. In one embodiment, the second obtaining module 1620 can be used to perform the operation S202 described above, which will not be described in detail here.

[0174] The third obtaining module 1630 is used to calculate the energy of the multiple frequency band signals respectively to obtain the first energy feature. In one embodiment, the third obtaining module 1630 can be used to perform the operation S203 described above, which will not be described in detail here.

[0175] The generating module 1640 is used to perform image processing on the time domain signal to generate a two-dimensional grayscale image. In one embodiment, the generating module 1640 can be used to perform the operation S204 described above, which will not be described in detail here.

[0176] The fourth obtaining module 1650 is used to extract the first energy feature using the energy feature processing sub-network to obtain the second energy feature. In one embodiment, the fourth obtaining module 1650 can be used to perform the operation S205 described above, which will not be described in detail here.

[0177] The fifth obtaining module 1660 is used to extract features from the two-dimensional grayscale image using the grayscale image processing subnetwork to obtain image features. In one embodiment, the fifth obtaining module 1660 can be used to perform the operation S206 described above, which will not be described in detail here.

[0178] The sixth obtaining module 1670 is used to process the second energy feature and the image feature using the fusion output sub-network to obtain the angular velocity sensor fault type. In one embodiment, the sixth obtaining module 1670 can be used to perform the operation S207 described above, which will not be described in detail here.

[0179] According to an embodiment of the present invention, any multiple modules among the first obtaining module 1610, the second obtaining module 1620, the third obtaining module 1630, the generating module 1640, the fourth obtaining module 1650, the fifth obtaining module 1660 and the sixth obtaining module 1670 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the first obtaining module 1610, the second obtaining module 1620, the third obtaining module 1630, the generating module 1640, the fourth obtaining module 1650, the fifth obtaining module 1660 and the sixth obtaining module 1670 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the first obtaining module 1610, the second obtaining module 1620, the third obtaining module 1630, the generating module 1640, the fourth obtaining module 1650, the fifth obtaining module 1660 and the sixth obtaining module 1670 may be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function may be executed.

[0180] It should be noted that the angular velocity sensor fault diagnosis device part in the embodiment of the present invention corresponds to the angular velocity sensor fault diagnosis method part in the embodiment of the present invention. The description of the angular velocity sensor fault diagnosis device part specifically refers to the angular velocity sensor fault diagnosis method part, which will not be repeated here.

[0181] Fig.17 A block diagram of an electronic device suitable for implementing a method for diagnosing a fault of an angular velocity sensor according to an embodiment of the present invention is shown. Fig.17 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0182] like Fig.17As shown, the electronic device 1700 according to an embodiment of the present invention includes a processor 1701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1702 or a program loaded from a storage part 1708 to a random access memory (RAM) 1703. The processor 1701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1701 may also include an onboard memory for caching purposes. The processor 1701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0183] In RAM 1703, various programs and data required for the operation of electronic device 1700 are stored. Processor 1701, ROM 1702 and RAM 1703 are connected to each other via bus 1704. Processor 1701 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1702 and / or RAM 1703. It should be noted that the program can also be stored in one or more memories other than ROM 1702 and RAM 1703. Processor 1701 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0184] According to an embodiment of the present invention, the electronic device 1700 may further include an input / output (I / O) interface 1705, which is also connected to the bus 1704. The electronic device 1700 may further include one or more of the following components connected to the input / output (I / O) interface 1705: an input portion 1706 including a keyboard, a mouse, etc.; an output portion 1707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1708 including a hard disk, etc.; and a communication portion 1709 including a network interface card such as a LAN card, a modem, etc. The communication portion 1709 performs communication processing via a network such as the Internet. A drive 1710 is also connected to the input / output (I / O) interface 1705 as needed. A removable medium 1711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1710 as needed, so that a computer program read therefrom is installed into the storage portion 1708 as needed.

[0185] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1709, and / or installed from the removable medium 1711. When the computer program is executed by the processor 1701, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0186] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0187] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0188] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1702 and / or the RAM 1703 described above and / or one or more memories other than the ROM 1702 and the RAM 1703 .

[0189] An embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the angular velocity sensor fault diagnosis method provided by the embodiment of the present invention.

[0190] When the computer program is executed by the processor 1701, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0191] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1709, and / or installed from a removable medium 1711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0192] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0193] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0194] The embodiments of the present invention have been described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment has been described above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The scope of the present invention is limited by the attached embodiments and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for diagnosing angular velocity sensor faults, characterized in that: include: Continuously collect data output by the angular velocity sensor within a preset time length to obtain a time domain signal; Performing wavelet packet decomposition on the time domain signal to obtain multiple frequency band signals; Calculating the energy of the multiple frequency band signals respectively to obtain a first energy feature; Performing image processing on the time domain signal to generate a two-dimensional grayscale image; Using an energy feature processing subnetwork to extract features from the first energy feature to obtain a second energy feature; Using a grayscale image processing subnetwork to extract features from the two-dimensional grayscale image to obtain image features; The second energy feature and the image feature are processed by using a fusion output sub-network to obtain a fault type of the angular velocity sensor.

2. The method according to claim 1, characterized in that: The imaging process of the time domain signal to generate a two-dimensional grayscale image comprises: Normalizing the time domain signal to obtain a standardized time series; Performing grayscale mapping on the standardized time series to obtain a one-dimensional grayscale sequence; The one-dimensional grayscale sequence is dimensionally upgraded to generate the two-dimensional grayscale image.

3. The method according to claim 1 or 2, characterized in that: The performing wavelet packet decomposition on the time domain signal to obtain multiple frequency band signals comprises: Determining a wavelet basis function according to the amplitude characteristics of the time domain signal; Determine the number of decomposition layers according to the sampling frequency and signal frequency band range of the time domain signal; According to the wavelet basis function and the decomposition level, the time domain signal is subjected to wavelet packet decomposition to obtain the multiple frequency band signals.

4. The method according to claim 3, characterized in that The grayscale image processing subnetwork includes a convolutional layer, a maximum pooling layer, a flattening layer and a first fully connected layer connected in sequence; The grayscale image processing subnetwork is used to extract features from the two-dimensional grayscale image to obtain image features, including: Using the convolutional layer to extract features from the two-dimensional grayscale image to obtain shallow image features; Down-sampling the shallow image features using the maximum pooling layer to obtain deep image features; Utilizing the flattening layer to perform dimension compression on the deep image features to obtain one-dimensional image features; The one-dimensional image features are fully connected using the first fully connected layer to obtain the image features.

5. The method according to claim 4, characterized in that The process of processing the second energy feature and the image feature by using the fusion output sub-network to obtain the fault type of the angular velocity sensor includes: Performing feature splicing on the second energy feature and the image feature to obtain a fusion feature; The fusion output subnetwork is used to fully connect and classify the fusion features to obtain the fault type of the angular velocity sensor.

6. The method according to claim 1, characterized in that The energy feature processing subnetwork is obtained according to the following operations: Constructing an energy feature training set and an energy feature test set, wherein each energy feature included in the energy feature training set and the energy feature test set is obtained by extracting energy features from a data sequence output by an angular velocity sensor in a predetermined state, and the energy features included in the energy feature training set and the energy feature test set are different; Iteratively training the initial fully connected neural network model using multiple energy features in the energy feature training set to obtain a trained fully connected neural network model, wherein the trained fully connected neural network model includes an input layer, a hidden layer, and a first output layer connected in sequence; Using the trained fully connected neural network model, respectively predicting the fault types of the multiple energy features in the energy feature test set to obtain multiple first fault type prediction results, and calculating a first mean square error based on the multiple first fault type prediction results and first labels corresponding to the multiple energy features in the energy feature test set; When the first mean square error is greater than a first preset threshold, using the trained fully connected neural network model as the fully connected neural network model for the next round, and returning to the operation of iteratively training the initial fully connected neural network model using multiple energy features in the energy feature training set; When the first mean square error is less than or equal to a first preset threshold, the input layer and the hidden layer in the trained fully connected neural network model are respectively determined as the input layer and the hidden layer in the energy feature processing subnetwork.

7. The method according to claim 1, characterized in that The grayscale image processing subnetwork is obtained according to the following operations: Constructing a grayscale image training set and a grayscale image test set, wherein each grayscale image included in the grayscale image training set and the grayscale image test set is obtained by performing image processing on a data sequence output by an angular velocity sensor in a predetermined state, and the grayscale images included in the grayscale image training set and the grayscale image test set are different; Iteratively training the initial convolutional neural network model using multiple grayscale images in the grayscale image training set to obtain a trained convolutional neural network model, wherein the trained convolutional neural network model includes a convolutional layer, a maximum pooling layer, a flattening layer, a first fully connected layer, and a second output layer connected in sequence; Using the trained convolutional neural network model, respectively predicting the fault types of the plurality of grayscale images in the grayscale image test set to obtain a plurality of second fault type prediction results, and calculating a second mean square error based on the plurality of second fault type prediction results and second labels corresponding to the plurality of grayscale images in the grayscale image test set; When the second mean square error is greater than a second preset threshold, the trained convolutional neural network model is used as the convolutional neural network model for the next round, and the operation of using the trained convolutional neural network model to respectively perform fault type prediction on multiple grayscale images in the grayscale image test set is returned; When the second mean square error is less than or equal to the second preset threshold, the convolution layer, maximum pooling layer, flattening layer and first fully connected layer in the trained convolutional neural network model are respectively determined as the convolution layer, maximum pooling layer, flattening layer and first fully connected layer in the grayscale image processing subnetwork.

8. The method according to claim 1, characterized in that The fusion output sub-network is obtained according to the following operations: Constructing a fusion feature training set and a fusion feature test set, wherein each fusion feature included in the fusion feature training set and the fusion feature test set is obtained by using the energy feature processing subnetwork to extract energy features corresponding to the target data sequence, using the grayscale image processing subnetwork to extract features from the two-dimensional grayscale image corresponding to the target data sequence, and then splicing the extracted features, wherein the target data sequence is a data sequence output by the angular velocity sensor under a predetermined state; Iteratively training the initial fusion output model using multiple fusion features in the fusion feature training set to obtain a trained fusion output model, wherein the trained fusion output model includes a second fully connected layer and a third output layer connected in sequence; Using the trained fusion output model, respectively predicting the fault types of the multiple fusion features in the fusion feature test set to obtain multiple third fault type prediction results, and calculating a third mean square error based on the multiple third fault type prediction results and third labels corresponding to the multiple fusion features in the fusion feature test set; When the third mean square error is greater than the third preset threshold, the trained fusion output model is used as the initial fusion output model for the next round, and the operation of iteratively training the initial fusion output model using the multiple fusion features in the fusion feature training set is returned; When the third mean square error is less than or equal to the third preset threshold, the second fully connected layer and the third output layer in the trained fusion output model are respectively determined as the second fully connected layer and the third output layer in the fusion output subnetwork.

9. An angular velocity sensor fault diagnosis device, characterized in that: include: The first obtaining module is used to continuously collect data output by the angular velocity sensor within a preset time length to obtain a time domain signal; The second obtaining module is used to perform wavelet packet decomposition on the time domain signal to obtain multiple frequency band signals; A third obtaining module is used to calculate the energy of the multiple frequency band signals respectively to obtain a first energy feature; A generating module, used for performing image processing on the time domain signal to generate a two-dimensional grayscale image; A fourth obtaining module, configured to extract features from the first energy feature using an energy feature processing subnetwork to obtain a second energy feature; A fifth obtaining module is used to extract features from the two-dimensional grayscale image using a grayscale image processing subnetwork to obtain image features; The sixth obtaining module is used to process the second energy feature and the image feature by using a fusion output sub-network to obtain the angular velocity sensor fault type.

10. An electronic device comprising: one or more processors; a memory for storing one or more programs, It is characterized in that when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

Images