Method and device for fault diagnosis and computer readable medium

Abstract

The invention discloses a method and device for diagnosing faults and a computer readable medium. The method comprises the steps that sensor signals related to a vehicle are collected in the form of time series data; obtaining a feature classified into a predetermined number of frequency indices by performing Fourier transform on a predetermined time interval in the time series data; scaling the feature to a value of a predetermined range; generating a graph showing values depending on the frequency index; generating a heat map from the map; training a normal trend graph model which represents the trend of data indicated as a normal state in the heat graph as a graph; and determining whether there is a fault by calculating a loss of the sensor data input based on the normal trend graph model.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of Korean Patent Application No. 10-2023-0095948 filed in the Korean Intellectual Property Office on July 24, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a method and apparatus for diagnosing vehicle faults. Background Art

[0004] Unsupervised learning is a method of machine learning, a learning method for discovering patterns or structures without the correct answer or label of the input data. In an unsupervised learning model, useful information is extracted only by self-learning with given data, so the information can be used to discover hidden features or inherent structures in the data. Techniques such as clustering and dimensionality reduction can be used for unsupervised learning. Clustering is the task of grouping data with similar characteristics into the same group, through which patterns or types of data can be discovered. In order to perform clustering, K-means, density-based spatial clustering of applications with noise (DBSCAN), hierarchical clustering, etc. can be used. Dimensionality reduction is a technique that reduces the complexity of data and facilitates visualization by converting high-dimensional data into a low-dimensional space. Through this technique, noise or unnecessary features can be removed while maintaining the key information of the data. Principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), and autoencoders can be used.

[0005] The above information disclosed in this Background section is provided only for enhancement of understanding of the background of the present disclosure and therefore the above information may contain information that does not form the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The present disclosure provides a method and device for diagnosing faults. Specifically, the method and device can diagnose faults of a vehicle based on a normal trend graph model that represents the trend of data as a graph by using only data representing a normal state through unsupervised learning after configuring a heat map for each sensor signal of the vehicle.

[0007] A method for diagnosing a fault may include collecting sensor signals related to a vehicle in the form of time series data. The method for diagnosing a fault may also include obtaining features classified into a predetermined number of frequency indexes by performing a Fourier transform on predetermined time intervals in the time series data. The method for diagnosing a fault may also include scaling the features to a predetermined range of values. The method for diagnosing a fault may also include generating a graph showing values ​​depending on the frequency index. The method for diagnosing a fault may also include generating a heat map from the graph. The method for diagnosing a fault may also include training a normal trend graph model, which represents the trend of data indicating a normal state in the heat map as a graph. The method for diagnosing a fault may also include determining whether a fault exists by calculating the loss of sensor data input based on the normal trend graph model.

[0008] The normal trend graph model may be represented as a graph having a shape having a uniform thickness and extending along an axis corresponding to a frequency index.

[0009] The graph may include a center line extending along an axis corresponding to the frequency index, and include a peripheral area formed around the center line to have a predetermined width and extending along the center line.

[0010] The first portion and the second portion may be arranged along an axis corresponding to the frequency index. The normal trend graph model may be represented as a first graph of a shape having a first thickness in the first portion and extending along the axis corresponding to the frequency index. The normal trend graph model is represented as a second graph of a shape having a second thickness different from the first thickness in the second portion and extending along the axis corresponding to the frequency index.

[0011] The sensor signal may include a sensor signal related to at least one of wheel speed, longitudinal acceleration, lateral acceleration, vertical acceleration, yaw rate, roll rate, pitch rate, steering wheel angle, steering wheel angular velocity, and vehicle sound.

[0012] Collecting the sensor signal may include collecting the sensor signal via a controller area network (CAN), wherein the CAN may include at least one of a chassis CAN (C-CAN), a gateway CAN (G-CAN), and a powertrain CAN (P-CAN).

[0013] Collecting the sensor signal may include collecting sounds generated in the vehicle by a recording device.

[0014] Scaling may include transforming the features to have a minimum value of 0 and a maximum value of 1 through Min-Max normalization.

[0015] The time series data may include only normal driving data, and the normal trend graph model is trained based on the normal driving data.

[0016] The loss can be calculated using the loss function expressed as:

[0017]

[0018] Among them, y i is the output value, is the predicted value, and α i is a predetermined α value.

[0019] The heat map may be converted into a graph having binary colors according to a predetermined ratio, and a predetermined alpha value may be determined based on the graph.

[0020] The first part and the second part may be arranged along an axis corresponding to the frequency index.The predetermined alpha value may include a first alpha value determined corresponding to the first part, and include a second alpha value determined corresponding to the second part and different from the first alpha value.

[0021] A non-transitory computer-readable medium may store a program or instruction that directs a processor of a computer to perform the following actions: collect sensor signals related to a vehicle in the form of time series data; obtain features classified into a predetermined number of frequency indexes by performing Fourier transform on predetermined time intervals in the time series data, scale the features to a predetermined range of values, generate a curve graph showing the values ​​depending on the frequency index, generate a heat map from the curve graph, train a normal trend graph model, the normal trend graph model represents the trend of data indicating a normal state in the heat map as a graph, and determine whether there is a fault by calculating the loss of sensor data input based on the normal trend graph model.

[0022] A device for diagnosing a fault may include a sensor signal acquisition module, which is configured to collect sensor signals related to the vehicle in the form of time series data. The device for diagnosing a fault may also include: a feature acquisition module, which is configured to obtain features classified into a predetermined number of frequency indexes by performing Fourier transform on a predetermined time interval in the time series data. The device for diagnosing a fault may also include a scaling module, which is configured to scale the features to a predetermined range of values. The device for diagnosing a fault may also include a heat map generation module, which is configured to generate a graph showing values ​​depending on the frequency index and generate a heat map from the graph. The device for diagnosing a fault may also include a learning module, which is configured to train a normal trend map model, and the normal trend map model represents the trend of data indicating a normal state in the heat map as a graph. The device for diagnosing a fault may also include a fault determination module, which is configured to determine whether there is a fault by calculating the loss of sensor data input based on the normal trend map model.

[0023] The normal trend graph model may be represented as a graph having a shape having a uniform thickness and extending along an axis corresponding to a frequency index.

[0024] The graph may include a center line extending along an axis corresponding to the frequency index, and may include a peripheral area formed around the center line to have a predetermined width and extending along the center line.

[0025] The first portion and the second portion may be arranged along an axis corresponding to the frequency index. The normal trend graph model may be represented as a first graph of a shape having a first thickness in the first portion and extending along the axis corresponding to the frequency index. The normal trend graph model may be a second graph of a shape having a second thickness different from the first thickness in the second portion and extending along the axis corresponding to the frequency index.

[0026] The loss can be calculated using the loss function expressed as:

[0027]

[0028] Among them, y i is the output value, is the predicted value, and α i is a predetermined α value.

[0029] The heat map may be converted into a graph having binary colors according to a predetermined ratio, and a predetermined alpha value may be determined based on the graph.

[0030] The first and second parts may be arranged along an axis corresponding to the frequency index, and the predetermined α value may include a first α value determined corresponding to the first part, and may include a second α value determined corresponding to the second part and different from the first α value.

[0031] According to the embodiment, after a heat map for each sensor signal of a vehicle is configured, based on a normal trend graph model that represents the trend of data as a graph through unsupervised learning using only data representing a normal state, a fault of the vehicle can be diagnosed with high accuracy by using a small memory, and a new type of fault can also be detected. In addition, as a very small-sized model, it is advantageous for mass production, and a fault diagnosis function can be added without separately preparing additional hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a block diagram for explaining an apparatus for diagnosing a fault according to an embodiment.

[0033] Figure 2 is a flowchart for explaining a method for diagnosing a fault according to an embodiment.

[0034] Figure 3is a diagram for explaining a feature acquisition module according to an embodiment.

[0035] Figure 4 and Figure 5 is a diagram for explaining a heat map generating module according to an embodiment.

[0036] Figure 6 is a diagram for explaining a normal trend graph model according to an embodiment.

[0037] Figure 7 is a diagram for explaining a normal trend graph model according to an embodiment.

[0038] Figure 8 is a diagram for explaining a normal trend graph model according to an embodiment.

[0039] Fig. 9 is a diagram for explaining a normal trend graph model according to an embodiment.

[0040] Fig.10 is a diagram for explaining a learning module according to an embodiment.

[0041] Fig.11 , Fig.12 and Fig.13 is a diagram showing a result of comparing performances related to a method and apparatus for diagnosing a fault according to an embodiment.

[0042] Fig.14 is a diagram for explaining a computing device according to an embodiment. DETAILED DESCRIPTION

[0043] The present disclosure is described more fully below with reference to the accompanying drawings, in which embodiments of the present disclosure are shown. As will be appreciated by those of ordinary skill in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present disclosure. Therefore, the drawings and descriptions are to be considered illustrative and not restrictive in nature. Throughout the specification, the same reference numerals represent the same elements.

[0044] Throughout the specification and claims, when a component "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise specified. Terms including ordinal numbers such as first, second, etc. are only used to describe various constituent elements and should not be interpreted as limiting these constituent elements. These terms are only used to distinguish one constituent element from other constituent elements.

[0045] In addition, terms such as "...part", "...portion", "...operator / device", or "module" disclosed in this specification may represent a unit that can process at least one function or operation described in this specification, and this may be implemented by hardware, software, or a combination thereof. When the "...part", "...portion", "...operator / device", "module", etc. of the present disclosure are described as having a purpose or performing an operation, function, etc., the "...part", "...portion", "...operator / device", "module", etc. should be considered as "configured to" meet that purpose or perform that operation or function. Each of the parts, devices, elements, etc. may be embodied separately as a processor and a memory (such as a non-transitory computer-readable medium) or included as part of the device together with the processor and the memory.

[0046] Figure 1 is a block diagram for explaining an apparatus for diagnosing a fault according to an embodiment.

[0047] refer to Figure 1 In order to diagnose a fault when the sensor signal of the vehicle deviates from the typical behavior seen in normal driving, the device for diagnosing a fault according to the embodiment may perform unsupervised learning only on the normal driving data, and may detect abnormal driving based on the learning of the normal driving data. Specifically, when the driving behavior of the vehicle is abnormal, for example, when abnormal vibration or noise occurs during the driving of the vehicle, the device for diagnosing a fault may perform fault detection without identifying the fault that has occurred. In other words, because the abnormal behavior will be reflected to the sensor data related to the vehicle behavior when a fault occurs in the vehicle, the device for diagnosing a fault may be operated to mainly consider the vehicle behavior as a fault when it deviates from the typical behavior seen in normal driving. The device for diagnosing a fault may also be operated to confirm whether a fault has occurred by auxiliary means of a detailed inspection of the vehicle. Through this, the device for diagnosing a fault may identify known faults, but may also detect new faults that are not yet known (for example, due to the application of new technologies).

[0048] To this end, the device for diagnosing a fault can obtain features from sensor signals related to the vehicle in a characteristic manner. The device for diagnosing a fault can also construct a heat map therefrom. The device for diagnosing a fault can also diagnose the fault of the vehicle based on a normal trend graph model that represents the trend of data indicating a normal state in the heat map as a graph. Therefore, it has been found that the use of an autoencoder, a generative adversarial network (GAN), a class 1 support vector machine (SVM) (hereinafter referred to as Fig.11 , Fig.12 and Fig.13 The fault diagnosis performance of the device for diagnosing a fault shows excellent determination accuracy with a very small memory usage rate compared to the case of (described in detail).

[0049] The apparatus for diagnosing a fault may include a sensor signal acquisition module 110 , a feature acquisition module 120 , a scaling module 130 , a heat map generation module 140 , a learning module 150 , and a fault determination module 160 .

[0050] The sensor signal acquisition module 110 may acquire sensor signals related to the vehicle in the form of time series data. Here, time series data is data measured or recorded at regular time intervals (e.g., 100 Hz, 10 ms sampling), and may include a continuous sequence of data points in chronological order. Meanwhile, the sensor signal may be various types of sensor signals that may be sensed relative to the vehicle. The sensor signal may include, for example, a sensor signal related to at least one of wheel speed, longitudinal acceleration, lateral acceleration, vertical acceleration, yaw rate, roll rate, pitch rate, steering wheel angle, steering wheel angular velocity, and vehicle sound.

[0051] In some embodiments, the sensor signal can be collected through a controller area network (CAN). CAN communication is a standardized protocol for transmitting data between electronic devices inside a vehicle, and can be used for fault diagnosis, error detection, transmission of driving-related data, etc. In some embodiments, CAN may include at least one of a chassis CAN (C-CAN), a gateway CAN (G-CAN), and a powertrain CAN (P-CAN). C-CAN can be a CAN bus responsible for communication between components related to the vehicle body (such as suspension, braking system, and vehicle stability system). G-CAN is a central gateway for connecting multiple CAN networks in a vehicle, facilitating data transmission and conversion between different CAN networks, and supporting integration between vehicle systems. P-CAN can be a CAN bus that handles communication between components related to the powertrain (such as an engine control module, a transmission control module, and a fuel management system).

[0052] Meanwhile, in some embodiments, the sensor signal may also be collected by a recording device. In other words, the recording device may be used to collect the sound generated in the vehicle.

[0053] The feature acquisition module 120 can obtain features classified into a predetermined number of frequency indexes by performing Fourier transform on a predetermined time interval in the time series data. The predetermined time interval can be defined by a window spanning a start point and an end point. The feature acquisition module 12 can perform Fourier transform on the raw data included in the window in the sensor signal acquired by the sensor signal acquisition module 110. In addition, the feature acquisition module 12 obtains features classified into a predetermined number of frequency indexes. At this time, when the number of time series data included in the window is n (n is an integer greater than or equal to 2), the (n / 2-1) pieces of data obtained by removing the first index corresponding to the sum of the signal values ​​from the n / 2 pieces of data obtained by removing the symmetry from the n amplitudes can be set as the final data to be used.

[0054] The scaling module 130 may scale the features obtained by the feature acquisition module 120 to a predetermined range of values. In some embodiments, regarding the feature values ​​obtained by the feature acquisition module 120, the scaling module 130 may convert the features to have a minimum value of 0 and a maximum value of 1 through Min-Max normalization.

[0055] The heat map generation module 140 may generate a graph showing the feature values ​​scaled by the scaling module 130 according to the frequency index, and may generate a heat map from the graph. A heat map for analyzing a signal trend of normal driving of a vehicle is a graphical representation that visualizes the distribution and density of data in colors, and the value of each data point may be represented as the intensity or saturation of the color. This heat map may be generated for each sensor signal of the vehicle.

[0056] The heat map generation module 140 may discretize the value of the vertical axis (i.e., the scaled value) with respect to the graph obtained after scaling the features obtained by the feature acquisition module 120 to a value within a predetermined range. For example, the vertical axis may be divided into a plurality of parts having the same size, a first value (e.g., 1) may be assigned to a part to which the scaled value belongs among the plurality of parts, and a second value (e.g., 0) may be assigned to a part to which the scaled value does not belong, to generate a first table.

[0057] Thereafter, a second table, a third table, . . . , and a k-th table (k is an integer greater than or equal to 2) may be generated by repeating the above operation with respect to another feature value obtained based on the newly collected data and scaling.

[0058] Then, when the first table, the second table, ..., and the kth table are superimposed, for items whose multiple parts and frequency indexes intersect each other in the table, the frequency counts of items in the same position of the first table, the second table, ..., and the kth table can be added up. Based on the total frequency count calculated as described above, a heat map can be generated by making the item with the highest frequency the darkest and the remaining items gradually brighter.

[0059] In some embodiments, a threshold value may be pre-set in the process of processing items with high frequency counts in a heat map in bold. For example, the threshold value may be set to a value corresponding to n% of k. For example, when k is 10,000 and n is set to 2, the threshold value may be determined to be 200. Thus, after the frequency counts of items in the same position of the first table, the second table, ..., and the kth table are summed, when the summed frequency count exceeds the threshold value, a heat map may be generated after replacing the frequency counts of the corresponding items with the threshold value. In other words, for example, for items whose summed frequency counts exceed 200 (e.g., items with frequency counts of 210, 204, 212, etc.), their frequency counts (e.g., 210, 204, 212) may all be replaced by 200. Here, the value of n may be defined as an appropriate value according to a specific implementation purpose or implementation environment. In this way, when a threshold value is introduced when generating a heat map, since the portion in the heat map that is different from the normal portion may be expressed as darkly as possible, the error indicating a fault that has not yet been detected may be maximized.

[0060] The learning module 150 can train a normal trend graph model by taking the data for generating the heat map (i.e., the first table, the second table, ..., the kth table) as input, wherein the normal trend graph model represents the trend of the data indicating the normal state in the heat map generated by the heat map generation module 140 as a graph. Here, the normal trend graph model can be a graph in the form of a band with thickness. When the learning module 150 generates the normal trend graph model, if there is a large amount of data deviating from the graph, it can be interpreted that the driving behavior of the vehicle is abnormal, or the vehicle may have a fault.

[0061] In some embodiments, the normal trend graph model may be represented as a graph having a uniform thickness and extending along an axis corresponding to the frequency index. In other words, the normal trend graph model may be trained by using the data used to generate the heat map generated by the heat map generation module 140 as training data, so that the normal trend graph model may be formed to have a uniform thickness and extend in the horizontal direction. Here, the time series data used as training data may include only normal driving data, and the normal trend graph model may be trained based on the normal driving data.

[0062] In some embodiments, the normal trend graph model may include a center line extending along an axis corresponding to the frequency index, and may include a peripheral area formed around the center line to have a predetermined width and extending along the center line. In other words, the normal trend graph model may be trained by using the data used to generate the heat map generated by the heat map generation module 140 as training data, so that the normal trend graph model may be formed to include the center line and the peripheral area and extend in the horizontal direction. The time series data used as training data may include only normal driving data, and the normal trend graph model may be trained based on the normal driving data.

[0063] In some embodiments, the normal trend graph model may also be generated to include multiple thicknesses. In more detail, a first portion and a second portion may be set along an axis corresponding to a frequency index. The normal trend graph model may be represented as a first graph of a shape having a first thickness in a first portion and extending along an axis corresponding to a frequency index; and may be represented as a second graph of a shape having a second thickness different from the first thickness in a second portion and extending along an axis corresponding to a frequency index.

[0064] The fault determination module 160 may determine whether there is a fault by calculating a loss regarding the sensor data input based on the normal trend graph model generated by the learning module 150 .

[0065] In some implementations, the loss may be calculated by a loss function represented by the following equation.

[0066]

[0067] Here, y i Can be an output value, can be the predicted value, and α i It can be a predetermined α value. The α value can be a value corresponding to the thickness of a peripheral area formed around the center line in the normal trend map model to have a predetermined width and extend along the center line. Therefore, the thickness of the normal trend map model can be a value obtained by doubling the α value. Therefore, the α value can be set to an appropriate value to adjust the thickness of the normal trend map model, and can also be used to adjust the boundary between things determined to be normal and things determined to be abnormal. The loss function can be used to calculate the error of the data used to generate the heat map (i.e., the first table, the second table... and the kth table), and normal or abnormal can be determined based on the average value.

[0068] Here, y i is the expected output value when any x is input, and y i Can refer to the output value of the predetermined data, i.e., the solution or ground truth. In addition, is the value that is output when x is input to the learned model, and in the sense that the learned model has made an inference for the input x, It can be called interference value. Learning can be done by reducing the i and In other words, when y i and The difference between i Internal time, y i and The difference between can be considered to be 0. Typically, after learning, the predicted value output from the learning model is directly used to classify a class of objects (e.g., distinguish between dogs and cats) or predict a continuous value (e.g., predict the current speed of a vehicle). However, according to an embodiment, the result output from the learning model Instead of using it as is, use the result y of the learned model i and the loss of the ground truth.

[0069] In some embodiments, the α value may be a value obtained by extracting the thickness of a portion to be included in the heat map as a band shape for each signal and taking half of the extracted thickness.

[0070] In some embodiments, the heat map may be converted into a graph having binary colors according to a predetermined ratio, and the alpha value may be determined based on the converted graph.

[0071] In some embodiments, the first portion and the second portion may be arranged along an axis corresponding to the frequency index.The alpha value may include a first alpha value determined corresponding to the first portion, and may include a second alpha value determined corresponding to the second portion but different from the first alpha value.

[0072] In other words, the device for diagnosing faults according to the embodiment can collect normal driving data, a heat map can be configured for each sensor signal, an α value can be set based on the heat map, and then learning can be performed. When the learning is completed, as many normal trend map models as the number of sensor signals can be generated. The device for diagnosing faults can set a threshold for each sensor signal based on normal test data, and then the loss can be estimated based on continuous driving data of a specific time unit (e.g., 20 minutes). Therefore, when the loss exceeds the threshold, a "potential fault" can be determined. By additional inspection of vehicles determined to be "potential faults", new types of faults can be specifically identified and diagnosed.

[0073] Figure 2 is a flowchart for explaining a method for diagnosing a fault according to an embodiment.

[0074] refer to Figure 2According to an embodiment, a method for diagnosing a fault may include step S201: collecting sensor signals related to the vehicle in the form of time series data. The method for diagnosing a fault may also include step S202: obtaining features classified into a predetermined number of frequency indexes by performing Fourier transform on predetermined time intervals in the time series data. The method for diagnosing a fault may also include step S203: scaling the features to a predetermined range of values. The method for diagnosing a fault may also include step S204: generating a curve graph showing the scaled feature values ​​according to the frequency index. The method for diagnosing a fault may also include step S205: generating a heat map from the curve graph. The method for diagnosing a fault may also include step S206: training a normal trend graph model, which represents the trend of data indicating a normal state in the heat map as a graph. The method for diagnosing a fault may also include step S207: determining whether there is a fault by calculating the loss of sensor data input based on the normal trend graph model.

[0075] For a more detailed description of this method, please refer to the above Figure 1 The description and the following Figures 3 to 14 The description is described above, and therefore a repeated description is omitted herein.

[0076] Figure 3 is a diagram for explaining a feature acquisition module according to an embodiment.

[0077] refer to Figure 3 , the feature acquisition module 120 may perform Fourier transform on the raw data included in the preset time window among the sensor signals in the form of time series data acquired by the sensor signal acquisition module 110. For example, referring to Figure 3 In the example shown in , relative to a certain time point t, the end point can be set within the range of t 1.28 seconds and t 5.12 seconds. Here, when the end point is t+1.28 seconds, when Fourier transform is performed, a total of 128 data can be included in the time window. The feature acquisition module 12 can obtain features classified as 63 frequency indexes. Here, for the frequency index, if there are 128 time series data included in the window, 63 data obtained by removing the first index corresponding to the sum of the signal values ​​from the 64 data obtained by removing the symmetry from the 128 amplitudes can be set as the final data to be used. When the end point is t+5.12 seconds, when Fourier transform is performed, a total of 512 data can be included in the time window, thereby, features classified as 255 frequency indexes can be obtained. In the present embodiment, the numbers 1.28 seconds and 5.12 seconds are not intended to limit the scope of the present disclosure, and can be changed to other values ​​according to the specific implementation purpose and environment.

[0078] Figure 4 and Figure 5 is a diagram for explaining a heat map generating module according to an embodiment.

[0079] refer to Figure 4 The heat map generation module 140 may generate a heat map for longitudinal acceleration, a heat map for lateral acceleration, a heat map for yaw rate, and a heat map for left front (FL) wheel speed. In each heat map, it can be seen that the feature value is scaled between 0 and 1.

[0080] refer to Figure 5 The heat map generation module 140 may be for the curve graph obtained after scaling the features obtained by the feature acquisition module 120 to a value within a predetermined range (in Figure 5 ), the value of the vertical axis (i.e., the scaled value) is discretized. For example, the vertical axis may be divided into a plurality of parts having the same size, a first value (e.g., 1) may be assigned to a portion to which the scaled value belongs in the plurality of parts, and a second value (e.g., 0) may be assigned to a portion to which the scaled value does not belong, to generate a first table. For example, the value of the vertical axis corresponding to frequency index 1 may be 0.11x (x is an integer), the value of the vertical axis corresponding to frequency index 3 may be 0.12x (x is an integer), and the value of the vertical axis corresponding to frequency index 63 may be 0.10x (x is an integer). In this case, in the column corresponding to frequency index 1, 1 may be assigned to the portion represented by "0.11", and 0 may be assigned to the remaining portion. In the case of the column corresponding to frequency index 3, 1 may be assigned to the portion represented by "0.12", and 0 may be assigned to the remaining portion. In the case of the column corresponding to frequency index 63, 1 may be assigned to the portion represented by "0.10", and 0 may be assigned to the remaining portion.

[0081] Thereafter, the second table, the third table, ..., and the kth table (k is an integer greater than or equal to 2) can be generated by repeating the above operation relative to another feature value obtained based on the newly collected data and the scaling. For each of these tables, the operation can be performed in the same manner as that of generating the first table.

[0082] Subsequently, when the first table, the second table, ..., and the kth table are superimposed, for multiple parts in the table and items whose frequency indexes intersect with each other, the frequency counts of the items in the same position of the first table, the second table, ..., and the kth table may be summed. For example, in the column corresponding to the frequency index 1, the frequency count of the part indicated by "0.11" and "0.00" may be 3000 (which is the largest), the frequency count of the part indicated by "0.10" may be 2600, and the frequency count of the part indicated by "0.01" may be 2000, and the frequency count of the part indicated by "0.12" may be 1200. At this time, in the column corresponding to the frequency index 1, the part indicated by "0.11" and "0.00" may be processed as the darkest, and the remaining items may be processed as gradually brighter in descending order of value. Furthermore, in the column corresponding to frequency index 2, the frequency count of the portion indicated by "0.11" and "0.00" may be 2000, which is the largest, the frequency count of the portion indicated by "0.12" may be 1900, the frequency count of the portion indicated by "0.10" may be 1500, and the frequency count of the portion indicated by "0.01" may be 1300. At this time, in the column corresponding to frequency index 2, the portions indicated by "0.11" and "0.00" may be processed as the darkest, and the remaining items may be processed as gradually brighter in descending order of value. When such an operation is repeated from the column corresponding to frequency index 3 to the column corresponding to frequency index 63, a value in Figure 5 The heat map is shown in the lower left corner.

[0083] Figure 6 is a diagram for explaining a normal trend graph model according to an embodiment.

[0084] refer to Figure 6 , the heat map shown on the left is for a normal left rear (RL) wheel speed and shows that a normal trend map model having a shape with uniform thickness and extending along an axis corresponding to a frequency index is generated by the learning module 150.

[0085] The heat map shown on the right is for suspension left rear (RL) wheel leakage left rear (RL) wheel speed, in which the normal trend map model shown on the left generated by the learning module 150 is superimposed. As indicated by the rectangle, the case where values ​​having significant errors in the values ​​of the heat map are included above a certain level can be interpreted as the possibility of abnormality or failure.

[0086] Figure 7 is a diagram for explaining a normal trend graph model according to an embodiment.

[0087] refer to Figure 7, the heat map shown on the left is for a normal left rear (RL) wheel speed and shows a normal trend map model including a center line. The normal trend map model may also include a peripheral area formed around the center line to have a predetermined width and extending along the center line. The normal trend map model has a shape extending along an axis corresponding to a frequency index and is generated by the learning module 150.

[0088] The heat map shown on the right is for suspension left rear (RL) wheel leakage left rear (RL) wheel speed, in which the normal trend map model shown on the left generated by the learning module 150 is superimposed. As indicated by the rectangle, the case where values ​​having significant errors in the values ​​of the heat map are included above a certain level can be interpreted as the possibility of abnormality or failure.

[0089] Figure 8 is a diagram for explaining a normal trend graph model according to an embodiment.

[0090] refer to Figure 8 , a heat map is used for lateral acceleration, wherein a first portion and a second portion may be set along an axis corresponding to a frequency index. A normal trend map model may be represented as a first map of a shape having a first thickness in a first portion and extending along an axis corresponding to a frequency index, and as a second map of a shape having a second thickness different from the first thickness in a second portion and extending along an axis corresponding to a frequency index.

[0091] Meanwhile, the α value may also be set to include a first α value determined corresponding to the first portion, and a second α value determined corresponding to the second portion but different from the first α value.

[0092] Fig. 9 is a diagram for explaining a normal trend graph model according to an embodiment.

[0093] refer to Fig. 9 , after converting the heat map into a curve graph having binary colors according to a predetermined ratio, a normal trend graph model according to an embodiment may be set based on the converted curve graph.

[0094] The value of the vertical axis (ie, the scaled value) may be relative to the curve graph ( Fig. 9) is discretized. For example, the vertical axis may be divided into a plurality of parts having the same size, a first value (e.g., 1) may be assigned to a part of the plurality of parts to which the scaling value belongs, and a second value (e.g., 0) may be assigned to a part to which the scaling value does not belong, to generate a first table. For example, the value of the vertical axis corresponding to frequency index 1 may be 0.11x (x is an integer), the value of the vertical axis corresponding to frequency index 3 may be 0.12x (x is an integer), and the value of the vertical axis corresponding to frequency index 63 may be 0.10x (x is an integer). In this case, in the column corresponding to frequency index 1, 1 may be assigned to the part indicated by "0.11", and 0 may be assigned to the remaining part. In the case of the column corresponding to frequency index 3, 1 may be assigned to the part indicated by "0.12", and 0 may be assigned to the remaining part. In the case of the column corresponding to frequency index 63, 1 may be assigned to the part indicated by "0.10", and 0 may be assigned to the remaining part.

[0095] Thereafter, the second table, the third table, ... and the kth table (k is an integer greater than or equal to 2) can be generated by repeating the above operation relative to another feature value obtained based on the newly collected data and scaling. For each of these tables, the operation can be performed in the same manner as that of generating the first table.

[0096] Subsequently, when the first table, the second table, ..., and the kth table are superimposed, for items in which multiple parts and frequency indexes in the tables intersect with each other, the frequency counts of items in the same position of the first table, the second table, ..., and the kth table may be summed. For example, in the column corresponding to the frequency index 1, the frequency count of the parts indicated by "0.11" and "0.00" may be 3000 (which is the largest), the frequency count of the part indicated by "0.10" may be 2600, the frequency count of the part indicated by "0.01" may be 2000, and the frequency count of the part indicated by "0.12" may be 1200. At this time, black may be assigned to items having values ​​higher than a predetermined ratio (e.g., 3%) of the corresponding items based on the same index, or may be assigned to items having values ​​higher than a predetermined threshold (e.g., 100) of the corresponding items. White may be assigned to items having values ​​less than a predetermined ratio or a predetermined threshold. When this operation is repeated, a color spectrum may be finally generated. Fig. 9 The heat map shown in the lower left corner.

[0097] Fig.10 is a diagram for explaining a learning module according to an embodiment.

[0098] refer to Fig.10, the learning module 150 prepares a training data set for each sensor signal and does not mix and use different sensor signals. For signal 1 corresponding to the first driving log for learning, specifically, the learning module 150 may extract and preprocess samples about the first window to add them to the training data set (sample #1-1), and then may extract and preprocess samples about the second window shifted by 0.1 seconds to add them to the training data set (sample #1-2). On the other hand, for signal 1 corresponding to the second driving log for learning, the learning module 150 may extract and preprocess samples about the third window to add them to the training data set (sample #2-1), and then may extract and preprocess samples about the fourth window shifted by a predetermined time to add them to the training data set (sample #2-2).

[0099] This process is a process of generating data for learning, and is intended to link and use the data of each signal for multiple learning logs. The reason for linking and using multiple of them is to learn a wide range of normal states that may occur in various situations by collecting and learning normal data for various possible driving conditions. At the same time, the reason for learning separately for each signal is that the heat map for each signal is different from the beginning, and through this, the effect of improving the accuracy of abnormal determination can be obtained. For example, for some abnormal data, if the learning model for signal 1 determines that the data is normal, but the learning model for signal 2 determines that the data is abnormal, the final result can be determined to be abnormal based on the logical AND or operation of the two results. This is because the abnormality does not affect one behavior of the vehicle (signal 1), but may affect another behavior (signal 2).

[0100] Fig.11 , Fig.12 and Fig.13 is a diagram showing a result of comparing performances related to a method and apparatus for diagnosing a fault according to an embodiment.

[0101] The fault diagnosis performance of the method and apparatus for diagnosing a fault according to the embodiment shows higher accuracy than that of the autoencoder, GAN, and 1-class SVM, and the following results of the determination accuracy of the suspension RL leakage and the determination accuracy of the suspension FL leakage are obtained.

[0102] (Table 1)

[0103]

[0104] The results according to the embodiment show a determination accuracy of 86% for suspension RL leakage and 54% for suspension FL leakage, which is excellent compared to autoencoders, GANs, and 1-class SVMs, where the required memory size is 1, which is the same as the input features.

[0105] Fig.11 The results of suspension RL leakage are shown, where 32 out of 37 data are detected as abnormal data for suspension RL leakage, only 22 data for autoencoder and 20 data for GAN are accurately detected, and in the case of SVM, no detection is performed appropriately.

[0106] Fig.12 Results regarding suspension FL leakage are shown, where 7 out of 13 pieces of data are detected as abnormal data for suspension FL leakage, in the case of the autoencoder and GAN, only 4 pieces of data are accurately detected, and in the case of the SVM, the detection is not properly performed.

[0107] Fig.13 Show the above combination Figure 1 The performance of the loss function described is used in the case of the loss function as in the implementation example ( Fig.13 The lower side of ) shows that the mean square error (MSE) is better than that of Fig.13 ) for better performance.

[0108] Fig.14 is a diagram for explaining a computing device according to an embodiment.

[0109] refer to Fig.14 , the method and apparatus for diagnosing a fault according to the embodiment may be implemented by using the computing device 50 .

[0110] The computing device 50 may include at least one of a processor 510, a memory 530, a user interface input device 540, a user interface output device 550, and a storage device 560. These elements may communicate with each other through a bus 520. The computing device 50 may include a network interface 570 electrically connected to the network 40. The network interface 570 may send or receive signals to or from other entities through the network 40.

[0111] The processor 510 may be implemented in various types, such as a microcontroller unit (MCU), an application processor (AP), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), etc. The processor 510 may be any semiconductor device configured to execute instructions stored in the memory 530 or the storage device 560. The processor 510 may be configured to implement the above-mentioned Figure 1-Figure 13 Describes the functions and methods.

[0112] The memory 530 and the storage device 560 may include various types of volatile or non-volatile storage media. For example, the memory may include a read-only memory (ROM) 531 and a random access memory (RAM) 532. In this embodiment, the memory 530 may be located inside or outside the processor 510, and the memory 530 may be connected to the processor 510 by various known means.

[0113] In some embodiments, at least some configurations or functions of the method and apparatus according to the embodiments may be implemented as a program or software executed by the computing device 50, and the program or software may be stored in a computer-readable medium.

[0114] In some embodiments, at least some configurations or functions of the method and apparatus according to the embodiments may be implemented by using hardware or circuits of the computing device 50 , or may also be implemented by using separate hardware or circuits that may be electrically connected to the computing device 50 .

[0115] According to the above-mentioned embodiment, after a heat map for each sensor signal of a vehicle is configured, based on a normal trend graph model that represents the trend of data as a graph through unsupervised learning using only data representing a normal state, a fault of the vehicle can be diagnosed with high accuracy by using a small memory, and a new type of fault can also be detected. In addition, as a very small-sized model, it is advantageous for mass production, and a fault diagnosis function can be added without separately preparing additional hardware.

[0116] While the present disclosure has been described in connection with what are presently considered to be practical embodiments, it is to be understood that the present disclosure is not limited to the disclosed embodiments, but is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for diagnosing a fault, the method comprising: Collecting sensor signals related to the vehicle in the form of time series data; Obtaining features classified into a predetermined number of frequency indices by performing Fourier transform on predetermined time intervals in the time series data; scaling the feature to a predetermined range of values; generating a graph showing the values ​​based on the frequency index; generating a heat map from the graph; training a normal trend graph model, the normal trend graph model representing the trend of the data indicated as a normal state in the heat map as a graph; as well as By calculating the loss of sensor data input based on the normal trend graph model, it is determined whether a fault exists.

2. The method according to claim 1, wherein: The normal trend graph model is represented as a graph having a shape having a uniform thickness and extending along an axis corresponding to the frequency index.

3. The method according to claim 2, wherein: The graph includes a center line extending along an axis corresponding to the frequency index, and includes a peripheral area formed around the center line to have a predetermined width and extending along the center line.

4. The method according to claim 2, wherein: Disposing a first portion and a second portion along an axis corresponding to the frequency index; The normal trend graph model is represented as a first graph of a shape having a first thickness in the first portion and extending along an axis corresponding to the frequency index; as well as The normal trend graph model is represented as a second graph of a shape having a second thickness different from the first thickness in the second portion and extending along an axis corresponding to the frequency index.

5. The method according to claim 1, wherein: The sensor signals include sensor signals related to at least one of wheel speed, longitudinal acceleration, lateral acceleration, vertical acceleration, yaw rate, roll rate, pitch rate, steering wheel angle, steering wheel angular velocity, and vehicle sound.

6. The method according to claim 5, wherein: Collecting the sensor signal includes collecting the sensor signal through a controller area network (CAN), The CAN includes at least one of a chassis CAN, a gateway CAN, and a powertrain CAN.

7. The method according to claim 5, wherein: Collecting the sensor signal includes collecting sounds generated in the vehicle through a recording device.

8. The method according to claim 1, wherein: Scaling involves transforming the features to have a minimum value of 0 and a maximum value of 1 through min-max normalization.

9. The method according to claim 1, wherein: The time series data includes only normal driving data, and the normal trend graph model is trained based on the normal driving data.

10. The method according to claim 1, wherein: The loss is calculated by the loss function expressed as the following equation: Among them, y i is the output value, is the predicted value, and α i is a predetermined α value.

11. The method according to claim 10, wherein: The heat map is converted into a curve graph having binary colors according to a predetermined ratio, and the predetermined alpha value is determined based on the curve graph.

12. The method according to claim 10, wherein: Disposing a first portion and a second portion along an axis corresponding to the frequency index; and The predetermined α values ​​include a first α value determined corresponding to the first portion, and include a second α value different from the first α value determined corresponding to the second portion.

13. A non-transitory computer-readable medium having a program or instruction recorded thereon, the program or instruction being used to instruct a processor to perform the following actions: Collecting sensor signals related to the vehicle in the form of time series data; Obtaining features classified into a predetermined number of frequency indices by performing Fourier transform on predetermined time intervals in the time series data; scaling the feature to a predetermined range of values; generating a graph showing the values ​​based on the frequency index; generating a heat map from the graph; training a normal trend graph model, the normal trend graph model representing the trend of the data indicated as a normal state in the heat map as a graph; as well as By calculating the loss of sensor data input based on the normal trend graph model, it is determined whether a fault exists.

14. A device for diagnosing a fault, the device comprising: A sensor signal acquisition module is configured to acquire sensor signals related to the vehicle in the form of time series data; a feature acquisition module configured to obtain features classified into a predetermined number of frequency indices by performing Fourier transform on predetermined time intervals in the time series data; a scaling module configured to scale the feature to a predetermined range of values; a heat map generation module configured to generate a graph indicating the values ​​depending on a frequency index and to generate a heat map from the graph; A learning module configured to train a normal trend graph model, wherein the normal trend graph model represents the trend of data indicated as a normal state in the heat map as a graph; as well as A fault determination module is configured to determine whether a fault exists by calculating a loss of sensor data input based on the normal trend graph model.

15. The device according to claim 14, wherein: The normal trend graph model is represented as a graph having a shape having a uniform thickness and extending along an axis corresponding to the frequency index.

16. The device according to claim 15, wherein: The graph includes a center line extending along an axis corresponding to the frequency index, and includes a peripheral area formed around the center line to have a predetermined width and extending along the center line.

17. The apparatus of claim 15, wherein: Disposing a first portion and a second portion along an axis corresponding to the frequency index; The normal trend graph model is represented as a first graph of a shape having a first thickness at the first portion and extending along an axis corresponding to the frequency index; as well as The normal trend graph model is represented as a second graph of a shape having a second thickness different from the first thickness at the second portion and extending along the axis corresponding to the frequency index.

18. The apparatus of claim 14, wherein: The loss is calculated by the loss function expressed as the following equation: Among them, y i is the output value, is the predicted value, and α i is a predetermined α value.

19. The apparatus according to claim 18, wherein: The heat map is converted into a curve graph having binary colors according to a predetermined ratio, and the predetermined alpha value is determined based on the curve graph.

20. The apparatus of claim 18, wherein: Disposing a first portion and a second portion along an axis corresponding to the frequency index; and The predetermined α values ​​include a first α value determined corresponding to the first portion, and include a second α value different from the first α value determined corresponding to the second portion.

Images