METHOD AND DEVICE FOR DIAGNOSING A FAULT

Abstract

A method for diagnosing a fault may comprise: collecting (S01) sensor signals relating to a vehicle in the form of time series data, obtaining (S202) a feature classified into a predetermined number of frequency indices by performing a Fourier transform on a predetermined time interval in the time series data, scaling (S203) the feature to a value of a predetermined range, generating (S204) a curve showing the value as a function of a frequency index, generating (S205) a heat map from the curve, training (S206) a normal trend figure model representing a trend of data indicating a normal state in the heat map as a figure, determining (S207) whether a fault exists by calculating a loss for a sensor data input based on the normal trend figure model.

Description

Technical field

[0001] The present disclosure and invention relates to a method and apparatus for diagnosing a fault of a vehicle. background

[0002] Unsupervised learning, a machine learning technique, is a learning method for discovering patterns or structures in the absence of correct answers or labels for input data. In an unsupervised learning model, useful information is extracted through self-learning with only given data, so that the information can be useful for finding hidden features or inherent structures in data. Techniques such as clustering and dimensionality reduction can be used for unsupervised learning. Clustering involves grouping data with similar properties into the same group, which can help discover patterns or categories of data. K-means, density-based spatial clustering with noise (DBSCAN), hierarchical clustering, and the like can be used to perform clustering.Dimensionality reduction is a technique that reduces data complexity and facilitates visualization by transforming high-dimensional data into a low-dimensional space. This technique can remove noise or unnecessary features while preserving the key information of the data. Principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), and autoencoders can be used.

[0003] The information disclosed in this Background section is provided solely for the purpose of facilitating a better understanding of the background of the present disclosure and may therefore contain information that is not part of the prior art as already known to one skilled in the art. Short explanation

[0004] The present disclosure or invention (hereinafter also referred to as "disclosure") provides a method and apparatus for diagnosing a fault. Specifically, after a heatmap has been established (e.g., created) for each sensor signal of a vehicle, the method and apparatus can diagnose a fault of a vehicle based on a normal trend figure model, which represents a trend of the data as a figure (e.g., a curve, a line), through unsupervised learning using only data representing normal states.

[0005] A method for diagnosing a fault, such as a malfunction and / or failure, may comprise collecting sensor signals relating to a vehicle (e.g., a motor vehicle, in particular a passenger car) in the form of time series data. The method for diagnosing the fault may also comprise obtaining a feature classified (e.g., divided, segmented) into a predetermined number of frequency indices by performing a Fourier transform on a predetermined time interval in the time series data. The method for diagnosing the fault may also comprise scaling the feature to a value within a predetermined range. The method for diagnosing the fault may also comprise generating a curve (e.g., a graph) showing the value as a function of a frequency index.The method for diagnosing the fault may also include generating a heatmap from the curve. The method for diagnosing the fault may also include training a normal trend figure model that represents the trend of data indicating a normal state in the heatmap as a figure. The method for diagnosing the fault may also include determining whether a fault exists by calculating a loss for a sensor data input based on the normal trend figure model.

[0006] The normal trend figure model can be represented as a figure of a shape having a uniform thickness and extending along an axis corresponding to the frequency index.

[0007] The figure may have a center line extending along the axis corresponding to the frequency index and a peripheral portion formed around the center line to have a predetermined width and extending along the center line.

[0008] A first section and a second section may be set (e.g., fixed, adjusted) along the axis corresponding to the frequency index. The normal trend figure model may be represented as a first figure of a shape having a first thickness in the first section and extending along the axis corresponding to the frequency index. The normal trend figure model may be represented as a second figure of a shape having a second thickness, different from the first thickness, in the second section and extending along the axis corresponding to the frequency index.

[0009] The sensor signals may include sensor signals relating to at least one of a wheel speed, a longitudinal acceleration, a lateral acceleration, a vertical direction acceleration, a yaw rate, a roll rate, a pitch rate, a steering wheel angle, a steering wheel angular velocity, or a vehicle noise.

[0010] Collecting the sensor signals may include collecting the sensor signals via a control unit network, CAN for short (derived from the English “Controller Area Network”), wherein the CAN may include at least one of a chassis CAN (C-CAN), a gateway CAN (G-CAN) or a powertrain CAN (P-CAN).

[0011] Collecting the sensor signals may include collecting the sound generated in the vehicle by a recording device.

[0012] The scaling may comprise transforming the feature to have a minimum value of 0 and a maximum value of 1 by minimum-maximum (min-max) normalization.

[0013] The time series data can only contain normal operation data (e.g., normal driving data), and the normal trend figure model is trained based on the normal operation data.

[0014] The loss can be calculated by a loss function expressed by the following equation: {0if |yi−y^i|<αie10*|yi−y^i|otherwise where y i is an output value, ŷ i is a predictive value and α i is a predetermined alpha value.

[0015] The heatmap can be converted into a curve with a binary color according to a predetermined ratio, and the predetermined alpha value is determined based on the curve.

[0016] A first section and a second section may be set (e.g., fixed, adjusted) along an axis corresponding to the frequency index. The predetermined alpha value may include a first alpha value determined according to the first section and a second alpha value determined according to the second section and different from the first alpha value.

[0017] A non-transitory computer-readable medium may store a program or instructions to direct a processor of a computer to perform the following actions: collecting sensor signals relating to a vehicle in the form of time series data, obtaining a feature classified (e.g., divided, segmented) into a predetermined number of frequency indices by performing a Fourier transform on a predetermined time interval in the time series data, scaling the feature to a value of a predetermined range, generating a curve showing the value based on a frequency index, generating a heat map from the curve, training a normal trend figure model that represents a trend (e.g., a history) of data indicative of a normal state in the heat map as a figure, and determining whether an error exists by calculating a loss for a sensor data input based on the normal trend figure model.

[0018] A device for diagnosing a fault may include a sensor signal collection module configured to collect sensor signals relating to a vehicle in the form of time series data. The device for diagnosing a fault may also include a feature acquisition module configured to acquire a feature classified (e.g., divided, segmented) into a predetermined number of frequency indices by performing a Fourier transform on a predetermined time interval in the time series data. The device for diagnosing a fault may also include a scaling module configured to scale the feature to a value within a predetermined range.The device for diagnosing a fault may also include a heatmap generation module configured to generate a curve showing the value as a function of a frequency index and generate a heatmap from the curve. The device for diagnosing a fault may also include a learning module configured to train a normal trend figure model that represents the trend of data indicating a normal state in the heatmap as a figure. The device for diagnosing a fault may also include a fault determination module configured to determine whether a fault exists by calculating a loss for a sensor data input based on the normal trend figure model.

[0019] The normal trend figure model can be represented as a figure of a shape having a uniform thickness and extending along an axis corresponding to the frequency index.

[0020] The figure may have a center line extending along the axis corresponding to the frequency index and a peripheral portion formed around the center line to have a predetermined width and extending along the center line.

[0021] A first section and a second section may be set (e.g., fixed, adjusted) along the axis corresponding to the frequency index. The normal trend figure model may be represented as a first figure of a shape having a first thickness in the first section and extending along the axis corresponding to the frequency index. The normal trend figure model may be represented as a second figure of a shape having a second thickness, different from the first thickness, in the second section and extending along the axis corresponding to the frequency index.

[0022] The loss can be calculated by a loss function expressed by the following equation: {0if |yi−y^i|<αie10*|yi−y^i|otherwise where y i is an output value, ŷ i is a predictive value and α iis a predetermined alpha value.

[0023] The heatmap can be converted into a curve with a binary color according to a predetermined ratio, and the predetermined alpha value is determined based on the curve.

[0024] A first section and a second section may be set (e.g., fixed, adjusted) along an axis corresponding to the frequency index. The predetermined alpha value may include a first alpha value determined according to the first section and a second alpha value determined according to the second section and different from the first alpha value.

[0025] According to one embodiment, after establishing a heat map for each sensor signal of a vehicle based on a normal trend figure model that represents a trend of the data as a figure through unsupervised learning using only data representing normal states, a vehicle fault can be diagnosed with high accuracy using a small memory, and a new type of fault can also be detected. Furthermore, since this model is very small in size, it is also advantageous for mass production, and a fault diagnosis function can be added without requiring the separate provision of additional hardware. Short description of the drawings Fig. 1 is a block diagram for explaining an apparatus for diagnosing a fault according to an embodiment. Fig.2 is a flowchart for explaining a method for diagnosing a fault according to an embodiment. Fig. 3 is a drawing for explaining a feature acquisition module according to an embodiment. Fig. 4 and Fig. 5 are drawings for explaining a heat map generation module according to an embodiment. Fig. 6 is a drawing for explaining a normal trend figure model according to an embodiment. Fig. 7 is a drawing for explaining a normal trend figure model according to an embodiment. Fig. 8 is a drawing for explaining a normal trend figure model according to an embodiment. Fig. 9 is a drawing for explaining a normal trend figure model according to an embodiment. Fig. 10 is a drawing for explaining a learning module according to an embodiment. Fig. 11, Fig. 12 and Fig. 13 are drawings showing results of a performance comparison related to a method and apparatus for diagnosing a fault according to embodiments. Fig. 14 is a drawing for explaining a data processing apparatus according to an embodiment. Detailed description

[0026] The present disclosure will be described in more detail below with reference to the accompanying drawings, in which embodiments of the present disclosure are illustrated. As those skilled in the art will appreciate, the described embodiments may be modified in many different ways without departing from the spirit or scope of the present disclosure. Accordingly, the drawings and description are to be considered as illustrative and not restrictive. Like reference characters designate like elements throughout the description.

[0027] When, in the description and claims, a part "includes" a particular element, this means that other elements may be present, not that other elements are excluded, unless otherwise stated. Terms that include ordinal numbers, such as first, second, and the like, are used only to describe numerous components and are not to be interpreted as limiting those components. The terms are used only to distinguish one component from other components.

[0028] In addition, terms such as "...part", "...section", "...or" or "-module" disclosed in the present description may mean a unit that can process at least one function or process described in this description, and this may be implemented by hardware, software, or a combination thereof. When a "...part", "...section", "...or" or "-module" or the like of the present disclosure is described as having a purpose or performing an operation, process, function, or the like, the "...part", "...section", "...or" or "-module" or the like should be considered herein as "configured" to fulfill that purpose or to perform that operation, process, or function. Each component, device, element, and the like may include a processor and a memory, such asa non-transitory computer-readable medium, as part of the device separately or integrated therewith.

[0029] Fig. 1 is a block diagram for explaining an apparatus for diagnosing a fault according to an embodiment.

[0030] Referring to Fig.1, according to one embodiment, a device 10 for diagnosing a fault, such as a malfunction and / or failure, may perform unsupervised learning only on normal operating data (e.g., normal driving data) and detect abnormal operation (e.g., abnormal driving) based on the learning on the normal operating data to diagnose a fault when a sensor signal of a vehicle deviates from typical behavior seen during normal operation. In particular, when the operating behavior (e.g., driving behavior) of the vehicle is abnormal, for example, when abnormal vibrations or noises occur during operation (e.g., driving) of the vehicle, the device 10 for diagnosing a fault may perform fault detection if an occurred fault is not identified.In other words, since the abnormal behavior is reflected in sensor data relating to vehicle behavior when a fault occurs in the vehicle, the fault diagnosis device 10 may operate to primarily consider it a fault when the vehicle behavior deviates from the typical behavior observed during normal operation (e.g., normal driving). The fault diagnosis device 10 may also operate to secondarily check whether the fault has occurred through a detailed inspection of the vehicle. In this way, the fault diagnosis device 10 may identify a known fault, but also detect a new fault that is not yet known, e.g., due to the application of a new technology.

[0031] For this purpose, the device 10 for diagnosing a fault can obtain a feature from the sensor signal that characteristically affects the vehicle. The device 10 for diagnosing a fault can also create a heat map from this. The device 10 for diagnosing a fault can also diagnose a fault of the vehicle based on a normal trend figure model (e.g., normal trend figure model / model for a figure of a normal trend - English "normal trend figure model"), which represents the trend (i.e., e.g., the course) of data indicating the normal state in the heat map as a figure (e.g., a curve, a line).Accordingly, it has been found that the fault diagnosis performance of the apparatus 10 for diagnosing a fault has excellent determination accuracy with very low memory consumption, compared with the case of using an autoencoder, a generative adversarial network (GAN) and a class 1 support vector machine (SVM), which will be described below with reference to FIG. Fig. 11, Fig. 12 and Fig. 13 is described.

[0032] The device 10 for diagnosing a fault may include the sensor signal collection module 110, a feature acquisition module 120, a scaling module 130, the heat map generation module 140, a learning module 150, and a fault determination module 160.

[0033] The sensor signal collection module 110 may collect the sensor signal relating to the vehicle in the form of time-series data. The time-series data is data that is measured or recorded at regular time intervals (e.g., 100 Hz, 10 ms sampling) and may comprise a continuous sequence of data points in time order. The sensor signal may comprise various types of sensor signals that can be acquired with respect to the vehicle. For example, the sensor signal may comprise the sensor signal relating to at least one of a wheel speed, a longitudinal acceleration, a lateral acceleration, a vertical acceleration, a yaw rate, a roll rate, a pitch rate, a steering wheel angle, a steering wheel angular velocity, or the vehicle noise.

[0034] In some embodiments, the sensor signal may be collected via a Controller Area Network (CAN). CAN communication is a standardized protocol used to transfer data between electronic devices within the vehicle and may be used for fault diagnosis, error detection, transmission of operational (e.g., driving-related) data, and the like. In some embodiments, CAN may include at least one of a chassis CAN (C-CAN), a gateway CAN (G-CAN), or a powertrain CAN (P-CAN). The C-CAN may be a CAN bus responsible for communication between components related to the body, such as the suspension, braking system, and vehicle stability system.The G-CAN is a central gateway used to connect multiple CAN networks in the vehicle, facilitate data transfer and conversion between different CAN networks, and support integration between vehicle systems. The P-CAN can be a CAN bus that handles communication between components related to a power transmission system, such as an internal combustion engine control module, a transmission control module, and a fuel management system.

[0035] In some embodiments, the sensor signal can also be collected by a recording device. In other words, the recording device can be used to capture the sound (e.g., noise) generated in the vehicle.

[0036] The feature acquisition module 120 may acquire the feature classified (e.g., divided, segmented) into a predetermined number of frequency indices by performing a Fourier transform on a predetermined time interval in the time series data. The predetermined time interval may be defined by a window including a start point and an end point. The feature acquisition module 120 may perform the Fourier transform on raw data included in the window among the sensor signals collected by the sensor signal collection module 110. Furthermore, the feature acquisition module 120 acquires the feature classified (e.g., divided, segmented) into the predetermined number of frequency indices.Here, when the number of time series data included in the window is n (n is an integer greater than or equal to 2), (n / 2 - 1) data items obtained by removing the first index corresponding to the sum of signal values ​​from n / 2 data items obtained by removing a symmetry of n sizes can be set as the final data to be used.

[0037] The scaling module 130 may scale the feature acquired by the feature acquisition module 120 to a value within a predetermined range. Based on the feature value acquired by the feature acquisition module 120, in some embodiments, the scaling module 130 may transform the feature to have a minimum value of 0 and a maximum value of 1 through minimum-maximum (min-max) normalization.

[0038] The heatmap generation module 140 can generate a curve (e.g., a graph) showing the feature value scaled by the scaling module 130 as a function of the frequency index, and can generate the heatmap (also called a "heat map") from the curve. The heatmap, used to analyze a signal trend for normal operation (e.g., normal driving) of the vehicle, is a graphical representation that visualizes the distribution and density of data in color, and the value of each data point can be expressed as the intensity or saturation of the color. Such a heatmap can be created for each sensor signal of the vehicle.

[0039] The heatmap generation module 140 may discretize the value of the vertical axis, i.e., the scaled value, with respect to the curve obtained after scaling the feature acquired by the feature acquisition module 120 to a value within a predetermined range. For example, the vertical axis may be divided into a plurality of sections of the same size, a first value (e.g., 1) may be assigned to a section of the plurality of sections to which the scaling value (i.e., e.g., the scaled value) belongs, and a second value (e.g., 0) may be assigned to a section to which the scaling value (i.e., e.g., the scaled value) does not belong, to generate a first table.

[0040] Thereafter, a second table, a third table, ..., and a k-th table (k is an integer greater than or equal to 2) can be generated by repeating the above-described process with respect to further feature values ​​obtained and scaled based on newly collected data.

[0041] Then, when the first table, the second table, ..., and the kth table are overlaid, for elements where the multiple sections and the frequency index intersect in the table, the frequency counts of the elements at the same position in the first table, the second table, ..., and the kth table can be summed. According to the overall frequency count calculated as above, the heat map can be generated by displaying the element with the highest frequency as the darkest and gradually lightening the remaining elements.

[0042] In some embodiments, a threshold may be set in advance in the process of processing items with high frequency values ​​in the heat map in bold. For example, the threshold may be set to a value corresponding to n% of k. For example, if k is 10,000 and n is set to 2, the threshold may be set to 200. Accordingly, after summing the frequency numbers of the items at the same position of the first table, the second table, ..., and the k-th table, the heat map may be generated if the summed frequency number exceeds the threshold after replacing the frequency number of the corresponding item with the threshold.In other words, for elements (e.g., elements whose frequency count is 210, 204, 212, or the like) whose summed frequency counts exceed 200, their frequency counts (e.g., 210, 204, 212) can all be replaced by 200. In this case, the value of n can be defined as an appropriate value according to a specific implementation purpose or implementation environment. In this way, by introducing the threshold when generating the heatmap, it is possible to maximize an error indicating a yet-undetected defect, since a part that differs from a normal part in the heatmap can be displayed as darkly as possible.

[0043] The learning module 150 may train the normal trend figure model, which represents the trend of the data indicating a normal state in the heat map generated by the heat map generation module 140 as a figure by using the data used for generating the heat map, i.e., the first table, the second table, ..., and the kth table, as input. Here, the normal trend figure model may be a figure (e.g., a curve, a line) in the shape of a band with a certain thickness. When the normal trend figure model is generated by the learning module 150, if there is a large amount of data deviating from the figure, it may be interpreted as indicating that the vehicle's drivability is abnormal or that the vehicle is likely to have a malfunction.

[0044] In some embodiments, the normal trend figure model may be represented (e.g., depicted) as a figure having a shape that has a uniform thickness and extends along an axis corresponding to the frequency index. In other words, the normal trend figure model may be trained by using the data used for generating the heat map generated by the heat map generation module 140 as training data, so that the normal trend figure model can be trained to have a uniform thickness and extend in a horizontal direction. Here, the time series data serving as the training data may include only the normal operation data (e.g., normal driving data), and the normal trend figure model may be trained based on the normal operation data.

[0045] In some embodiments, the normal trend figure model may have a central line extending along the axis corresponding to the frequency index and a peripheral region (e.g., peripheral region, surrounding region) formed around the central line to have a predetermined width and extend along the central line. In other words, the normal trend figure model may be trained by using the data used for generating the heat map generated by the heat map generation module 140 as the training data, so that the normal trend figure model may be trained to have the central line and the peripheral region and extend in the horizontal direction. The time series data serving as the training data may include only the normal operation data (e.g.,normal driving data), and the normal trend figure model can be trained based on the normal operation data.

[0046] In some embodiments, the normal trend figure model may also be generated to have a plurality of thicknesses. Specifically, a first section and a second section may be set (e.g., fixed, adjusted) along the axis corresponding to the frequency index. The normal trend figure model may be represented (e.g., depicted) as a first figure of a shape having a first thickness in the first section and extending along the axis corresponding to the frequency index, and represented (e.g., depicted) as a second figure of a shape having a second thickness, different from the first thickness, in the second section and extending along the axis corresponding to the frequency index.

[0047] The error determination module 160 may determine whether an error exists by calculating a loss with respect to the sensor data input based on the normal trend figure model generated by the learning module 150.

[0048] In some embodiments, the loss may be calculated by a loss function expressed by the following equation. {0if |yi−y^i|<αie10*|yi−y^i|otherwise

[0049] Here you can i be an output value, can ŷ i be a predictive value and can α ia predetermined alpha value. The alpha value can be a value corresponding to (e.g., equal to) a thickness of the boundary region formed in the normal trend figure model around the center line so that it has the predetermined width and extends along the center line. Accordingly, the thickness of the normal trend figure model can be a value obtained by doubling the alpha value. The alpha value can therefore be set (e.g., fixed, adjusted) to an appropriate value to adjust the thickness of the normal trend figure model and can also be used to adjust the boundary between what is determined to be normal and what is determined to be abnormal. The loss function can be used to calculate the error for the data used to generate the heatmap, i.e., the first table, the second table, etc., and the k-th table, and a normality or abnormality (e.g., membership in normal values ​​or in abnormal values) can be determined based on the average value.

[0050] Here is y i an expected output value when any x is entered, and can y i represent the output value of the predetermined data, i.e., a solution or a ground truth. In addition, ŷ i a value which is output when x is entered into a learning model and can ŷ i can be called an interference value in the sense that the learning model has drawn a conclusion for the input x. The learning can be in a direction of reducing the difference between y i and ŷ i In other words, if the difference between y i and ŷ i within α i the difference between y i and ŷ ibe regarded as 0. In general, after learning, the prediction value output by the learning model is used directly to classify a class of objects (e.g., distinguishing a dog from a cat) or to predict a continuous value (e.g., predicting the current speed of the vehicle). However, according to embodiments, the result output by the learning model is not used unchanged, but the result ŷ i of the learning model and the loss of ground truth y i are used.

[0051] In some embodiments, the alpha value may be a value obtained by extracting the thickness of a portion to be provided as a band shape in the heat map for each signal and taking half of the extracted thickness.

[0052] In some embodiments, the heatmap may be converted into a curve (e.g., a graph) with a binary color according to a predetermined ratio, and the alpha value may be determined based on the converted curve.

[0053] In some embodiments, the first portion and the second portion may be set (e.g., fixed, adjusted) along the axis corresponding to the frequency index. The alpha value may include a first alpha value determined according to the first portion and a second alpha value determined according to the second portion but different from the first alpha value.

[0054] In other words, a fault diagnosis device according to one embodiment may collect normal operation data (e.g., normal driving data), establish a heat map for each sensor signal, set the alpha value based on the heat map, and then perform learning. After learning is complete, as many normal trend figure models as the number of sensor signals may be generated. A fault diagnosis device may set (e.g., establish, adjust) a threshold for each of the sensor signals based on normal test data, and then determine, for example, estimate, a loss based on continuous operation data (e.g., driving data) for a certain unit of time (e.g., 20 minutes). Thus, a "potential fault" (e.g., a potential malfunction / failure) may be determined when the loss exceeds the threshold.In particular, additional inspection of the vehicle classified as a “potential fault” may identify and diagnose a new type of fault.

[0055] Fig. 2 is a flowchart for explaining a method for diagnosing a fault according to an embodiment.

[0056] With reference to Fig.2, a method for diagnosing a fault according to one embodiment may include a step S201 of collecting the sensor signal relating to the vehicle in a form of the time series data. The method for diagnosing a fault may also include a step S202 of obtaining the feature classified (e.g., divided, disaggregated) into the predetermined number of frequency indices by performing the Fourier transform on the predetermined time interval in the time series data. The method for diagnosing a fault may also include a step S203 of scaling the feature to a value of the predetermined range. The method for diagnosing a fault may also include a step S204 of generating a curve showing the scaled feature value as a function of the frequency index.The method for diagnosing a fault may also include a step S205 of generating the heatmap from the curve. The method for diagnosing a fault may also include a step S206 of training the normal trend figure model, which represents the trend of the data indicating the normal state in the heatmap as a figure. The method for diagnosing a fault may also include a step S207 of determining whether a fault exists by calculating the loss with respect to the sensor data input based on the normal trend figure model.

[0057] For a more detailed description of the method, please refer to the description above with respect to Fig. 1 and the description to be described below with regard to Fig. 3 to 14, so duplicate descriptions have been omitted here.

[0058] Fig.3 is a drawing for explaining a feature acquisition module according to an embodiment.

[0059] Referring to Fig. 3, the feature acquisition module 120 may perform the Fourier transform on raw data contained in a preset time window in the sensor signals in the form of time series data collected by the sensor signal collection module 110. With reference to a Fig.3, for a given time t, the end point can be set within the range of t 1.28 seconds (e.g., t + 1.28 seconds) and t 5.12 seconds (e.g., t + 5.12 seconds). In the case where the end point is t + 1.28 seconds, a total of 128 data items can be included in the time window when performing the Fourier transform. The feature acquisition module 120 can acquire features classified (e.g., divided, segmented) into 63 frequency indices. Here, for the frequency index, when 128 time series data are included in the window, 63 data items obtained by removing the first index corresponding to a sum of the signal values ​​from 64 data obtained by removing the symmetry of 128 quantities can be set as the final data to be used.When time t is + 5.12 seconds and the Fourier transform is performed, a total of 512 data items can be included in the time window, and from this, the feature classified (e.g., divided, segmented) into 255 frequency indices can be obtained. In the present embodiment, the numbers 1.28 seconds and 5.12 seconds are not intended to limit the scope of the present disclosure and may be changed to other values ​​depending on the specific implementation purpose and environment.

[0060] Fig. 4 and Fig. 5 are drawings for explaining the heat map generation module according to an embodiment.

[0061] Referring to Fig.4, the heatmap generation module 140 can generate the heatmap related to the longitudinal acceleration, the heatmap related to the lateral acceleration, the heatmap related to the yaw rate, and the heatmap related to the front left wheel speed (FL wheel speed for short, e.g., the speed of the left front wheel). In each heatmap, it can be seen that the values ​​of the features are scaled between 0 and 1.

[0062] With reference to Fig. 5, the heatmap generation module 140 may calculate the value of the vertical axis, ie, the scaled value, with respect to the curve obtained after scaling the feature acquired by the feature acquisition module 120 to a value of the predetermined range (the value shown in Fig.5 in the upper left corner). For example, the vertical axis can be divided into a plurality of sections of equal size, the first value (e.g., 1) can be assigned to a section of the plurality of sections to which the scaling value belongs, and the second value (e.g., 0) can be assigned to a section to which the scaling value does not belong, in order to generate the first table. For example, the value of the vertical axis corresponding to frequency index 1 can be 0.11x (x is an integer), the value of the vertical axis corresponding to frequency index 3 can be 0.12x (x is an integer), and the value of the vertical axis corresponding to frequency index 63 can be 0.10x (x is an integer). In this case, in a column corresponding to frequency index 1, a section indicated by "0.11" (e.g.,a table field in row "0.11" and in the column corresponding to frequency index 1) can be assigned the value 1 and the remaining sections the value 0. In the case of a column belonging to frequency index 3, a section specified as "0.12" (e.g. a table field in row "0.12" and in the column corresponding to frequency index 3) can be assigned the value 1 and the remaining sections the value 0. In the case of a column belonging to frequency index 63, a section specified as "0.10" (e.g. a table field in row "0.10" and in the column corresponding to frequency index 63) can be assigned the value 1 and the remaining sections the value 0.

[0063] Subsequently, the second table, the third table, ..., and the kth table (k is an integer greater than or equal to 2) can be generated by repeating the above procedure for additional feature values ​​obtained and scaled based on newly collected data. For each of these tables, the procedure can be performed in the same way as for generating the first table.

[0064] Subsequently, when the first table, the second table, ..., and the kth table are superimposed, for elements where the multiple sections and the frequency index in the table intersect, frequency counts of elements at the same position in the first table, the second table, ..., and the kth table can be summed. For example, in a column belonging to the frequency index of 1, the frequency count of the sections (i.e., the rows) indicated by "0.11" and "0.00" may be 3000, which is the largest value; the frequency count of the section indicated by "0.10" may be 2600; the frequency count of the section indicated by "0.01" may be 2000; and the frequency count of the section indicated by "0.12" may be 1200.In a column with a frequency index of 1, the segments indicated by "0.11" and "0.00" can be processed to be the darkest, and the remaining segments can be processed to become gradually brighter in descending order of value. Furthermore, in a column with a frequency index of 2, the frequency count of the segments indicated by "0.11" and "0.002" can be 2000, which is the largest value; the frequency count of the segment indicated by "0.12" can be 1900; the frequency count of the segment indicated by "0.10" can be 1500; and the frequency count of the segment indicated by "0.01" can be 1300.In a column corresponding to a frequency index of 2, the sections indicated by "0.11" and "0.00" can be processed to be the darkest, and the remaining elements can be processed to become gradually brighter in descending order of value. If such a process is repeated from the column corresponding to a frequency index of 3 to the column corresponding to a frequency index of 63, the value indicated in the lower left corner of . Fig. The heatmap shown in Figure 5 can be generated.

[0065] Fig. 6 is a drawing for explaining the normal trend figure model according to an embodiment.

[0066] Referring to Fig.6 is the heat map shown on the left for the normal rear left wheel speed (RL wheel speed for short, e.g., the speed of the left rear wheel) and shows that the normal trend figure model of the shape having a uniform thickness and extending along the axis corresponding to the frequency index is generated by the learning module 150.

[0067] The heatmap shown on the right is for a rear-left wheel speed (RL-wheel speed) at a leakage (e.g., leakage) of the rear-left wheel suspension (also "suspension-rear-left (RL) wheel leakage rear-left (RL) wheel speed", also "suspension-RL-leakage" for short, English "suspension rear-left (RL) wheel leakage rear-left (RL) wheel speed"), where the normal trend figure model generated by the learning module 150 shown on the left is superimposed. As indicated by a rectangle, a case in which values ​​with a significant error are included in the heatmap values ​​at a certain level or greater can be interpreted as an abnormality / anomaly or a possibility of a failure.

[0068] Fig. 7 is a drawing for explaining the normal trend figure model according to an embodiment.

[0069] Referring to Fig.7, the heat map on the left side for the normal rear left wheel speed (RL wheel speed) is shown, showing that the normal trend figure model includes the center line. The normal trend figure model may also include the edge region formed around the center line to have the predetermined width and extend along the center line. The normal trend figure model has a shape extending along the axis corresponding to the frequency index and is generated by the learning module 150.

[0070] The heatmap shown on the right is for the rear left wheel speed (RL wheel speed) in the event of a leak (e.g., leakage) of the rear left wheel suspension, with the normal trend figure model generated by the learning module 150, shown on the left, superimposed. As indicated by a rectangle, a case in which values ​​with a significant error are included in the heatmap values ​​at a certain level or greater can be interpreted as an abnormality or a possibility of a failure.

[0071] Fig. 8 is a drawing for explaining the normal trend figure model according to an embodiment.

[0072] In Fig.8 shows the heat map for lateral acceleration, where the first section and the second section may be set (e.g., fixed, adjusted) along the axis corresponding to the frequency index. The normal trend figure model may be represented / displayed as the first figure of a shape having the first thickness in the first section and extending along the axis corresponding to the frequency index, and may be represented / displayed as the second figure of a shape having the second thickness, different from the first thickness, in the second section and extending along the axis corresponding to the frequency index.

[0073] Furthermore, the alpha value can also be set (e.g., fixed, adjusted) to have the first alpha value determined according to the first section and the second alpha value determined according to the second section but different from the first alpha value.

[0074] Fig. 9 is a drawing for explaining the normal trend figure model according to an embodiment.

[0075] With reference to Fig. 9, after converting the heat map into the curve with the binary color (also binary color) according to the predetermined ratio, the normal trend figure model can be set based on the converted curve according to an embodiment.

[0076] The value of the vertical axis, ie the scaled value, can be calculated with respect to the curve obtained after scaling the feature obtained by the feature acquisition module 120 to a value of the predetermined range (shown in the upper left corner in Fig.9). For example, the vertical axis may be divided into the plurality of sections having the same size, the first value (e.g., 1) may be assigned to a section among the plurality of sections to which the scaling value belongs, and the second value (e.g., 0) may be assigned to a section to which the scaling value does not belong, to generate the first table. For example, the vertical axis value corresponding to the frequency index of 1 may be 0.11x (x is an integer), the vertical axis value corresponding to the frequency index of 3 may be 0.12x (x is an integer), and the vertical axis value corresponding to the frequency index of 63 may be 0.10x (x is an integer).In this case, in a column belonging to the frequency index of 1, a section indicated by "0.11" can be assigned the value 1, and the remaining sections can be assigned the value 0. In a column belonging to the frequency index of 3, a section indicated by "0.12" can be assigned the value 1, and the remaining sections can be assigned the value 0. In the case of a column belonging to the frequency index of 63, a section indicated by "0.10" can be assigned the value 1, and the remaining sections can be assigned the value 0.

[0077] Subsequently, the second table, the third table, ..., and the kth table (k is an integer greater than or equal to 2) can be generated by repeating the above process for additional feature values ​​obtained and scaled based on newly collected data. For each of these tables, the process can be performed in the same way as for generating the first table.

[0078] Subsequently, when the first table, the second table, ..., and the k-th table are superimposed, for elements where the multiple sections and the frequency index in the table intersect, frequency counts of elements at the same position in the first table, the second table, ..., and the k-th table may be summed up. For example, in a column belonging to the frequency index of 1, the frequency count of the sections indicated by "0.11" and "0.00" may be 3000, which is the largest, the frequency count of the section indicated by "0.10" may be 2600, the frequency count of the section indicated by "0.01" may be 2000, and the frequency count of the section indicated by "0.12" may be 1200. At this time, a black color may be assigned to the positions whose values ​​are above the predetermined ratio (e.g.3%) for the respective elements based on the same index, or to the positions whose values ​​are above the predetermined thresholds (e.g., 100) for the respective elements. A white color can be assigned to elements whose values ​​are below the predetermined ratio or the predetermined threshold. If this process is repeated, eventually the value shown in the lower left corner of . Fig. The heatmap shown in Figure 9 can be generated.

[0079] Fig. 10 is a drawing for explaining a learning module according to an embodiment.

[0080] Referring to Fig.10, the learning module 150 prepares the training data set for each sensor signal, and the learning module 150 does not mix or use different sensor signals. With respect to a signal 1 belonging to a first operating log (e.g., a driving log or logbook) for learning, the learning module 150 may, in particular, extract and preprocess samples relating to a first window to add them to the training data set (sample #1-1), and may then extract and preprocess samples relating to a second window, which is shifted by 0.1 seconds, to add them to the training data set (sample #1-2). On the other hand, with respect to the signal 1 (and / or, e.g., signal 2) belonging to a second operating log (e.g.,driving log) for learning, extract and preprocess samples with respect to a third window to add them to the training dataset (Sample #2-1), and can then extract and preprocess samples with respect to a fourth window, which is shifted by a predetermined time, to add them to the training dataset (Sample #2-2).

[0081] This process is a process for generating data for learning and is designed to concatenate and use data for each signal for a plurality of learning logs. The reason for concatenating and using a plurality of them is to learn a wide range of normal states that can occur in numerous situations by collecting and learning normal data for numerous possible operating circumstances. The reason for learning each signal separately is that the heatmap for each signal is different from the start, which can achieve improved accuracy in detecting abnormalities / anomalies. For example, if the learning model for signal 1 classifies the data as normal, but the learning model for signal 2 classifies the data as abnormal, the final result can be classified as abnormal by performing a logical OR operation on these two results.This is because an abnormality / anomaly does not affect one behavior of the vehicle (signal 1), but may affect another behavior (signal 2).

[0082] Fig. 11, Fig. 12 and Fig. 13 are drawings showing the results of a performance comparison related to a method and an apparatus for diagnosing a fault according to the embodiments.

[0083] The fault diagnosis performance of a method and apparatus for diagnosing a fault according to embodiments showed higher accuracy than autoencoder, GAN, and Class 1 SVM, and the following results for the detection accuracy for suspension RL leakage and the detection accuracy for suspension FL leakage (for short, e.g., front left wheel suspension leakage) were derived. [Table 1] Learning model Determination accuracy for suspension RL leakage Determination accuracy for suspension FL leakage Required memory size Embodiment 86 % 54 % 1 Autoencoder 60 % 31 % 101.8 GAN 54 % 31 % 2154.7 Class 1 SVM 0 % 0 % 1621

[0084] The result according to one embodiment showed a detection accuracy of 86% for suspension RL leakage and a detection accuracy of 54% for suspension FL leakage, which is excellent compared to autoencoder, GAN and 1-class SVM, with a required memory size of 1, which is equal to the input feature.

[0085] Fig. Figure 11 shows the results for suspension RL leakage, where 32 data items out of 37 data items were detected as abnormal data for suspension RL leakage, only 22 data items were correctly detected for the autoencoder and only 20 data items for GAN, and in the case of SVM, the detection was not performed correctly.

[0086] Fig.Figure 12 shows the results for suspension FL leakage, where seven data items out of 13 data items were detected as abnormal data for suspension FL leakage, only 4 data items were accurately detected in the case of autoencoder and GAN, and the detection was not performed correctly in the case of SVM.

[0087] Fig. 13 shows the performance of the above in conjunction with Fig. 1, where the case of using the loss function as in the embodiments (shown on the lower side of Fig. 13) showed better performance than the case of using the mean square error (MSE) (shown on the upper side of Fig. 13).

[0088] Fig. 14 is a drawing for explaining a data processing apparatus according to an embodiment.

[0089] With reference to Fig.14, a method and apparatus for diagnosing a fault according to embodiments may be implemented using a data processing device 50.

[0090] The data processing device (e.g., a computer system, a computing system) 50 may include at least one of a processor 510, a memory 530, a user interface input device 540, a user interface output device 550, and / or a (data) storage device (e.g., a mass storage device) 560. The elements may communicate with each other via a bus 520. The data processing device 50 may include a network interface 570 electrically connected to a network 40. The network interface 570 may send or receive signals to and from other devices via the network 40.

[0091] The processor 510 may be implemented in numerous ways, such as a microcontroller (MCU), an application processor (AP), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), and the like. The processor 510 may be any semiconductor device configured to execute instructions stored in the memory 530 or the storage device 560. The processor 510 may be configured to perform the operations described above with reference to Fig. 1 to 13 described functions and procedures.

[0092] Memory 530 and storage device 560 may include numerous types of volatile or non-volatile storage media. For example, memory may include read-only memory (ROM) 531 and random access memory (RAM) 532. In the present embodiment, memory 530 may be located internally or externally of processor 510, and memory 530 may be connected to processor 510 by numerous known means.

[0093] In some embodiments, at least some particular configurations or functions of a method and apparatus according to the embodiments may be implemented as a program or software executed by the data processing device 50, and the program or software may be stored in a computer-readable medium.

[0094] In some embodiments, at least some particular configurations or functions of a method and apparatus according to the embodiments may be implemented using hardware or circuitry of the computing device 50 or may also be implemented using separate hardware or circuitry that may be electrically connected to the computing device 50.

[0095] According to an embodiment described above, after establishing a heat map for each sensor signal of a vehicle based on a normal trend figure model that represents a trend of the data as a figure through unsupervised learning using only data representing normal states, a vehicle fault can be diagnosed with high accuracy using a small memory, and a new type of fault can also be detected. Furthermore, since this model is very small in size, it is advantageous for mass production, and a fault diagnosis function can be added without the need to separately provide additional hardware.

[0096] Although this disclosure has been described in connection with the embodiments presently contemplated as practical, it is to be understood that the disclosure is not limited to the disclosed embodiments, but is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

[1] A method for diagnosing a fault, the method comprising: Collecting (S01) sensor signals relating to a vehicle in the form of time series data, Obtaining (S202) a feature classified into a predetermined number of frequency indices by performing a Fourier transform on a predetermined time interval in the time series data, Scaling (S203) the feature to a value of a predetermined range, Creating (S204) a curve showing the value as a function of a frequency index, Creating (S205) a heatmap from the curve, Training (S206) a normal trend figure model that represents a trend of data indicating a normal state in the heatmap as a figure, Determining (S207) whether an error exists by calculating a loss for a sensor data input based on the normal trend figure model. [2] The method of claim 1, wherein the normal trend figure model is represented as a figure of a shape having a uniform thickness and extending along an axis corresponding to the frequency index. [3] The method according to claim 2, wherein the figure has a center line extending along the axis corresponding to the frequency index and a border region formed around the center line to have a predetermined width and extending along the center line. [4] The method of claim 2, wherein: a first section and a second section are set along the axis corresponding to the frequency index, wherein the normal trend figure model is represented as a first figure of a shape having a first thickness in the first section and extending along the axis corresponding to the frequency index, and wherein the normal trend figure model is represented as a second figure of a shape having a second thickness different from the first thickness in the second section and extending along the axis corresponding to the frequency index. [5] The method according to any one of claims 1 to 4, wherein the sensor signals comprise sensor signals relating to at least one of a wheel speed, a longitudinal acceleration, a lateral acceleration, a vertical direction acceleration, a yaw rate, a roll rate, a pitch rate, a steering wheel angle, a steering wheel angular velocity, or a vehicle noise. [6] The method according to claim 5, wherein collecting the sensor signals comprises collecting the sensor signals via a control unit network, CAN for short, wherein the CAN comprises at least one of a chassis CAN, abbreviated to C-CAN, a gateway CAN, abbreviated to G-CAN, and / or a powertrain CAN, abbreviated to P-CAN. [7] The method of claim 5 or 6, wherein collecting the sensor signals comprises collecting the sound generated in the vehicle by a recording device. [8] The method of any one of claims 1 to 7, wherein said scaling comprises transforming said feature to have a minimum value of 0 and a maximum value of 1 by minimum-maximum (min-max) normalization. [9] Method according to one of claims 1 to 8, wherein the time series data comprise only normal operating data, in particular normal driving data, and the normal trend figure model is trained on the basis of the normal operating data. [10] A method according to any one of claims 1 to 9, wherein the loss is calculated by a loss function which is expressed as an equation {0if |yi−y^i|<αie10*|yi−y^i|otherwise where y i is an output value, ŷ i is a predictive value and α i is a predetermined alpha value. [11] The method of claim 10, wherein the heat map is converted into a curve having a binary color according to a predetermined ratio, and the predetermined alpha value is determined based on the curve. [12] A method according to claim 10 or 11, wherein: a first section and a second section are set along an axis corresponding to the frequency index, and the predetermined alpha value comprises a first alpha value determined according to the first section and a second alpha value determined according to the second section and different from the first alpha value. [13] A non-transitory computer-readable medium having stored thereon a program or instructions for directing a processor to perform any of the following actions: Collecting sensor signals relating to a vehicle in the form of time series data, Obtaining a feature classified into a predetermined number of frequency indices by performing a Fourier transform on a predetermined time interval in the time series data, Scaling the feature to a value within a predetermined range, Generating a curve showing the value based on a frequency index, Creating a heatmap from the curve, Training a normal trend figure model that represents a trend of data indicating a normal state in the heatmap as a figure, and Determine whether an error exists by calculating a loss for a sensor data input based on the normal trend figure model. [14] Device (10) for diagnosing a fault, the device comprising: a sensor signal collection module (110) which is configured to collect sensor signals relating to a vehicle in the form of time series data, a feature acquisition module (120) configured to acquire a feature classified into a predetermined number of frequency indices by performing a Fourier transform on a predetermined time interval in the time series data, a scaling module (130) configured to scale the feature to a value within a predetermined range, a heatmap generation module (140) configured to generate a curve showing the value as a function of a frequency index and to generate a heatmap from the curve, a learning module (150) configured to train a normal trend figure model that represents a trend of data indicating a normal state in the heatmap as a figure, and a fault determination module (160) configured to determine whether a fault exists by calculating a loss for a sensor data input based on the normal trend figure model. [15] The apparatus (10) of claim 14, wherein the normal trend figure model is represented as a figure of a shape having a uniform thickness and extending along an axis corresponding to the frequency index. [16] The device (10) of claim 15, wherein the figure has a center line extending along the axis corresponding to the frequency index and a border region formed around the center line to have a predetermined width and to extend along the center line. [17] Device (10) according to claim 15, wherein: a first section and a second section are set along the axis corresponding to the frequency index, the normal trend figure model is represented as a first figure of a shape having a first thickness in the first section and extending along the axis corresponding to the frequency index, and the normal trend figure model is represented as a second figure of a shape having a second thickness different from the first thickness in the second section and extending along the axis corresponding to the frequency index. [18] Apparatus (10) according to any one of claims 14 to 17, wherein the loss is calculated by a loss function expressed as an equation {0if |yi−y^i|<αie10*|yi−y^i|otherwise where y i is an output value, ŷ i is a predictive value and α i is a predetermined alpha value. [19] The apparatus (10) of claim 18, wherein the heat map is converted into a curve having a binary color according to a predetermined ratio, and the predetermined alpha value is determined based on the curve. [20] Device (10) according to claim 18 or 19, wherein: a first section and a second section are set along an axis corresponding to the frequency index, wherein the predetermined alpha value comprises a first alpha value determined according to the first portion and a second alpha value determined according to the second portion and different from the first alpha value.

Images