Time-series data processing method, training method, processing system, and training system

Abstract

To provide a time-series data processing method, etc., that make it possible to reduce information missing during data clipping in the two-dimensional conversion processing of a TimesNet.SOLUTION: A time-series data processing method includes: inputting input time-series data to a trained model 130; performing two-dimensional conversion processing for converting the input time-series data into two-dimensional data; performing convolution processing for inputting the two-dimensional data to a convolution layer; converting the output of the convolution layer into mono-dimensional data and outputting output time-series data; and outputting a feature amount based on the output time-series data. The two-dimensional conversion processing divides the input time-series data into a plurality of divided time-series data in an integral value clipping cycle larger than a cycle corresponding to the frequency of interest, and configures two-dimensional data from the plurality of divided time-series data.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] The present invention relates to a time-series data processing method, a learning method, a processing system, a learning system, and the like. [Background technology]

[0002] Non-Patent Document 1 discloses a time series data analysis method using machine learning called TimesNet. Specifically, the method proposes a method for efficiently extracting features from time series data by converting time series data, such as one-dimensional vibration data, into two-dimensional data, arranging both short-term and long-term features in adjacent regions in the two-dimensional data, and inputting the two-dimensional data into a convolutional layer.

[0003] This method focuses on a certain frequency and generates two-dimensional data by extracting and arranging time series data for each period corresponding to the frequency. At this time, the period is expressed as a real number, including non-integer numbers. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Haixu Wu, “TIMESNET: TEMPORAL 2D-VARIATION MODELING FOR GENERAL TIME SERIES ANALYSIS”, arXiv:2210.02186, October 5, 2022 Summary of the Invention [Problem to be solved by the invention]

[0005] However, since the number of data pieces to be extracted is an integer, the period and the number of extracted data pieces do not match, which raises the risk of information being missed when extracting data. [Means for solving the problem]

[0006] One aspect of the present disclosure includes the steps of: inputting input time series data to a trained model that has been trained to extract feature quantities of the input time series data from the input time series data; performing two-dimensional conversion processing to convert the input time series data into two-dimensional data; performing convolution processing to input the two-dimensional data into a convolution layer, converting the output of the convolution layer into one-dimensional data, and outputting output time series data; and outputting feature quantities based on the output time series data, wherein the step of performing the two-dimensional conversion processing relates to a time series data processing method in which the neural network divides the input time series data into a plurality of divided time series data at cutout periods of integer values ​​greater than a period corresponding to a frequency of interest, and constructing the two-dimensional data from the plurality of divided time series data.

[0007] Another aspect of the present disclosure includes the steps of: inputting input time series data to a learning model that extracts features of the input time series data from the input time series data; performing two-dimensional conversion processing to convert the input time series data into two-dimensional data; performing convolution processing to input the two-dimensional data to a convolution layer, converting an output of the convolution layer into one-dimensional data, and outputting output time series data; and performing learning to output features based on the output time series data, wherein the step of performing the two-dimensional conversion processing relates to a learning method that divides the input time series data into a plurality of divided time series data at cutout periods of integer values ​​greater than a period corresponding to a frequency of interest, and constructs the two-dimensional data from the plurality of divided time series data.

[0008] Yet another aspect of the present disclosure relates to a processing system including: a memory unit that includes a neural network and stores a trained model that has been trained to extract feature quantities of input time series data from the input time series data; and a processing unit that inputs the input time series data to the trained model and acquires the feature quantities based on an output of the neural network, wherein the neural network performs a two-dimensional conversion process to convert the input time series data into two-dimensional data; and a convolution process to input the two-dimensional data to a convolution layer, converting the output of the convolution layer into one-dimensional data and outputting output time series data, wherein the two-dimensional conversion process includes the steps of: dividing the input time series data into a plurality of divided time series data at a cutout period of an integer value greater than a period corresponding to a frequency of interest; and constructing the two-dimensional data from the plurality of divided time series data.

[0009] Furthermore, still another aspect of the present disclosure relates to a learning system including a neural network, a storage unit that stores a learning model that includes input time series data and extracts feature quantities of the input time series data from the input time series data, and a processing unit that inputs the input time series data to the learning model and trains the learning model to output feature quantities based on an output of the neural network, wherein the neural network performs a two-dimensional conversion process to convert the input time series data into two-dimensional data, and a convolution process to input the two-dimensional data to a convolution layer, converting an output of the convolution layer into one-dimensional data and outputting output time series data, and the two-dimensional conversion process includes a step of dividing the input time series data into a plurality of divided time series data at a cutout period of an integer value greater than a period corresponding to a frequency of interest, and a step of constructing the two-dimensional data from the plurality of divided time series data. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram of a sensor that detects vibrations of an object. [Figure 2] An example of a processing system configuration. [Figure 3] An example of a learning system configuration. [Figure 4] An example of the configuration of a learning model during learning. [Figure 5] First example of the configuration of a trained model during inference. [Figure 6] A second example of the configuration of a trained model during inference. [Figure 7] A detailed example of the configuration of TimesNet as a predictor. [Figure 8] A detailed example of the configuration of TimesNet as a classifier. [Figure 9] A detailed example of the configuration of a feature extractor. [Figure 10] A detailed example of the Times Block configuration. [Figure 11] A diagram explaining the operation of the Times Block. [Figure 12] A diagram explaining the operation of the Times Block. [Figure 13] A diagram explaining the issues with 2D conversion processing in conventional TimesNet. [Figure 14] A diagram explaining the issues with 2D conversion processing in conventional TimesNet. [Figure 15] 5A to 5C are diagrams illustrating a first example of two-dimensional conversion processing in this embodiment. [Figure 16] 5A to 5C are diagrams illustrating a first example of two-dimensional conversion processing in this embodiment. [Figure 17] 6A to 6C are diagrams illustrating a second example of two-dimensional conversion processing in this embodiment. [Figure 18] 5A to 5C are diagrams illustrating the effect of two-dimensional conversion processing in this embodiment. [Figure 19] Amplitude spectrum of the dataset used for task evaluation. [Figure 20] Evaluation results for classification tasks. [Figure 21] Evaluation results for the prediction task. [Figure 22] Example of frequency peak graph and entropy per epoch. [Figure 23]This figure shows that when the selected frequency is determined from the peak frequency of the spectrum, as in the conventional Times Net, the change in the selected frequency becomes smaller as learning progresses. DETAILED DESCRIPTION OF THE INVENTION

[0011] Preferred embodiments of the present disclosure will be described in detail below. Note that the embodiments described below do not unduly limit the scope of the claims, and not all of the configurations described in the embodiments are necessarily essential components.

[0012] 1. Processing and learning systems In the following, an example will be described in which the time-series data to be analyzed is vibration data measured by a vibration sensor. However, as will be described later, the method of this embodiment can be applied to various time-series data other than vibration data.

[0013] 1 is an explanatory diagram of a sensor that detects vibrations of an object. The object 10 includes, for example, a vibration source 11 that generates vibrations through mechanical operation. The sensor 140 detects the vibrations of the object 10 caused by the vibration source 11. Examples of the vibration source 11 include a motor, an engine, or a turbine. Alternatively, the object 10 may not include the vibration source 11, and vibrations may be applied from outside the object 10, and the sensor 140 may detect the vibrations.

[0014] The object 10 may be, for example, the vibration source 11 itself, i.e., a motor, engine, turbine, or the like. Alternatively, the object 10 may be a machine, equipment, device, or the like that includes the vibration source 11, such as a printer, air conditioner, robot, pump, belt conveyor, processing equipment, or other household or industrial equipment, a moving object such as an automobile or airplane, or industrial equipment such as a power generator or manufacturing plant. Alternatively, the object 10 may be a structure that vibrates due to an external force, such as a building, road, or bridge.

[0015] Sensor 140 detects acceleration, velocity, displacement, angular acceleration, angular velocity, or angle, and outputs a signal indicating the detected physical quantity as vibration information. Sensor 140 may be a sensor that detects one type of physical quantity, or a sensor that detects multiple types of physical quantities. Sensor 140 may also be a sensor that detects a physical quantity on one axis, or a sensor that detects physical quantities on two or more axes. Sensor 140 outputs vibration information from one or multiple channels. One channel is a channel that outputs a signal of one type of physical quantity on one axis.

[0016] The sensor 140 may be an acceleration sensor or gyro sensor using a quartz crystal oscillator as a detection element, or an acceleration sensor or gyro sensor using a MEMS as a detection element. The sensor 140 may also be an IMU (Integrated Mechanical Unit) that combines an acceleration sensor and a gyro sensor into a single unit. The sensor 140 may detect velocity or displacement by integrating acceleration detected by the detection element, or may use a detection element that detects velocity, etc. The sensor 140 may detect angular acceleration or angle by differentiating or integrating angular velocity detected by the detection element, or may use a detection element that detects angular acceleration, etc. One example of an acceleration sensor is a sensor that detects acceleration by measuring the vibration frequency, which changes in response to stress applied to a quartz crystal oscillator. One example of a gyro sensor is a sensor that detects angular velocity by detecting the Coriolis force applied to a quartz crystal oscillator. Another example of an acceleration sensor or gyro sensor is a sensor in which a mass and electrodes are formed by MEMS and that detects acceleration or angular velocity by detecting the electrostatic capacitance between the electrodes, which changes in response to the inertial force applied to the mass.

[0017] It is assumed that the sensor 140 is attached so as to be in contact with the object 10, but this is not limiting, and it is sufficient that vibrations are transmitted from the object 10 to the sensor 140. One sensor unit may be attached to the object 10, or multiple sensor units may be attached to the object 10. The multiple sensor units may be attached to different positions on the object 10, or may be attached to the same position on the object 10.

[0018] 2 shows an example of the configuration of a processing system. The processing system 100 infers the state of an object by analyzing vibration data, which is time-series data. The processing system 100 includes a processing unit 110, a storage unit 120, a sensor 140, and a presentation unit 150. Note that the sensor 140 may be provided outside the processing system 100 and connected to the processing system 100 via a cable, a network, or the like.

[0019] The processing unit 110 includes an acquisition unit 111 and an inference unit 113 .

[0020] The acquisition unit 111 receives a physical quantity signal output by the sensor 140, thereby acquiring vibration data of the object from the sensor 140. The sensor 140 may output either an analog signal or digital data. When the acquisition unit 111 receives an analog signal, it may include an A / D converter that A / D converts the analog signal into digital data. The acquisition unit 111 outputs vibration data, which is vibration information of the object, to the inference unit 113. The vibration data is time-series data of acceleration, velocity, displacement, angular acceleration, angular velocity, or angle detected by the sensor 140.

[0021] Vibration data, which is time-series data, is input to the inference unit 113. The inference unit 113 infers the vibration state from the vibration data using the trained model 130. The vibration state is a state related to at least one of condition monitoring of an object, quality control, and predictive maintenance. Detailed examples of these states will be described later. The inference unit 113 inputs the vibration data to the trained model 130 and causes the trained model 130 to output an inference result. The trained model 130 includes a TimesNet, as described later. The TimesNet includes a feature extractor that extracts features from the vibration data using multiple TimesBlocks. The TimesNet also includes at least one of a predictor that predicts future vibration data from the features and a classifier that classifies the vibration state from the features. The trained model 130 may also include a classifier that classifies the vibration state from the vibration data predicted by the predictor.

[0022] The processing unit 110 outputs presentation information based on the inference result to the presentation unit 150. The presentation unit 150 presents the presentation information to the user. The presentation unit 150 is, for example, a display, a speaker, a lamp, or a vibrator. The presentation information is a numerical value, a character, a color, an image, a sound, a light, or a vibration. The processing unit 110 may store the vibration state classification result, the vibration state detection result, or the presentation information generated therefrom in a memory or storage. The memory or storage may be common to the storage unit 120 described below.

[0023] The storage unit 120 stores the trained model 130. The learning system generates the trained model 130 by training the trained model in advance. The generated trained model 130 is stored in the storage unit 120. Various learning methods, such as supervised learning or unsupervised learning, may be adopted, and may be selected appropriately depending on the architecture of the model or the content that the model is to infer.

[0024] Various configurations may be adopted as the hardware configuration of the processing system 100. The processing system 100 is, for example, a computer, or a cloud system in which multiple computers are connected via a network or the like. The computer is not limited to a general-purpose computer such as a personal computer, but may also be a computer dedicated to performing the vibration analysis of this embodiment, or a computer incorporated into a specific device or the like.

[0025] The processing unit 110 is, for example, a processor. The processor includes, for example, one or more of a CPU, GPU, microcomputer, DSP, ASIC, FPGA, etc. CPU is an abbreviation for Central Processing Unit. GPU is an abbreviation for Graphics Processing Unit. DSP is an abbreviation for Digital Signal Processor. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array. The storage unit 120 stores a program in which the functions of each unit of the processing unit 110 are described. The processor executes the program to realize the functions of each unit of the processing unit 110 as processing.

[0026] The processing unit 110 is not limited to the software processing described above, and may be a circuit in which the functions of each unit are implemented in hardware. In this case, the storage unit 120 does not need to store a program.

[0027] The storage unit 120 is a memory or a register. The memory is a volatile memory such as RAM, or a non-volatile memory such as OTP memory or EEPROM. RAM is an abbreviation for Random Access Memory. OTP is an abbreviation for One Time Programmable. EEPROM is an abbreviation for Electrically Erasable Programmable Read Only Memory.

[0028] The trained model 130 and the above program may be stored in a non-transitory information storage medium that is a computer-readable medium. The information storage medium may be, for example, an optical disk, a memory card, a hard disk drive, or a non-volatile semiconductor memory.

[0029] 3 shows an example of the configuration of a learning system. The learning system 200 generates a trained model 130 by training a learning model 230. The learning system 200 includes a processing unit 210, a memory unit 220, an operation unit 260, and a display unit 270.

[0030] The storage unit 220 stores the learning model 230 and the learning data 225. The architecture of the learning model 230 and the trained model 130 is basically the same, but may differ in some parts. Details will be described later with reference to Figures 4 to 6, etc. The learning data 225 is data used for training the learning model 230. The learning data 225 is data used for training the learning model 230 and includes a large number of vibration data. This may include not only correct labels corresponding to each piece of vibration data in the case of supervised learning, but also unlabeled data assuming unsupervised learning or semi-supervised learning scenarios.

[0031] The processing unit 210 includes an acquisition unit 211 and a learning processing unit 214. The acquisition unit 211 reads out learning data 225 from the storage unit 220. The learning processing unit 214 inputs vibration data included in the learning data 225 into a learning model 230 and trains the learning model 230 using an appropriate loss function depending on the type of task. In a classification task, the error between the output of the model and the correct label included in the learning data is calculated as the loss function. In a time series prediction task, the error between the future value of the time series itself of the learning data and the predicted value output by the model is used as the loss function.

[0032] The operation unit 260 and the display unit 270 are what is called a user interface. The operation unit 260 is a device that allows the user to operate the learning system 200, and is, for example, a keyboard, a mouse, or a touch panel. The display unit 270 is a display that displays information, and is, for example, a liquid crystal display device.

[0033] 2. Model configuration example An example of a model configuration is shown below. However, this is an example of a model configuration to which the two-dimensional conversion method described in Figure 13 and subsequent figures can be applied, and the application of the two-dimensional conversion method of this embodiment is not limited to the model configuration described below. In other words, the two-dimensional conversion method described in Figure 13 and subsequent figures can be applied to any model that extracts features using time blocks.

[0034] 4 shows an example of the configuration of a learning model during learning. The learning model 230 includes a Times Net 405. The Times Net 405 is a deep learning model that includes a neural network, and includes a feature extractor 410 and a predictor 420. The feature extractor 410 is also called an encoder.

[0035] The feature extractor 410 is composed of multiple time blocks connected in series and extracts features from the input vibration data. The extracted features are time series data, and have a data length obtained by adding the input length of the vibration data to the predicted length predicted by the predictor 420. In this embodiment, when converting the input time series data into two-dimensional data, each time block divides the time series data of the component of a period of interest into multiple divided time series data with cut-out lengths of integer values ​​greater than the period of interest. This prevents information loss during data extraction. This point will be described in detail in Figure 13 and subsequent figures.

[0036] The predictor 420 predicts future values ​​of the vibration data from the features extracted by the feature extractor 410, and outputs vibration data including the predicted future value. Note that the target of prediction is not limited to future values, and may be a value older than the vibration data input to the times net 405, or a value at a time between two vibration data input to the times net 405.

[0037] The learning process for the learning model 230 will now be described. The acquisition unit 211 of the learning system 200 reads vibration data from the learning data 225 from the storage unit 220 and inputs it to the learning processing unit 214. The learning processing unit 214 inputs the vibration data to the feature extractor 410, which extracts features from the vibration data, and the predictor 420 predicts future values ​​of the vibration data from the features. The learning processing unit 214 trains the learning model 230 by updating the internal parameters of the feature extractor 410 and the predictor 420 based on the predicted values.

[0038] The learning process is, for example, supervised learning, and takes the form of a time series prediction task. Here, "correct data" refers to future time series values ​​of vibration data. In this context, "label" referred to in the context of supervised learning means the actual future time series values ​​of vibration data, and is an adaptation of the conventional concept of label to the context of time series prediction. For example, if the sequence length of vibration data included in the learning data 225 is 1000, vibration data with a sequence length of 800 may be input to the learning model 230, and the remaining vibration data with a sequence length of 200 may be used as the correct label.

[0039] FIG. 5 shows a first example configuration of a trained model at the time of inference. The trained model 130 includes a TimesNet 305. The TimesNet 305 is a deep learning model including a neural network, and includes a feature extractor 310, a predictor 320, and a classifier 330. The feature extractor 310 is also called an encoder. Note that the predictor 320 may be omitted.

[0040] 4 after the learning process has been performed by the learning system 200. The predictor 320 is the predictor 420 in FIG. 4 after the learning process has been performed by the learning system 200.

[0041] The acquisition unit 111 of the processing system 100 inputs vibration data acquired from the sensor 140 to the inference unit 113. The inference unit 113 inputs the vibration data to the feature extractor 310, which extracts features from the vibration data, and the predictor 320 predicts future values ​​of the vibration data from the features. The classifier 330 classifies the vibration state from the features output by the feature extractor 310. The classifier 330 classifies the vibration state of the object as to which of multiple classes the object belongs to. The multiple classes may be, for example, classes that distinguish between abnormality types such as abnormality A, B, and C, or classes that distinguish between normal and abnormal, or normal and fault. The inference result of the classifier is, for example, the probability of each vibration state, or a flag indicating which vibration state it is.

[0042] Learning of the classifier 330 can be performed, for example, as follows. The feature extractor 310 used when learning the classifier 330 is the feature extractor 410 that has been trained through the learning process described with reference to FIG. 4. The acquisition unit 211 of the learning system 200 reads vibration data from the learning data 225 from the storage unit 220 and inputs the data to the learning processing unit 214. The learning processing unit 214 inputs the vibration data to the feature extractor 310, which extracts features from the vibration data, and the untrained classifier 330 outputs a classification result from the features. The learning processing unit 214 trains the classifier 330 by updating the internal parameters of the classifier 330 based on the classification result.

[0043] The learning process for the classifier 330 is, for example, supervised learning, and the correct label is the probability of each vibration state, or a flag indicating which vibration state it is, etc. The learning processing unit 214 calculates a loss function from the classification result and the correct label output by the classifier 330, and trains the classifier 330 using the loss function.

[0044] Fig. 6 shows a second configuration example of the trained model at the time of inference. The trained model 130 includes a Times Net 305 and a classifier 360. Below, differences from the first configuration example in Fig. 5 will be mainly explained.

[0045] The inference unit 113 of the processing system 100 inputs the features extracted by the feature extractor 310 to the classifier 360. The classifier 360 classifies the vibration state from the features. The classifier 360 classifies the vibration state of the object into one of multiple classes. The multiple classes may be classes that distinguish between abnormality types such as abnormalities A, B, and C, or classes that distinguish between normal and abnormal, or normal and fault. The inference result of the classifier is, for example, the probability of each vibration state, or a flag indicating which vibration state it is.

[0046] The classifier 360 can be trained, for example, as follows. The feature extractor 310 used during training of the classifier 360 is the feature extractor 410 trained by the training process described with reference to FIG. 4. The acquisition unit 211 of the training system 200 reads vibration data from the training data 225 from the storage unit 220 and inputs the data to the training processing unit 214. The training processing unit 214 inputs the vibration data to the feature extractor 310, which extracts features from the vibration data. The features obtained from this trained feature extractor are used as input to the untrained classifier 360. The classifier 360 classifies the vibration state based on these features. The training processing unit 214 trains the classifier 360 by updating the internal parameters of the classifier 360 based on the classification results.

[0047] The learning process for the classifier 360 is, for example, unsupervised learning, in which the classifier 360 is trained to perform classification based on clustering. In this context, an unsupervised learning algorithm such as k-means is appropriate. Note that the supervised learning algorithm SVM (Support Vector Machine) is sometimes used for classification tasks in different contexts, and this description assumes the use of an unsupervised algorithm such as k-means in the context of clustering.

[0048] Figure 7 shows a detailed example of the configuration of a Times Net 305 as a predictor. Times Net 305 includes a normalization layer, an embedding layer, a first fully connected layer, a feature extractor, a second fully connected layer, and a non-normalization layer. Note that Times Net 405 during training has a similar configuration.

[0049] The sequence length of vibration data for one axis input to the Times Net 305 is called the "input length." For example, when vibration is measured using a three-axis sensor, vibration data of input length x 3 is input to the Times Net 305.

[0050] The normalization layer normalizes the vibration data input to the Times Net 305. The embedding layer performs dimensional expansion or dimensional reduction of the normalized vibration data. Here, an example is shown in which the internal dimensionality is 32 and 3 dimensions are expanded to 32 dimensions. The embedding layer outputs data of the input length × 32. The first fully connected layer is a linear layer that multiplies the data from the embedding layer by a weight matrix and adds a bias, outputting data of (input length + prediction length) × 32. The "prediction length" is the length of the vibration data predicted from the vibration data of the input length.

[0051] The feature extractor converts the data from the first fully connected layer into two-dimensional data (number of segments × period) × 32, performs processing such as convolution on the two-dimensional data, and then converts it into one-dimensional data (input length + prediction length) × 32, outputting the data. As an example, the (input length + prediction length) × 32 data input to the feature extractor contains 32 pieces of time series data (input length + prediction length). The feature extractor converts one piece of time series data (input length + prediction length) into two-dimensional data (number of segments × period), and by doing this 32 times, generates (number of segments × period) × 32 data. The process of converting to one dimension is similar.

[0052] The second fully connected layer is a linear layer that multiplies the data from the feature extractor by a weight matrix and adds a bias, outputting data of (input length + prediction length) × 3. The denormalization layer denormalizes the data from the second fully connected layer and outputs predicted data of (input length + prediction length) × 3.

[0053] In the TimesNet 305 in Fig. 7, the embedding layer, the first fully connected layer, the feature extractor, and the second fully connected layer are trainable layers. The TimesNet 305 in Fig. 7 functions as a predictor as a whole, and in terms of correspondence with Fig. 5 and Fig. 6, the second fully connected layer and the non-normalized layer in Fig. 7 correspond to the predictor 320 in Fig. 5 and Fig. 6.

[0054] 8 shows a detailed configuration example of a Times Net as a classifier. Times Net 305 includes an embedding layer, a feature extractor, an activation layer, and a fully connected layer.

[0055] The embedding layer expands or reduces the dimension of the vibration data of the input length × 3 input to the Times Net 305. Here, an example is shown in which 3 dimensions are expanded to 32 dimensions. The embedding layer outputs data of the input length × 32.

[0056] The feature extractor converts the data from the embedding layer into two-dimensional data (number of cuts x period) x 32, performs processing such as convolution on the two-dimensional data, and then converts it into one-dimensional data, outputting input length x 32 data.

[0057] The activation layer is a nonlinear layer that is a function of the data from the feature extractor. The activation layer is incorporated to learn more complex functions. The fully connected layer multiplies the data from the activation layer by a weight matrix and adds a bias, and outputs classification data that is the classification result of the vibration state.

[0058] In TimesNet 305 in Fig. 8, the embedding layer, feature extractor, and fully connected layer are trainable layers. TimesNet 305 in Fig. 8 functions as a classifier as a whole, but in terms of correspondence with Fig. 5 and Fig. 6, the activation layer and fully connected layer in Fig. 8 correspond to classifier 330 in Fig. 5 and Fig. 6.

[0059] Fig. 9 shows a detailed configuration example of the feature extractor shown in Figs. 5 to 8. Feature extractor 310 includes time blocks 311-1 to 311-n and adders 312-1 to 312-n, where n is an integer equal to or greater than 2. Note that feature extractor 410 during learning has a similar configuration.

[0060] The input time series data of the times block 311-1 is time series data input to the feature extractor 310. The times block 311-1 extracts features from the input time series data and outputs the features as output time series data. The adder 312-1 adds the input time series data and output time series data of the times block 311-1.

[0061] The input time series data of the times block 311-2 is the output time series data of the adder 312-1. The times block 311-2 extracts features from the input time series data and outputs the features as output time series data. The adder 312-2 adds the input time series data and output time series data of the times block 311-2.

[0062] Thereafter, the time blocks 311-3 to 311-n and the adders 312-3 to 312-n perform similar processing. The feature extractor 310 outputs the output time series data of the adder 312-n as the final feature.

[0063] Fig. 10 shows an example of a detailed configuration of a times block. The times block 311 shown in Fig. 10 corresponds to each of the times blocks 311-1 to 311-n shown in Fig. 9. As shown in Fig. 10, the times block 311 includes a Fourier transform unit 510, k transform units 520, a convolution processing unit 550, and an adder 560. k is an integer equal to or greater than 2. Each transform unit 520 includes a two-dimensional transform unit 530 and a one-dimensional transform unit 540. The operation of the times block 311 in Fig. 10 will be described below with reference to Figs. 11 and 12.

[0064] FIG. 11 shows an example of time series data input to a time block. Here, the time series data is shown as a waveform, but in reality, the time series data is a data string of discrete signal values. The Fourier transform unit 510 performs a Fourier transform on the input time series data to convert it into a spectrum, and sets k frequencies of interest based on the spectrum. The Fourier transform unit 510 outputs the k frequencies of interest and the input time series data to k transform units 520. Each transform unit 520 receives one of the k frequencies of interest and the input time series data. The Fourier transform unit 510 also outputs the amplitude value of the spectrum at each of the k frequencies of interest to the weight calculation unit 570.

[0065] The target frequency may be, for example, a frequency set based on the spectrum, or may be a pre-fixed frequency. The frequency set based on the spectrum is, for example, a peak frequency in the spectrum. The pre-fixed frequency is a frequency at which vibration characteristics are likely to appear, and is, for example, a frequency determined experimentally based on anomaly detection accuracy, etc.

[0066] The k frequencies of interest are referred to as the first to k-th frequencies. As shown in FIG. 11, the two-dimensional conversion units 530 of the k conversion units 520 convert the input time series data into first to k-th two-dimensional data corresponding to the first to k-th frequencies. The first two-dimensional data will be described as an example. The first frequency and the input time series data are input to the two-dimensional conversion unit 530 of any one of the k conversion units 520. The two-dimensional conversion unit 530 divides the input time series data at a first period corresponding to the first frequency. FIG. 11 shows an example in which the input time series data is divided into four. For example, if the divided time series data is a row vector, the two-dimensional conversion unit generates two-dimensional data by arranging the row vectors of the divided time series data in four columns. Note that the relationship between rows and columns may be reversed. The second to k-th two-dimensional data are generated in a similar manner.

[0067] 12, the convolution processing unit 550 inputs the first to k-th two-dimensional data to the convolution layer. The convolution layer performs convolution processing on each piece of two-dimensional data and outputs the first to k-th two-dimensional data after convolution to the one-dimensional conversion units 540 of the k conversion units 520. The one-dimensional conversion units 540 of each conversion unit 520 receive one of the first to k-th two-dimensional data as input.

[0068] Note that convolution processing unit 550 may have first to k-th convolution layers. Convolution processing unit 550 may input first two-dimensional data to the first convolution layer, second two-dimensional data to the second convolution layer, ..., k-th two-dimensional data to the k-th convolution layer.

[0069] As shown in FIG. 12, the one-dimensional conversion units 540 of the k conversion units 520 convert the first to k-th two-dimensional data after convolution into the first to k-th output one-dimensional data. The first output one-dimensional data will be described as an example. The first two-dimensional data after convolution is input to the one-dimensional conversion unit 540 of any one of the k conversion units 520. Here, it is assumed that row vectors are time-series data. The one-dimensional conversion unit 540 generates the first output one-dimensional data by serially arranging the row vectors in the first two-dimensional data after convolution in one dimension. The second to k-th output one-dimensional data are generated in a similar manner.

[0070] 12, the adder 560 generates output time-series data by weighting and adding the first to k-th output one-dimensional data. The weighting coefficients are input to the adder 560 from the weighting calculation unit 570. The weighting calculation unit 570 calculates a weighting coefficient for each output one-dimensional data based on the amplitude value of the spectrum from the Fourier transform unit 510. Specifically, the weighting calculation unit 570 calculates a weighting coefficient for the first output one-dimensional data based on the amplitude value of the spectrum at the first frequency. The same applies to the weighting coefficients for the second to k-th output one-dimensional data. The weighting calculation unit 570 is, for example, a Softmax function.

[0071] 3. 2D conversion processing The details of the 2D conversion process in Times Block will be explained below. First, the issues with the 2D conversion process in the conventional Times Net will be explained using Fig. 13 and Fig. 14.

[0072] Figures 13 and 14 show an example where the input length of the time series data is 33 and the optimal extraction period is 5.5. Each square in the time series data represents one piece of data, and the number inside the square indicates the position of the data in the time series. The optimal extraction period is the frequency of interest converted into a period and expressed as a data length.

[0073] As shown in Figure 13, in conventional TimesNet, when the optimal extraction period is a non-integer, the decimal point is truncated to determine the extraction period. In other words, the extraction period is 5.0. When time series data with an input length of 33 is divided at an extraction period of 5.0 and a matrix is ​​created using these as row vectors, the result is two-dimensional data with 7 rows and 5 columns. Since there is no data in columns 4 and 5 of the 7th row, they are padded with fixed value data Pad.

[0074] In Figure 13, one square in the matrix represents one piece of data. The vertical dotted lines schematically indicate the boundary of the optimal extraction period. For example, there is a vertical dotted line in the middle of the sixth piece of data in the second row and first column, but this does not indicate a data boundary, but rather a period boundary. Every other optimal extraction period is hatched. In conventional data extraction, the start positions of the optimal extraction periods are shifted diagonally within the matrix and are not aligned vertically.

[0075] As shown in Figure 14, the two-dimensional data in Figure 13 is convolved to generate two-dimensional convolved data. The "c" at the end of each data number indicates that the data is a feature extracted by convolution. The two-dimensional convolved data is then rearranged back into one dimension to generate output one-dimensional data.

[0076] As described above, features can be extracted from time-series data by converting one-dimensional data into two-dimensional data and then performing convolution processing. That is, in two-dimensional data, each row vector is time-series data and has features in the short-term time domain, while adjacent row vectors have features in the long-term time domain. By extracting these through convolution, various vibration features can be extracted.

[0077] However, from the viewpoint of extracting features at a frequency of interest, conventional methods may miss information about the features. That is, because the start positions of the optimal extraction periods are not aligned vertically within the matrix, the features of the signal at the frequency of interest are not aligned vertically, and there is a possibility that the features of the signal at the frequency of interest will not be properly extracted when convolution is performed.

[0078] 15 and 16 are diagrams illustrating a first example of two-dimensional conversion processing in this embodiment. As with Fig. 13 etc., an example is shown in which the input length of time series data is 33 and the optimal extraction period is 5.5.

[0079] In this embodiment, when the optimal extraction period is a non-integer, the decimal point is rounded up to the nearest integer. In other words, the extraction period is 6.0. Time series data with an input length of 33 is divided using an extraction period of 6.0. If the first data of the next extraction period comes before the last data of the row vector, padding is performed and the extraction period begins from the next row. For example, since the second extraction period ends at row 2 and column 5, row 2 and column 6 are padded with fixed value data Pad, and the 12th data, which is the first data of the next extraction period, is placed at the beginning of row 3. This process continues. In this way, two-dimensional data with 6 rows and 6 columns is generated. In this example, the fixed value data Pad is, for example, zero, but it may also be a fixed value other than zero.

[0080] As shown in Fig. 16, convolution processing is performed on the two-dimensional data in Fig. 15 to generate convoluted two-dimensional data. The convoluted two-dimensional data is then rearranged back into one dimension to generate output one-dimensional data.

[0081] A specific example of a calculation method for data extraction in two-dimensional transformation will be described. The time series data and two-dimensional data in FIG. 15 will be used as an example. Let i be an integer between 1 and 6. The optimal extraction period is 5.5, and the extraction period is 6.0, which is 5.5 rounded up to the nearest whole number. The extraction start position of the i-th row is (i-1) × 5.5 + 1, rounded to the nearest whole number. For example, the extraction start positions of the first, second, and third rows are the first, seventh, and twelfth data points in the time series data. The extraction end position of the i-th row is (the extraction start position of the i-th row) + (6.0 - 1.0). For example, the extraction end positions of the first, second, and third rows are the sixth, twelfth, and 17th data points in the time series data. The extraction end position of the second row is the same as the twelfth data point, which is the extraction start position of the third row. In this case, the data at the cutout end position of the second row is padded with fixed value data Pad.

[0082] FIG. 17 is a diagram illustrating a second example of the two-dimensional conversion process according to this embodiment. In this second example, padding is performed using data from the extraction start position of the next row, rather than using fixed-value data Pad. For example, the extraction end position of the second row is the 12th data point in the time-series data using the calculation method described above, which is the same as the extraction start position of the third row. In this example, the extraction end position of the second row is simply the 12th data point in the time-series data. Note that, when the last 33rd data point in the time-series data is used midway through the row, as in the sixth row, the remaining data is padded with fixed-value data Pad.

[0083] FIG. 18 is a diagram illustrating the effect of the two-dimensional conversion processing in this embodiment. The top diagram shows two-dimensional data created using the conventional method described in FIG. 13. The middle diagram shows two-dimensional data created using the first example of the two-dimensional conversion processing in this embodiment described in FIG. 15. The bottom diagram shows two-dimensional data created using the second example of the two-dimensional conversion processing in this embodiment described in FIG. 17. The areas ARa, ARb, and ARc indicated by dashed dotted lines are areas from which features are appropriately extracted in convolution. Here, the kernel size of the convolution is 3 × 3.

[0084] As shown in the top diagram, in the conventional method, the extraction start positions are not aligned vertically. Furthermore, the boundaries of the extraction period include a discrepancy between the optimal extraction position and the actual extraction position, and such boundaries of the extraction period are included within the area ARa. For these reasons, there is a possibility that the signal features of the target frequency will not be correctly extracted during convolution.

[0085] As shown in the middle and bottom diagrams, the method of this embodiment aligns the extraction start positions vertically. Furthermore, the boundaries of the extraction period are excluded from the regions ARb and ARc. For these reasons, the convolution allows accurate extraction of the signal characteristics of the target frequency.

[0086] 4. Experimental Example An example of a comparative experiment of task evaluation using the conventional Times Net and the method of this embodiment is shown. Fig. 19 shows the amplitude spectrum of the dataset used for task evaluation. Fig. 20 shows the evaluation results of the classification task.

[0087] The dataset was acquired as follows. The dry pump under test consisted of a main pump and a mechanical booster pump, each equipped with bearings and gears. The dry pump was an Ebara AA70W dry pump. To measure the vibrations generated during operation, a 6-axis digital output IMU sensor was attached to the top of the main pump. The IMU sensor detects acceleration and angular velocity along the X, Y, and Z axes. The IMU sensor was attached to the main pump so that its X axis was aligned with the axial direction of the main pump, its Y axis was aligned horizontally relative to the axis, and its Z axis was aligned vertically relative to the axis. The IMU sensor was a Seiko Epson M-G354, with an acceleration sensor bandwidth of approximately 200 Hz. The IMU sensor was attached using thin, strong, double-sided tape, which was confirmed to have no effect on measurements. The data output by the IMU sensor was acquired by a PC using UART as the communication protocol for data acquisition. Dedicated logger software installed on a PC acquired XYZ-axis acceleration data, XYZ-axis angular velocity data, and temperature data at 2000 samples per second for 5.5 seconds. Of these data, the acceleration data was used for this task evaluation.

[0088] Artificial failure data was created by shifting the phase of the X-axis acceleration in the XYZ-axis acceleration data. As shown in Figure 19, the spectrum of the measured data has peaks at 85 Hz, 314 Hz, and 398 Hz. Failure data for three abnormal modes was created by creating data with the phase shifted of the 85 Hz frequency component, data with the phase shifted of the 314 Hz frequency component, and data with the phase shifted of the 398 Hz frequency component. When inputting into the model, each failure data was shifted by one point and cut out to a width of 96 points. These 96-point data sets were used as learning data, validation data, and test data in a ratio of 7:1:2.

[0089] As explained in Figure 4, the model training was performed on the task of predicting future time series. The deep learning models used were the conventional TimesNet model, a model with the clipping position correction explained in Figure 15, and a model with the clipping position correction and padding modification explained in Figure 17, and the inference accuracy of these three models was compared. In the test, a classification task was performed using the model configuration explained in Figure 5. That is, test data was input into the trained model, and the resulting features were classified using a classifier. SVM was used as the classifier. The internal dimensionality of the model was set to 16, the number of training epochs was set to 15, and (input length, prediction length) = (96, 96).

[0090] As shown in Figure 20, the classification performance between two classes of failure modes was evaluated. Figure 20 shows the results of an experiment to classify 85 Hz abnormal modes from 314 Hz abnormal modes, and the results of an experiment to classify 85 Hz abnormal modes from 398 Hz abnormal modes. Inference was performed five times for each experiment, and the average and standard deviation of the five results were shown as percentages. Bold underlined numbers indicate the best-performing value in each experiment. Bold ununderlined numbers indicate the second-best-performing value in each experiment.

[0091] 20, the classification accuracy using the cutout correction of this embodiment was improved compared to the classification accuracy using the conventional method between the abnormal modes of 85 Hz and 314 Hz, and between the abnormal modes of 85 Hz and 398 Hz. It is believed that the correction of the cutout position better organizes the information given to the network, and the feature quantities of the vibration data are extracted more accurately.

[0092] Furthermore, with regard to padding techniques, accuracy improved when padding values ​​were taken from the sequence compared to when padding with zeros. By taking padding values ​​from the sequence, the continuity of the data is maintained, and it is thought that inference accuracy improved as learning proceeded while maintaining the continuity of the data.

[0093] Figure 21 shows the evaluation results for the prediction task. The dataset was the acceleration data of the dry pump described in Figure 19. For the evaluation, the model configuration described in Figures 4 and 5 was used. The accuracy of the prediction task was evaluated by calculating an evaluation function between the prediction results output by the predictor and the correct data. The internal dimensionality of the model was 32, and the number of training epochs was 15. MSE and MAE were used as evaluation functions. MSE stands for mean square error, and MAE stands for mean absolute error. Lower MSE and MAE values ​​indicate higher prediction accuracy. Experiments were conducted for (input length, prediction length) = (96,96), (96,192), (96,336), and (96,720). Each experiment was performed five times, and the average and standard deviation of the five results are shown. Bold underlined values ​​indicate the best performance in each experiment. Bold ununderlined values ​​indicate the second best performance in each experiment.

[0094] 21, the prediction accuracy was improved by the clipping correction of this embodiment compared to the prediction accuracy using the conventional method for all prediction lengths tested. The reason for this improvement is thought to be the same as that described for the classification task.

[0095] In this embodiment, the time series data processing method uses a trained model 130. The trained model 130 has been trained to extract feature quantities of input time series data from the input time series data. The time series data processing method includes the steps of inputting the input time series data to the trained model 130 and performing two-dimensional conversion processing to convert the input time series data into two-dimensional data. The time series data processing method includes the steps of performing convolution processing to input the two-dimensional data into a convolution layer, converting the output of the convolution layer into one-dimensional data, and outputting output time series data, and outputting feature quantities based on the output time series data. The step of performing two-dimensional conversion processing divides the input time series data into a plurality of divided time series data at cutout periods of integer values ​​greater than a period corresponding to a frequency of interest, and constructs two-dimensional data from the plurality of divided time series data.

[0096] As explained in FIG. 13 , in the conventional method, the extraction period is an integer value smaller than the period corresponding to the frequency of interest. This causes a problem in that the extraction start position shifts from row to row. According to this embodiment, the extraction period is an integer value larger than the period corresponding to the frequency of interest, so that the extraction start position does not shift from row to row, as shown in FIG. 15 . This makes it possible to prevent information from being missed when extracting data.

[0097] Specifically, as described in FIG. 18, by aligning the extraction start positions of the divided time series data vertically in the two-dimensional data, the signal features of the frequency of interest are aligned vertically in the two-dimensional data. This allows the signal features of the frequency of interest to be correctly extracted in the convolution. Also, as described in FIG. 18, the boundaries of the extraction period can be excluded from the areas ARb and ARc where appropriate convolution results can be obtained. By excluding the boundary portion containing the deviation between the optimal extraction position and the actual extraction position from the area, the signal features of the frequency of interest can be correctly extracted in the convolution.

[0098] In the above-described embodiment, an example has been described in which the state of vibration is inferred based on vibration data measured by a vibration sensor. However, the method of this embodiment can be applied to cases in which analysis of time series data is performed based on various time series data. The time series data may be, for example, time series data measured by a sensor other than a vibration sensor. The sensor other than a vibration sensor may be, for example, a temperature sensor, a microphone, a voltmeter, or an ammeter. Alternatively, the time series data may be meteorological data such as temperature or air pressure, or marketing data such as sales, price, or inventory.

[0099] In the time-series data processing method of this embodiment, the extraction period may be the smallest integer value among integer values ​​greater than the period corresponding to the frequency of interest.

[0100] 15, padding is performed after the data of the extraction period in each row has ended. According to this embodiment, padding can be minimized by setting the smallest integer value among integer values ​​larger than the period corresponding to the frequency of interest as the extraction period.

[0101] In addition, in the time series data processing method of this embodiment, the step of performing two-dimensional conversion processing may divide the data so that the difference between the phase at the start of extraction of the first divided time series data and the phase at the start of extraction of the second or subsequent divided time series data is equal to or less than a predetermined value.

[0102] For example, in the example of Figure 15, if the phase at the start of the extraction of the first row is zero, the phase at the start of the extraction of the second row corresponds to 0.5 data intervals, and the phase at the start of the extraction of the third row is again zero. If the extraction period is the smallest integer value greater than the period corresponding to the frequency of interest, the phase difference corresponds to a maximum of one data interval. As described with reference to Figure 13, in conventional methods, the extraction period shifts from row to row, resulting in a phase difference greater than one data interval. However, in this embodiment, the phase difference can be kept to one data interval or less, making it possible to appropriately extract the characteristics of the signal at the frequency of interest.

[0103] In the time series data processing method of this embodiment, the step of performing two-dimensional conversion processing may include filling the remaining data in the divided time series data with predetermined data when the extraction period is m and the data before the mth data in the divided time series data ends at a period corresponding to the frequency of interest, where m is an integer of 2 or more.

[0104] The predetermined data may be data of a fixed value or the leading data of the next divided time-series data.

[0105] 17, the extraction period is m=6, and the sixth data in the divided time series data in the second row is the twelfth data in the input time series data, which is the same as the first data in the divided time series data in the third row. In this case, the sixth data in the divided time series data in the second row is padded with fixed value data Pad as in FIG. 15, or padded with the first data of the divided time series data in the third row as in the example of FIG. 17.

[0106] As described above, the extraction period is an integer value greater than the period corresponding to the frequency of interest. Therefore, there is a possibility that the period corresponding to the frequency of interest will end and the next period will begin before the mth period in the divided time series data. According to this embodiment, in such a case, by padding the remaining data with predetermined data, it is possible to roughly align the extraction start position with the start position of the period corresponding to the frequency of interest. For example, in FIG. 15 , in the divided time series data on the second row, the period corresponding to the frequency of interest ends with the fifth data. By padding the sixth data, the extraction start position on the third row becomes the start position of the period corresponding to the frequency of interest.

[0107] In the time-series data processing method of this embodiment, the trained model 130 may include a neural network. The two-dimensional conversion process may be performed by the neural network.

[0108] Note that the "neural network" here corresponds to the feature extractor in Figure 5, etc., or the time blocks in Figures 9 and 10. "Input time series data" corresponds to the time series data input to one time block, and "output time series data" corresponds to the time series data output by one time block. The "feature" in "outputting feature based on output time series data" refers to the feature obtained by multiple time blocks sequentially processing the time series data, that is, the output of the feature extractor as a whole, which is composed of multiple time blocks.

[0109] In the time-series data processing method of the present embodiment, the neural network may include a plurality of convolution layers corresponding to a plurality of frequencies of interest. In the convolution processing, the neural network may input two-dimensional data of each frequency of interest to a corresponding one of the plurality of convolution layers.

[0110] According to this embodiment, two-dimensional data can be convolutionally processed using a separate convolutional layer for each frequency of interest. This allows convolutional processing to be performed in a dedicated convolutional layer for each frequency of interest, and each convolutional layer is trained so that features corresponding to each frequency of interest can be appropriately extracted during training. This allows the neural network to more accurately extract features from time-series data.

[0111] In the time-series data processing method of the present embodiment, the input time-series data may be measurement data of a physical quantity related to vibration measured from an object. The time-series data processing method may include a step of inferring a vibration state related to at least one of condition monitoring, quality control, and predictive maintenance of the object based on the feature.

[0112] According to this embodiment, the feature quantity based on the output time-series data of the neural network can be used to infer the vibration state of the object 10. In this embodiment, the feature of the signal at the frequency of interest can be appropriately extracted in the convolution process on the two-dimensional data, so that improvement in the accuracy of inferring the vibration state can be expected.

[0113] A processing system 100 that executes the above-described time series data processing method may also be configured. That is, the processing system 100 includes a storage unit 120 that stores a trained model 130 and a processing unit 110. The trained model 130 includes a neural network and is trained to extract feature quantities of input time series data from the input time series data. The processing unit 110 inputs the input time series data to the trained model 130 and acquires feature quantities based on the output of the neural network. The neural network performs a two-dimensional conversion process to convert the input time series data into two-dimensional data, a convolution process to input the two-dimensional data into a convolution layer, convert the output of the convolution layer into one-dimensional data, and output the output time series data. The two-dimensional conversion process includes a step of dividing the input time series data into a plurality of divided time series data at an extraction period of an integer value greater than a period corresponding to a frequency of interest, and a step of constructing two-dimensional data from the plurality of divided time series data.

[0114] In this embodiment, the learning method uses a learning model 230. The learning model 230 extracts features of the input time series data from the input time series data. The learning method includes the steps of inputting the input time series data to the learning model 230 and performing two-dimensional conversion processing to convert the input time series data into two-dimensional data. The learning method includes the steps of performing convolution processing to input the two-dimensional data to a convolution layer, converting the output of the convolution layer into one-dimensional data and outputting output time series data, and performing learning to output features based on the output time series data. The step of performing two-dimensional conversion processing divides the input time series data into a plurality of divided time series data at cutout periods of integer values ​​greater than a period corresponding to a frequency of interest, and constructs two-dimensional data from the plurality of divided time series data.

[0115] In the learning method of this embodiment, the extraction period may be the smallest integer value among integer values ​​greater than the period corresponding to the frequency of interest.

[0116] In addition, in the learning method of this embodiment, the step of performing two-dimensional conversion processing may divide the data so that the difference between the phase at the start of extraction of the first divided time series data and the phase at the start of extraction of the second or subsequent divided time series data is equal to or less than a predetermined value.

[0117] Furthermore, in the learning method of this embodiment, in the step of performing the two-dimensional conversion processing, if the extraction period is m and the data before the mth data in the divided time series data ends at a period corresponding to the frequency of interest, the remaining data in the divided time series data may be filled with predetermined data.

[0118] In the learning method of this embodiment, the predetermined data may be data of a fixed value or the leading data of the next divided time-series data.

[0119] In the learning method of this embodiment, the learning model 230 may include a neural network, and the two-dimensional conversion process may be performed by the neural network.

[0120] In the learning method of this embodiment, the neural network may include a plurality of convolutional layers corresponding to a plurality of frequencies of interest. In the convolutional processing, the neural network may input two-dimensional data of each frequency of interest to a corresponding one of the plurality of convolutional layers.

[0121] The effects of the above learning method are similar to those of the time-series data processing method described above, and therefore a description thereof will be omitted.

[0122] A learning system 200 that executes the above learning method may also be configured. The learning system 200 includes a storage unit 220 that stores a learning model 230 and a processing unit 210. The learning model 230 includes a neural network and extracts features of the input time series data from the input time series data. The processing unit 210 inputs the input time series data to the learning model 230 and trains the learning model 230 to output features based on the output of the neural network. The neural network performs a two-dimensional conversion process to convert the input time series data into two-dimensional data, a convolution process to input the two-dimensional data into a convolution layer, convert the output of the convolution layer into one-dimensional data, and output the output time series data. The two-dimensional conversion process includes a step of dividing the input time series data into a plurality of divided time series data at an extraction period of an integer value greater than a period corresponding to a frequency of interest, and a step of constructing two-dimensional data from the plurality of divided time series data.

[0123] 5. Fixing the target frequency When converting time series data into two-dimensional data in the time block, data is extracted at a period corresponding to the frequency of interest. The frequency of interest may be determined from the peak frequency of the spectrum, as in the conventional time block, or may be fixed to a preset frequency. Fixing the frequency of interest will be explained below.

[0124] The left figure in Figure 22 shows an example of a frequency peak graph, and the right figure in Figure 23 shows an example of entropy for each epoch. The frequency peak graph on the left shows the probability that each frequency will be selected as the target frequency when determined from the peak frequency of the spectrum, as in the conventional Times Net. The entropy on the right shows the value of entropy loss at the end of each epoch.

[0125] Figure 23 shows that when the selected frequency is determined from the peak frequency of the spectrum, as in the conventional Times Net, the change in the selected frequency becomes smaller as the learning progresses. The upper and lower graphs in Figure 23 each show different learning. Each learning has 15 epochs. The graphs in the first, second, and third columns from the left are frequency peak graphs for the 1st, 8th, and 15th epochs. The graph on the far right shows the entropy for each epoch.

[0126] In both the upper and lower experiments, the frequencies selected at the 8th and 15th epochs are almost the same, and the entropy remains almost unchanged. In other words, as learning progresses, the frequency selected as the target frequency is fixed, and the entropy loss converges. This shows that, rather than determining the target frequency from the spectrum for each learning, it is possible to perform each learning step by fixing the target frequency to one determined in advance through an experiment such as that shown in Figure 23. In the inference stage, the fixed target frequency used during learning is used.

[0127] Furthermore, when the frequency of interest is determined from the spectrum for each learning, the convergence point of the entropy loss changes for each learning, as shown in Fig. 23. On the other hand, by fixing the frequency of interest at which the entropy loss converges to a small value, as shown in the upper part of Fig. 23, for example, the effect of learning can be increased and inference accuracy can be improved.

[0128] All or some of the multiple frequencies of interest may be set to fixed values. The frequencies of interest that are not fixed may be determined from the peak frequency of the spectrum.

[0129] In the time-series data processing method of the present embodiment, the step of performing two-dimensional conversion processing generates two-dimensional data for each of a plurality of frequencies of interest, at least some of which are fixed.

[0130] Furthermore, in the learning method of this embodiment, the neural network generates two-dimensional data for each of a plurality of frequencies of interest, at least some of which are fixed.

[0131] According to this embodiment, by fixing at least some of the frequencies of interest among a plurality of frequencies of interest, it is possible to enhance the learning effect and improve inference accuracy. Specifically, as described in Fig. 23, it is possible to fix the frequencies of interest so that the entropy loss converges to a small value, thereby improving inference accuracy.

[0132] 6. Examples of vibration analysis for condition monitoring, quality control and predictive maintenance Below, examples of vibration analysis relating to condition monitoring, quality control, and predictive maintenance will be shown as examples to which the processing system 100 of this embodiment can be applied.

[0133] Condition monitoring refers to a technology that continuously or periodically monitors the current operating state or performance of machinery or equipment. The main purpose of condition monitoring is to monitor specific parameters such as vibration, sound, temperature, or pressure, and evaluate the health of the machinery or equipment. Examples of systems that perform condition monitoring include the following (a) to (f).

[0134] (a) A system in which sensors are attached to industrial motors and an alarm is issued if abnormal vibrations or temperature increases are detected.

[0135] (b) A system that monitors the structural integrity of a building by installing vibration sensors in various parts of the building and monitoring vibrations during earthquakes or strong winds.

[0136] (c) A system in which vibration sensors are installed to monitor the performance of the gas turbine and alert the operator when abnormal vibrations are detected.

[0137] (d) A system in which vibration sensors are fitted to monitor the health of heavy machinery in a mine or quarry, aiding in the early detection of wear or failure.

[0138] (e) A system in which vibration sensors are installed to monitor the health of each machine on a factory's production line and issue an alarm when abnormal vibrations are detected.

[0139] (f) A system that uses vibration sensors attached to the blades or gearbox of a wind turbine to detect abnormal wear or damage.

[0140] Quality control refers to the process of verifying whether a product or service meets the established quality standards or required specifications. Examples of quality control systems include the following (g) to (k).

[0141] (g) A system that uses cameras or sensors on the production line to check the quality of products, such as dimensions, color, or shape, in real time and automatically rejects products that do not meet the standards.

[0142] (h) A system that uses a vibration sensor to check the vibration pattern when a product made up of multiple parts is operated after the assembly process. If abnormal vibrations are detected, an assembly defect or a component malfunction is suspected.

[0143] (i) A system in which a manufactured motor or generator is operated and vibrations during operation are measured with a vibration sensor. If vibrations exceeding a certain standard are detected, internal imbalance or damage may be suspected.

[0144] (j) A system that verifies whether newly manufactured electronic equipment has the specified vibration resistance by placing the electronic equipment on a vibration test bench and monitoring the effects of vibration using a vibration sensor.

[0145] (k) A system that uses vibration sensors to monitor vibrations during operation to ensure that new railcars or aircraft operate as designed. If abnormal vibrations are detected, the cause can be identified and used to improve quality.

[0146] Predictive maintenance refers to an approach that involves collecting and analyzing operational or status data of equipment or machinery. The purpose of predictive maintenance is to predict the risk of future failure or performance degradation. Examples of systems that perform predictive maintenance include the following (l) to (q):

[0147] (l) A system that attaches sensors to industrial robots, analyzes sensor data when the industrial robot is operating and sensor data from past failures, and detects signs that a specific part is about to fail, thereby predictively scheduling the replacement of that part.

[0148] (m) A system that uses sensors attached to railcar wheels to monitor the degree of wear and predict when wheels need to be replaced based on data indicating excessive wear.

[0149] (n) A system that uses sensors to monitor the condition of pipes or valves in an oil plant and predict the possibility of future leaks.

[0150] (o) A system that analyzes sensor data while an elevator or escalator is in operation and predicts the risk of failure before parts need to be replaced.

[0151] (p) A system for monitoring the efficiency of industrial cooling equipment or air conditioners based on sensor data and predicting component wear or failure.

[0152] (q) A system that collects and analyzes sensor data while agricultural machinery is operating, predicts part wear or failure, and assists in scheduling appropriate maintenance activities.

[0153] Although the present embodiment has been described in detail above, it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novel features and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure. For example, a term described at least once in the specification or drawings together with a different term having a broader or equivalent meaning may be replaced with that different term anywhere in the specification or drawings. Furthermore, all combinations of the present embodiment and modifications are also included within the scope of the present disclosure. Furthermore, the configurations and operations of the processing system, learning system, trained model, learning model, sensor, target, etc. are not limited to those described in the present embodiment, and various modifications are possible. [Explanation of symbols]

[0154] 10...object, 11...vibration source, 100...processing system, 110...processing unit, 111...acquisition unit, 113...inference unit, 120...memory unit, 130...trained model, 140...sensor, 150...presentation unit, 200...learning system, 210...processing unit, 211...acquisition unit, 214...learning processing unit, 220...memory unit, 225...learning data, 230...learning model, 260...operation unit, 270...display unit, 305...TimesNet, 310 ...Feature extractor, 311...Times block, 311-1 to 311-n...Times block, 312-1 to 312-n...Adder, 320...Predictor, 330...Classifier, 360...Classifier, 405...Times net, 410...Feature extractor, 420...Predictor, 510...Fourier transform unit, 520...Transform unit, 530...Two-dimensional transform unit, 540...One-dimensional transform unit, 550...Convolution processing unit, 560...Adder, 570...Weight calculation unit

Claims

1. Using a trained model trained to extract features of input time series data from the input time series data, inputting the input time series data into the trained model; performing a two-dimensional conversion process to convert the input time series data into two-dimensional data; performing a convolution process of inputting the two-dimensional data into a convolution layer, converting an output of the convolution layer into one-dimensional data, and outputting output time-series data; outputting a feature amount based on the output time-series data; Including, The step of performing the two-dimensional transformation process includes: A time series data processing method characterized in that the input time series data is divided into a plurality of divided time series data at cut-out periods of integer values ​​greater than a period corresponding to a frequency of interest, and the two-dimensional data is constructed from the plurality of divided time series data.

2. In claim 1, A time series data processing method, wherein the extraction period is the smallest integer value among integer values ​​larger than the period corresponding to the target frequency.

3. In claim 1, The step of performing the two-dimensional transformation process includes: The phase at the beginning of the first divided time series data segment, The phase at the start position of the second or subsequent divided time series data, The time series data processing method according to claim 1, wherein the division is performed so that the difference between the time series data and the data is equal to or smaller than a predetermined value.

4. In claim 3, The step of performing the two-dimensional transformation process includes: the extraction period is m (m is an integer of 2 or more), If the data before the m-th data of the divided time series data ends at a period corresponding to the frequency of interest, A time series data processing method, characterized in that remaining data in the divided time series data is filled with predetermined data.

5. In claim 4, A time series data processing method, wherein the predetermined data is data of a fixed value or the leading data of the next divided time series data.

6. In any one of claims 1 to 5, the trained model includes a neural network; The time series data processing method, wherein the two-dimensional conversion processing is performed by the neural network.

7. In claim 6, The step of performing the two-dimensional transformation process includes: A time series data processing method, comprising generating the two-dimensional data for each of a plurality of frequencies of interest, at least some of which are fixed.

8. In claim 7, the neural network includes a plurality of convolution layers corresponding to the plurality of frequencies of interest; In the convolution process, The neural network a time-series data processing method, comprising inputting the two-dimensional data of each of the frequencies of interest into a corresponding one of the plurality of convolution layers.

9. In claim 1, the input time-series data is measurement data of a physical quantity related to vibration measured from an object, A time series data processing method comprising a step of inferring a state of the vibration related to at least one of condition monitoring, quality control, and predictive maintenance of the object based on the feature amount.

10. Using a learning model that extracts features of the input time series data from the input time series data, inputting the input time series data into the learning model; performing a two-dimensional conversion process to convert the input time series data into two-dimensional data; performing a convolution process of inputting the two-dimensional data into a convolution layer, converting an output of the convolution layer into one-dimensional data, and outputting output time-series data; performing learning so as to output feature quantities based on the output time-series data; Including, The step of performing the two-dimensional transformation process includes: Dividing the input time series data into a plurality of divided time series data at cutout periods of integer values ​​greater than a period corresponding to a frequency of interest; A learning method, characterized in that the two-dimensional data is constructed from the plurality of divided time-series data.

11. In claim 10, A learning method characterized in that the extraction period is the smallest integer value among integer values ​​larger than the period corresponding to the target frequency.

12. In claim 10, The step of performing the two-dimensional transformation process includes: The phase at the beginning of the first divided time series data segment, The phase at the start position of the second or subsequent divided time series data, The learning method is characterized in that the division is performed so that the difference between the two is equal to or less than a predetermined value.

13. In claim 12, The step of performing the two-dimensional transformation process includes: the extraction period is m (m is an integer of 2 or more), If the data before the m-th data of the divided time series data ends at a period corresponding to the frequency of interest, A learning method characterized in that remaining data in the divided time series data is filled with predetermined data.

14. In claim 13, A learning method, wherein the predetermined data is data of a fixed value or the leading data of the next divided time series data.

15. In any one of claims 10 to 14, the learning model includes a neural network; A learning method characterized in that the two-dimensional conversion processing is performed by the neural network.

16. In claim 15, A learning method characterized in that the neural network generates the two-dimensional data for each of a plurality of frequencies of interest, at least some of which are fixed.

17. In claim 16, the neural network includes a plurality of convolution layers corresponding to the plurality of frequencies of interest; In the convolution process, A learning method characterized in that the neural network inputs the two-dimensional data for each of the target frequencies to a corresponding one of the plurality of convolution layers.

18. a memory unit that stores a trained model including a neural network and trained to extract feature quantities of input time series data from the input time series data; a processing unit that inputs the input time series data to the trained model and acquires the feature amount based on an output of the neural network; Including, The neural network performing a two-dimensional conversion process to convert the input time series data into two-dimensional data; performing a convolution process of inputting the two-dimensional data into a convolution layer, converting an output of the convolution layer into one-dimensional data, and outputting output time-series data; and The step of performing the two-dimensional transformation process includes: Dividing the input time series data into a plurality of divided time series data at cutout periods of integer values ​​greater than a period corresponding to a frequency of interest; constructing the two-dimensional data from the plurality of divided time-series data; A processing system comprising:

19. a memory unit that stores a learning model including a neural network and that extracts features of input time series data from the input time series data; a processing unit that inputs the input time series data to the learning model and trains the learning model so as to output features based on an output of the neural network; Including, The neural network performing a two-dimensional conversion process to convert the input time series data into two-dimensional data; performing a convolution process of inputting the two-dimensional data into a convolution layer, converting an output of the convolution layer into one-dimensional data, and outputting output time series data; and The step of performing the two-dimensional transformation process includes: Dividing the input time series data into a plurality of divided time series data at cutout periods of integer values ​​greater than a period corresponding to a frequency of interest; constructing the two-dimensional data from the plurality of divided time-series data; A learning system comprising:

Images