Prediction device, prediction method, and program

Abstract

The prediction device includes an extraction unit (11) that extracts first and second multidimensional features including features of each of a plurality of frequency components included in the time-series signals of the first and second sensors, respectively; a training unit (12) that inputs the first multidimensional feature to an encoder and the second multidimensional feature to a decoder, where a transformer model including the encoder and the decoder learns a relationship between the first and second multidimensional features; a prediction unit (13) that inputs the first multidimensional feature to the encoder, where the trained transformer model predicts and outputs the second multidimensional feature; and a reconstruction unit (14) that generates the time-series signals of the second sensor based on the second multidimensional feature obtained from the output of the transformer model.

Description

[Technical Field]

[0001] The present disclosure relates to a prediction apparatus, a prediction method, and a computer-readable medium. [Background technology]

[0002] When a bridge is subjected to heavy traffic loads, it ages faster and deteriorates. Bridge health can be monitored using displacement signals, which are a key indicator of bridge damage and internal structural characteristics. Acceleration sensors (accelerometers) have a longer lifespan than displacement sensors (e.g., strain gauges), so it is desirable to predict the time series signals of displacement sensors from the time series signals of accelerometers.

[0003] Non-Patent Document 1 discloses a technique for calculating a displacement signal from an acceleration signal. The displacement x is expressed by equation (1).

number

number

number

[0004] Non-Patent Document 2 discloses a technique for estimating a displacement signal from an acceleration signal using a Kalman filter.

[0005] Non-Patent Document 3 discloses a technology for predicting a displacement signal using a convolutional neural network. The displacement signal is estimated from the acceleration signal by having a U-net, which is a convolutional neural network, learn the relationship between the acceleration signal and a signal obtained by numerically integrating the acceleration signal twice. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] H. Sekiya et. al., "Technique for Determining Bridge Displacement Response Using MEMS Accelerometers", Sensors, 2016 [Non-patent document 2] T. Nagayama et. Al., "A numerical study on bridge deflection estimation using multi-channel acceleration measurement", Journal of Structural Engineering, 2017 [Non-patent document 3] Atta et.al., "Estimation of highway bridges' deflection from acceleration measurement by using a machine learning approach", JSCE 2022 Summary of the Invention [Problem to be solved by the invention]

[0007] Regarding Non-Patent Document 1, noise accumulates in the acceleration signal when performing two numerical integrations, making it impossible to accurately calculate the displacement signal, and there is high uncertainty about the integral boundary conditions of the numerical integration, which are important for vehicle entry and exit time information.

[0008] Regarding Non-Patent Document 2, modal information such as mode shapes and structural information related to the sensor installation positions are required. In addition, adjusting the hyperparameters of the Kalman filtering model is difficult due to the uncertainty of the signal and noise distributions, and even if the hyperparameters are fine-tuned, there is a problem that dynamic bias occurs in the estimated displacement signal.

[0009] Regarding Non-Patent Document 3, the convolution filter makes it impossible to learn complex relationships over long periods of time. Furthermore, in order to understand the behavior of N-dimensional time series signals, multiple channel signals from accelerometers and inclinometers are required. Furthermore, there is a problem in that it is not possible to predict complex temporal changes in bridge displacement due to complex traffic patterns.

[0010] In view of the above circumstances, an object of the present disclosure is to provide a prediction device, a prediction method, and a computer-readable medium that improve the accuracy of predicting a time-series signal of a second sensor from a time-series signal of a first sensor. [Means for solving the problem]

[0011] The present disclosure provides: extraction means for extracting first and second multidimensional features including features of each of a plurality of frequency components included in the time-series signals of the first and second sensors, respectively; training means for inputting the first multi-dimensional features to an encoder and the second multi-dimensional features to a decoder, wherein a Transformer model including the encoder and decoder learns a relationship between the first multi-dimensional features and the second multi-dimensional features; a prediction means for inputting the first multidimensional features to the encoder, wherein the trained transformer model predicts and outputs the second multidimensional features; a reconstruction means for generating a time series signal of the second sensor based on the second multidimensional feature obtained from the output of the transformer model; A prediction device comprising:

[0012] The present disclosure provides: extracting first and second multidimensional features including features of each of a plurality of frequency components included in the time-series signals of the first and second sensors, respectively; inputting the first multi-dimensional features to an encoder and the second multi-dimensional features to a decoder, wherein a transformer model including the encoder and decoder learns a relationship between the first multi-dimensional features and the second multi-dimensional features; inputting the first multi-dimensional features to the encoder, wherein the trained transformer model predicts and outputs the second multi-dimensional features; generating a time series signal of the second sensor based on the second multidimensional features obtained from the output of the transformer model; The present invention provides a prediction method including:

[0013] The present disclosure provides: A process of extracting first and second multidimensional features including features of each of a plurality of frequency components included in the time-series signals of the first sensor and the second sensor, respectively; inputting the first multi-dimensional features to an encoder and the second multi-dimensional features to a decoder, wherein a transformer model including the encoder and decoder learns a relationship between the first multi-dimensional features and the second multi-dimensional features; inputting the first multidimensional features to the encoder, wherein the trained transformer model predicts and outputs the second multidimensional features; generating a time series signal of the second sensor based on the second multidimensional features obtained from the output of the transformer model; A non-transitory computer-readable medium is provided that stores a program that causes a computer to execute the above. [Effects of the Invention]

[0014] A prediction device, a prediction method, and a computer-readable medium according to the present disclosure improve the accuracy of predicting a time-series signal of a second sensor from a time-series signal of a first sensor. [Brief explanation of the drawings]

[0015] [Figure 1]FIG. 1 is a block diagram illustrating a configuration of a prediction device according to a first embodiment. [Figure 2] FIG. 10 is a block diagram illustrating the configuration of a prediction device according to a second embodiment. [Figure 3] FIG. 10 is a schematic diagram illustrating the operation of a multidimensional feature extraction unit according to the second embodiment. [Figure 4] FIG. 10 is a schematic diagram illustrating the operation of the transformer according to the second embodiment. [Figure 5] 10 is a graph illustrating the effect of the second embodiment. [Figure 6] 10 is a graph illustrating the effect of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and for clarity of explanation, duplicate explanations will be omitted as necessary.

[0017] Embodiments according to the present disclosure will be described below with reference to the drawings. Note that in the following description and drawings, appropriate omissions and simplifications are made for clarity. Furthermore, throughout the drawings, the same elements are represented by the same reference numerals (or symbols), and redundant descriptions thereof will be omitted as necessary. Furthermore, in this disclosure, unless otherwise specified, "at least one of A or B (A / B)" may mean either A or B, or both A and B. Similarly, when "at least one" is used in reference to three or more elements, it may mean any one of these elements, or any multiple elements (including all elements). Furthermore, in the description of this disclosure, it should be noted that elements described using singular forms such as "a," "an," "the," and "one" may refer to multiple elements unless otherwise specified.

[0018] (Embodiment 1) 1 is a block diagram showing the configuration of a prediction device 1 according to embodiment 1. The prediction device 1 includes an extraction unit 11, a training unit 12, a prediction unit 13, and a reconstruction unit .

[0019] The extraction unit 11 extracts first and second multidimensional features including features of each of a plurality of frequency components included in the time-series signals of the first and second sensors, respectively.

[0020] The second sensor and the first sensor may be attached to the bridge. The first and second sensors may be different types of sensors, including but not limited to a displacement sensor and an acceleration sensor. The first and second sensors may include optical fiber cables attached to a distributed acoustic sensor (DAS) installed along the bridge.

[0021] The training unit 12 inputs the first multi-dimensional feature to the encoder and the second multi-dimensional feature to the decoder, and the Transformer model including the encoder and decoder learns the relationship between the first multi-dimensional feature and the second multi-dimensional feature.

[0022] The prediction unit 13 inputs the first multidimensional feature to the encoder, and the trained transformer model predicts and outputs the second multidimensional feature.

[0023] The reconstruction unit 14 generates a time-series signal of the second sensor based on the second multidimensional feature obtained from the output of the transformer model.

[0024] Since the first multidimensional features are extracted without using a convolution filter, the prediction device 1 can accurately predict the time-series signal of the second sensor.

[0025] The prediction device 1 includes a processor, a memory, and a storage device (not shown). The storage device stores a computer program that implements the processing of the prediction method according to this embodiment. The processor then loads the computer program from the storage device into the memory and executes the computer program. This allows the processor to implement the functions of an extraction unit 11, a training unit 12, a prediction unit 13, and a reconstruction unit 14.

[0026] Alternatively, the extraction unit 11, the training unit 12, the prediction unit 13, and the reconstruction unit 14 may each be realized by dedicated hardware. Furthermore, some or all of the components of each device may be realized by general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and a program. Furthermore, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), etc., may be used as the processor.

[0027] Furthermore, when some or all of the components of the prediction device 1 are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or distributed. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in a form in which each is connected via a communication network. Furthermore, the functions of the prediction device 1 may be provided in a SaaS (Software as a Service) format.

[0028] (Embodiment 2) 2 is a diagram illustrating the configuration of a prediction device 100 according to embodiment 2. This embodiment 2 describes one specific example of embodiment 1, but the specific example of embodiment 1 is not limited to this embodiment.

[0029] The prediction device 100 includes a training phase configuration 110 and a test phase configuration 120. The Transformer model 40 is trained using the configuration 110. The Transformer model 40 predicts the time series signal of the second sensor using the configuration 120. The prediction device 100 is a specific example of the prediction device 1.

[0030] The architecture 110 comprises a preprocessing unit 111 , a multidimensional feature extraction unit 112 , and a training unit 113 .

[0031] The pre-processing unit 111 performs resampling of the signal 21 measured by the first sensor, for example an acceleration sensor.

[0032] Similarly, the pre-processing unit 111 performs a re-sampling of the signal 22 measured by a second sensor, for example a displacement sensor.

[0033] The multidimensional feature extraction unit 112 is a specific example of the extraction unit 11. The multidimensional feature extraction unit 112 applies a sliding window of a predetermined length to divide the original acceleration signal 21 into partial time-series signals. The partial time-series signals correspond to the time-series signals of the first sensor. The multidimensional feature extraction unit 112 applies a sliding window of a predetermined length to divide the original displacement signal 22 into partial time-series signals. The partial time-series signals correspond to the time-series signals of the second sensor.

[0034] The multidimensional feature extraction unit 112 extracts first and second multidimensional features for training the transformer model 40. The multidimensional feature extraction unit 112 extracts first multidimensional features from the time-series signal of the first sensor using a fast Fourier transform. The first multidimensional features include features of each of multiple frequency components included in the time-series signal of the first sensor. Similarly, the multidimensional feature extraction unit 112 extracts second multidimensional features from the time-series signal of the second sensor using a fast Fourier transform. The multidimensional features include features of each of multiple frequency components included in the time-series signal of the second sensor.

[0035] The operation of the multidimensional feature extraction unit 112 will be described with reference to Fig. 3. The multidimensional feature extraction unit 112 performs a fast Fourier transform on the time-series signal 30 of the first sensor or the second sensor. The multidimensional feature extraction unit 112 extracts the time-series signal 30 from the frequency components f1 to f 10 The multidimensional feature extraction unit 112 extracts a first multidimensional feature or a second multidimensional feature that includes the features of each of the frequency components 301 to 310.

[0036] 2, training unit 113 is a specific example of training unit 12. Training unit 113 causes Transformer model 40 to learn a relationship between a first multi-dimensional feature and a second multi-dimensional feature. Transformer model 40 includes an encoder and a decoder.

[0037] The configuration 120 includes a preprocessing unit 121, a multidimensional feature extraction unit 122, a prediction unit 123, and a signal reconstruction unit .

[0038] The pre-processing unit 121 removes noise from the time-series signal 23 of the first sensor.

[0039] The multidimensional feature extraction unit 122 extracts multidimensional features from the time-series signal of the first sensor. The multidimensional feature extraction unit 122 performs the same processing as the multidimensional feature extraction unit 112. The multidimensional feature extraction unit 122 may perform a fast Fourier transform.

[0040] The prediction unit 123 is a specific example of the prediction unit 13. The prediction unit 123 inputs first multidimensional features of a time-series signal to the encoder of the transformer model 40. The prediction unit 123 obtains second multidimensional features output from the decoder of the transformer model 40.

[0041] The signal reconstruction unit 124 is a specific example of the reconstruction unit 14. The signal reconstruction unit 124 generates a time-series signal 24 of the second sensor based on a plurality of frequency components corresponding to the second multidimensional features acquired from the decoder.

[0042] The operation of the transformer model 40 will be described with reference to FIG. 4. First, the multidimensional feature extraction unit 112 of the prediction device 100 decomposes the time-series signal 31 into frequency components 311 to 316. The time-series signal 31 represents an acceleration signal. The multidimensional feature extraction unit 112 extracts a first multidimensional feature 310 from the frequency components 311 to 316. The first multidimensional feature 310 is represented by a matrix with 6 rows and 6 columns. For example, six sample data of the frequency component 311 are stored in the first row.

[0043] Similarly, the multidimensional feature extraction unit 112 decomposes the time-series signal 32 into frequency components 321 to 326. The time-series signal 32 represents a displacement signal. The multidimensional feature extraction unit 112 extracts a second multidimensional feature 320 from the frequency components 321 to 326.

[0044] The training unit 113 of the prediction device 100 provides the first multidimensional features 310 to the encoder 41 of the Transformer model 40, and provides the second multidimensional features 320 to the decoder 42 of the Transformer model 40. The first multidimensional features 310 and the second multidimensional features 320 are associated with each other at the same time. In this way, the encoder 41 and the decoder 42 are trained.

[0045] The prediction unit 123 of the prediction device 100 inputs the first multidimensional features to the encoder 41 and receives the second multidimensional features 320a output from the decoder .

[0046] The signal reconstruction unit 124 of the prediction device 100 generates frequency components 321a to 326a based on the second multidimensional feature 320a. The signal reconstruction unit 124 generates a time-series signal 32a of the second sensor by summing the frequency components 321a to 326a.

[0047] The operation of the prediction device 100 will be described with reference to equations (4), (5), and (6). The time-series signal y(t) in equation (4) contains multiple frequency components. s represents the sampling frequency. f n represents the frequency. n represents the phase.

number

[0048] The multidimensional feature extraction unit 122 extracts a first multidimensional feature from the time-series signal y(t) by decomposing the time-series signal y(t) into a plurality of frequency components expressed by equation (5).

number

[0049] The signal reconstruction unit 124 can generate the time-series signal y^(t) from the second multidimensional features by summing up a plurality of frequency components as expressed by equation (6).

number

[0050] The effect of the second embodiment will be described with reference to Figures 5 and 6. Figure 5 shows an actual displacement signal. The solid line represents the actually measured signal, and the white dotted line represents the average signal. The vertical axis represents normalized signal strength, and the horizontal axis represents the number of samples (sampling) or time.

[0051] Figure 6 shows the displacement signal estimated from the acceleration signal. The displacement signal may be estimated from the vibration signal generated by the DAS, i.e., the Rayleigh-based backscattered vibration signal. The displacement signal shown in Figure 6 is close to the actual displacement signal shown in Figure 5.

[0052] According to the second embodiment, the displacement signal can be accurately predicted from the acceleration signal. Because the temporal dynamic behavior is decomposed into the frequency dimension, the transformer model learns complex long-term sequences. In addition, the transformer model also learns the relationship between vibrations because inter-vibration frequencies (frequency components) have inherent strong correlations. The second embodiment can be applied to weight-induced bridge movement and traffic volume estimation.

[0053] The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0054] Within the scope of the claims of the present disclosure, various combinations and selections of various disclosed elements (including elements of each example, elements of each drawing, etc.) are possible. In other words, the present disclosure naturally includes various variations and modifications that can be made by a person skilled in the art according to the overall disclosure including the claims and technical concepts. [Explanation of symbols]

[0055] 1,100 Prediction Device 11, 114 Extraction part 12, 113 Training Department 13, 123 Prediction Department 14 Reconstruction part 124 Signal reconstruction unit 110, 120 configuration 111, 121 Pretreatment section 112, 122 Multidimensional feature extraction unit 30 Time Series Signals 301~310, 311~316, 321~326, 321a~326a frequency components 31 First time series signal 32, 32a Second time series signal 40 Transformer Model 41 Encoder 42 Decoder

Claims

1. extraction means for extracting first and second multidimensional features including features of each of a plurality of frequency components included in the time-series signals of the first and second sensors, respectively; training means for inputting the first multi-dimensional features to an encoder and the second multi-dimensional features to a decoder, wherein a transformer model including the encoder and decoder learns a relationship between the first multi-dimensional features and the second multi-dimensional features; a prediction means for inputting the first multidimensional features to the encoder, wherein the trained transformer model predicts and outputs the second multidimensional features; a reconstruction means for generating a time series signal of the second sensor based on the second multidimensional feature obtained from the output of the transformer model; A prediction device comprising:

2. The first sensor is an acceleration sensor, the second sensor is a displacement sensor, the displacement signal is obtained by integrating the acceleration signal twice, and the inherent correlation characteristics between the sensor signals are learned and used for prediction. The prediction device according to claim 1 .

3. The first sensor and the second sensor are attached to a bridge structure on a roadway for the purpose of determining bridge characteristics and traffic characteristics. The prediction device according to claim 2 .

4. The extraction means applies a sliding window of a predetermined length to the original time-series signals of the first sensor and the original time-series signals of the second sensor, and then extracts the first multidimensional features and the second multidimensional features for training the Transformer model. The prediction device according to claim 1 .

5. The extracting means extracts the first and second multidimensional features by performing a fast Fourier transform. The prediction device according to claim 1 .

6. extracting first and second multidimensional features including features of each of a plurality of frequency components included in the time-series signals of the first and second sensors, respectively; inputting the first multi-dimensional features to an encoder and the second multi-dimensional features to a decoder, wherein a transformer model including the encoder and decoder learns a relationship between the first multi-dimensional features and the second multi-dimensional features; inputting the first multi-dimensional features to the encoder, wherein the trained transformer model predicts and outputs the second multi-dimensional features; generating a time series signal of the second sensor based on the second multidimensional features obtained from the output of the transformer model; A forecasting method including:

7. A process of extracting first and second multidimensional features including features of each of a plurality of frequency components included in the time-series signals of the first and second sensors, respectively; inputting the first multi-dimensional features to an encoder and the second multi-dimensional features to a decoder, wherein a transformer model including the encoder and decoder learns a relationship between the first multi-dimensional features and the second multi-dimensional features; inputting the first multidimensional features to the encoder, wherein the trained transformer model predicts and outputs the second multidimensional features; generating a time series signal of the second sensor based on the second multidimensional features obtained from the output of the transformer model; A non-transitory computer-readable medium that stores a program that causes a computer to execute the program.

Images