Analog waveform processing apparatus, system, method, and program

Abstract

To provide an analog waveform reading apparatus for recognizing a graph with a desired accuracy, a system, a method, and a program.SOLUTION: A waveform processing unit 21 constituting an analog waveform processing apparatus can executes an analog waveform processing. The analog waveform processing is a convolutional neural network obtained through supervised learning. Waveform data is formed from a seismogram image using an acquired parameter and the seismogram image. A first neural network is composed of 21 layers. Data to be input is a seismogram image which is one grayscale image having W pixels in X direction and H pixels in Y direction.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a reading device, system, method, and program that read an analog waveform recorded as an image and convert it into electronic data.

Background Art

[0002] Conventionally, a line graph recognition device that scans a graph image and converts it into numerical data is known. This device uses known shape recognition technology to recognize a plurality of straight lines that make up a line graph and records the coordinates of the nodes of the plurality of straight lines (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, an analog waveform recorded as an image may include figures, characters, stains, etc. other than the waveform. In such a case, it may not be possible to recognize the graph with the desired accuracy.

[0005] The present invention has been made in view of such problems, and an object thereof is to obtain an analog waveform processing device, system, method, and program capable of recognizing a graph with desired accuracy.

Means for Solving the Problems

[0006] The analog waveform processing apparatus according to the first invention of the present application includes a first convolution unit that performs a convolution operation on an image having data in the X and Y directions to output first data, a reduction unit that reduces the dimension in the Y direction of the first data to output second data, and a second convolution unit that performs a convolution operation on the second data to output third data composed of one-dimensional data.

[0007] Preferably, the reduction unit reduces the dimension in the Y direction of the first data to one dimension in the first data and outputs second data.

[0008] Preferably, the reduction unit reduces the dimension in the Y direction of the first data to one dimension by performing a convolution operation on each value in the X direction of the first data and outputs second data.

[0009] The reduction unit may output second data by reducing the dimension in the Y direction of the first data to one dimension by outputting the largest value with respect to the Y direction for each value in the X direction of the first data.

[0010] Preferably, after performing a convolution operation on the image, the first convolution unit calculates the residual between the input value and the output value and outputs the residual as the first data.

[0011] Preferably, after expanding the data in the X direction with respect to the second data, the second convolution unit performs a convolution operation and outputs third data.

[0012] The apparatus further includes preprocessing means for performing preprocessing on the image. The preprocessing means preferably includes at least one of range specifying means for specifying a range in the image that the first convolution unit processes, baseline specifying means for specifying a baseline that serves as a reference for the waveform amplitude in the waveform recorded in the image, arc correction means for correcting an arc component included in the waveform in the waveform recorded in the image, single swing correction means for correcting a deviation of the baseline that serves as a reference for the waveform amplitude in the waveform recorded in the image, and scaling means for reducing in the amplitude direction (for example, the Y direction) in the waveform recorded in the image.

[0013] It further includes correction means for correcting the third data, and the correction means includes at least one of correct point designating means for designating a plurality of points and complementing between the designated plurality of points, point moving means for moving the position of any point among the points constituting the third data, point adding means for adding the position of any point to the third data, point deleting means for deleting any point among the points constituting the third data, and amplitude stretching means for stretching or shrinking the waveform constituting the third data in the amplitude direction (for example, the Y direction).

[0014] The analog waveform processing system according to the second invention of the present application includes a client for reading an image having data in the X and Y directions, and the analog waveform processing device for processing the image, and the client is characterized by processing and outputting the third data.

[0015] Preferably, the client transmits the image to the analog waveform processing device via a network, and the analog waveform processing device transmits the third data to the client via the network.

[0016] The analog waveform processing method according to the third invention of the present application includes a first convolution step of performing a convolution operation on an image having data in the X and Y directions to output first data, a reduction step of reducing the dimension in the Y direction of the first data to output second data, and a second convolution step of performing a convolution operation on the second data to output third data composed of one-dimensional data.

[0017] Preferably, in the reduction step, for each value in the X direction in the first data, the dimension in the Y direction is reduced to one dimension by performing a convolution operation to output the second data.

[0018] Alternatively, in the reduction step, for each value in the X direction in the first data, the dimension in the Y direction may be reduced to one dimension by outputting the largest value with respect to the Y direction to output the second data.

[0019] The analog waveform processing program according to the fourth invention of the present application includes: a first convolution step of performing a convolution operation on an image having data in the X and Y directions to output first data; a reduction step of reducing the dimension in the Y direction of the first data to output second data; and a second convolution step of performing a convolution operation on the second data to output third data composed of one-dimensional data.

Effects of the Invention

[0020] According to the present invention, an analog waveform processing apparatus, system, method, and program capable of recognizing a graph with a desired accuracy can be obtained.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 6C

Figure 6D

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19A

Figure 19B

Figure 19C

Figure 19D

Embodiments for Carrying Out the Invention

[0022] Hereinafter, with reference to FIGS. 1 to 10, an analog waveform processing system 10, its method, and a program according to a first embodiment of the present invention will be described. The analog waveform processing system 10 mainly includes a server 20 including a waveform processing unit 21 and a client 30 including a seismic waveform reading unit 31.

[0023] The client 30 is a so-called personal computer directly operated by a user and is connected to the server 20 via a network such as the Internet. Currently, seismic waveforms are recorded as digital data, but in the past, there were those recorded as analog data on smoked recording paper or ink-written recording paper. Hereinafter, a method of recording seismic waveforms using smoked recording paper or ink-written recording paper is referred to as a smoked recording method or the like, and a seismic waveform recorded on smoked recording paper or ink-written recording paper is referred to as an analog seismic waveform. An image reading means 36 such as a digital camera or a scanner is connected to the client 30. The client 30 digitizes the analog seismic waveform via the image reading means 36 to obtain a recording paper image.

[0024] The client 30 mainly includes a seismic waveform reading unit 31 capable of executing a seismic waveform reading program. The seismic waveform reading program mainly includes a preprocessing means 32, a dot sequence generation means 33, a correction means 34, and an output means 35, and displays a recording paper image or digital data on the screen. Referring to FIG. 2, a recording paper image 31a is displayed on the right side of the screen, and a menu 31b is displayed on the left side.

[0025] The preprocessing means 32 performs preprocessing on the recording paper image and includes at least one of a range specifying means, a baseline specifying means, an arc correction means, a single swing correction means, and a scaling means.

[0026] The range specifying means specifies the range of the waveform to be digitized in the recording paper image displayed on the screen. At least a part of the specified range is digitized. In the soot writing recording method or the like, a plurality of recording needles arranged side by side pivot around their respective fulcrums according to the intensity of seismic motion (hereinafter also referred to as amplitude), and record the seismic motion on the soot writing recording paper wound around a single rotating cylinder. Therefore, a plurality of waveforms are recorded on a single recording paper. Thus, the user needs to specify the range to be digitized among the plurality of waveforms. Alternatively, even if only one waveform is recorded on a single recording paper, if there is a range that the user wants to digitize within one waveform, it is necessary to specify the range to be digitized. Referring to FIG. 2, after the user selects "Range Specification" in menu 31b, referring to the recording paper image 31a displayed on the right side and clicking and dragging a mouse or the like, the range to be digitized is specified from the plurality of waveforms recorded in the recording paper image. In FIG. 2, the range indicated by the broken line is the range input by the user. The range specifying means holds the position data on the recording paper image of the range input by the user in the memory.

[0027] The baseline specifying means specifies the baseline that serves as the reference for the waveform amplitude in the waveform recorded in the recording paper image. When a single recording needle gradually shifts its position and makes multiple revolutions around a cylinder wound with recording paper to record the amplitude, or when the recording paper tilts when reading the recording paper to create a recording paper image, etc., the baseline that forms the zero point of the waveform amplitude may not be parallel to one side of the recording paper image. Thus, the user needs to specify the baseline on the recording paper image. Referring to FIG. 3, after the user selects "Baseline Specification" in menu 31b, referring to the recording paper image 31a displayed on the right side and clicking and dragging a mouse or the like, a line serving as the reference for the waveform amplitude is input. The broken line shown in FIG. 3 is the line input by the user. The baseline specifying means holds the position data on the recording paper image of the line input by the user in the memory as the baseline that serves as the reference for the waveform amplitude.

[0028] The arc correction means corrects the arc components included in the waveform recorded in the recording paper image, and the single-sided swing correction means corrects the deviation of the baseline serving as the reference for the waveform amplitude, that is, the single-sided swing, in the waveform recorded in the recording paper image. Since the recording needle pivots around the fulcrum, the waveform includes arc components due to the pivoting. Therefore, in order to obtain accurate earthquake waveform data, it is necessary to correct this arc component and then digitize the data. Also, for some reason, the line connecting the tip of the recording needle and the fulcrum may not be parallel to the rotation direction of the cylinder around which the recording paper is wound, that is, a waveform including a single-sided swing may be recorded. Therefore, in order to obtain accurate data, it is necessary to correct the influence of the single-sided swing and then digitize the data. Referring to FIG. 4, when the user selects "Arc & Single-sided Swing Correction" in menu 31b, the arc correction means and the single-sided swing correction means display a slider bar 41 on the screen at the lower part of the recording paper image 31a displayed on the right side (see FIG. 4). The user refers to the displayed recording paper image and drags the slider 42 of the slider bar 41 with a mouse or the like to input the correction parameters for the influence of the arc and the single-sided swing. The arc correction means corrects the recording paper image displayed on the screen based on the input correction parameters and displays the corrected recording paper image. Thereby, the input correction parameters are immediately reflected in the recording paper image on the screen. The arc correction means and the single-sided swing correction means hold the correction parameters input by the user in the memory.

[0029] The scaling means reduces the waveform recorded in the recording paper image in the amplitude direction (Y direction). The waveform processing unit 21, which will be described later, tends to recognize a large amplitude of the waveform included in the recording paper image as a small amplitude. Therefore, in the recording paper image, the reading result may be improved by reducing the amplitude direction (Y direction) while maintaining the size in the time direction (X direction), that is, reducing the amplitude. The user refers to the displayed recording paper image and clicks and drags a mouse or the like to input the size of the reduced waveform amplitude. In response to the user's operation, the recording paper image displayed on the screen expands and contracts in the amplitude direction (Y direction) (see Fig. 5). The scaling means calculates the reduction rate of the waveform amplitude based on the size input by the user and holds it in the memory.

[0030] After all the processes of the range specifying means, the baseline specifying means, the arc correction means, the single swing correction means, and the scaling means are completed, or after at least one process is completed, the preprocessing means 32 corrects the image using the position data obtained by the range specifying means and the baseline specifying means, the correction parameters obtained by the arc correction means and the single swing correction means, and the reduction rate obtained by the scaling means, which are held in the memory.

[0031] The dot sequence generation means 33 transmits the image data corrected by the preprocessing means 32 to the server 20 and receives the third data output by the waveform processing unit 21, which will be described later. The waveform processing unit 21, which will be described later, outputs the waveform line recorded in the recording paper image as digital data such as vector data, for example, waveform data in SVG format. That is, the waveform data is dot sequence data created by the waveform processing unit 21, has vector data of points and lines, and these vector data represent the waveform. The seismic waveform reading program superimposes the waveform composed of vector data on the recording paper image and displays it on the screen.

[0032] The client 30 can record the third data output by the waveform processing unit 21 in the past as an SVG file. The dot sequence generation means 33 can read the recorded SVG file and generate a dot sequence.

[0033] The correction means 34 includes at least one of a correct point designating means, a point moving means, a point adding means, a point deleting means, and an amplitude stretching and shrinking means, and corrects the waveform data created by the point sequence generating means 33.

[0034] In the correct point designating means, when the waveform data, that is, the vector data does not exist on the seismic waveform while the user views the waveform and the seismogram image displayed on the screen, the user designates two or more points as correct positions on the seismic waveform as points. The correct point designating means refers to the pixel values between the plurality of points designated by the user as points at correct positions, and interpolates them by dynamic programming so as to follow the waveform line.

[0035] In the point moving means, when the waveform data does not exist on the seismic waveform while the user views the waveform and the seismogram image displayed on the screen, the user clicks and drags a mouse or the like to move an arbitrary point of the vector data to a correct position on the seismic waveform. The point moving means moves the position of the arbitrary point, that is, records the position moved by the user as new vector data of the arbitrary point.

[0036] In the point adding means, when there are not enough points on the waveform data while the user views the waveform and the seismogram image displayed on the screen, the user clicks an arbitrary point at a position on the line of the waveform data using a mouse or the like. The point adding means adds the position of the point clicked by the user as vector data, and complements the space between the added point and the two adjacent points on both sides with vector data representing a line.

[0037] In the point deleting means, while the user views the waveform and the seismogram image displayed on the screen, the user clicks an arbitrary point on the waveform data using a mouse or the like. The point deleting means deletes the vector data of the point clicked by the user and the line connected to that point, and complements the space between the points that were connected to the deleted point with vector data representing a line.

[0038] In the amplitude expansion / contraction means, while viewing the waveform and the chart paper image displayed on the screen, the user clicks two points in the X direction (time direction) using a mouse or the like to specify the range for expanding / contracting the amplitude. Then, the user clicks and drags one point to specify the expansion / contraction scale indicating the degree of expansion / contraction. The amplitude expansion / contraction means expands / contracts the waveform included in the X direction range specified by the user in the Y direction according to the specified expansion / contraction scale, and records the expanded / contracted value as vector data. Thereby, the waveform composed of vector data is expanded / contracted in the amplitude direction (Y direction).

[0039] The dot sequence data and the chart paper image corrected using the above means are sent to the output means 35.

[0040] The output means 35 is means for processing and outputting the dot sequence data corrected by the user into data with an arbitrary resolution and time interval. For the processing, the image resolution, time resolution, and resampling interval of the dot sequence data are used as parameters. The image resolution is the number of dots per inch in the X direction and the number of dots per inch in the Y direction or the length (mm) per pixel. The time resolution is the length (mm) per arbitrary time. The arbitrary time is, for example, 1 minute. The resampling interval is the desired sampling interval in the time direction, and either 0.10 seconds or 0.20 seconds is selected.

[0041] The method for a user to specify the image resolution, time resolution, and resampling interval will be described below with reference to FIGS. 6A to 6D. Here, the imaging paper image records its image resolution as data in the image file. When the user selects "Unit Conversion & CSV Output" in Menu 31b (see FIG. 2 etc.), the output means 35 reads out the image resolution from the imaging paper image and calculates the ratio between the number of pixels (dots) of the imaging paper image and the actual size of the imaging paper image. Then, a dialog 61 is displayed on the screen, which displays the image resolution in text box 61a and text box 61b, and displays the ratio between the number of pixels of the imaging paper image and the actual size of the imaging paper image in text box 61c (see FIG. 6A). The user can set the image resolution, time resolution, and resampling interval by directly entering specific numerical values into text boxes 61a to 61d. For the resampling interval, 0.10 seconds is selected as the default value. The values in text box 61b and text box 61c both represent the ratio between the number of pixels of the imaging paper image and the actual size of the imaging paper image. When either one of text box 61b and text box 61c is changed by the user, the output means 35 automatically changes the other value accordingly. In dialog 61, when the user clicks the "Specify from Image" button 61e, dialog 62 is displayed on the screen (see FIG. 6B). On this screen, the user enters the time width as a numerical value in the numerical box 62a provided at the upper part of the screen, and then clicks two points on the image with the mouse etc. (see FIG. 6C). The distance between the two points clicked on the image corresponds to the length corresponding to the entered time width. When the specification is completed, the user closes dialog 62 by clicking the "OK" button. Then, the output means 35 displays the actual size per minute (mm / min) calculated from the distance between the two points specified by the user and the time width in text box 61d (see FIG. 6D). When the OK button of dialog 61 is clicked by the user, the image resolution, time resolution, and resampling interval are input to the output means 35, and the output means 35 resamples the dot sequence data at the specified time interval and outputs the data in CSV format.As described above, since the output means 35 automatically displays the image resolution in the dialog 61 and 0.10 seconds has been selected for the resampling interval, the user only needs to specify at least the time resolution. Note that the resampling interval can also be set using the value described in the file input from outside the apparatus.

[0042] Next, the server 20 will be described. The server 20 mainly includes a waveform processing unit 21. The waveform processing unit 21 forms an analog waveform processing apparatus, receives preprocessed image data from the client 30, and forms waveform data. The waveform processing unit 21 can execute an analog waveform processing program and an analog waveform processing method. The analog waveform processing program and the analog waveform processing method implement a convolutional neural network by supervised learning, and form waveform data from the ruled paper image using the learned parameters and the ruled paper image. Hereinafter, with reference to FIG. 7, the first neural network forming the analog waveform processing program will be described in detail.

[0043] The first neural network consists of 21 layers, and the input data is a ruled paper image composed of a single grayscale image having W pixels in the X direction and H pixels in the Y direction. The first to fourth layers at the beginning form at least a part of the first convolutional part and are mainly composed of convolutional layers.

[0044] The first layer has a convolutional layer, a batch normalization layer, and a ReLU activation function layer. The convolutional layer pads the ruled paper image with padding 3, applies a filter having a kernel size of 7x7 with a stride width of 1 thereto, and obtains 32-channel data. Then, the first layer applies batch normalization and the ReLU activation function to the obtained data and outputs it.

[0045] The second layer has a convolutional layer, a batch normalization layer, and a ReLU activation function layer. The convolutional layer pads the 32-channel data received from the first layer with padding 1, applies a filter with a kernel size of 4x4 to it with a stride of 2, and obtains 64-channel data. Then, the second layer applies batch normalization and the ReLU activation function to the obtained data and outputs it.

[0046] The third layer has a convolutional layer, a batch normalization layer, and a ReLU activation function layer. The convolutional layer pads the 64-channel data received from the second layer with padding 1, applies a filter with a kernel size of 4x4 to it with a stride of 2, and obtains 128-channel data. Then, the third layer applies batch normalization and the ReLU activation function to the obtained data and outputs it.

[0047] The fourth layer has a convolutional layer, a batch normalization layer, and a ReLU activation function layer. The convolutional layer pads the 128-channel data received from the third layer with padding 1, applies a filter with a kernel size of 4x4 to it with a stride of 2, and obtains 256-channel data. Then, the fourth layer applies batch normalization and the ReLU activation function to the obtained data and outputs it. The data after applying the fourth layer is 256 pieces of data with the number of pixels converted to 1 / 8 in the X and Y directions, that is, 256 pieces of data with W / 8 pixels in the X direction and H / 8 pixels in the Y direction.

[0048] Next, nine Residual layers forming the 5th to 13th layers are applied. The 5th to 13th layers form at least a part of the first convolutional section. The Residual layer is a layer that calculates the residual between the input value and the output value, and includes two convolutional layers with padding 1, kernel size 3x3, and stride 1, and a ReLU activation function layer. First, the first convolutional layer is applied to the input value, then the ReLU activation function layer is applied to this, and then the second convolutional layer and the ReLU activation function layer are further applied. And the residual obtained by subtracting the resulting output value from the input value is output. This is executed in the 5th to 13th layers. The data after applying the 13th layer is 256 pieces of data whose number of pixels in the X and Y directions is converted to 1 / 8, that is, 256 pieces of data having W / 8 pixels in the X direction and H / 8 pixels in the Y direction, and the data size is the same as the data after applying the 4th layer.

[0049] Here, up to the 1st to 13th layers are substantially composed of only convolutional layers. Therefore, it is possible to calculate regardless of the values of the number of pixels (W, H) in the X and Y directions. Alternatively, by appropriately adjusting parameters such as padding, kernel size, and stride width, it is possible to adjust the memory usage amount, calculation amount, and the like.

[0050] Next, a Y-direction MaxPooling layer forming the 14th layer is applied. The 14th layer forms at least a part of the reduction section, and extracts and outputs the largest value in the Y direction for each value in the X direction. As a result, the dimension in the Y direction is reduced to one dimension. By performing this for all 256 channels, data of 256 channels with a dimension of W / 8 in the X direction and a dimension of 1 in the Y direction is obtained. Here, the MaxPooling layer is applied to extract the position of the pixel that is most wavy in the Y direction. Therefore, by applying the 1st to 13th layers to the seismic paper image before the 14th layer, the 1st to 13th layers can be learned so as to output pixels characteristic of forming a seismic waveform in backpropagation.

[0051] Next, apply the UnPooling layer that forms the 15th layer. The 15th layer forms at least a part of the second convolutional section and expands the data only in the X direction with respect to the data output by the 14th layer. More specifically, apply 1D UnPooling with padding 0, kernel size 2, and stride width 2 to the data received from the 14th layer. By applying a kernel size of 2, the value of 1 pixel included in the previous layer is expanded to 1x2, and the data is expanded only in the X direction. Perform the above processing for all 256 channels.

[0052] Next, apply the convolutional layer that forms the 16th layer. The 16th layer forms at least a part of the second convolutional section and has a convolutional layer, a batch normalization layer, and an activation function ReLU layer. The convolutional layer applies padding 1 to each of the 256 channels of data received from the 15th layer, applies a filter with a kernel size of 3 with a stride width of 1 to obtain 128-channel data, applies batch normalization and the activation function ReLU to this, and then outputs it.

[0053] Next, apply the UnPooling layer that forms the 17th layer. The 17th layer forms at least a part of the second convolutional section. Since the processing of the 17th layer is the same as that of the 15th layer, that is, it performs UnPooling processing on the data from the previous layer and outputs it, the description is omitted.

[0054] Next, apply the convolutional layer that forms the 18th layer. The 18th layer forms at least a part of the second convolutional section and has a convolutional layer, a batch normalization layer, and an activation function ReLU layer. The convolutional layer applies padding 1 to the 128-channel data received from the 17th layer, applies a filter with a kernel size of 3 with a stride width of 1 to obtain 64-channel data, applies batch normalization and the activation function ReLU to this, and then outputs it.

[0055] Next, an UnPooling layer forming the 19th layer is applied. The 19th layer forms at least a part of the second convolutional section. Since the processing of the 19th layer, like that of the 15th layer, performs UnPooling processing on the data from the previous layer and outputs it, the description thereof is omitted.

[0056] Next, a convolutional layer forming the 20th layer is applied. The 20th layer forms at least a part of the second convolutional section and includes a convolutional layer, a batch normalization layer, and a ReLU activation function layer. The convolutional layer applies a filter with a kernel size of 3 and a stride of 1 to the 64-channel data received from the 19th layer after padding by 1, obtains 32-channel data, applies the batch normalization layer and the ReLU activation function layer thereto, and then outputs the result.

[0057] Next, a convolutional layer forming the 21st layer is applied. The 21st layer forms at least a part of the second convolutional section and includes a convolutional layer, a batch normalization layer, and a ReLU activation function layer. The convolutional layer applies a filter with a kernel size of 7 and a stride of 1 to the 32-channel data received from the 20th layer after padding by 3, obtains 1-channel data, applies the batch normalization layer and the ReLU activation function layer thereto, and then outputs the result.

[0058] As a result, data having W pixels in the X direction and one value in the Y direction, where the value in the Y direction represents the position of the seismic wave, is obtained. The position of the seismic wave here refers to the position where the seismic wave is drawn in the preprocessed image data. The waveform processing unit 21 transmits the dot sequence data output by the first neural network to the seismic waveform reading unit 31 in the client 30.

[0059] A processing method executed by the analog waveform processing system will be described with reference to FIG. 8. Before executing this processing, the first neural network has been completed using the learning data described later and functions as the waveform processing unit 21.

[0060] First, in step S801, the server 20 and the client 30 are started, thereby starting the system. In the next step S802, the user inputs the seismogram image obtained using the image reading means (Digital camera or scanner, etc.) 36 or an existing seismogram image into the seismic waveform reading unit 31. The seismic waveform reading unit 31 displays the input seismogram image on the screen.

[0061] In the next step S803, while referring to the seismogram image displayed on the screen, the user clicks and drags a mouse or the like to specify a range to be digitized from a plurality of waveforms. The range specifying means holds, in the memory, the position data on the seismogram image of the range input by the user as the range for the waveform processing unit 21 to perform processing from among the plurality of waveforms recorded in the seismogram image.

[0062] In the next step S804, while referring to the seismogram image displayed on the screen, the user clicks and drags a mouse or the like to input a line serving as a reference for the waveform amplitude. The baseline specifying means holds, in the memory, the position data on the seismogram image of the line input by the user as the baseline serving as a reference for the waveform amplitude.

[0063] In the next step S805, while referring to the seismogram image displayed on the screen, the user drags the slider of the slider bar with a mouse or the like to input correction parameters for the influence of the arc and single-sided swing. The arc correction means and the single-sided swing correction means hold the correction parameters input by the user in the memory.

[0064] In the next step S806, if the user determines that the amplitude of the waveform included in the seismogram image is large, in step S807, scaling is performed. While referring to the seismogram image displayed on the screen, the user clicks and drags a mouse or the like to input the magnitude of the reduced waveform amplitude. The scaling means calculates the reduction rate of the waveform amplitude based on the magnitude input by the user and holds it in the memory.

[0065] In the next step S808, the seismic waveform reading unit 31 preprocesses the seismogram image using the parameters acquired by the range specifying means, baseline specifying means, arc correction means, single-swing correction means, and / or scaling means, and the seismogram image, and transmits it to the server 20. The waveform processing unit 21 in the server 20 applies the first neural network to the received preprocessed seismogram image to create waveform data. Then, the created waveform data is transmitted to the client 30. The seismic waveform reading unit 31 in the client 30 overlays the waveform data on the preprocessed seismogram image and displays it on the screen.

[0066] In the next step S809, if the user determines that the waveform data needs to be corrected, in step S810, at least one of the correct point specifying means, point moving means, point adding means, point deleting means, and amplitude stretching / shrinking means is executed using the correction means 34 to correct the waveform data.

[0067] In the next step S811, the seismic waveform reading unit 31 outputs the corrected waveform data as a file in CSV format. Then, the process ends in step S812.

[0068] Next, the training data and the verification data will be described. Among the seismogram images with vector data accumulated by the Earthquake Research and Promotion Center of the Incorporated Administrative Agency, Earthquake Prediction and Comprehensive Research Promotion Foundation, 158 seismogram images in which one waveform with a certain amplitude and a plurality of horizontal lines (lines parallel to the time axis) are drawn are selected. Among these, 136 are used as the first training data and 22 are used as the first verification data. Then, for the waveform part with the largest amplitude among the waveforms recorded in the seismogram image, waveform data was formed using the waveform processing unit 21. Note that the arcs and single swings included in the seismogram image have been corrected. FIG. 9 shows an example of the first training data and the first verification data. The seismic waveform indicated by the arrow in FIG. 9 is the object to be read, that is, the object for forming waveform data. Here, the first training data and the first verification data include the seismogram image that forms the input signal for the first neural network and the vector data that forms the teacher signal.

[0069] As Example 1, the result of training the first neural network with the first training data will be described. The first neural network trained with the first training data had a root mean square error of 10.43 at epoch number 1213. FIG. 10 shows the waveform data obtained by reading the seismogram image shown in FIG. 9 with the trained first neural network, where the waveform data is drawn as a dashed line and the waveform line recorded on the seismogram image is drawn as a solid line. The waveform data approximates the waveform line recorded on the seismogram image, and the waveform line could be reproduced with good accuracy.

[0070] According to this embodiment, in a seismogram image in which one waveform with a certain amplitude and a plurality of horizontal lines (lines parallel to the time axis) are drawn, the seismic waveform can be read with favorable accuracy to form waveform data.

[0071] Also, even if the analog waveform itself recorded on the seismogram image is blurred and has uneven line thickness and uneven color density, the seismic waveform can be read with favorable accuracy to form waveform data. Further, even if the analog waveform recorded on the seismogram image includes figures, characters, stains, etc. other than the waveform, the seismic waveform can be read with favorable accuracy to form waveform data.

[0072] Next, the second neural network according to the second embodiment will be described. For the same configurations as those in the first embodiment, the description will be omitted. In this embodiment, the differences are that a 1x1 convolutional layer is used instead of the Y-direction MaxPooling layer forming the 14th layer of the first neural network, and the second training data and the second verification data are used.

[0073] The second neural network forming the waveform processing unit 21 will be described. The 14th layer of the second neural network includes a 1x1 convolutional layer. The 14th layer has a 1x1 convolutional layer, a batch normalization layer, and an activation function ReLU. The 1x1 convolutional layer first swaps the channel axis and the Y axis for the 256-channel data received from the 13th layer. Padding 0 is applied to the data with the axes swapped, and a filter with a 1x1 kernel size is applied with a stride of 1, and then the channel axis and the Y axis are swapped again. As a result, 256-channel data with the dimension in the Y direction reduced to 1 dimension is obtained. Then, after applying batch normalization and the activation function ReLU to the obtained data, the 14th layer outputs it. Through the above processing, 256-channel data with the dimension in the X direction being W and the dimension in the Y direction being 1 dimension is obtained. Here, the 1x1 convolutional layer is applied to reduce the amount of data while reflecting the Y coordinate value of the input signal in the output data and extracting the feature amount in the Y direction.

[0074] Next, the training data and the validation data will be described. In addition to the first training data and the first validation data according to the first embodiment, among the seismogram images with vector data accumulated by the Earthquake Research Center of the Incorporated Administrative Agency, Earthquake Prediction and Comprehensive Research Promotion Association, those that include waveforms with relatively small amplitudes compared to the earthquake waveforms used in the first embodiment, and the seismogram images contain multiple earthquake waveforms, the waveform lines forming the earthquake waveforms overlap, and there is vector data of the waveform line with a small amplitude among the overlapping multiple waveform lines, 259 new sheets were added. Among these, 233 sheets were used as training data and 26 sheets were used as validation data. That is, together with the first training data, the second training data was 369 sheets and the second validation data was 48 sheets. Then, for the part of the waveform recorded in the seismogram image that includes the overlapping part of multiple waveform lines and has a small amplitude waveform part, waveform data was formed using the waveform processing unit 21. Note that the arcs and single swings included in the seismogram image have been corrected. Fig. 11 shows an example of the second training data and the second validation data. In Fig. 11, the earthquake waveform indicated by the arrow is the object to be read, that is, the object for forming waveform data.

[0075] As Example 2, the results of training a second neural network with second training data will be described. The second neural network trained with the second training data had a root mean square error of 8.63 at epoch number 608. FIG. 12 shows the waveform data obtained by reading the seismogram image shown in FIG. 9 with the trained second neural network, with the waveform data obtained thereby shown as a dashed line and the waveform line recorded in the seismogram image shown as a solid line. FIG. 13 shows the waveform data obtained by reading the seismogram image shown in FIG. 11 with the trained second neural network, with the waveform data obtained thereby shown as a dashed line and the waveform line recorded in the seismogram image shown as a solid line. Even when two waveforms overlap, the waveform data approximates the waveform line to be read, and the waveform line can be accurately reproduced.

[0076] According to the present embodiment, among the waveforms recorded in the seismogram image, when reading a waveform line with a relatively small amplitude among the waveform lines forming the seismic waveforms, which includes waveforms with relatively small amplitudes and a plurality of seismic waveforms in the seismogram image and the waveform lines forming the seismic waveforms overlap, the seismic waveforms can be read with favorable accuracy and waveform data can be formed.

[0077] Here, for reference, FIG. 14 is shown. FIG. 14 shows the waveform data obtained by reading the seismogram image shown in FIG. 11 with the first neural network trained according to Example 1, with the waveform data obtained thereby shown as a dashed line and the waveform line recorded in the seismogram image shown as a solid line.

[0078] Next, the comparative examples will be described. As Comparative Example 1, the first neural network was trained with the second training data. The first neural network trained with the second training data had a root mean square error of 11.46 at epoch number 336. FIG. 15 shows the waveform data obtained by reading the recording paper image shown in FIG. 9 with the trained first neural network, with the broken line representing the waveform data and the solid line representing the waveform line recorded on the recording paper image. FIG. 16 shows the waveform data obtained by reading the recording paper image shown in FIG. 11 with the trained first neural network, with the broken line representing the waveform data and the solid line representing the waveform line recorded on the recording paper image.

[0079] As Comparative Example 2, the second neural network was trained with the first training data. The second neural network trained with the first training data had a root mean square error of 9.02 at epoch number 733. FIG. 17 shows the waveform data obtained by reading the recording paper image shown in FIG. 9 with the trained second neural network, with the broken line representing the waveform data and the solid line representing the waveform line recorded on the recording paper image. FIG. 18 shows the waveform data obtained by reading the recording paper image shown in FIG. 11 with the trained second neural network, with the broken line representing the waveform data and the solid line representing the waveform line recorded on the recording paper image.

[0080] Next, the epoch numbers and the root mean square errors according to Example 1, Example 2, Comparative Example 1, and Comparative Example 2 are shown. [Table 1] The list of seismogram images used as learning and verification data is shown in FIGS. 19A to 19D. In the figures, "First" means that it was used as the first learning data and the first verification data, and "Second" means that it was used as the second learning data and the second verification data. Also, "○" means that it was used as learning data, and "◎" means that it was used as verification data. Among the three seismic waves included in one seismogram image, "X component" means the seismic wave in the E-W direction, "Y component" means the seismic wave in the N-S direction, and "Z component" means the seismic wave in the U-D direction.

[0081] Example 1 is the first neural network that learned the first learning data. Example 1 became a neural network with a smaller error compared to the first neural network that learned the second learning data, which is Comparative Example 1.

[0082] Example 2 is the second neural network that learned the second learning data. Example 2 became a neural network with a smaller error compared to the first neural network that learned the second learning data, which is Comparative Example 1, and also became a neural network with a smaller error compared to the second neural network that learned the first learning data, which is Comparative Example 2. Also, Example 2 showed a significantly smaller error compared to Example 1, Comparative Examples 1 and 2.

[0083] As a result, the second neural network includes a seismogram image in which one waveform with a certain amplitude and a plurality of horizontal lines (lines parallel to the time axis) are drawn, and waveforms with relatively small amplitudes, and the seismogram image includes a plurality of seismic waveforms, and the waveform lines forming the seismic waveforms overlap, and there is vector data of the waveform line with a small amplitude among the overlapping waveform lines. It was shown that the second neural network has high recognition ability for any seismogram image.

[0084] Although the analog waveform processing system 10 has been described on the assumption that a client and a server are connected via a network, the analog waveform processing system 10 may be executed by a single computer.

[0085] In any of the embodiments, the first to fourth layers can be replaced with one or more layers composed of a combination of a convolutional layer, batch normalization, and an activation layer. Also, the batch normalization layer may not be used and may be appropriately used depending on the depth of the layer. The activation function is not limited to ReLU.

[0086] In any of the embodiments, the number of Residual layers in the fifth to thirteenth layers is not limited to nine and may be one or more. Also, for the two convolutional layers and the activation function ReLU layer included in the Residual layer, the number of convolutional layers is not limited to two, and the activation function is not limited to ReLU. Although the Residual layer has been described as a non-bottleneck design, a bottleneck design may be used.

[0087] In any of the embodiments, the combinations of the fifteenth and sixteenth layers, the seventeenth and eighteenth layers, and the nineteenth and twentieth layers are not limited to three and may be one or more. Also, without providing the twenty-first layer, one channel may be output by a convolutional layer before the twenty-first layer.

[0088] The parameters and numerical values shown in this specification and the drawings are examples and are not limited to these values.

[0089] Although embodiments of the present invention have been described with reference to the accompanying drawings, it will be apparent to those skilled in the art that modifications can be made to the structure and relationships of each part without departing from the scope and spirit of the described invention.

Description of Reference Numerals

[0090] 10 Analog waveform processing system 20 Server 21 Waveform processing unit 30 Client 31 Seismic waveform reading section 31a Recording paper image 31b Menu

Claims

Claims 1. A first convolution unit that performs a convolution operation on a graph image composed of two-dimensional data in the X and Y directions showing an analog waveform, and outputs first data composed of two-dimensional data in the X and Y directions of a plurality of channels; A reduction unit that outputs second data of a plurality of channels in which the number of pixels in the X direction remains the same and the Y direction is reduced to one pixel with respect to the first data; A second convolution unit that performs a convolution operation on the second data to expand the data only in the X direction and outputs third data composed of single-channel data having one pixel in the Y direction; The first convolution unit performs a first convolution operation on the graph image so that the image size of the graph image becomes smaller to obtain a first output value, and performs a second convolution operation without changing the number of channels and the image size to obtain a second output value, executes a difference calculation for calculating a difference between the second output value and the first output value, and outputs the first data composed of two-dimensional data in the X and Y directions of a plurality of channels by repeating the second convolution operation and the difference calculation a plurality of times; The reduction unit outputs, as the second data, the largest value in the Y direction for each value in the X direction in the first data, or the reduction unit outputs, as the second data, a value obtained by reducing the Y direction of the first data to one pixel by performing a convolution operation using a 1x1 convolution layer after swapping the channel axis and the Y axis; The second convolution unit outputs the third data composed of single-channel data having one pixel in the Y direction and the same number of pixels as the input image width in the X direction obtained by performing an expansion and a convolution operation on the X direction of the second data; An analog waveform processing device. Claims 2. The analog waveform processing device further comprises preprocessing means for performing preprocessing on the graph image, The preprocessing means includes: Range specifying means for acquiring, as a range for the first convolution unit to process, a range specified by a user in the graph image; Baseline specifying means for acquiring, as a baseline serving as a reference for the waveform amplitude, a line input by a user in the analog waveform recorded in the graph image; Arc correction means for correcting an arc component included in the analog waveform based on a correction parameter input by a user and applying the correction to the graph image showing the analog waveform. Single-sided swing correction means for correcting the deviation of the baseline serving as the reference for the amplitude of the analog waveform in the graph image based on the correction parameters input by the user. Scaling means for expanding and contracting the graph image showing the analog waveform in the amplitude direction based on the input by the user. The analog waveform processing apparatus according to claim 1, comprising at least one of the above.

3. Further comprising correction means for correcting the third data, The correction means includes: Correct answer point designation means for designating a plurality of points specified by the user as points at the correct positions and complementing the intervals between the specified plurality of points. Point movement means for recording the position of any point among the points constituting the third data moved by the user as new vector data of the any point. Point addition means for adding any point specified by the user to the third data. Point deletion means for deleting the points specified by the user among the points constituting the third data. Amplitude expansion and contraction means for expanding and contracting the waveform constituting the third data in the amplitude direction according to the expansion and contraction scale specified by the user. The analog waveform processing apparatus according to claim 1 or 2, comprising at least one of the above.

4. A client for reading the graph image, And the analog waveform processing apparatus according to any one of claims 1 to 3 for processing the graph image, The client outputs the third data by overlaying it on the graph image. Analog waveform processing system.

5. The client transmits the graph image to the analog waveform processing apparatus via a network, The analog waveform processing apparatus transmits the third data to the client via a network. The analog waveform processing system according to claim 4.

6. A first convolution step of performing a convolution operation on a graph image composed of two-dimensional data in the X and Y directions showing an analog waveform to output first data composed of two-dimensional data in the X and Y directions of a plurality of channels; A reduction step of outputting second data of a plurality of channels in which the number of pixels in the X direction remains the same and the Y direction is reduced to one pixel for the first data; And a second convolution step of performing a convolution operation on the second data to output third data composed of single-channel data having data expanded only in the X direction and one pixel in the Y direction. The first convolution step performs a first convolution operation on the graph image so that the image size of the graph image becomes smaller to obtain a first output value, performs a second convolution operation without changing the number of channels and the image size to obtain a second output value, executes a difference calculation for calculating a difference between the second output value and the first output value, and outputs, as the first data, two-dimensional data in the X direction and the Y direction of a plurality of channels by repeating the second convolution operation and the difference calculation a plurality of times. The reduction step outputs, as the second data, the largest value with respect to the Y direction for each value in the X direction in the first data, or the reduction step outputs, as the second data, a value obtained by reducing the Y direction of the first data to one pixel by performing a convolution operation using a transposition of the channel axis and the Y axis and a 1x1 convolution layer. The second convolution step outputs the third data composed of single-channel data having one pixel in the Y direction and the same number of pixels as the input image width in the X direction obtained by performing an expansion and a convolution operation on the X direction of the second data. An analog waveform processing method executed by a computer.

7. The method further includes a preprocessing step of performing preprocessing on the graph image. The preprocessing step includes: a range specifying step of obtaining, as a range to be processed by the first convolution unit, a range specified by a user in the graph image; a baseline specifying step of obtaining, as a baseline serving as a reference for the waveform amplitude, a line input by a user in the analog waveform recorded in the graph image; an arc correction step of correcting an arc component included in the analog waveform based on a correction parameter input by a user to the graph image showing the analog waveform; a single-sided swing correction step of correcting a deviation of a baseline serving as a reference for the amplitude of the analog waveform based on a correction parameter input by a user to the graph image; a scaling step of stretching or shrinking the graph image showing the analog waveform in the amplitude direction based on an input by a user. The analog waveform processing method according to claim 6, comprising at least one of the above.

8. The method further includes a correction step of correcting the third data. The correction step includes: A correct point designation step of designating a plurality of points specified by a user as points at correct positions and complementing between the specified plurality of points, A point movement step of recording, as new vector data of an arbitrary point, the position of an arbitrary point among the points constituting the third data that has been moved by the user, A point addition step of adding an arbitrary point specified by the user to the third data, A point deletion step of deleting a point specified by the user among the points constituting the third data, An amplitude expansion / contraction step of expanding / contracting the waveform constituting the third data in the amplitude direction according to the expansion / contraction scale specified by the user The analog waveform processing method according to claim 6 or 7, comprising at least one of the above.

9. A first convolution step of performing a convolution operation on a graph image composed of two-dimensional data in the X direction and the Y direction showing an analog waveform to output first data composed of two-dimensional data in the X direction and the Y direction of a plurality of channels; A reduction step of outputting second data of a plurality of channels in which the number of pixels in the X direction remains the same and the Y direction is reduced to one pixel with respect to the first data; A second convolution step of performing a convolution operation on the second data to output third data composed of single-channel data having data expanded only in the X direction and one pixel in the Y direction, The first convolution step performs a first convolution operation on the graph image so that the image size of the graph image becomes smaller to obtain a first output value, performs a second convolution operation without changing the number of channels and the image size to obtain a second output value, executes a difference calculation for calculating the difference between the second output value and the first output value, and repeats the second convolution operation and the difference calculation a plurality of times, and outputs as the first data composed of two-dimensional data in the X direction and the Y direction of a plurality of channels, The reduction step outputs, as the second data, the largest value in the Y direction for each value in the X direction in the first data, or the reduction step outputs, as the second data, the value obtained by reducing the Y direction of the first data to one pixel by performing a convolution operation using a transposed layer of the channel axis and the Y axis and a 1x1 convolution layer. The second convolution step outputs the third data, which consists of single-channel data with one pixel in the Y direction and the same number of pixels as the input image width in the X direction, obtained by performing an expansion and convolution operation on the second data with respect to the X direction. An analog waveform processing program executed by a computer.

10. The method further comprises a preprocessing step of performing preprocessing on the graph image before the first convolution step. The preprocessing step includes: A range specifying step of obtaining, as a range to be processed by the first convolution unit, a range specified by a user in the graph image. A baseline specifying step of obtaining, as a baseline serving as a reference for the waveform amplitude, a line input by a user in the analog waveform recorded in the graph image. An arc correction step of correcting an arc component included in the analog waveform based on correction parameters input by a user, and applying the correction to the graph image showing the analog waveform. A single-sided deviation correction step of correcting a deviation of a baseline serving as a reference for the amplitude of the analog waveform based on correction parameters input by a user, and applying the correction to the graph image. A scaling step of stretching or shrinking the graph image showing the analog waveform in the amplitude direction based on an input by a user. The analog waveform processing program according to claim 9, comprising at least one of the above steps.

11. The method further comprises a correction step of correcting the third data. The correction step includes: A correct point specifying step of setting a plurality of points specified by a user as points at correct positions and complementing the area between the specified plurality of points. A point movement step of recording, as new vector data for an arbitrary point, the position of the arbitrary point among the points constituting the third data that has been moved by a user. A point addition step of adding an arbitrary point specified by a user to the third data. A point deletion step of deleting a point specified by a user among the points constituting the third data. An amplitude stretching / shrinking step of stretching or shrinking the waveform constituting the third data in the amplitude direction according to a stretching / shrinking scale specified by a user. The analog waveform processing program according to claim 9 or 10, comprising at least one of the above steps.

Images