Method, apparatus, and program for machine learning to combine point data with raster data

The method uses convolutional neural networks to convert point data into raster data, addressing linear dependence and redundancy issues in kriging, ensuring accurate and reliable conversion and handling.

JP7579695B2Active Publication Date: 2024-11-08IHI CORP +1
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2020215187
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-12-29
Filing Date
2020-12-24
Publication Date
2024-11-08
Estimated Expiration
2040-12-24

AI Technical Summary

Technical Problem

Kriging methods for interpolating spatially dependent data introduce linear dependence and redundancy, which are undesirable in data conversion processes.

Method used

A method involving a first convolutional neural network to expand point data into raster data, followed by a second convolutional neural network to combine with standard raster data, using trained weight and mask matrices to avoid linear dependence and redundancy.

Benefits of technology

Point data is converted to raster data without linear dependence on prior distributions and without introducing redundancy, enabling accurate and reliable conversion and handling as standard raster data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007579695000001
    Figure 0007579695000001
  • Figure 0007579695000002
    Figure 0007579695000002
  • Figure 0007579695000003
    Figure 0007579695000003
Patent Text Reader

Abstract

To provide a method of, an apparatus for, and a program for converting dot data to raster data without linearly depending on previous distribution and further without introducing redundancy to interpolated data.SOLUTION: According to a method, an apparatus and a program of the present invention, a dot matrix that is partially vacant is acquired to generate extrapolation raster data matrix by expanding the dot data by a first convolution neural network. By a second convolution neural network, the extrapolation raster data matrix and a standard raster data matrix are combined to generate a matrix of aggregated raster data. Thus, based on the dot data, prediction for the entire field of the raster data is generated.SELECTED DRAWING: Figure 10
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a method, apparatus, and program for machine learning to combine point data with raster data. [Background technology]

[0002] Whether arising from measurements, predictions, or other processes, data may have some type of spatial dependency.

[0003] Examples of spatially dependent data include data obtained by measurements at observation stations or by mobile sensors, such as the outdoor air temperature sensors installed in many vehicles, that correspond to specific points in space (or time). Census observations usually represent areas (e.g., the population of a county or the average income of a particular zip code) and have spatial dependencies. Images taken by drones, satellites or planes consist of a raster of adjacent pixels, where each pixel usually corresponds to a point at the center of the pixel (technically, this value could also correspond to an aggregate of the whole pixel), and have spatial dependencies.

[0004] The three data examples mentioned above are called "point data", "polygon data" and "raster data" respectively ("point data" is technically a sub-category of "polygon"). When processing data in these formats, it is often desirable to process them jointly. In particular, it is often necessary to convert point data (or polygons in general) into raster data that also contains values ​​for areas where no data was included in the original observation.

[0005] For example, if we have data from three nearby stations, we can combine the observations to estimate the median value. Common methods for making such estimates include simple nearest-neighbor, bilinear, and other interpolation methods, as well as kringling techniques. Summary of the Invention [Problem to be solved by the invention]

[0006] Kriging is a method of interpolation in which the interpolated values ​​are modeled by a Gaussian process governed by the covariance of the priors. Under appropriate assumptions about the prior distribution, kriging provides a linear unbiased prediction of intermediate values; that is, kriging produces an interpolation that is close to the most likely value based on the prior distribution. However, kriging has the challenge of producing an interpolation that is linearly dependent on the prior distribution, which tends to introduce redundancy in the interpolated data.

[0007] The present disclosure has been made in view of the above-mentioned circumstances, and aims to provide a method, an apparatus, and a program capable of converting point data into raster data without linearly depending on a prior distribution and without introducing redundancy into the interpolated data. [Means for solving the problem]

[0008] The disclosed method for machine learning to combine point data with raster data includes: <x0>Then, the first convolutional neural network is used to expand the point data and extrapolate the raster data matrix. <X n Then, the second convolutional neural network generates the matrix <X n > and a matrix of standard raster data <X r Combined with a matrix of aggregated raster data <X a > to generate a new

[0009] The first convolutional neural network may be a recurrent convolutional neural network.

[0010] The first convolutional neural network has the weight matrix <w>, 1st row <o>, the mask matrix <M i > (where the subscript i indicates the number of times the process is executed, and is an integer from 1 to n). Here, <o>is the weight matrix <w>The mask matrix is ​​the same size as and has all elements set to 1. <M i > is made up of elements that take the values ​​0 or 1. Mask matrix <m0>The range of elements that are 1 in the matrix <x0>The mask matrix <M i-1 > 1 in the matrix <X i-1 > the mask matrix <M i-1 The first convolutional neural network may be configured as a matrix <X i-1 >, weight matrix <w>, 1st row <o>, and the mask matrix <M i-1 > multiplying by the matrix <X i > the mask matrix <M n The process may be repeated until all elements of > are 1.

[0011] The first convolutional neural network may be trained based on teacher data consisting of past raster data and sets of past point data corresponding to the past raster data.

[0012] The first convolutional neural network may be configured with n (n is an integer equal to or greater than 1) extension layers, where the i-th (i is an integer from 1 to n) extension layer has a weight matrix <W i >, weight matrix <W i A matrix of the same size as >, with all elements being 1 <O i A mask matrix consisting of elements that take values ​​>, 0 or 1. <M i When training the first convolutional neural network, a mask matrix <M i-1 > 1 in the matrix <X i-1 > the mask matrix <M i-1 > may be set. The i-th enhancement layer is the matrix <X i-1 >, weight matrix <W i >, 1st row <O i > and the mask matrix <M i-1 > multiplying by the matrix <X i > the mask matrix <M n > , all elements of the weight matrix <W i > It may also be a device that allows the user to learn.

[0013] When the first convolutional neural network is made to make an inference, the first convolutional neural network may expand the input point data to generate raster data that is completely free of voids. Here, the input point data is a matrix <x0>may have the same non-empty range as the non-empty range of

[0014] The first convolutional neural network may be implemented by artificial neurons.

[0015] According to the present disclosure, an apparatus for machine learning combining point data with raster data includes a memory and at least one or more processors, where the memory stores executable instructions for performing a method of operating a computer to generate predictions over a field of raster data based on the point data. The processor is coupled to the memory and is capable of processing the executable instructions. According to the method, a matrix of point data that is partially empty is generated. <x0>Then, the first convolutional neural network is used to expand the point data and extrapolate the raster data matrix. <X n Then, the second convolutional neural network generates the matrix <X n > and a matrix of standard raster data <X r Combined with a matrix of aggregated raster data <X a > to generate a new

[0016] The present disclosure provides a machine learning program for combining point data with raster data, the program comprising: <x0>Then, the first convolutional neural network is used to expand the point data and extrapolate the raster data matrix. <X n Then, the second convolutional neural network generates the matrix <X n > and a matrix of standard raster data <X r Combined with a matrix of aggregated raster data <X a > to generate a new Effect of the Invention

[0017] According to the present disclosure, point data can be converted to raster data without linear dependency on the prior distribution and without introducing redundancy into the interpolated data. [Brief description of the drawings]

[0018] [Figure 1] FIG. 2 illustrates a grid partially filled with point data. [Diagram 2] FIG. 2 illustrates a grid filled with raster data. [Diagram 3] FIG. 1 illustrates a series of grids filled with interpolated point data by an operation of deconvolution through a convolution filter layer. [Figure 4] FIG. 2 illustrates the configuration of a first neural network used to expand point data to fill a grid. [Diagram 5] FIG. 13 is a diagram showing details of an extension network for obtaining raster data from point data. [Figure 6] FIG. 13 illustrates a first formula used to expand point data to fill a grid. [Figure 7] FIG. 13 illustrates the configuration of a second neural network used to expand the point data to fill the grid. [Figure 8] FIG. 10 shows details of an enhancement layer used to obtain raster data from point data. [Figure 9] FIG. 13 illustrates a second formula used to expand point data to fill a grid. [Figure 10] FIG. 1 illustrates a flowchart of a method for expanding point data to fill a grid. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, some exemplary embodiments will be described with reference to the drawings. In addition, common parts in each drawing are given the same reference numerals, and duplicated explanations will be omitted.

[0020] Prediction of field variables has been achieved by measuring a series of values ​​of the variable at points in a grid, applying an equation that incorporates time as a variable, and extrapolating in time from a grid that is partially filled by point data. Interpolation and extrapolation of field variables has been achieved by interpolating and extrapolating between the points that fill the grid.

[0021] FIG. 1 illustrates a grid 100 that is partially filled with point data. In FIG. 1, the grid with point data is marked with a "+" sign and shaded in a dark color. The point data is obtained by measurements at a given location and time. Here, sensors that obtain the point data can be sensors mounted on satellites, airplanes, automobiles, mobile phones, etc. Various methods have been used to interpolate between predicted points to generate a field prediction, for example "kriging".

[0022] One way to obtain the field variables is to directly obtain the variables at each point of the grid over the entire grid using scattering techniques such as laser, radar, x-ray, ultrasound, etc. To perform predictions by extrapolation in the time direction, equations with time as a variable are applied to the obtained field variables.

[0023] 2 illustrates a grid 200 that is completely filled with raster data (field data) from in situ measurements or predictions of variables such as the output of a satellite, drone, or numerical weather forecast system. When applying equations to a grid that is completely filled with values, no interpolation is required between the points of the grid.

[0024] The values ​​of the field variables obtained by scattering techniques may not be as accurate as those obtained by direct measurements at each point. The present disclosure allows point data to be combined with raster data, and the field equations can be applied to the combined data set obtained by combining the point data with the raster data. This method is superior to the method of kriging to perform interpolation on the point data, which is achieved by rasterizing or expanding the point data to fill a grid. Superior expansion of the point data is achieved by applying a neural network, in particular a convolutional filter layer, to the point data.

[0025] A neural network includes multiple computer processors configured to work together to implement one or more machine learning algorithms, with either synchronous or asynchronous communication between the computer processors, simulating the behavior of thousands to millions of neurons connected by axons and synapses.

[0026] The connections between neurons can be either forcing, inhibiting, or neutral in terms of their influence on the activation state of the connected units. Each unit has a summation function that combines together the values ​​of all its inputs. There is an activation function (e.g., sigmoid function, rectified linear function, softmax function, etc.) for each unit so that a limit must be exceeded before a signal is propagated to other neurons. Activation functions are also called threshold or limiting functions. Neural networks can perform supervised, unsupervised, or semi-supervised machine learning.

[0027] For example, a neural network includes an input layer where data is input, an output layer where output values ​​are output, and at least one or more hidden layers between the input layer and the output layer, and signals propagate in the order of the input layer, hidden layer, and output layer. Each of the input layer, hidden layer, and output layer is composed of one or more units. The units between the layers are connected to each other, and each unit has an activation function. A weighted sum is calculated based on multiple inputs to the unit, and the value of the activation function, whose variable is the sum, becomes the output of the unit.

[0028] A convolutional filter layer is a type of neural network feature in which each unit of the convolutional filter layer samples only a selected portion of the input layer. Convolutional filter layers are often used in applications related to computer vision or image processing because their limited sampling mimics the performance of neurons in the optic nerve bundle.

[0029] The convolutional filter layer can be used to downsample a complex image to generate a simplified image or to classify images by performing a convolution operation (forward operation), and can be used to upsample a simplified image or a classification of images to generate a more complex image by performing a deconvolution operation (backward operation).

[0030] FIG. 3 illustrates a series of grids filled with point data interpolated by the deconvolution operation by a convolution filter layer. First, point data at measurement points is placed in a grid, as shown in FIG. 3(a). In FIG. 3(a), grids with point data are marked with a "+" symbol and shaded in a dark color. Grids that do not contain data are marked with a "0" symbol and are not shaded.

[0031] By performing deconvolution using a convolution filter layer on the grid data shown in Fig. 3(a), grid data (extrapolated raster data) shown in Fig. 3(b) is obtained. By performing deconvolution using a convolution filter layer on the grid data shown in Fig. 3(b), grid data (extrapolated raster data) shown in Fig. 3(c) is obtained. By performing deconvolution using a convolution filter layer on the grid data shown in Fig. 3(c), grid data (extrapolated raster data) shown in Fig. 3(d) is obtained.

[0032] In this way, by performing deconvolution (dilation processing) with the trained convolutional filter layer, the trained convolutional filter layer dilates the sensor data across the entire grid in several steps.

[0033] It is assumed that a pre-trained convolution filter layer is prepared by having the convolution filter layer learn based on training data consisting of past raster data and sets of past point data corresponding to the past raster data.

[0034] The objective function to be learned by the convolutional filter layer is an arbitrary function that represents the correspondence between "raster data" and "point data" that are input to or output from the convolutional filter layer. The weight matrix used in the extended network in Figure 5 can also be reliably trained using the backpropagation method.

[0035] Therefore, by repeating the expansion process, it is possible to obtain raster data 302 shown in Fig. 3(b), raster data 304 shown in Fig. 3(b), and raster data 306 shown in Fig. 3(d) based on the point data 300 shown in Fig. 3(a). As a result, the entire grid can be filled with data. In other words, the point data can be converted into raster data and rasterized.

[0036] In this way, by converting point data into raster data using the extension process, the point data can be handled in the same way as standard raster data (standard raster data). As a result, the rasterized point data can be handled in the same way as data from different input channels to a standard convolutional neural network. For example, consider a case where a convolutional neural network has wavelength data (standard raster data) such as "red," "green," and "blue" as input channels. This convolutional neural network can also have wavelength data (point data) such as "infrared" and "ultraviolet" as input channels.

[0037] In the embodiments of Figures 4 to 6, convolutional filter layers having the same weight matrix are repeatedly applied. On the other hand, in the embodiments of Figures 7 to 9, up to n convolutional filter layers are applied. Different weight matrices may be used between the n convolutional filter layers.

[0038] The dilation achieved by the convolutional filter layer rasterizes the point data but avoids linear dependencies and redundancies in the gridded data. Convolutional filters utilize summation operations, which are weighted by the number of points for which data is available.

[0039] [First Neural Network] Figure 4 shows point data <x0>FIG. 4 illustrates the configuration of a first neural network 400 used to expand the point data <x0>is gridded and input to the augmented network 500. The augmented network 500 has a weight matrix <w>The point data is expanded and extrapolated to raster data by repeatedly applying deconvolution with a convolution filter layer defined by <X n > to generate a new

[0040] Weight matrix <w>It is assumed that the convolutional filter layer defined by has already been trained based on past raster data and training data consisting of a set of past point data corresponding to the past raster data. Note that the past raster data can include measurements, predictions, or both.

[0041] And extrapolating raster data <X n > is standard raster data <X r > (standard raster data obtained by a conventional method) are input to the convolutional neural network 402. The convolutional neural network 402 uses the extrapolated raster data <X n > Point data <x0>, and standard raster data <X r Aggregated raster data incorporating <X a >.

[0042] This procedure is similar to the technique used in the "image inpainting" problem. For reference, see Liu et al., "Image Inpainting for Irregular Holes Using Partial Convolutions" December 2013 (section 3.1). Instead of working with images with a portion of pixels missing in the "image inpainting" problem, this disclosure can assume working with sparse data. Thus, instead of filling in small missing regions, we extend random point data to improve the range around the region where point data was obtained.

[0043] Additionally, the extension works on generic measurement sets: while a set of sensor measurements (e.g., temperature) may constitute a single band, the method can be applied to any set of parameters recorded at different locations in space and time.

[0044] For example, for image data, values ​​in different bands are generally correlated. In an "image inpainting" problem, values ​​in all bands are missing in the same location. However, data may typically be unrelated between different bands, and the missing locations in different bands may be unrelated to each other. Thus, the disclosed method is also applicable when different data sources are missing data in different locations (in which case one mask would be used per data source / input layer). This is different from the "image inpainting" problem, where all layers are missing pixels in the same location.

[0045] The mask used for the expansion <m>depends not only on the location in space but also on the input of the problem. For example, consider a situation where we have multiple input bands defined in an n-dimensional space. The space of input data may be 2 or 3 dimensional. These inputs can be represented as n+1 dimensional tensors <x0>where the extra dimension is the dimension of the parameters to distinguish the different input bands from each other. In this case, the mask <m>is also an n+1-dimensional tensor.

[0046] If all layers have missing data in the same place, there is no need to distinguish the bands from each other, so mask <m>is simply an n-dimensional tensor.

[0047] 5 is a diagram showing details of an expansion network 500 for obtaining raster data from point data. The expansion network 500 includes point data 308 or point data <x0>(gridded point data) is input.

[0048] The augmented network 500 may be implemented as a recurrent convolutional neural network (RCNN), which has a weight matrix <w>, the mask matrix <M i > (the subscript i indicates the number of times the process is executed by the extended network 500), <o>The mask matrix <M i > is composed of elements that take the value 0 or 1. <o>is the weight matrix <w>and all elements are 1.

[0049] In addition, the mask matrix <m0>The range of elements that are 1 in the matrix <x0>During processing, the mask matrix <M i-1 The range of elements that are >1 in the matrix <X i-1 > the mask matrix <M i-1 > is set.

[0050] The expansion network 500 includes a first multiplier 502 and a second multiplier 504. The first multiplier 502 is a matrix <x0>or matrix <X i-1 > for the weight matrix <w>Multiply by , and the temporary raster data <P i The second multiplier 504 calculates the mask matrix <M i-1 > for one matrix <o>Multiply by the point weight <V i > to calculate.

[0051] The first multiplier 502 is a matrix <x0>or matrix <X i-1 Which of the matrices is multiplied depends on the number of times the process of the extended network 500 is executed. <x0>If the expansion network 500 is executed for the second or subsequent times, the matrix <X i-1 > is multiplied.

[0052] The extended network 500 is a temporary raster data <P i > and point weight <V i Based on the matrix <X i >, where the matrix <X i The element value is temporary raster data. <P i > element values ​​are weighted <V i The weight is calculated by dividing the value of the element by the value of the element. <V i If the element value of is 0, the matrix <X i > The value of the element is set to 0.

[0053] In addition, the augmented network 500 is <V i Based on the mask matrix <M i > where the point weights <V i If the element value of is 0, the mask matrix <M i The element values ​​of are set to 0, and the point weights <V i If the element value of is not 0, the mask matrix <M i The value of the > element is set to 1.

[0054] The augmented network 500 is a mask matrix <M i The above process is repeated until all elements of the mask matrix are 1. <M i If the value of the subscript i when all elements of > are 1 is represented as n, then the subscript i can take any integer value from 1 to n.

[0055] FIG. 6 is a diagram illustrating a first formula used to expand point data to fill a grid. In FIG. <g>or <x0>is the point data at the measurement point <x0>is a matrix of <w>is the weight matrix held by the augmented network 500 <w>It is. <X i > is point data <x0>A matrix of extrapolated raster data obtained by repeatedly executing the process by the expansion network 500 on <X i >It is.

[0056] The weight matrix held by the augmented network 500 is <w>Depending on the weight matrix, the results may show characteristics such as anisotropy. <w>can express a process that prioritizes interpolation in a particular direction within a raster.

[0057] [Second Neural Network] Figure 7 shows point data <x0>FIG. 7 illustrates the configuration of a second neural network 700 used to expand the point data <x0>is gridded and input to the enhancement layer. The i-th enhancement layer has a weight matrix <W i The point data is expanded and extrapolated to raster data by sequentially applying deconvolution with convolution filter layers defined by <X i Here, the entire set of extension layers from the first to the nth constitutes the first convolutional neural network.

[0058] Weight matrix <W i The convolutional filter layer defined by > is assumed to have been trained based on past raster data and training data consisting of sets of past point data corresponding to the past raster data. Note that the past raster data can include measurements, predictions, or both.

[0059] And extrapolating raster data <X n > is standard raster data <X r > (standard raster data obtained by a conventional method) are input to the convolutional neural network 702. The convolutional neural network 702 uses the extrapolated raster data <X n > Point data <x0>, and standard raster data <X r Aggregated raster data incorporating <X a >.

[0060] FIG. 8 shows details of the enhancement layers used to obtain raster data from point data. <X i-1 > is input, and from the i-th extension layer, data <X i That is, in the first enhancement layer, point data 308 or point data <x0>(gridded point data) is input, and from the first extension layer, the extrapolated raster data <x1>The i-th extension layer is the extrapolated raster data. <X i > is input, and from the i-th extension layer, the extrapolated raster data <X i > is output.

[0061] The i-th augmentation layer has the weight matrix <W i >, mask matrix <M i >, 1st row <O i > has a mask matrix <M i > is composed of elements that take the value 0 or 1. <O i > is the weight matrix <W i > and all elements are 1.

[0062] In addition, the mask matrix <m0>The range of elements that are 1 in the matrix <x0>The mask matrix <M i-1 The range of elements that are >1 in the matrix <X i-1 > the mask matrix <M i-1 > is set.

[0063] The i-th enhancement layer includes a third multiplier 802 and a fourth multiplier 804. The third multiplier 802 multiplies the matrix <X i-1 > for the weight matrix <W i > to create temporary raster data. <P i The fourth multiplier 804 calculates the mask matrix <M i-1 > for one matrix <O i > multiply the point weight <V i > to calculate.

[0064] The i-th extension layer is the temporary raster data. <P i > and point weight <V i Based on the matrix <X i >, where the matrix <X i The element value is temporary raster data. <P i > element values ​​are weighted <V i The weight is calculated by dividing the value of the element by the value of the element. <V i If the element value of is 0, the matrix <X i > The value of the element is set to 0.

[0065] Furthermore, the i-th extension layer is <V i Based on the mask matrix <M i > where the point weights <V i If the element value of is 0, the mask matrix <M i The element values ​​of are set to 0, and the point weights <V i If the element value of is not 0, the mask matrix <M i The value of the > element is set to 1.

[0066] FIG. 9 illustrates a second formula used to expand the point data to fill the grid. In FIG. <g>or <x0>is the point data at the measurement point <x0>is a matrix of <W i > is the weight matrix maintained by the i-th augmentation layer <W i >It is. <X i > is a matrix of data <X i-1 A matrix of extrapolated raster data obtained by performing processing on the i-th enhancement layer for <X i >It is.

[0067] Note that the weight matrix held by the i-th enhancement layer is <W i Depending on the weight matrix, the results may show characteristics such as anisotropy. <W i > can be used to express a process of interpolating by giving priority to a particular direction within a raster.

[0068] [Processing Procedure] 10 is a flowchart of a method for expanding point data to fill a grid. According to the method for expanding point data to fill a grid of the present disclosure, first, in step S1002, a partially empty matrix of point data is <x0>is obtained.

[0069] In step S1004, the point data is expanded by the first convolutional neural network to obtain a matrix of extrapolated raster data. <xn>is generated.

[0070] Then, in step S1006, a matrix of the extrapolated raster data is generated by the second convolutional neural network. <X n > and a matrix of standard raster data <X r > and are combined to produce a matrix of aggregated raster data. <X a > is generated.

[0071] [Effects of the embodiment] As described in detail above, the method of the present disclosure is directed to a partially empty matrix of point data. <x0>Then, the first convolutional neural network is used to expand the point data and extrapolate the raster data matrix. <X n Then, the second convolutional neural network generates the matrix <X n > and a matrix of standard raster data <X r Combined with a matrix of aggregated raster data <X a This operates the computer to generate predictions across a field of raster data based on point data.

[0072] This allows us to convert point data to raster data without linear dependence on the prior distribution and without introducing redundancy into the interpolated data, and moreover, we can treat the point data in the same way as standard raster data.

[0073] The first convolutional neural network may be a recurrent convolutional neural network, which allows for generating raster data based on point data for grids that do not contain data, and more specifically, converting point data to raster data without linear dependency on prior distributions and without introducing redundancy into the interpolated data, as compared to a case where a user determines an interpolation method for grids that do not contain data.

[0074] The first convolutional neural network has the weight matrix <w>, 1st row <o>, the mask matrix <M i > (where the subscript i indicates the number of times the process is executed, and is an integer from 1 to n). <o>is the weight matrix <w>The mask matrix is ​​the same size as and has all elements set to 1. <M i > is made up of elements that take the values ​​0 or 1. Mask matrix <m0>The range of elements that are 1 in the matrix <x0>The mask matrix <M i-1 > 1 in the matrix <X i-1 > the mask matrix <M i-1 The first convolutional neural network may be configured as a matrix <X i-1 >, weight matrix <w>, 1st row <o>, and the mask matrix <M i-1 > multiplying by the matrix <X i > the mask matrix <M n The process may be repeated until all elements of > are 1.

[0075] This allows the process to be repeated until there are no more grids containing no data, and ensures that raster data can be generated based on point data for grids containing no data.

[0076] The first convolutional neural network may be trained based on teacher data consisting of past raster data and sets of past point data corresponding to the past raster data. This allows the first convolutional neural network to learn a relationship between the past raster data and the past point data. As a result, the point data can be converted to raster data without linear dependency on a prior distribution and without introducing redundancy into the interpolated data, as compared to a case where an interpolation method for grids that do not include data is determined by a user.

[0077] The first convolutional neural network may be configured with n (n is an integer equal to or greater than 1) extension layers, where the i-th (i is an integer from 1 to n) extension layer has a weight matrix <W i >, weight matrix <W i A matrix of the same size as >, with all elements being 1 <O i A mask matrix consisting of elements that take values ​​>, 0 or 1. <M i When training the first convolutional neural network, a mask matrix <M i-1 > 1 in the matrix <X i-1 > the mask matrix <M i-1 > may be set. The i-th enhancement layer is the matrix <X i-1 >, weight matrix <W i >, 1st row <O i > and the mask matrix <M i-1 > multiplying by the matrix <X i > the mask matrix <M n > , all elements of the weight matrix <W i > It may also be a device that allows the user to learn.

[0078] This allows raster data to be generated based on point data for grids that do not contain data. Moreover, compared to a case where the interpolation method for grids that do not contain data is determined by the user, the point data can be converted into raster data without linear dependency on the prior distribution and without introducing redundancy into the interpolated data. Furthermore, the weight matrix <W i >The more degrees of freedom there are, the greater the range of ways the relationship between raster data and point data can be expressed, making it possible to more accurately convert point data to raster data without introducing redundancy into the interpolated data.

[0079] When the first convolutional neural network is made to make an inference, the first convolutional neural network may expand the input point data to generate raster data that is completely free of voids. Here, the input point data is a matrix <x0>This allows the process to continue iteratively until there are no more grids that contain no data, ensuring that raster data can be generated based on the point data for those grids that do not contain data.

[0080] The first convolutional neural network may be implemented by artificial neurons, simulating the behavior of thousands to millions of neurons connected by axons and synapses, effectively representing the relationship between the raster data and the point data.

[0081] An apparatus according to the present disclosure includes a memory and at least one processor, where the memory stores executable instructions for performing the above-mentioned method, and the processor is coupled to the memory and capable of processing the executable instructions.

[0082] The program according to the present disclosure includes executable instructions for executing the above-described method. Note that the program according to the present disclosure may be stored on a computer-readable and writable recording medium.

[0083] Each of the functions described in the above embodiments may be implemented by one or more processing circuits, including programmed processors, electrical circuits, and even devices such as application specific integrated circuits (ASICs), or circuit components arranged to perform the described functions.

[0084] Although several embodiments have been described, the embodiments can be modified or modified based on the above disclosure. All components of the above embodiments and all features described in the claims may be individually extracted and combined as long as they are not mutually inconsistent. [Explanation of symbols]

[0085] 402,702 Convolutional Neural Network (Second Convolutional Neural Network) 500 Extended Network (First Convolutional Neural Network) 801 Extension Layer < / o> < / w> < / w> < / o> < / o> < / w> < / xn> < / g> < / w> < / w> < / w> < / w> < / g> < / o> < / w> < / w> < / o> < / o> < / w> < / m> < / m> < / m> < / w> < / w> < / o> < / w> < / w> < / o> < / o> < / w>

Claims

1. 1. A method of operating a computer to generate predictions over a field of raster data based on point data, comprising: The matrix of point data <X 0 >, The first convolutional neural network is repeatedly applied to expand the point data in a direction in which the number of empty elements is reduced, and the matrix of the extrapolated raster data <X n >, The second convolutional neural network calculates the matrix of the extrapolated raster data <X n > and the matrix of standard raster data <X r > to combine with the matrix <X a >A method for generating

2. 2. The method of claim 1 , The method of claim 1, wherein the first convolutional neural network is a recurrent convolutional neural network.

3. 3. The method of claim 2, The first convolutional neural network includes a weight matrix <W>, an 1 matrix <O> having the same size as the weight matrix <W> and all elements of which are 1, and a mask matrix <M> having elements that take the value of 0 or 1. i > (where the subscript i indicates the number of times the process is executed, and is an integer from 1 to n), The mask matrix <M 0 The range of elements that are 1 in the matrix <X 0 >, and In the process, The mask matrix <M i-1 The range of elements that are 1 in the matrix <X i-1 , so that the range of the mask matrix <M i-1 > is set, The first convolutional neural network is a matrix <X i-1 >, the weight matrix <W>, the one matrix <O>, and the mask matrix <M i-1 > gives the matrix <X i >, The mask matrix <M n The above process is repeated until all elements of are 1 n times.

4. 4. The method of claim 3, A method for training the first convolutional neural network based on training data including past raster data and a set of past point data corresponding to the past raster data.

5. 2. The method of claim 1 , The first convolutional neural network is composed of n extension layers (n is an integer equal to or greater than 1); The i-th (i is an integer from 1 to n) enhancement layer has a weight matrix <W i >, the weight matrix <W i >, a matrix with all elements being 1. i >, a mask matrix <M consisting of elements that take the value 0 or 1 i > When training the first convolutional neural network, The mask matrix <M i-1 The range of elements that are 1 in the matrix <X i-1 , so that the range of the mask matrix <M i-1 > is set, The i-th enhancement layer has a matrix <X i-1 >, the weight matrix <W i >, the first matrix <O i >, and the mask matrix <M i-1 > gives the matrix <X i >, The mask matrix <M n The weight matrix <W> is added to the i-th enhancement layer so that all elements of i >How to make it learn.

6. 6. The method of claim 5, When the first convolutional neural network is made to make an inference, the first convolutional neural network performs the inference process by 0 augmenting the input point data with a non-empty portion extent equal to the non-empty portion extent of > to generate completely sky-free raster data.

7. The method according to any one of claims 1 to 6, The method of claim 1, wherein the first convolutional neural network is implemented by artificial neurons.

8. a memory storing executable instructions for performing a method for operating a computer to generate predictions over a field of raster data based on the point data; at least one or more processors coupled to the memory and capable of processing the executable instructions, The method comprises: The matrix of point data <X 0 >, The first convolutional neural network is repeatedly applied to expand the point data in a direction in which the number of empty elements is reduced, and the matrix of the extrapolated raster data <X n >, The second convolutional neural network calculates the matrix of the extrapolated raster data <X n > and the matrix of standard raster data <X r > to combine with the matrix <X a >. An apparatus for generating

9. 1. A program comprising executable instructions for performing a method for operating a computer to generate predictions over a field of raster data based on point data, the method comprising: The method comprises: The matrix of point data <X 0 >, The first convolutional neural network is repeatedly applied to expand the point data in a direction in which the number of empty elements is reduced, and the matrix of the extrapolated raster data <X n >, The second convolutional neural network calculates the matrix of the extrapolated raster data <X n > and the matrix of standard raster data <X r > to combine with the matrix <X a >A program that generates

Citation Information

Patent Citations

  • Number-of-targets estimation device, number-of-targets estimation method, and program

    JP2018072938A

  • Missing value interpolation device, missing value interpolation method and program

    JP2019125110A

  • Automated cardiac volume segmentation

    JP2019504659A