Learning device, learning method, estimation device, estimation method, estimation program, and estimation system

Abstract

To provide a learning device, a learning method, a learning program, an estimation device, an estimation method, an estimation program, a learning system, and an estimation system which facilitate long-term observation of a wave height.SOLUTION: A learning device 4A for performing learning so as to be capable of estimating a wave height at an arbitrary point of time at a spot on the sea includes: a learning data acquisition part 33 for acquiring learning data including image data obtained by photographing a shore related to the spot, estimation data of a wave height at the spot, and observation data of the wave height at the spot; and a learning processing part 34 for making a first learning device perform machine learning, using a data set for learning obtained by collecting the learning data. The learning processing part 34 makes the first learning device perform learning so as to output second estimation data of the wave height at the point of time at the spot by inputting image data at the point of time when the shore is photographed and first estimation data of the wave height at the point of time at the spot.SELECTED DRAWING: Figure 2A

Description

[Technical Field]

[0001] The present invention relates to a learning device, a learning method, an estimation device, an estimation method, an estimation program, and an estimation system relating to ocean wave height. [Background technology]

[0002] In coastal management, it is important to understand wave information. Conventionally, there is a wave-related technology described in Patent Document 1. The technology described in Patent Document 1 estimates the wave direction as well as at least one of the wave period, wave height, and time variation of water level based on an image taken of a buoy installed on the water. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-124388 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 requires the installation of a buoy on the water, which can make wave observation using the buoy difficult depending on the location and time of year.

[0005] From this perspective, the present invention provides a learning device, a learning method, an estimation device, an estimation method, an estimation program, and an estimation system that facilitate long-term observation of wave height. [Means for solving the problem]

[0006] A learning device according to the present invention is a device that learns to be able to estimate wave height at any point on the ocean. The learning device includes a learning data acquisition unit and a learning processing unit. The learning data acquisition unit acquires learning data including image data of a coastline associated with the location, estimated wave height data at the location, and observed wave height data at the location. The learning processing unit performs machine learning on a first learning device using a learning dataset that is a compilation of the learning data. The learning processing unit trains the first learning device to output second estimated data of wave height at the point in time by inputting image data of the coast at the point in time and first estimated data of wave height at the point in time.

[0007] By using the learning device according to the present invention, it is possible to estimate wave height without installing a buoy on the sea. This makes it possible to easily observe wave height regardless of location or time. In particular, it is easy to observe over a long period of time.

[0008] The estimated data included in the learning data may be obtained by a second learning device that is trained to output estimated data of wave height at the point by inputting wind speed data for a range including the point. This method makes it possible to estimate offshore wave heights that are not visible in images, resulting in high estimation accuracy. It also significantly reduces calculation time compared to methods that rely on numerical calculations.

[0009] The estimated data included in the training data may be obtained using one or more wave prediction models. Furthermore, the estimated data included in the learning data may be obtained by coupling a first wave prediction model for shallow waters with a second wave prediction model for deep waters. In this way, it is possible to estimate the wave height offshore, which is not visible in the image, with high estimation accuracy.

[0010] The image data may be an orthomosaic image of the coast. This increases the accuracy of wave height estimation.

[0011] A learning method according to the present invention is a method for enabling a computer to estimate wave height at a point on the ocean at any time. This learning method includes a learning data acquisition step and a learning processing step. In the learning data acquisition step, learning data including image data of a coastline related to the location, estimated data of wave height at the location, and observed data of wave height at the location is acquired. In the learning processing step, a first learning device is trained by machine learning using a learning dataset that compiles the learning data. In the learning processing step, the first learning device is trained to output second estimated data of wave height at the point in time by inputting image data of the coast at the point in time and first estimated data of wave height at the point in time.

[0012] By using the learning method according to the present invention, it is possible to estimate wave height without installing a buoy on the sea. This makes it possible to easily observe wave height regardless of location or time. In particular, it is easy to observe over a long period of time.

[0013] The estimation device according to the present invention is a device that estimates wave height at a certain point on the sea at an arbitrary time. This estimation device includes an estimation processing unit that inputs image data of a coast related to the point at the time and first estimated data of wave height at the point at the time into a trained first learning device, and outputs second estimated data of wave height at the point at the time. The trained first learning device is trained using a training dataset that is a compilation of training data including image data of the coast at the time, estimated data of wave height at the point, and observed data of wave height at the point.

[0014] An estimation method according to the present invention is a method in which a computer estimates wave height at a certain point on the ocean at any given time. This estimation method includes an estimation processing step of inputting image data of a coast related to the point at the time and first estimated data of wave height at the point at the time into a trained first learning device, and outputting second estimated data of wave height at the point at the time. The trained first learning device is trained using a training dataset that is a compilation of training data including image data of the coast at the time, estimated data of wave height at the point, and observed data of wave height at the point.

[0015] The estimation program according to the present invention is a program for estimating wave height at a certain point on the ocean at an arbitrary time. This estimation program causes a computer to function as an estimation processing unit that inputs image data of a coast related to the point at the time and first estimated data of wave height at the point at the time into a trained first learning device, and outputs second estimated data of wave height at the point at the time. The trained first learning device has been trained using a training dataset that is a compilation of training data including image data of the coast at the time, estimated data of wave height at the point, and observed data of wave height at the point.

[0016] An estimation system according to the present invention includes the above-described estimation device and an imaging device that captures the image data for estimation data.

[0017] The estimation device, estimation method, estimation program, and estimation system according to the present invention enable wave height estimation without installing a buoy on the sea. This makes it possible to easily observe wave height regardless of location or time. In particular, it is easy to observe over a long period of time. [Effects of the Invention]

[0018] According to the present invention, long-term observation of wave height is easy. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a schematic configuration diagram of a wave height estimation system according to a first embodiment of the present invention. [Figure 2A] 1 is a functional configuration diagram of a learning device according to a first embodiment of the present invention. [Figure 2B] 1 is a functional configuration diagram of an estimation device according to a first embodiment of the present invention. [Figure 3] 3 is a flowchart for explaining the overall operation of the wave height estimation system according to the first embodiment of the present invention. [Figure 4] FIG. 4 is a schematic configuration diagram of a wave height estimation system according to a second embodiment of the present invention. [Figure 5A] 1 is a functional configuration diagram of a learning device according to a first embodiment of the present invention. [Figure 5B] 1 is a functional configuration diagram of an estimation device according to a first embodiment of the present invention. [Figure 6] This is the result of conventional numerical calculation. [Figure 7] 10 shows the results of a verification test using the method of the first embodiment. [Figure 8] 10 shows the results of a verification test using the method of the second embodiment. [Figure 9] This shows the results of a verification test when coupling SWAN and WAVEWATCH III is used as the method of the second embodiment. [Figure 10] FIG. 1 is a diagram showing the area of ​​wind speed data used in the verification test. [Figure 11] This is an image of the SWAN calculation domain used in the validation test. [Figure 12] FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device according to each embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] [First embodiment] <Configuration of wave height estimation system according to the first embodiment> The configuration of a wave height estimation system 1 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic configuration diagram of the wave height estimation system 1 according to the first embodiment. The wave height estimation system 1 estimates the wave height at any point on the sea (for example, offshore).

[0021] As shown in FIG. 1, the wave height estimation system 1 includes a photographing device 2 and information processing devices 3 and 4. The photographing device 2 is installed on the coast. The installation locations of the information processing devices 3 and 4 are not limited and may be any location. For example, the information processing device 3 is installed in a location close enough to the photographing device 2 to be connected via a LAN (Local Area Network). On the other hand, the information processing device 4 is connected to the information processing device 3 via a WAN (Wide Area Network). The information processing device 3 and the information processing device 4 may also be configured as a single device.

[0022] The direction and angle of view of the camera device 2 are set so that it can capture images of the coast. The camera device 2 captures still images or moving images and outputs the captured still images or moving images. When capturing still images, the camera device 2 is capable of continuous shooting at a predetermined time interval (for example, one second intervals). The captured still images or moving images (sometimes referred to simply as "images" without distinction) are sent to the information processing device 3. The camera device 2 is, for example, a network camera capable of creating RGB image data. It is preferable to use a camera of a type and with functions appropriate for the environment in which the image is captured; for example, if it is nighttime, it is desirable that the camera device 2 be a camera capable of night-time shooting.

[0023] It is preferable that the photographing direction of the photographing device 2 can be changed. For example, the photographing device 2 can rotate on a horizontal plane around the vertical axis. As will be described in detail later, an orthomosaic image (sometimes shortened to "orthoimage") is created from the images photographed by the photographing device 2. An orthomosaic image can be created by projectively transforming images, and is an image that is displayed at the correct size and position without tilt, as if the object were viewed from directly above. It is also possible to photograph the target coastline using multiple photographing devices 2, and create an orthomosaic image using the images photographed by each photographing device 2.

[0024] The information processing devices 3 and 4 are devices (computers) equipped with information processing functions. The information processing devices 3 and 4 may be, for example, personal computers (PCs) or application servers communicably connected to the personal computers. The information processing devices 3 and 4 may also be devices constituting a cloud system. The information processing devices 3 and 4 realize various functions (including learning functions) related to wave height estimation by program execution processing using a CPU (Central Processing Unit) or the like.

[0025] The information processing device 3 acquires image data (e.g., RGB data) of a photographed coastline from the photographing device 2 and transmits the acquired image data to the information processing device 4. The information processing device 3 acquires image data from the photographing device 2, for example, in real time or at short time intervals (e.g., every few minutes to several hours), and transmits the image data at long time intervals (e.g., every day to every few days) or in response to a request from the information processing device 4. The information processing device 3 may have a function to control and manage the photographing device 2.

[0026] The information processing device 4 estimates offshore wave height using artificial intelligence (AI) technology. The information processing device 4 acquires image data of a photograph of the coast from the photographing device 2 via the information processing device 3. Note that the information processing device 4 may also acquire image data directly from the photographing device 2. The information processing device 4 also acquires offshore wind speed data. The source from which the information processing device 4 acquires wind speed data is not limited; for example, the information processing device 4 acquires wind speed data from the Mesoscale Numerical Weather Forecast Model GPV (hereinafter sometimes referred to as "MSM") operated by the Japan Meteorological Agency via the Internet. Note that the information processing device 4 acquires wave height observation data (correct answer labels) in addition to wind speed data and image data during the learning stage of the learning device (also referred to as a "learning model" or "AI model"). The observation data (correct answer labels) are omitted from FIG. 1.

[0027] The information processing device 4 estimates offshore wave height based on offshore wind speed data and image data of the coast. In this embodiment, the significant wave height is assumed. The significant wave height is calculated by averaging the heights of one-third of the waves observed at a certain point, starting from the highest wave height, when successive waves are observed one by one. The wave height estimated by the information processing device 4 may be a statistical value other than the significant wave height (e.g., a simple average value), or may be a representative value over a predetermined period of time (e.g., the maximum wave height). The significant wave height estimated by the information processing device 4 offshore may be used, for example, for business activities or research (e.g., studying coastal geographical change characteristics).

[0028] The functions of the information processing device 4 will be described with reference to FIGS. 2A and 2B (and also with reference to FIG. 1 as appropriate). FIG. 2A is a functional configuration diagram of the information processing device 4 according to the first embodiment, showing functions related to the learning stage. FIG. 2B is a functional configuration diagram of the information processing device 4 according to the first embodiment, showing functions related to the estimation stage. In this embodiment, the description will be made assuming that processing related to learning of a learner (processing in the learning stage) and processing related to estimation using a trained learner (processing in the estimation stage) are performed by a single device. In other words, the information processing device 4 performs learning of the learner and estimation using a trained learner. An information processing device 4 focusing on functions related to learning of a learner (functions in the learning stage) will be particularly referred to as a "learning device 4A," and an information processing device 4 focusing on functions related to estimation using a trained learner (functions in the estimation stage) will be particularly referred to as an "estimation device 4B."

[0029] It is also possible to perform the process related to learning of the learning device (processing at the learning stage) and the process related to estimation using the trained learning device (processing at the estimation stage) using separate devices. In this case, the device that performs the process related to learning of the learning device is the "learning device 4A," and the device that performs the process related to estimation using the trained learning device is the "estimation device 4B." The estimation device 4B acquires the trained learning device from the learning device 4A and estimates the wave height using the acquired learning device.

[0030] The learning device 4A shown in FIG. 2A is a device that trains a learning device to output the significant wave height offshore at a certain time based on image data of a coastal image taken at that time and wind speed data offshore at that time. In this embodiment, taking into account the ease of tuning the learning device's parameters, the significant wave height is calculated in two stages rather than estimating the final significant wave height from wind speed data and image data. That is, in the first stage, a provisional significant wave height is calculated from wind speed data, and in the second stage, the final significant wave height is estimated from the image data and the provisional significant wave height calculated in the first stage. Note that if the ease of tuning the parameters is not a consideration, the final significant wave height can also be directly estimated based on wind speed data and image data.

[0031] As shown in FIG. 2A, the learning device 4A mainly includes a first-stage learning unit 10, a first-stage estimation unit 20, and a second-stage learning unit 30. The first-stage learning unit 10 and the first-stage estimation unit 20 are functions related to the first stage of training a learning device to output a provisional significant wave height from wind speed data. The second-stage learning unit 30 is a function related to the second stage of training a learning device to output a final significant wave height from image data and the provisional significant wave height calculated in the first stage.

[0032] The first-stage learning unit 10 includes a wind speed data processing unit 11, a learning data acquisition unit 12, and a learning processing unit 13.

[0033] The wind speed data processing unit 11 extracts wind speeds for every three hours from 24 hours before each time included in an arbitrary first period from wind speed data obtained from the Japan Meteorological Agency. The wind speed data covers a range including the point where the significant wave height is to be estimated. The wind speed data is, for example, vector data consisting of a U component (east-west component) and a V component (north-south component).

[0034] The learning data acquisition unit 12 acquires the observed values ​​of significant wave height for each time period included in the first period from NOWPHAS (Nationwide Ocean Wave information network for Ports and HArbourS) and sets them as correct labels. NOWPHAS is a coastal wave information network that conducts regular wave observations at dozens of locations. In addition, the learning data acquisition unit 12 converts the wind speed and significant wave height into an input format for deep learning to create a learning dataset.

[0035] The learning processing unit 13 has a learning device 14. The learning device 14 is a neural network consisting of an input layer, an intermediate layer, and an output layer. Explanatory variables are input to the input layer, and a target variable is output from the output layer. The intermediate layer plays a role in relating the two. There are no restrictions on the number of intermediate layers and the number of neurons in each intermediate layer, and they can be set arbitrarily. In this embodiment, it is assumed that an LSTM (Long Short-Term Memory) model is used for the learning device 14. LSTM is an extended recurrent network (RNN) and is excellent at learning and predicting time-series data. The learning processing unit 13 inputs the data set created by the learning data acquisition unit 12 to the learning device 14, and causes the learning device 14 to learn the relationship between the observed values ​​of wind speed and significant wave height.

[0036] The first stage estimation unit 20 includes a wind speed data processing unit 21, an estimation data acquisition unit 22, and an estimation processing unit .

[0037] The wind speed data processing unit 21 extracts wind speeds for three-hour intervals starting from 24 hours before each time included in an arbitrary second period from the wind speed data obtained from the Japan Meteorological Agency. The second period is, for example, a period different from the first period described in the wind speed data processing unit 11, and is preferably a period that does not overlap. Note that the first period and the second period may partially overlap. The wind speed data is for a range that includes the point where the significant wave height is to be estimated. The wind speed data is, for example, vector data composed of a U component (east-west component) and a V component (north-south component).

[0038] The estimation data acquisition unit 22 converts the wind speed into an input format for deep learning and creates a data set for estimation.

[0039] The estimation processing unit 23 has a trained learning device 24. The learning device 24 is a learning device obtained by training the learning device 14. The estimation processing unit 23 inputs the data set for the second period created by the estimated data acquisition unit 22 into the trained learning device 24, and outputs an estimated value of the significant wave height for each time included in the second period.

[0040] The second-stage learning unit 30 includes an image processing unit 31, a wave height data processing unit 32, a learning data acquisition unit 33, and a learning processing unit .

[0041] The image processing unit 31 processes images from multiple directions (here, five directions, "Angle A" to "Angle E," are assumed) obtained from the photographing device 2 (see FIG. 1) installed on the coast, and creates orthomosaic images for each time included in the second period. The orthomosaic images show coasts related to the offshore point for which significant wave heights are to be estimated. A coast related to an offshore point is a coast that is affected by waves generated at the offshore point, such as a coast close to the offshore point.

[0042] The wave height data processing unit 32 processes the estimated values ​​of the significant wave height at each time included in the second period obtained by the first-stage estimation unit 20. For example, the wave height data processing unit 32 extracts only the significant wave height at the time corresponding to the orthomosaic image from the estimated values ​​of the significant wave height obtained by the first-stage estimation unit 20, and also performs normalization processing.

[0043] The training data acquisition unit 33 acquires orthomosaic images for each time included in the second period from the image processing unit 31, and also acquires estimated values ​​of significant wave height for each time included in the second period from the wave height data processing unit 32. The training data acquisition unit 33 acquires observed values ​​of significant wave height for each time included in the second period from NOWPHAS and sets them as correct labels. The training data acquisition unit 33 also converts the orthomosaic images for the same time, the estimated values ​​of significant wave height, and the observed values ​​of significant wave height into an input format for deep learning, and creates a training dataset. The training data acquisition unit 33 may store the training dataset in a memory unit in advance and acquire the training dataset from the memory unit as needed. The memory unit for storing the training dataset in advance may be located externally.

[0044] The learning processing unit 34 has a learning device 35. The learning device 35 is a neural network consisting of an input layer, a hidden layer, and an output layer. Explanatory variables are input to the input layer, and a target variable is output from the output layer. The hidden layer plays a role in relating the two. There are no restrictions on the number of hidden layers and the number of neurons in each hidden layer, and they can be set as desired. In this embodiment, it is assumed that the learning device 35 is constructed using a convolutional neural network (CNN), which is often used in the field of image recognition. The learning processing unit 34 inputs the data set created by the training data acquisition unit 33 to the learning device 35, and causes the learning device 35 to learn the relationship when the orthomosaic image and the estimated value of significant wave height are used as explanatory variables and the observed value of significant wave height is used as the objective function.

[0045] The estimation device 4B shown in Fig. 2B is a device that outputs an estimated value of the significant wave height at a certain point in time offshore, using trained learners 24 and 45 created by the learning device 4A (see Fig. 2A). The estimation device 4B mainly includes a first-stage estimation unit 20 and a second-stage estimation unit 40.

[0046] The first-stage estimation unit 20 of the estimation device 4B has the same functions as the learning device 4A (see FIG. 2A). The data input to the first-stage estimation unit 20 of the estimation device 4B is different from that of the learning device 4A. In the estimation stage, wind speeds every three hours starting from 24 hours before a certain point in time to be estimated are input to the first-stage estimation unit 20. The certain point in time is any point in time that is not included in the first period or the second period. The first-stage estimation unit 20 then outputs an estimated value (first estimated data) of the significant wave height at the certain point in time offshore of interest.

[0047] The second-stage estimation unit 40 includes an image processing unit 41, a wave height data processing unit 42, an estimation data acquisition unit 43, and an estimation processing unit 44.

[0048] The image processing unit 41 processes images taken in multiple directions (here, five directions are assumed) obtained from the photographing device 2 (see Figure 1) installed on the coast, and creates an orthomosaic image for any point in time. The orthomosaic image shows the coast related to the offshore point for which the significant wave height is to be estimated.

[0049] The wave height data processing unit 42 acquires the estimated value of the significant wave height at any point in time obtained by the first stage estimation unit 20 and performs normalization processing.

[0050] The estimation data acquisition unit 43 acquires an orthomosaic image at any time from the image processing unit 41, and also acquires an estimated value of significant wave height at any time from the wave height data processing unit 42. The estimation data acquisition unit 43 converts the orthomosaic image at the same time and the estimated value of significant wave height into an input format for deep learning, and creates a dataset for estimation. The estimation data acquisition unit 43 may store the estimation dataset in a storage unit in advance and acquire the estimation dataset from the storage unit as needed. The storage unit for storing the estimation dataset in advance may be located externally.

[0051] The estimation processing unit 44 has a trained learning device 45. The learning device 45 is a learning device obtained by training the learning device 35 (see FIG. 2A). The estimation processing unit 44 inputs a data set at an arbitrary time point created by the estimated data acquisition unit 43 to the trained learning device 45, and outputs a final estimated value (second estimated data) of the significant wave height at the arbitrary time point.

[0052] <Operation of the wave height estimation system according to the first embodiment> The operation of the wave height estimation system 1 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart for explaining the overall operation of the wave height estimation system 1 according to the first embodiment. The operation of the wave height estimation system 1 can be mainly divided into a trained learning device construction step S10 and a wave height estimation step S20.

[0053] (Process S10 for constructing a trained learning machine) Input values ​​such as coastal image data and wind speed data, as well as NOWPHAS observation values ​​(ground truth labels), are obtained and converted into an input format for deep learning (step S11). The dataset created using these is divided into training data and evaluation data, and the learning machine is trained using the training data.

[0054] Next, the parameters are tuned (step S12), and the estimation accuracy is evaluated by inputting evaluation data to the learning device trained using the training data (step S13). The parameters are tuned until the estimation accuracy is sufficient, and a trained learning device is constructed.

[0055] (Wave height estimation process S20) Data for the date and time to be estimated is prepared and input into the trained learning device (step S21). The estimation results are confirmed (step S22), and the output wave height is used to examine topographical change characteristics, etc. (step S23).

[0056] As described above, the wave height estimation system 1 according to the first embodiment can estimate wave height without installing a buoy on the sea. This makes it possible to easily observe wave height regardless of location or time. In particular, it is easy to observe over a long period of time.

[0057] Furthermore, the wave height estimation system 1 can estimate significant wave heights offshore that are not visible in images by inputting not only images of the coast but also significant wave heights that take wind speed into account. Furthermore, the estimation accuracy is high, and calculation time can be significantly reduced compared to methods that rely on numerical calculations.

[0058] Furthermore, the wave height estimation system 1 improves the accuracy of wave height estimation by inputting an orthomosaic image of the coast. We calculated the root mean squared error (RMSE) when using a single-directional image and when using an orthomosaic image. As a result, the RMSE for "Angle A" in Figure 2A was 0.404 m, the RMSE for "Angle B" was 0.349 m, the RMSE for "Angle C" was 0.352 m, the RMSE for "Angle D" was 0.383 m, and the RMSE for "Angle E" was 0.538 m, resulting in a RMSE of 0.325 m for the orthomosaic image. Thus, it can be concluded that using an orthomosaic image reduces the RMSE and improves the accuracy of significant wave height estimation compared to when using single-directional images (Angles A to E).

[0059] [Second embodiment] <Configuration of wave height estimation system according to the second embodiment> The configuration of a wave height estimation system 101 according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a schematic configuration diagram of the wave height estimation system 101 according to the second embodiment. The wave height estimation system 101 estimates the wave height at an arbitrary point on the sea (for example, offshore).

[0060] As shown in Fig. 4, the wave height estimation system 101 according to the second embodiment includes a photographing device 2 and information processing devices 3 and 104. That is, the wave height estimation system 101 has a configuration in which the information processing device 4 (see Fig. 1) in the wave height estimation system 1 according to the first embodiment is replaced with the information processing device 104. The following description will focus on the differences from the first embodiment, and a description of the same configuration as the first embodiment will be omitted.

[0061] The information processing device 104 is a device (computer) equipped with an information processing function. The information processing device 104 may be, for example, a personal computer (PC) or an application server communicably connected to the personal computer. The information processing device 104 may also be a device constituting a cloud system. The information processing device 104 realizes various functions (including learning functions) related to wave height estimation by program execution processing using a CPU (Central Processing Unit) or the like.

[0062] The information processing device 104 estimates offshore wave height using artificial intelligence technology (AI technology). As in the first embodiment, significant wave height is assumed as the wave height in this embodiment. The information processing device 4 in the first embodiment uses a learning device to determine the provisional significant wave height in the first stage, and then uses a further learning device to estimate the final significant wave height from the image data and the provisional significant wave height determined in the first stage in the second stage. The information processing device 104 according to the second embodiment uses a wave prediction model to determine the provisional significant wave height in the first stage, and then uses a learning device to estimate the final significant wave height from the image data and the provisional significant wave height determined in the first stage in the second stage.

[0063] The information processing device 104 acquires image data of a coast captured by the photographing device 2 via the information processing device 3. The information processing device 104 may also acquire image data directly from the photographing device 2. The information processing device 4 also acquires input data required for the wave estimation model. The input data required for the wave estimation model differs depending on the type of wave estimation model. The information processing device 104 acquires the input data required for the wave estimation model, for example, from the Japan Meteorological Agency via the Internet. The information processing device 104 also acquires wave height observation data (correct answer labels) in addition to the input data and image data for the wave estimation model during the learning stage of the learner. In FIG. 4, the description of the observation data (correct answer labels) is omitted.

[0064] The functions of the information processing device 104 will be described with reference to Figs. 5A and 5B (and Fig. 4 as appropriate). Fig. 5A is a functional configuration diagram of the information processing device 104 according to the second embodiment, showing functions related to the learning stage. Fig. 5B is a functional configuration diagram of the information processing device 104 according to the second embodiment, showing functions related to the estimation stage. As with the first embodiment, this embodiment will be described assuming that processing related to learning of a learning device (processing in the learning stage) and processing related to estimation using a trained learning device (processing in the estimation stage) are performed by a single device.

[0065] An information processing device 104 that focuses on the function related to learning of a learning device (function at the learning stage) is particularly referred to as a "learning device 104A," and an information processing device 104 that focuses on the function related to estimation using a trained learning device (function at the estimation stage) is particularly referred to as an "estimation device 104B." Note that the processing related to learning of a learning device (processing at the learning stage) and the processing related to estimation using a trained learning device (processing at the estimation stage) can be performed by separate devices.

[0066] The learning device 104A shown in Figure 5A is a device that trains a learning device to output the significant wave height at a certain time point offshore based on image data of a coastal photograph taken at that time point and an estimated value of the significant wave height that is the calculation result of the wave prediction model 51.

[0067] As shown in FIG. 5A, the learning device 104A includes a wave estimation unit 50 and a learning unit 60. The wave estimation unit 50 has one or more wave estimation models 51. The wave estimation unit 50 calculates estimated values ​​related to waves (including estimated values ​​of significant wave height) using one or more wave estimation models 51. Note that a device other than the learning device 104A may have the wave estimation model 51, and the learning device 104A may acquire the calculation results of the wave estimation model 51 as needed.

[0068] The wave prediction model 51 is preferably a third-generation wave prediction model. Examples of third-generation wave prediction models include "SWAN (Simulating WAves Nearshore)" and "WAVEWATCH III." SWAN was developed for shallow waters, while WAVEWATCH III was developed for deep waters. The wave prediction unit 50 inputs meteorological information and water depth data into SWAN, for example, and outputs the significant wave height. The output significant wave height is input to the calculation result processing unit 62. The significant wave height may also be calculated by coupling SWAN and WAVEWATCH III. In this case, for example, the open ocean is calculated using WAVEWATCH III, and the significant wave height is estimated using the calculation results of WAVEWATCH III as the boundary conditions for SWAN.

[0069] 5A, the learning unit 60 has an image processing unit 61, a calculation result processing unit 62, a learning data acquisition unit 63, and a learning processing unit 64. The image processing unit 61, the calculation result processing unit 62, the learning data acquisition unit 63, and the learning processing unit 64 are functions corresponding to the image processing unit 31, the pulse-height data processing unit 32, the learning data acquisition unit 33, and the learning processing unit 34 of the second-stage learning unit 30 according to the first embodiment.

[0070] The image processing unit 61 processes images from multiple directions (here, five directions are assumed) obtained from the photographing device 2 (see FIG. 4) installed on the coast, and creates an orthomosaic image from a certain point in time in the past.

[0071] The calculation result processing unit 62 processes the estimated value of the significant wave height included in the calculation result obtained by the wave prediction unit 50. For example, the calculation result processing unit 62 extracts only the significant wave height at the time corresponding to the orthomosaic image from the estimated value of the significant wave height obtained by the wave prediction unit 50, and also performs normalization processing.

[0072] The training data acquisition unit 63 acquires an orthomosaic image from a certain point in time past from the image processing unit 61, and also acquires an estimated value of significant wave height at a certain point in time past from the calculation result processing unit 62. The training data acquisition unit 63 acquires observed values ​​of significant wave height from a certain point in time past from NOWPHAS and sets them as correct labels. The training data acquisition unit 63 also converts the orthomosaic image from the same time, the estimated value of significant wave height, and the observed value of significant wave height into an input format for deep learning, and creates a training dataset. The training data acquisition unit 63 may store the training dataset in a memory unit in advance and acquire the training dataset from the memory unit as needed. The memory unit for storing the training dataset in advance may be located externally.

[0073] The learning processing unit 64 has a learning device 65. The learning device 65 is a neural network consisting of an input layer, an intermediate layer, and an output layer. In this embodiment, it is assumed that the learning device 65 is constructed using a convolutional neural network (CNN), which is often used in the field of image recognition. The learning processing unit 64 inputs the data set created by the learning data acquisition unit 63 to the learning device 65, and causes the learning device 65 to learn the relationship when the orthomosaic image and the estimated value of significant wave height are used as explanatory variables and the observed value of significant wave height is used as an objective function.

[0074] The estimation device 104B shown in Fig. 5B is a device that outputs an estimated value of the significant wave height at a certain point in time offshore, using a trained learning device 75 created by the learning device 104A (see Fig. 5A). The estimation device 104B includes a wave prediction unit 50 and an estimation unit 70.

[0075] The wave estimation unit 50 of the estimation device 104B has the same functions as the learning device 104A (see FIG. 5A). The wave estimation unit 50 uses one or more wave estimation models 51 to calculate estimated values ​​related to waves (including estimated values ​​of significant wave height).

[0076] 5B, the estimation unit 70 has an image processing unit 71, a calculation result processing unit 72, an estimated data acquisition unit 73, and an estimation processing unit 74. The image processing unit 71, the calculation result processing unit 72, the estimated data acquisition unit 73, and the estimation processing unit 74 are functions corresponding to the image processing unit 41, the wave height data processing unit 42, the estimated data acquisition unit 43, and the estimation processing unit 44 of the second-stage estimation unit 40 according to the first embodiment.

[0077] The image processing unit 71 processes images from multiple directions (here, five directions are assumed) obtained from the photographing device 2 (see FIG. 4) installed on the coast, and creates an orthomosaic image at a certain point in time.

[0078] The calculation result processing unit 72 acquires the estimated value of the significant wave height obtained by the wave prediction unit 50 and performs normalization processing.

[0079] The estimation data acquisition unit 73 acquires an orthomosaic image at any time from the image processing unit 71, and also acquires an estimated value of significant wave height at any time from the calculation result processing unit 72. The estimation data acquisition unit 73 converts the orthomosaic image at the same time and the estimated value of significant wave height into an input format for deep learning, and creates a dataset for estimation. The estimation data acquisition unit 73 may store the estimation dataset in a storage unit in advance and acquire the estimation dataset from the storage unit as needed. The storage unit for storing the estimation dataset in advance may be located externally.

[0080] The estimation processing unit 74 has a trained learning device 75. The learning device 75 is a learning device obtained by training the learning device 65 (see FIG. 5A). The estimation processing unit 74 inputs a data set at an arbitrary time point created by the estimated data acquisition unit 73 to the trained learning device 75, and outputs a final estimated value of the significant wave height at the arbitrary time point.

[0081] As described above, the wave height estimation system 101 according to the second embodiment can estimate wave height without installing a buoy on the sea. This makes it possible to easily observe wave height regardless of location or time. In particular, it is easy to observe over a long period of time.

[0082] Furthermore, the wave height estimation system 101 can estimate significant wave heights offshore that are not visible in the image by inputting not only images of the coast but also the calculation results of a wave prediction model. As with the first embodiment, the estimation accuracy is high. Note that the effect of inputting an orthomosaic image is also the same as in the first embodiment.

[0083] In order to verify the effects of the learning devices 4A and 104A and the estimation devices 4B and 104B according to the respective embodiments, verification tests using actual data were carried out, which will now be described. In the verification tests, (1) estimation of significant wave height using SWAN as a conventional numerical calculation, (2) estimation using coastal images and wind speed data as a method according to the first embodiment, and (3) estimation using coastal images and SWAN output as a method according to the second embodiment were performed.

[0084] FIG. 6 shows the estimation results by conventional numerical calculation, FIG. 7 shows the estimation results by the method of the first embodiment, and FIG. 8 shows the estimation results by the method of the second embodiment. It can be seen that the estimation results by the method of the first embodiment (see FIG. 7) and the estimation results by the method of the second embodiment (see FIG. 8) are more accurate than the estimation results by conventional numerical calculation (see FIG. 6).

[0085] Furthermore, in the method of the second embodiment, a test was conducted to verify the effect of coupling SWAN and WAVEWATCH III as the wave prediction model 51. Figure 9 shows the wave height estimation results at Cape Shionomisaki during the typhoon period. Figure 9 compares the wave heights observed by NOWPHAS, the wave heights estimated using only SWAN, and the wave heights estimated by SWAN, which were calculated using WAVEWATCH III for the open ocean and used as the boundary conditions for SWAN. Figure 9 shows that the values ​​estimated using the coupling of WAVEWATCH III and SWAN are closer to the observed values, confirming improved estimation accuracy. This demonstrates that, even when the wave prediction portion using SWAN is replaced with a different model in the method of the second embodiment, wave heights can be estimated with sufficient accuracy.

[0086] The details of the verification test will be explained below. The imaging device 2 captures images in five directions, changing direction every 10 minutes. Images are captured at one-second intervals for five minutes of the 10 minutes allocated for each direction, allowing a maximum of 300 images to be acquired per hour in each direction. The verification test used images of the coast taken between June 2012 and December 2018.

[0087] Images taken in each direction by camera 2 were averaged every hour. The images were then projected and cropped using the georeferencing function of ESRI ArcGIS to create an orthomosaic image. Control points were used during this process. The pixel count of the created orthomosaic image was 1600 x 400px, but this was reduced to 160 x 40px to avoid memory shortages when applying deep learning. Estimation accuracy was verified using the root mean square error (RMSE).

[0088] The wind speed data domain used in the validation test is shown in Figure 10. An image of the SWAN calculation domain used in the validation test is shown in Figure 11.

[0089] The information processing devices 3, 4, and 104 according to each embodiment are realized by a computer having a configuration as shown in Fig. 12, for example. Fig. 12 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing devices 3, 4, and 104 according to each embodiment. The computer has a CPU 401, a ROM (Read Only Memory) 402, a RAM 403, an HDD (Hard Disk Drive) 404, an input / output I / F (Interface) 405, a communication I / F 406, and a media I / F 407. These components are connected by a bus.

[0090] Although the embodiment of the present invention has been described above, the present invention is not limited to this and can be practiced within the scope of the claims. [Explanation of symbols]

[0091] 1,101 Wave height estimation system (learning system, estimation system) 2. Imaging equipment 3,4,104 Information processing equipment 4A,104A Learning Device 4B,104B Estimation device 10 First Stage Learning Section 11 Wind speed data processing section 12 Learning data acquisition unit 13 Learning processing unit 20 First stage estimation section 21 Wind speed data processing section 22 Estimation data acquisition section 23 Estimation processing unit 30 Second Stage Learning Section 31 Image processing section 32 Wave height data processing section 33 Learning data acquisition unit 34 Learning processing unit 40 Second stage estimation section 41 Image processing section 42 Wave height data processing section 43 Estimation data acquisition section 44 Estimation processing unit 50 Wave Estimation Department 60 Learning Department 61 Image processing section 62 Calculation result processing section 63 Learning data acquisition unit 64 Learning processing unit 70 Estimation part 71 Image processing section 72 Calculation result processing section 73 Estimation Data Acquisition Unit 74 Estimation processing unit

Claims

1. A learning device that learns to be able to estimate wave height at any point on the ocean, a learning data acquisition unit that acquires learning data including image data of a coastline related to the location, estimated data of wave height at the location, and observed data of wave height at the location; a learning processing unit that performs machine learning on a first learning device using a learning dataset that is a compilation of the learning data, the learning processing unit inputs image data of the coast at the time point and first estimated data of wave height at the point at the time point, and causes the first learning device to learn so as to output second estimated data of wave height at the point at the time point; A learning device characterized by:

2. the estimated data included in the learning data is obtained by a second learning device that is trained to output estimated data of wave height at the point by inputting wind speed data in a range including the point; 2. The learning device according to claim 1 .

3. The estimated data included in the training data is obtained using one or more wave prediction models.

2. The learning device according to claim 1.

4. The estimated data included in the training data is obtained by coupling a first wave prediction model for shallow waters and a second wave prediction model for deep waters.

2. The learning device according to claim 1 .

5. 2. The learning device according to claim 1, wherein the image data is an orthomosaic image of the coast.

6. A learning method in which a computer learns to be able to estimate wave height at any point on the ocean, comprising: a learning data acquisition step of acquiring learning data including image data of a coastline related to the location, estimated data of wave height at the location, and observed data of wave height at the location; a learning processing step of performing machine learning on a first learning device using a learning dataset that is a compilation of the learning data, In the learning process, the first learning device is trained to output second estimated data of wave height at the point of time by inputting image data of the coast at the point of time and first estimated data of wave height at the point of time. A learning method characterized by:

7. An estimation device for estimating wave height at a point on the sea at any time, an estimation processing unit that outputs second estimated data of the wave height at the point at the time by inputting image data of a coast related to the point at the time when the image data was taken and first estimated data of the wave height at the point at the time into a trained first learning device; The trained first learning device is trained using a learning dataset that is a compilation of learning data including image data of the coast, estimated data of wave height at the point, and observed data of wave height at the point. An estimation device characterized by:

8. An estimation method in which a computer estimates wave height at a point on the ocean at any time, comprising: an estimation processing step of inputting image data of a coastline related to the location at the time point and first estimated data of wave height at the location at the time point into a trained first learning device, and outputting second estimated data of wave height at the location at the time point; The trained first learning device is trained using a learning dataset that is a compilation of learning data including image data of the coast, estimated data of wave height at the point, and observed data of wave height at the point. An estimation method characterized by:

9. An estimation program for estimating wave height at any point on the ocean, Computer, image data of a coastline related to the location at the time point and first estimated data of wave height at the location at the time point are input to the trained first learning device, thereby causing the first learning device to function as an estimation processing unit that outputs second estimated data of wave height at the location at the time point, The trained first learning device is trained using a learning dataset that is a compilation of learning data including image data of the coast, estimated data of wave height at the point, and observed data of wave height at the point. An estimation program characterized by:

10. The estimation device according to claim 7 ; an imaging device that captures the image data for estimation data; An estimation system comprising:

Images