Sea wave height determination method and device, storage medium and electronic device

Abstract

The invention discloses a sea wave height determination method and device, a storage medium and an electronic device.The method comprises the steps that sea wave video data collected by image collection equipment deployed at a preset position in real time are obtained, the original video frame rate of the sea wave video data is a preset value, and the original video frame rate of the sea wave video data is the preset value; the preset position comprises at least one of an offshore wind power tower, an ocean platform and a ship; inputting the sea wave video data and a first spectrogram corresponding to each frame of image in the sea wave video data into a trained first convolutional neural network to obtain an output result; and performing moving average processing on the continuous output results to determine the sea wave height corresponding to the sea wave video data. The problems of high cost and limited coverage range of traditional sea wave height monitoring methods such as buoys and radars are solved.

Description

Technical Field

[0001] This application relates to the field of ocean monitoring technology. Specifically, it relates to a method and device for determining wave height, a storage medium, and an electronic device. Background Art

[0002] With the development of offshore wind power, port construction, and ocean engineering, the demand for real-time monitoring of ocean environmental parameters is increasing day by day. Wave height is one of the key indicators, and its accurate monitoring helps to optimize offshore operation plans, improve equipment operation efficiency, and reduce risks. Traditional monitoring methods such as buoys and radars have high accuracy, but are costly and have limited coverage.

[0003] In related technologies, there is no effective solution to the problem that traditional monitoring methods for wave height, such as buoys and radars, are costly and have limited coverage. Summary of the Invention

[0004] Embodiments of this application provide a method and device for determining wave height, a storage medium, and an electronic device to at least solve the problem in the prior art that traditional monitoring methods for wave height, such as buoys and radars, are costly and have limited coverage.

[0005] According to an embodiment of the embodiments of this application, a method for determining wave height is provided, including: obtaining real-time wave video data collected by an image acquisition device deployed at a preset position, where the original video frame rate of the wave video data is a preset value, and the preset position includes at least one of the following: an offshore wind turbine tower, an ocean platform, a ship; inputting the wave video data and a first spectrogram corresponding to each frame image in the wave video data into a trained first convolutional neural network to obtain an output result; performing a moving average process on a continuous plurality of the output results to determine the wave height corresponding to the wave video data.

[0006] In an exemplary embodiment, before inputting the wave video data and a first spectrogram corresponding to each frame image in the wave video data into a trained first convolutional neural network to obtain an output result, the method further includes: obtaining an untrained second convolutional neural network, where the second convolutional neural network includes: an input layer, a convolutional layer connected to the input layer, a pooling layer connected to the convolutional layer, and a fully connected layer connected to the pooling layer; training the second convolutional neural network with target training data expanded by adopting a data augmentation technique to obtain the first convolutional neural network.

[0007] In an exemplary embodiment, before training the second convolutional neural network with the target training data expanded by adopting data augmentation technology to obtain the first convolutional neural network, the method further includes: obtaining basic training data, where the basic training data includes: sea wave images and sea wave height measurement values corresponding to the sea wave images, and the sea wave images have been subjected to preprocessing operations; performing fast Fourier transform on each sea wave image in the basic training data to obtain a second spectrogram corresponding to each sea wave image; determining the target training data based on the second spectrogram and the basic training data.

[0008] In an exemplary embodiment, training the second convolutional neural network with the target training data expanded by adopting data augmentation technology to obtain the first convolutional neural network includes: an input step: inputting the sea wave images in the target training data and the second spectrograms corresponding to the sea wave images into the second convolutional neural network to obtain prediction values; a calculation step: calculating the loss between the prediction values and the sea wave height measurement values corresponding to the sea wave images in the target training data through a preset loss function, and performing backpropagation gradient update on the second convolutional neural network through the loss; an update step: updating the target parameters of the second convolutional neural network according to the backpropagation gradient update through an Adam optimizer, where the target parameters at least include: weights and biases; repeatedly executing the input step, the calculation step, and the update step until the number of loops meets a preset number of iterations to obtain the first convolutional neural network.

[0009] In an exemplary embodiment, inputting the sea wave video data and the first spectrograms corresponding to each frame image in the sea wave video data into the trained first convolutional neural network to obtain an output result includes: processing each frame image and the first spectrogram into multi-channel features through an input layer in the first convolutional neural network; extracting high-dimensional features corresponding to the multi-channel features through a convolutional layer in the first convolutional neural network, where the high-dimensional features include: ripple detail features, periodic features, and spatial features; splicing the high-dimensional features and the energy distribution information corresponding to the first spectrogram according to a splicing weight to obtain a joint feature vector; determining the output result through the joint feature vector.

[0010] In an exemplary embodiment, performing moving average processing on a continuous plurality of the output results to determine the sea wave height corresponding to the sea wave video data includes: determining the number of results of a continuous plurality of the output results; performing moving average processing on a continuous plurality of the output results through a sliding window with a window size of the number of results to obtain the sea wave height corresponding to the sea wave video data.

[0011] According to another embodiment of the embodiments of the present application, there is also provided a device for determining the wave height, including: an acquisition module, configured to acquire the wave video data collected in real time by an image acquisition device deployed at a preset position, where the original video frame rate of the wave video data is a preset value, and the preset position includes at least one of the following: an offshore wind power tower, an ocean platform, a ship; an input module, configured to input the wave video data and the first spectrogram corresponding to each frame image in the wave video data into a trained first convolutional neural network to obtain an output result; a determination module, configured to perform a moving average process on a plurality of consecutive output results to determine the wave height corresponding to the wave video data.

[0012] According to another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, and the computer program is configured to execute the above method when running.

[0013] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the above processor executes the above method through the computer program.

[0014] According to another aspect of the embodiments of the present application, there is also provided a computer program product, including a computer program, where the computer program implements the steps in any one of the above method embodiments when executed by a processor.

[0015] In the embodiments of the present application, the wave video data collected in real time by an image acquisition device deployed at a preset position is acquired, where the original video frame rate of the wave video data is a preset value, and the preset position includes at least one of the following: an offshore wind power tower, an ocean platform, a ship; the wave video data and the first spectrogram corresponding to each frame image in the wave video data are input into a trained first convolutional neural network to obtain an output result; a moving average process is performed on a plurality of consecutive output results to determine the wave height corresponding to the wave video data. Through the above embodiments, large-scale real-time monitoring can be achieved through ordinary image acquisition devices, and the automatic feature extraction ability of deep learning and the physical interpretability of frequency domain analysis are utilized, which not only avoids the high cost of traditional sensor layout, but also is superior to traditional methods in terms of accuracy and robustness. It solves the problem in the prior art that traditional monitoring methods for wave height, such as buoys and radars, are costly and have limited coverage. Description of the Drawings

[0016] The accompanying drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0017] Figure 1 is a hardware structure block diagram of a computer terminal for a method of determining the wave height according to an embodiment of the present application;

[0018] Figure 2 is a flowchart of a method of determining the wave height according to an embodiment of the present application;

[0019] Figure 3 is a structure block diagram of a device for determining the wave height according to an embodiment of the present application. Detailed implementation manners

[0020] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned accompanying drawings are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices; "a plurality" means two or more.

[0022] The method embodiments provided by the embodiments of the present application can be executed on a computer terminal or a similar computing device or a cloud platform or an independent physical server or a software platform, wherein the above-mentioned software platform runs through one or more servers. Taking running on a computer terminal as an example, Figure 1 is a hardware structure block diagram of a computer terminal for a method of determining the wave height according to an embodiment of the present application. As Figure 1 shown, the computer terminal may include one or more ( Figure 1Only one processor 102 and a memory 104 for storing data are shown. In one exemplary embodiment, the computer terminal may further include a transmission device 106 for communication functions and an input / output device 108. Among them, the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. Those of ordinary skill in the art can understand that Figure 1 The structure shown is only schematic and does not limit the structure of the above computer terminal. For example, the computer terminal may further include more or fewer components than Figure 1 shown in, or have the same functions as Figure 1 shown in or different configurations with more functions than Figure 1 shown in.

[0023] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, implements the above method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories may be connected to the computer terminal through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0024] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the computer terminal. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (Radio Frequency, abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0025] In this embodiment, a method for determining the sea wave height is provided, which is applied to the above computer terminal. Figure 2 is a flowchart of the method for determining the sea wave height according to the embodiments of the present application. The process includes the following steps:

[0026] Step S202: Obtain the wave video data collected in real time by an image acquisition device deployed at a preset location. The original video frame rate of the wave video data is a preset value. The preset location includes at least one of the following: an offshore wind turbine tower, an offshore platform, and a ship.

[0027] Step S204: Input the wave video data and the first spectrogram corresponding to each frame image in the wave video data into the trained first convolutional neural network to obtain an output result.

[0028] Step S206: Perform a moving average process on a plurality of consecutive output results to determine the wave height corresponding to the wave video data.

[0029] Through the above steps, obtain the wave video data collected in real time by an image acquisition device deployed at a preset location. The original video frame rate of the wave video data is a preset value. The preset location includes at least one of the following: an offshore wind turbine tower, an offshore platform, and a ship. Input the wave video data and the first spectrogram corresponding to each frame image in the wave video data into the trained first convolutional neural network to obtain an output result. Perform a moving average process on a plurality of consecutive output results to determine the wave height corresponding to the wave video data. Through the above embodiments, large-scale real-time monitoring can be achieved by ordinary image acquisition devices, and the automatic feature extraction ability of deep learning and the physical interpretability of frequency domain analysis are utilized. This not only avoids the high cost of traditional sensor layout but also is superior to traditional methods in terms of accuracy and robustness. It solves the problem in the prior art that traditional monitoring methods for wave height, such as buoys and radars, are expensive and have limited coverage.

[0030] In an exemplary embodiment, before inputting the wave video data and the first spectrogram corresponding to each frame image in the wave video data into the trained first convolutional neural network to obtain an output result, the method further includes: obtaining an untrained second convolutional neural network, where the second convolutional neural network includes: an input layer, a convolutional layer connected to the input layer, a pooling layer connected to the convolutional layer, and a fully connected layer connected to the pooling layer; training the second convolutional neural network with target training data expanded by using data augmentation techniques to obtain the first convolutional neural network.

[0031] Specifically, before training the second convolutional neural network with the target training data expanded by adopting data augmentation techniques to obtain the first convolutional neural network, the method further includes: obtaining basic training data, where the basic training data includes: sea wave images and the corresponding sea wave height measurement values of the sea wave images, and the sea wave images have been subjected to preprocessing operations; performing fast Fourier transform on each sea wave image in the basic training data to obtain a second spectrogram corresponding to each sea wave image; determining the target training data based on the second spectrogram and the basic training data.

[0032] It can be understood that to train the first convolutional neural network, first, the architecture of the untrained second convolutional neural network needs to be defined, which may include, for example, an input layer, a convolutional layer, a pooling layer, and a fully connected layer. The size of the input layer should match the size of the input data (sea wave images and spectrograms). The convolutional layer is used to extract local features of the images and spectrograms, such as ripple details, periodicity, and spatial features. The pooling layer is used to reduce the dimension and computational complexity of the features, and the fully connected layer is used to learn the non-linear combinations between the features for the final sea wave height estimation.

[0033] After, before, or at the same time as determining the network architecture of the second convolutional neural network, the target training data for training the second convolutional neural network can be determined. Regarding the target training data, it can be understood that:

[0034] The basic training data contains the actually collected sea wave images and their corresponding sea wave height measurement values. These sea wave images should be preprocessed, such as resizing, normalizing, and denoising, to ensure the quality of the data. The sea wave height measurement values corresponding to the images are used as label data during training.

[0035] Furthermore, perform fast Fourier transform (FFT) on each sea wave image to obtain the spectrogram (second spectrogram) of the image. Combine the sea wave images in the basic training data with the corresponding second spectrograms and sea wave height measurement values to form a target training dataset.

[0036] In addition, to expand the scale of the training dataset and improve the generalization ability of the model, it may be necessary to perform data augmentation on the basic training data. Data augmentation techniques include rotation, scaling, flipping, brightness adjustment, noise addition, etc.

[0037] Finally, use the target training dataset to train the second convolutional neural network, optimize the network weights to minimize the difference between the predicted sea wave height value and the actual sea wave height measurement value, and the trained model is the first convolutional neural network.

[0038] Optionally, the training actions for the second convolutional neural network may include:

[0039] Input step: Input the ocean wave images in the target training data and the corresponding second spectrogram of the ocean wave images into the second convolutional neural network to obtain a prediction value; Calculation step: Calculate the loss between the prediction value and the measured ocean wave height value corresponding to the ocean wave images in the target training data through a preset loss function, and perform backpropagation gradient update on the second convolutional neural network through the loss; Update step: Update the target parameters of the second convolutional neural network according to the backpropagation gradient through the Adam optimizer, where the target parameters at least include: weights and biases; Loop and execute the input step, the calculation step, and the update step until the number of loops meets the preset number of iterations to obtain the first convolutional neural network.

[0040] In an exemplary embodiment, input the ocean wave video data and the first spectrogram corresponding to each frame image in the ocean wave video data into the trained first convolutional neural network to obtain an output result, including: Process the each frame image and the first spectrogram into multi-channel features through the input layer in the first convolutional neural network; Extract the high-dimensional features corresponding to the multi-channel features through the convolutional layer in the first convolutional neural network, where the high-dimensional features include: ripple detail features, periodic features, spatial features; Concatenate the high-dimensional features and the energy distribution information corresponding to the first spectrogram according to the concatenation weights to obtain a joint feature vector; Determine the output result through the joint feature vector.

[0041] That is to say, the input layer receives the preprocessed grayscale image (i.e., each frame image above) and its corresponding spectrogram to form multi-channel features. The convolutional layer performs convolution operations on the input multi-channel features through multiple convolutional kernels to extract features related to ocean waves, including the detail features of the ripples (corresponding to the high-frequency components in the spectrogram), periodic features (corresponding to the low-frequency components in the spectrogram), and spatial features (such as the position and range of the waves in the image). After feature extraction, the high-dimensional features output by the convolutional layer are concatenated with the energy distribution information of the spectrogram to form a joint feature vector. The "concatenation weights" actually refer to the weights learned by the model during training on how to fuse features from different sources to adapt to dynamic environmental changes and optimize the prediction results. Finally, the joint feature vector will be passed to the fully connected layer, and the fully connected layer maps the high-dimensional features to a continuous output space to predict the ocean wave height value of each frame image. In practical applications, this step may also include the application of activation functions and the calculation of losses in the output layer to ensure that the predicted value is as close as possible to the actual ocean wave height.

[0042] Through the above steps, the CNN can effectively learn from the frequency-domain and spatial-domain features of the image and predict the wave height. Meanwhile, by combining the energy distribution information of the spectrogram, the accuracy and robustness of the prediction are improved.

[0043] In an exemplary embodiment, performing a moving average process on a plurality of consecutive output results to determine the wave height corresponding to the wave video data includes: determining the number of the plurality of consecutive output results; performing a moving average process on the plurality of consecutive output results through a sliding window with the window size being the number of the results, to obtain the wave height corresponding to the wave video data.

[0044] It should be noted that the number of output results for the moving average process is the size of the sliding window. For example, if the window size is set to 10, then at any moment, the system will consider the most recent 10 wave height prediction values.

[0045] At the beginning of the process, initialize a window, which can be an array or a list, for storing consecutive output results. As new prediction values are generated, the window slides over the output result sequence. When the 11th prediction value is generated, the system removes the 1st prediction value in the window and then adds the 11th prediction value to the window, so as to always keep 10 consecutive prediction values in the window. At each time point, calculate the average value of all prediction values in the window. This can be achieved by adding up all the values in the window and then dividing by the window size (in this example, 10). The obtained moving average value is smoother, reducing the influence of fluctuations in the immediate prediction and more stably reflecting the average wave height. Finally, take the calculated moving average value as the wave height at the current time point, and output or store it for subsequent analysis or real-time monitoring.

[0046] The process of the sliding window is continuous. Whenever a new prediction value is generated, the window slides and updates, and the moving average value is calculated and updated accordingly. This cycle continues until all output results are processed. In this way, even when the sea conditions change greatly or the prediction values fluctuate due to environmental noise, the moving average process can provide a relatively stable and reliable wave height assessment, which is of great significance for the safety planning and equipment management of offshore operations. In addition, the moving average process can also help the monitoring system filter out some outliers, further improving the robustness of the prediction results.

[0047] To better understand the process of the above method for determining the wave height, the following further describes the above method for determining the wave height in combination with optional embodiments, but it is not used to limit the technical solutions of the embodiments of the present application.

[0048] An alternative embodiment of the present application proposes a sea wave height monitoring scheme based on a convolutional neural network (CNN) combined with frequency domain analysis (equivalent to the method for determining the sea wave height in the above embodiment). The sea wave images are captured in real time by a camera device (equivalent to the image acquisition device in the above embodiment), the wave features are extracted by using frequency domain transformation, and the sea wave height is predicted in combination with CNN. The automatic feature extraction ability of deep learning and the physical interpretability of frequency domain analysis are fully utilized, which is not only superior to traditional methods in terms of accuracy and robustness, but also has the advantages of low cost and large-scale monitoring.

[0049] The alternative embodiment of the present application includes the following steps:

[0050] Step 1: Data acquisition and preprocessing.

[0051] 1) Video and image acquisition.

[0052] Install a high-resolution camera device (i.e., image acquisition device) on an offshore wind turbine tower, an ocean platform or a ship to collect sea wave video data in real time. The original video frame rate of the acquisition is 30fps (30 frames per second) to ensure that the dynamic changes of the sea waves are captured in detail.

[0053] 2) Image preprocessing.

[0054] Denoising processing: Eliminate the noise in the image, such as water droplets or light fluctuations, through Gaussian filtering.

[0055] Frame selection: Uniformly extract key frames from the video to ensure that the samples are representative.

[0056] Normalization: Adjust the brightness and contrast of the image to make all samples consistent during feature extraction.

[0057] Step 2: Frequency domain feature extraction.

[0058] Through the fast Fourier transform (FFT), the image is transformed from the spatial domain to the frequency domain, and the periodic features of the waves are extracted, including:

[0059] Grayscale conversion: Convert each frame of the image into a grayscale image to simplify the calculation.

[0060] Fast Fourier transform: Apply two-dimensional FFT to the grayscale image to generate a spectrogram.

[0061] Frequency component analysis: Extract the energy distribution in the high-frequency region to characterize the texture complexity and periodicity of the sea waves.

[0062] Among them, the two-dimensional fast Fourier transform is expressed as:

[0063]

[0064] Among them, f(x, y) represents the gray value of the image, F(u, v) is the frequency domain representation, and M and N represent the width and height of the image respectively.

[0065] Step 3: CNN model construction.

[0066] 1) Network architecture design.

[0067] Input layer: Input the preprocessed image and its spectrogram to form multi-channel features.

[0068] Convolution layer: Use multiple convolutional kernels to extract ripple details, periodicity, and spatial features.

[0069] Pooling layer: Reduce the feature dimension through the max pooling layer, retain key information, and prevent overfitting.

[0070] Fully connected layer: Map the high-dimensional features to the output of the wave height, and the prediction result is the wave height value of each frame.

[0071] 2) Definition of the loss function (i.e., the preset loss function in the above embodiment).

[0072] The loss function is defined using the mean squared error (MSE) to quantify the gap between the predicted height and the true height:

[0073]

[0074] Among them, yi is the true height, is the predicted height, and N is the number of samples.

[0075] 3) During the training process, the Adam optimizer is used to adjust the network parameters, and the learning rate is set to 0.001. Parallel training is performed on the GPU, and data augmentation techniques (such as random rotation and cropping) are used to expand the sample size and improve the generalization ability of the model.

[0076] Step 4: Multi-view feature fusion.

[0077] 1) Image feature and frequency domain feature fusion.

[0078] After the high-dimensional features in the convolution layer, add the energy distribution information of the spectrogram, and form a joint feature vector through feature concatenation.

[0079] The fusion process adapts to dynamic environmental changes through weight adjustment.

[0080] 2) Time series feature processing.

[0081] Through the sliding window technique, stack the prediction results of consecutive multiple frames to smooth short-term fluctuations and improve the stability evaluation of the wave height.

[0082] Step 5: Real-time prediction and visualization.

[0083] 1) Real-time monitoring.

[0084] The video frames collected per second are input into the CNN model to output the predicted value of the sea wave height.

[0085] Moving average processing is performed on multiple consecutive predicted values to smooth the prediction errors of short-term fluctuations.

[0086] 2) Dynamic visualization.

[0087] The change curve of the sea wave height is displayed in real time on the monitoring platform, and the height level is marked with colors (for example, green is the safe area and red is the warning area).

[0088] 3) Early warning system.

[0089] Set the height threshold. When extreme sea waves are detected, an alarm is triggered to guide the offshore equipment to stop operating or adjust the operation plan.

[0090] Based on the above, the optional embodiments of this application have the following advantages:

[0091] Fusion of frequency-domain physical characteristics: Extract physical features through FFT to make up for the lack of understanding of physical mechanisms by the CNN model, enabling the CNN to combine the frequency-domain information of the image during the prediction process, capture the periodic characteristics of sea waves, and improve the prediction accuracy.

[0092] Possess multi-perspective learning ability: Use convolutional layers to extract multi-scale features and automatically analyze complex sea wave forms such as wave crests and wave troughs.

[0093] Low cost and high coverage: Real-time monitoring over a large range can be achieved through ordinary camera equipment, with low cost, avoiding the high cost of deploying traditional sensors.

[0094] End-to-end learning framework: Directly from the original image input to the sea wave height output, without manually defining features, with a high degree of automation.

[0095] Optionally, the optional embodiments of this application can be applied to, including but not limited to: dynamic real-time monitoring of offshore wind farms, research on nearshore sea wave changes, ocean science research, shipping route optimization, and disaster prevention and early warning, etc. With its high efficiency and low cost, it can be widely deployed in coastal and ocean areas to provide scientific support for ocean environmental management.

[0096] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present application.

[0097] The embodiment of the present application also provides a structural block diagram of a device for determining the sea wave height. Figure 3 It is a structural block diagram of a device for determining the sea wave height according to an embodiment of the present application; as Figure 3 shown, it includes:

[0098] An acquisition module 32, configured to acquire real-time sea wave video data collected by an image acquisition device deployed at a preset position, wherein the original video frame rate of the sea wave video data is a preset value, and the preset position includes at least one of the following: an offshore wind turbine tower, an offshore platform, a ship;

[0099] An input module 34, configured to input the sea wave video data and a first spectrogram corresponding to each frame image in the sea wave video data into a trained first convolutional neural network to obtain an output result;

[0100] A determination module 36, configured to perform a moving average process on a plurality of consecutive output results to determine the sea wave height corresponding to the sea wave video data.

[0101] Through the above device, real-time sea wave video data collected by an image acquisition device deployed at a preset position is acquired, wherein the original video frame rate of the sea wave video data is a preset value, and the preset position includes at least one of the following: an offshore wind turbine tower, an offshore platform, a ship; the sea wave video data and a first spectrogram corresponding to each frame image in the sea wave video data are input into a trained first convolutional neural network to obtain an output result; a moving average process is performed on a plurality of consecutive output results to determine the sea wave height corresponding to the sea wave video data. Through the above embodiments, large-scale real-time monitoring can be achieved through ordinary image acquisition devices, and the automatic feature extraction ability of deep learning and the physical interpretability of frequency domain analysis are utilized, which not only avoids the high cost of traditional sensor layout, but also is superior to traditional methods in terms of accuracy and robustness. It solves the problem in the prior art that traditional monitoring methods for sea wave height, such as buoys and radars, are expensive and have limited coverage.

[0102] In an exemplary embodiment, the device further includes a training module, which, before inputting the ocean wave video data and the first spectrogram corresponding to each frame image in the ocean wave video data into the trained first convolutional neural network to obtain an output result: obtains an untrained second convolutional neural network, where the second convolutional neural network includes: an input layer, a convolutional layer connected to the input layer, a pooling layer connected to the convolutional layer, and a fully connected layer connected to the pooling layer; trains the second convolutional neural network with target training data expanded by adopting a data augmentation technique to obtain the first convolutional neural network.

[0103] In an exemplary embodiment, the training module is further configured to: obtain basic training data, where the basic training data includes: ocean wave images and ocean wave height measurement values corresponding to the ocean wave images, and the ocean wave images have been subjected to preprocessing operations; perform fast Fourier transform on each ocean wave image in the basic training data to obtain a second spectrogram corresponding to each ocean wave image; determine the target training data based on the second spectrogram and the basic training data.

[0104] In an exemplary embodiment, the training module is further configured to: input step: input the ocean wave images in the target training data and the second spectrograms corresponding to the ocean wave images into the second convolutional neural network to obtain predicted values; calculation step: calculate the loss between the predicted values and the ocean wave height measurement values corresponding to the ocean wave images in the target training data through a preset loss function, and perform backpropagation gradient update on the second convolutional neural network through the loss; update step: update the target parameters of the second convolutional neural network according to the corresponding gradients of the backpropagation gradient update through an Adam optimizer, where the target parameters at least include: weights and biases; repeatedly execute the input step, the calculation step, and the update step until the number of loops meets a preset number of iterations to obtain the first convolutional neural network.

[0105] In an exemplary embodiment, the input module is further configured to: process each frame image and the first spectrogram into multi-channel features through the input layer in the first convolutional neural network; extract high-dimensional features corresponding to the multi-channel features through the convolutional layer in the first convolutional neural network, where the high-dimensional features include: ripple detail features, periodic features, and spatial features; splice the high-dimensional features and the energy distribution information corresponding to the first spectrogram according to a splicing weight to obtain a joint feature vector; determine the output result through the joint feature vector.

[0106] In an exemplary embodiment, the determination module is further configured to: determine the number of results of a continuous plurality of the output results; perform a moving average process on the continuous plurality of the output results through a sliding window with the number of results as the window size, to obtain the wave height corresponding to the sea wave video data.

[0107] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the above program, when running, executes the method of any one of the above.

[0108] Optionally, in this embodiment, the above storage medium may be set to store program code for performing the following steps:

[0109] S1, obtain sea wave video data collected in real time by an image acquisition device deployed at a preset position, wherein the original video frame rate of the sea wave video data is a preset value, and the preset position includes at least one of the following: an offshore wind turbine tower, an offshore platform, a ship;

[0110] S2, input the sea wave video data and the first spectrogram corresponding to each frame of the sea wave video data into a trained first convolutional neural network to obtain an output result;

[0111] S3, perform a moving average process on a continuous plurality of the output results to determine the wave height corresponding to the sea wave video data.

[0112] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0113] Optionally, the above electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0114] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0115] S1, obtain sea wave video data collected in real time by an image acquisition device deployed at a preset position, wherein the original video frame rate of the sea wave video data is a preset value, and the preset position includes at least one of the following: an offshore wind turbine tower, an offshore platform, a ship;

[0116] S2, input the sea wave video data and the first spectrogram corresponding to each frame of the sea wave video data into a trained first convolutional neural network to obtain an output result;

[0117] S3. Perform a moving average process on multiple consecutive said output results to determine the wave height corresponding to the wave video data.

[0118] Optionally, in this embodiment, the above storage medium may include, but is not limited to: USB flash drives, read-only memories (ROM), random access memories (RAM), external hard drives, magnetic disks, or optical discs, etc., all kinds of media that can store program codes.

[0119] An embodiment of the present application also provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0120] Another embodiment of the present application also provides a computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0121] An embodiment of the present application also provides a computer program. The computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in any one of the above method embodiments.

[0122] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein.

[0123] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described herein can be executed in a different order, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. In this way, the present application is not limited to any specific combination of hardware and software.

[0124] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various modifications and variations can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application shall be included within the protection scope of the present application.

Claims

1. A method for determining the wave height, characterized in that Including: Obtain the wave video data collected in real time by an image acquisition device deployed at a preset location. The original video frame rate of the wave video data is a preset value. The preset location includes at least one of the following: an offshore wind turbine tower, an ocean platform, a ship; Input the wave video data and the first spectrogram corresponding to each frame image in the wave video data into the trained first convolutional neural network to obtain an output result; Perform a moving average process on a continuous plurality of the output results to determine the wave height corresponding to the wave video data.

2. The method for determining the wave height according to claim 1, wherein Before inputting the wave video data and the first spectrogram corresponding to each frame image in the wave video data into the trained first convolutional neural network to obtain an output result, the method further includes: Obtain an untrained second convolutional neural network, where the second convolutional neural network includes: an input layer, a convolutional layer connected to the input layer, a pooling layer connected to the convolutional layer, and a fully connected layer connected to the pooling layer; Train the second convolutional neural network with the target training data expanded by using data augmentation techniques to obtain the first convolutional neural network.

3. The method for determining the wave height according to claim 2, wherein Before training the second convolutional neural network with the target training data expanded by using data augmentation techniques to obtain the first convolutional neural network, the method further includes: Obtain basic training data, where the basic training data includes: wave images and the measured wave height values corresponding to the wave images, and the wave images have been subjected to preprocessing operations; Perform a fast Fourier transform on each wave image in the basic training data to obtain a second spectrogram corresponding to each wave image; Determine the target training data based on the second spectrogram and the basic training data.

4. The method for determining the wave height according to claim 2, characterized in that, Training the second convolutional neural network with the target training data expanded by using data augmentation techniques to obtain the first convolutional neural network includes: Input step: Input the wave images in the target training data and the second spectrograms corresponding to the wave images into the second convolutional neural network to obtain predicted values; Calculation step: Calculate the loss between the predicted values and the measured wave height values corresponding to the wave images in the target training data through a preset loss function, and perform backpropagation gradient update on the second convolutional neural network through the loss; Update step: Update the target parameters of the second convolutional neural network according to the corresponding gradients of the backpropagation gradient update by an Adam optimizer, where the target parameters at least include: weights and biases; Loop and execute the input step, the calculation step, and the update step until the number of loops meets a preset number of iterations to obtain the first convolutional neural network.

5. The method for determining the wave height according to claim 1, wherein Inputting the wave video data and the first spectrogram corresponding to each frame image in the wave video data into the trained first convolutional neural network to obtain an output result includes: Process each frame image and the first spectrogram into multi-channel features through the input layer in the first convolutional neural network; Extract high-dimensional features corresponding to the multi-channel features through the convolutional layer in the first convolutional neural network, where the high-dimensional features include: ripple detail features, periodic features, and spatial features; Concatenate the high-dimensional features and the energy distribution information corresponding to the first spectrogram according to the concatenation weight to obtain a joint feature vector; Determine the output result through the joint feature vector.

6. The method for determining the wave height according to claim 1, characterized in that Perform a moving average process on multiple consecutive output results to determine the wave height corresponding to the wave video data, including: Determine the number of results of multiple consecutive output results; Perform a moving average process on multiple consecutive output results through a sliding window with a window size equal to the number of results to obtain the wave height corresponding to the wave video data.

7. A device for determining the wave height, characterized in that Include: An acquisition module for acquiring wave video data collected in real time by an image acquisition device deployed at a preset location, where the original video frame rate of the wave video data is a preset value, and the preset location includes at least one of the following: an offshore wind turbine tower, an offshore platform, a ship; An input module for inputting the wave video data and the first spectrogram corresponding to each frame image in the wave video data into a trained first convolutional neural network to obtain an output result; A determination module for performing a moving average process on multiple consecutive output results to determine the wave height corresponding to the wave video data.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, where the program, when running, executes the method described in any one of claims 1 to 6 above.

9. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to execute the method described in any one of claims 1 to 6 through the computer program.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method described in any one of claims 1 to 6 are implemented.

Images