Method, system, medium and device for monitoring sea wave height of offshore boosting platform

Abstract

The invention provides a sea wave height monitoring method, system, medium and device for an offshore boosting platform. The method comprises the following steps: acquiring an original image of an offshore target area; constructing a sea wave image data set based on the original image, and marking the sea wave image data set; obtaining a sea wave feature extraction model based on the sea wave image data set; training the sea wave feature extraction model to obtain an optimized sea wave feature extraction model; and calculating and storing the sea wave height based on the optimized sea wave feature extraction model. The sea wave height can be efficiently and accurately monitored in real time, the workload of workers is reduced, and the monitoring data provides a basis for follow-up analysis of the sea wave motion law.

Description

Technical Field

[0001] The present invention belongs to the field of marine engineering technology and relates to a method for detecting sea wave height, and in particular to a method, system, medium and device for monitoring sea wave height of an offshore booster platform. Background Art

[0002] Due to the rich offshore wind energy resources, the scale and power generation of offshore wind farms are much larger than those of onshore wind power, and they do not occupy land resources. At the same time, wind power generation does not consume any fossil fuels, and the scale benefits and environmental benefits are very significant, which has led to the construction of more and more offshore wind power platforms. At the same time, wind power platforms are set up at sea. In order to ensure the safety of the platforms and meet some research needs, it is necessary to monitor the wave height at the location of the wind power platforms.

[0003] Since the waves are in motion, they cannot be collected and recorded manually. Wave images are collected through video, and then deep learning is used to detect moving targets from background images to achieve wave detection. However, due to the interference of background color and a large number of surrounding waves, traditional target detection methods have weak targeting for wave detection and a large number of redundant windows, making it difficult to extract the underlying features of the image. Summary of the invention

[0004] In view of the shortcomings of the prior art described above, the purpose of the present application is to provide a method, system, medium and device for monitoring the wave height of an offshore booster platform, which are used to solve the problems of the prior art in the process of implementing the wave height monitoring technology, that is, the inability to perform manual collection and recording, and the wave images collected through video are interfered by the background and a large amount of surrounding waves, resulting in low accuracy and low efficiency of wave height detection.

[0005] To achieve the above-mentioned purpose and other related purposes, in a first aspect, the present application provides a method for monitoring wave height of an offshore booster platform, comprising the following steps: obtaining an original image of a target area at sea; constructing a wave image dataset based on the original image, and annotating the wave image dataset; obtaining a wave feature extraction model based on the wave image dataset; training the wave feature extraction model to obtain an optimized wave feature extraction model; and calculating and saving the wave height based on the optimized wave feature extraction model.

[0006] In an implementation of the first aspect, obtaining an original image of an offshore target area includes the following steps: obtaining video data with a scale column of an offshore platform through a monitoring probe; and intercepting an original image of the video data at a preset time.

[0007] In an implementation of the first aspect, constructing an ocean wave image dataset based on the original image, and annotating the ocean wave image dataset comprises the following steps: randomly selecting a number of samples from the original image as an ocean wave image dataset; wherein the ocean wave image dataset comprises a training dataset and a test dataset; manually annotating ocean wave rectangular frames as a training dataset; and using unselected ocean wave image rectangular frames as a test dataset.

[0008] In an implementation of the first aspect, obtaining a wave feature extraction model based on the wave image dataset includes the following steps: constructing a deep neural network model, and adding an attention module to the residual module in the deep neural network model to perform image processing on the wave image dataset; wherein the attention module includes: a channel attention module and a spatial attention module; aggregating the wave image dataset based on the residual module to obtain a first feature vector of the wave image; and splicing the first feature vector based on the residual module to obtain a second feature vector to construct a wave feature extraction model.

[0009] In an implementation of the first aspect, aggregating the ocean wave image dataset based on the residual module to obtain a first feature vector of the ocean wave image includes the following steps: performing spatial pooling based on samples in the ocean wave image dataset to obtain two first channel vectors; adding the two first channel vectors based on a convolutional neural network to obtain a first weight coefficient; and obtaining a first feature vector of the ocean wave image based on the first weight coefficient.

[0010] In an implementation of the first aspect, the first feature vectors are spliced ​​based on the residual module to obtain a second feature vector to construct a wave feature extraction model, including the following steps: pooling the channel dimension based on the first feature vector to obtain two second channel vectors; splicing the two second channel vectors to obtain a second weight coefficient; and obtaining a second feature vector of the wave image based on the second weight coefficient to construct a wave feature extraction model.

[0011] In an implementation of the first aspect, based on the optimized wave feature extraction model, calculating and saving the wave height includes the following steps: cropping a newly acquired wave image and marking its wave rectangular frame; calculating a first pixel value of the cropped wave image based on the wave rectangular frame; calculating the original pixel value of the wave height before the wave image is cropped; and calculating a true wave height value based on the first pixel value and the original pixel value.

[0012] In an implementation of the first aspect, the calculation formula for the true height value of the wave is:

[0013] height=(h-mark) / scale

[0014] The first pixel value calculation formula is:

[0015] y=y1+(y2-y1)*0.5

[0016] The original pixel value calculation formula is:

[0017] h=y+supple

[0018] Among them, y1 and y2 represent the vertical axis coordinate values ​​of the pixel coordinates of the upper left vertex and the lower right vertex of the rectangular box respectively; supplement represents the pixel value clipped on the vertical axis; mark represents the pixel value of the wave at the 0-meter scale line; scale represents the pixel value per meter.

[0019] In a second aspect, the present application provides an offshore boost platform wave height monitoring system, comprising: an acquisition module for acquiring original images of an offshore target area; a data processing module for constructing a wave image data set based on the original images; a feature extraction module for acquiring a wave feature extraction model based on the wave image data set; a model training module for training the wave feature extraction model to obtain an optimized wave feature extraction model; and a calculation module for calculating and saving the wave height based on the optimized wave feature extraction model.

[0020] In the last aspect, the present application provides an offshore booster platform wave height monitoring device, comprising: a processor and a memory. The memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the offshore booster platform wave height monitoring device executes the offshore booster platform wave height monitoring method.

[0021] As described above, the method, system, medium and device for monitoring the wave height of an offshore booster platform of the present invention have the following beneficial effects:

[0022] (1) The present application provides a method for monitoring the wave height of an offshore booster platform, which uses an incremental learning algorithm based on deep learning to extract a small number of images as a training data set for annotation, and then uses an incremental method to automatically add wave images to the training data set to achieve the purpose of expanding the data set, thereby improving the accuracy, generalization and training efficiency of the wave extraction model.

[0023] (2) This application increases the attention of the wave extraction model to the core wave target by adding a channel attention mechanism and a spatial attention mechanism, effectively suppressing the interference of surrounding background information, so that it can more accurately identify the position of the waves, generate a rectangular frame, and calculate the wave height; and the wave extraction model has a strong processing capability, so that it can better handle the occlusion and background interference in the wave image, accurately extract the wave crest position, and thus the wave height has a high accuracy rate; the generalization ability of the wave extraction model is improved, making it suitable for different wave scenes.

[0024] (3) The system of the present application can efficiently and accurately monitor the wave height in real time, reducing the workload of staff, and the monitoring data provides a basis for subsequent analysis of the laws of wave movement. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic flow chart of a method for monitoring sea wave height on an offshore booster platform according to an embodiment of the present invention.

[0026] Figure 2 It is a schematic diagram of the flow of S11 in the method for monitoring the sea wave height of an offshore booster platform of the present invention.

[0027] Figure 3 It is a schematic diagram of the flow of S13 in the method for monitoring the sea wave height of an offshore boost platform of the present invention.

[0028] Figure 4 It shows a schematic diagram of the residual block structure of the series channel attention module and the spatial attention module in the offshore boost platform wave height monitoring method of the present invention.

[0029] Figure 5 Shown is a schematic diagram of the structure of the channel attention module in the offshore boost platform wave height monitoring method of the present invention.

[0030] Figure 6 Shown is a schematic diagram of the structure of the spatial attention module in the offshore boost platform wave height monitoring method of the present invention.

[0031] Figure 7 It is a schematic diagram showing the principle structure of an offshore booster platform wave height monitoring system in one embodiment of the present invention.

[0032] Figure 8 It is a schematic diagram showing the principle structure of an offshore boost platform wave height monitoring device in one embodiment of the present invention.

[0033] Component number description

[0034] 71 Acquisition Module

[0035] 72 Data processing module

[0036] 73 Feature extraction module

[0037] 74 Model training module

[0038] 75 Computing Module

[0039] 81 Processor

[0040] 82 Memory

[0041] Steps S11 to S15 DETAILED DESCRIPTION

[0042] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0043] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0044] The method for monitoring sea wave height on an offshore booster platform provided in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present application.

[0045] See also Figure 1 , which is a schematic diagram of a flow chart of a method for monitoring sea wave height on an offshore booster platform according to an embodiment of the present invention. Figure 1 As shown, this embodiment provides a method for monitoring wave height of an offshore booster platform.

[0046] The method for monitoring the wave height of an offshore booster platform specifically comprises the following steps:

[0047] S11, obtain the original image of the target area at sea. Figure 2 , which is a schematic diagram of the flow of S11 in the method for monitoring the sea wave height of an offshore booster platform of the present invention. Figure 2 As shown, the S11 comprises the following steps:

[0048] S111, obtaining video data with a scale column of the offshore platform through a monitoring probe.

[0049] In this embodiment, a camera is installed in the target area, and a network camera is used to capture video data of the offshore platform with a scale column.

[0050] S112, capturing an original image of the video data according to a preset time.

[0051] In this embodiment, the wave video frames are captured and the images are cropped at fixed time intervals to complete the collection of the wave images at the scale column.

[0052] Specifically, it is preferred to intercept or crop the wave video frames at a time interval of 1 second to obtain a cropped image.

[0053] S12, constructing an ocean wave image dataset based on the original image, and annotating the ocean wave image dataset.

[0054] In this embodiment, several samples are randomly selected from the original image as a wave image dataset; wherein the wave image dataset includes a training dataset and a test dataset; the wave rectangular frames are manually annotated as the training dataset; and the unselected wave image rectangular frames are used as the test dataset. The wave rectangular frames are standardized.

[0055] Specifically, a small number of images are randomly selected from the original images as the original wave image dataset, and then some of the wave images are randomly selected from the original wave image dataset, and the wave rectangular boxes are manually marked as the training dataset, and the remaining images are used as the test dataset. The data ratio of the training dataset to the test dataset is 9:1. For example, the intercepted original image dataset has 1,000 images, and 900 image samples are randomly selected as the training dataset; the remaining 100 image samples are used as the test dataset. The training dataset is used to train the model, and the test dataset is used to evaluate the performance and computing power of the model.

[0056] S13, obtaining a wave feature extraction model based on the wave image dataset. Figure 3 , which is a schematic diagram of the flow of S13 in the method for monitoring the sea wave height of an offshore booster platform of the present invention. Figure 3 As shown, the S13 comprises the following steps:

[0057] S131, construct a deep neural network model, and add an attention module to the residual module in the deep neural network model, and perform image processing on the wave image dataset. The attention module includes: a channel attention module and a spatial attention module. Figure 4 , which is a schematic diagram of the residual block structure of the series channel attention module and the spatial attention module in the offshore boost platform wave height monitoring method of the present invention.

[0058] In this embodiment, the samples of the ocean wave image dataset are subjected to standardized preprocessing, including format conversion, pixel adjustment and grayscale processing. For example, the rectangular frame of the samples of the ocean wave image dataset is converted into an image of a uniform format, the image pixels may be adjusted to uniform pixels, and then the rectangular frame image may be converted into a grayscale image using a grayscale algorithm.

[0059] Specifically, the format conversion of the rectangular frames of the samples of the ocean wave image dataset is because the formats of the sample images obtained by different devices may be quite different. Therefore, it is necessary to convert the sample images of different formats into PNG format.

[0060] The format conversion of the wave sample images is mainly due to the large number of image databases. Therefore, the relevant tools in the PIL image processing library in Python can be used for batch processing, adjusting the image format to PNG and the image pixels to 800*600.

[0061] Then, the grayscale calculation formula is used to grayscale the wave sample image. The grayscale calculation formula is:

[0062]

[0063] Among them, Gray represents the grayscale value of the sample grayscale image obtained after grayscale processing; R, G, and B respectively represent the color values ​​of the three primary colors in the PNG format of the sample image.

[0064] Finally, based on the obtained sample grayscale image, the ocean wave rectangular frame sample image database is obtained.

[0065] Furthermore, in this embodiment, the wave model adopts a deep neural network model. An attention module is added to the residual module in the deep neural network model. The wave rectangular frame sample image database obtained after the wave image rectangular frame processing is input into the deep neural network model as a training data set for training.

[0066] Specifically, the rectangular frame of the wave image is processed by a neural network to obtain rectangular frame feature matrices of different scales. In order to reduce the performance loss problem caused by the transmission and decoding of multiple rectangular frame features, an attention module is used to perform feature fusion on the rectangular frame feature matrices of the wave image of different scales. The attention module is used to introduce attention into the neural network model, which enables the model to grasp the key points to improve the model's understanding ability. The CBAM module is preferred in this embodiment. The CBAM module is a lightweight attention module that combines the channel and space attention mechanism modules.

[0067] In existing deep learning-based object detection methods, most feature extraction modules are prone to missed detection and false detection of targets when the background is complex and the image features are not significant. At the same time, for the detection of wave targets in wave images, due to the interference of the background wave color and the surrounding wave features, the core wave target in the image is not obvious, which easily leads to the incorrect labeling of the wave rectangle. Therefore, a channel attention module and a spatial attention module are added in series in each residual block of the network model.

[0068] Specifically, the deep neural network model includes: a convolution layer, a batch normalization layer, an activation layer, a channel attention module, and a spatial attention module. A series of channel attention modules and spatial attention modules are added to each residual block of the neural network model. The series of channel attention modules and spatial attention modules are set before the downsampling layer with a convolution kernel of 3×3 in the residual module to aggregate the local information of the waves in the wave image.

[0069] The rectangular frame of the wave image is input into the deep neural network, passes through the sampling layer with a convolution kernel of 1×1, and is normalized by the BN layer. It is then transmitted to the convolution layer with a convolution kernel of 3×3, and then normalized. The local information of the waves in the wave image is aggregated through the channel attention module and the spatial attention module respectively. Finally, after the downsampling layer with a convolution kernel of 3×3, the extracted features are obtained.

[0070] It should be noted that the deep neural network model can be processed using a convolutional neural network (CNN), or using a recurrent neural network (RNN), a deep belief network (DBN), a deep autoencoder (AutoEncoder), and a generative adversarial network (GAN).

[0071] S132, aggregating the ocean wave image dataset based on the residual module to obtain a first feature vector of the ocean wave image. Figure 5 , which is a schematic diagram of the structure of the channel attention module in the offshore boost platform wave height monitoring method of the present invention.

[0072] In this embodiment, spatial pooling is performed based on samples in the ocean wave image data set to obtain two first channel vectors; the two first channel vectors are added based on a convolutional neural network to obtain a first weight coefficient; and a first feature vector of the ocean wave image is obtained based on the first weight coefficient.

[0073] Specifically, the wave image is processed through the channel attention module, and the maximum pooling layer and the average pooling layer are fused.

[0074] The implementation steps are as follows:

[0075] (1) Performing global maximum pooling and average pooling on the input H×W×C wave feature F, thereby obtaining two 1×1×C first channel vectors;

[0076] (2) The two first channel vectors are respectively input into a two-layer convolutional neural network, and the two first channel vectors output by the convolutional neural network are added;

[0077] (3) The added first channel vector is subjected to the first Sigmoid activation function to obtain the weight coefficient Mc;

[0078] (4) Mc multiplied by the input feature F can obtain the new feature G after scaling H×W×C.

[0079] The calculation formula for the channel attention involved above is:

[0080]

[0081] Among them, st represents the exchange of the positions of channels and depths to perform a one-dimensional convolution operation. After the convolution operation, a restoration operation is required to re-exchange the positions of channels and depths; Mc represents the weight coefficient.

[0082] Channel attention uses global average pooling when extracting features in the channel direction. It retains the global distribution of wave features while suppressing useless information in the channel direction, which is beneficial to the detection of core wave targets.

[0083] S133, concatenating the first feature vectors based on the residual module to obtain a second feature vector to construct a wave feature extraction model. Figure 6 , which is a schematic diagram of the structure of the spatial attention module in the offshore boost platform wave height monitoring method of the present invention.

[0084] In this embodiment, channel dimension pooling is performed based on the first feature vector to obtain two second channel vectors; the two second channel vectors are concatenated to obtain a second weight coefficient; and the second feature vector of the wave image is obtained based on the second weight coefficient to construct a wave feature extraction model.

[0085] Specifically, the spatial attention module processes the most critical areas in the image. The implementation steps are as follows:

[0086] (1) Perform maximum pooling and average pooling of one channel dimension on the H×W×C first feature vector G output by the channel attention module, thereby obtaining two H×W×1 second channel vectors;

[0087] (2) Concatenate the two H×W×1 second channel vectors;

[0088] (3) After concatenation, the convolutional layer is passed through 7×7, and the second Sigmoid activation function is used to obtain the second weight coefficient Ms;

[0089] (4) The second weight coefficient Ms is multiplied by the input first eigenvector G to obtain the scaled second eigenvector, thereby constructing a wave feature extraction model.

[0090] The spatial attention calculation formula involved in the above steps is:

[0091] M s (f) = σ(f 7×7 ([AvgPool(F); MaxPool(f)]))

[0092] Among them, σ represents the Sigmoid activation function; f 7×7 Represented as a convolution operation with a convolution kernel of 7.

[0093] This spatial attention mechanism retains the key information of the spatial position of the wave features, suppresses the useless background and surrounding wave feature information, and can effectively reduce the background noise introduced in the fusion and reduce the aliasing effect caused by the fusion.

[0094] By connecting the channel attention module and the spatial attention module in series, we can obtain features on different channels and aggregate important local information of the wave image, so that the neural network model can pay more attention to the feature fusion between channels and achieve better wave detection effect.

[0095] In this embodiment, the series-connected channel attention module and spatial attention module are arranged before the downsampling layer with a convolution kernel of 3×3 in the residual module of the DarkNet network to aggregate the local information of the waves in the wave image and better realize the detection of the wave rectangular frame.

[0096] S14, training the ocean wave feature extraction model to obtain an optimized ocean wave feature extraction model.

[0097] In this embodiment, the remaining 100 test data set samples are input into the constructed wave feature extraction model for evaluation, thereby obtaining an optimized wave feature extraction model.

[0098] S15, calculating and saving the wave height based on the optimized wave feature extraction model.

[0099] In this embodiment, the newly collected wave image is cropped and its wave rectangular frame is marked; the first pixel value of the cropped wave image is calculated based on the wave rectangular frame; the original pixel value of the wave height before the wave image is cropped is calculated; and the real wave height value is calculated based on the first pixel value and the original pixel value.

[0100] Specifically, first mark the rectangular frame of the ocean wave, set the vertical axis pixel value of the top and bottom of the image to 0, the pixel coordinates of the upper left vertex and the lower right vertex of the rectangular frame to (x1, y1) and (x2, y2) respectively, supplement is the pixel value clipped on the vertical axis, mark is the pixel value at the 0-meter scale line of the ocean wave, and scale represents the pixel value per meter.

[0101] Then calculate the first pixel value y of the current wave height in the cropped image, and the calculation formula is:

[0102] y=y1+(y2-y1)*0.5

[0103] Then calculate the original pixel value h of the original wave height before cropping the image, and the calculation formula is:

[0104] h=y+supple

[0105] Finally, the true height of the wave is calculated using the following formula:

[0106] height=(h-mark) / scale

[0107] Among them, y1 and y2 represent the vertical axis coordinate values ​​of the pixel coordinates of the upper left vertex and the lower right vertex of the rectangular box respectively; supplement represents the pixel value clipped on the vertical axis; mark represents the pixel value of the wave at the 0-meter scale line; scale represents the pixel value per meter.

[0108] It should be noted that the staff developed the image preprocessing model based on Python's PIL image processing library and built the feature extraction model based on the Pytorch library.

[0109] The wave height monitoring method for an offshore booster platform provided in the present application uses video data of an offshore platform with a scale column taken by a network camera, intercepts wave video frames at fixed intervals and crops pictures, and completes the collection of wave pictures at the scale column. Then, a small number of pictures are randomly selected as the original wave picture data set, and then some of the wave pictures are randomly selected from the original wave picture data set, and the wave rectangular frames are manually annotated as training sets, and the remaining pictures are used as test sets. The data ratio of the training set to the test set is 9:1. The built wave feature extraction model is trained with the training set, and the parameters of the wave extraction model are adjusted through repeated training, including: batch size, convolution kernel of the convolution layer, sliding window size, learning rate, and loss function, etc., and then the trained wave extraction model is saved. In view of the different wave positions in different time periods, the wave extraction model is incrementally trained, and the test set is automatically annotated with wave rectangular frames using the trained wave extraction model. Then, the annotated pictures are added to the original training set, and the wave extraction model is trained again to obtain the weight of the final model. This can significantly reduce the workload of annotating pictures, improve the generalization of the model, and make the wave extraction model more accurate in annotating the wave rectangle. At the same time, the trained wave extraction model can be used for wave height monitoring.

[0110] The protection scope of the method for monitoring wave height on an offshore booster platform described in the embodiment of the present application is not limited to the execution order of the steps listed in the present embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.

[0111] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the following is achieved: Figure 1 The method for monitoring the wave height of an offshore booster platform.

[0112] At any possible level of technical detail combination, the present application may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present application.

[0113] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0114] The computer-readable program described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives a computer-readable program instruction from the network, and forwards the computer-readable program instruction for storage in the computer-readable storage medium in each computing / processing device. The computer program instruction for performing the operation of the present application can be an assembly instruction, an instruction set architecture (ISA) instruction, a machine instruction, a machine-related instruction, a microcode, a firmware instruction, a state setting data, an integrated circuit configuration data, or a source code or object code written in any combination of one or more programming languages, wherein the programming language includes an object-oriented programming language, such as Smalltalk, C++, etc., and a procedural programming language, such as "C" language or similar programming language. Computer readable program instructions can be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In certain embodiments, by using the state information of computer readable program instructions to personalize electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs), the electronic circuits can execute computer readable program instructions, thereby realizing various aspects of the present application.

[0115] The embodiment of the present application also provides an offshore booster platform wave height monitoring system, which can implement the offshore booster platform wave height monitoring method described in the present application. However, the implementation device of the offshore booster platform wave height monitoring method described in the present application includes but is not limited to the structure of the offshore booster platform wave height monitoring system listed in the present embodiment. All structural deformations and replacements of the prior art made according to the principles of the present application are included in the protection scope of the present application.

[0116] The offshore booster platform wave height monitoring system provided by this embodiment will be described in detail below with reference to the drawings.

[0117] This embodiment provides a wave height monitoring system for an offshore booster platform, comprising:

[0118] See also Figure 7 , which is a schematic diagram showing the principle structure of an offshore booster platform wave height monitoring system in one embodiment of the present invention. Figure 7 As shown, the offshore boost platform wave height monitoring system includes: a collection module 71, a data processing module 72, a feature extraction module 73, a model training model 74 and a calculation module 75.

[0119] The acquisition module 71 is used to acquire original images of the target area at sea.

[0120] The video data with a scale column of the offshore platform is acquired through a monitoring probe, and the original image of the video data is intercepted according to a preset time.

[0121] In this embodiment, a camera is installed in the target area, and a network camera is used to shoot video data of the offshore platform with a scale column. The wave video frames are intercepted and the pictures are cropped at fixed time intervals to complete the collection of the wave pictures at the scale column.

[0122] The data processing module 72 is used to construct an ocean wave image dataset based on the original image.

[0123] In this embodiment, several samples are randomly selected from the original image as a wave image dataset; wherein the wave image dataset includes a training dataset and a test dataset; the wave rectangular frames are manually annotated as the training dataset; and the unselected wave image rectangular frames are used as the test dataset. The wave rectangular frames are standardized.

[0124] Specifically, a small number of images are randomly selected from the original images as the original wave image dataset, and then some wave images are randomly selected from the original wave image dataset, and the wave rectangular boxes are manually marked as the training dataset, and the remaining images are used as the test dataset. The ratio of the training dataset to the test dataset is 9:1. The training dataset is used to train the model, and the test dataset is used to evaluate the performance and computing power of the model.

[0125] The feature extraction module 73 is used to obtain a wave feature extraction model based on the wave image dataset.

[0126] A deep neural network model is constructed, and an attention module is added to the residual module in the deep neural network model, and image processing is performed on the wave image dataset at the same time. The attention module includes: a channel attention module and a spatial attention module.

[0127] In this embodiment, the samples of the ocean wave image data set are subjected to standardized preprocessing, including format conversion, pixel adjustment and grayscale processing. The ocean wave model adopts a deep neural network model. An attention module is added to the residual module in the deep neural network model. The ocean wave rectangular frame sample image database obtained after the ocean wave image rectangular frame processing is input into the deep neural network model as a training data set for training.

[0128] Specifically, the deep neural network model includes: a convolution layer, a batch normalization layer, an activation layer, a channel attention module, and a spatial attention module. A series of channel attention modules and spatial attention modules are added to each residual block of the neural network model. The series of channel attention modules and spatial attention modules are set before the downsampling layer with a convolution kernel of 3×3 in the residual module to aggregate the local information of the waves in the wave image.

[0129] The ocean wave image dataset is aggregated based on the residual module to obtain a first feature vector of the ocean wave image.

[0130] In this embodiment, spatial pooling is performed based on samples in the ocean wave image data set to obtain two first channel vectors; the two first channel vectors are added based on a convolutional neural network to obtain a first weight coefficient; and a first feature vector of the ocean wave image is obtained based on the first weight coefficient.

[0131] Specifically, the wave image is processed through the channel attention module, and the maximum pooling layer and the average pooling layer are fused. The implementation steps are: the input H×W×C wave feature F is subjected to spatial global maximum pooling and average pooling, thereby obtaining two 1×1×C first channel vectors; the two first channel vectors are respectively input into a two-layer convolutional neural network, and the two first channel vectors output by the convolutional neural network are added; the added first channel vector is subjected to the first Sigmoid activation function to obtain the weight coefficient Mc; Mc is multiplied by the input feature F to obtain the scaled H×W×C new feature G.

[0132] Channel attention uses global average pooling when extracting features in the channel direction. It retains the global distribution of wave features while suppressing useless information in the channel direction, which is beneficial to the detection of core wave targets.

[0133] The first feature vectors are concatenated based on the residual module to obtain a second feature vector to construct a wave feature extraction model.

[0134] In this embodiment, channel dimension pooling is performed based on the first feature vector to obtain two second channel vectors; the two second channel vectors are concatenated to obtain a second weight coefficient; and the second feature vector of the wave image is obtained based on the second weight coefficient to construct a wave feature extraction model.

[0135] Specifically, the spatial attention module processes the most critical area in the image, and the implementation steps are as follows: the H×W×C first eigenvector G output by the channel attention module is subjected to maximum pooling and average pooling of one channel dimension respectively, thereby obtaining two H×W×1 second channel vectors; the two H×W×1 second channel vectors are concatenated; after concatenation, they are passed through a 7×7 convolution layer, and the second Sigmoid activation function is used to obtain the second weight coefficient Ms; the second weight coefficient Ms is multiplied by the input first eigenvector G to obtain the scaled second eigenvector, thereby constructing a wave feature extraction model.

[0136] This spatial attention mechanism retains the key information of the spatial position of the wave features, suppresses the useless background and surrounding wave feature information, and can effectively reduce the background noise introduced in the fusion and reduce the aliasing effect caused by the fusion.

[0137] In summary, by connecting the channel attention module and the spatial attention module in series, we can obtain the features on different channels and aggregate the important local information of the wave image, so that the neural network model can pay more attention to the feature fusion between channels and achieve better wave detection effect.

[0138] The model training module 74 is used to train the ocean wave feature extraction model to obtain an optimized ocean wave feature extraction model.

[0139] The calculation module 75 is used to calculate and save the wave height based on the optimized wave feature extraction model.

[0140] In this embodiment, the newly collected wave image is cropped and its wave rectangular frame is marked; the first pixel value of the cropped wave image is calculated based on the wave rectangular frame; the original pixel value of the wave height before the wave image is cropped is calculated; and the real wave height value is calculated based on the first pixel value and the original pixel value.

[0141] The following is an example of an implementation scenario.

[0142] Collect data frame by frame from the wave monitoring video, build a wave data set, design and train a wave feature extraction model, identify the position of the wave crest, calculate the current wave height information according to the scale, and use it to monitor and record the wave height of the offshore wind power platform in real time, provide a basis for subsequent analysis of the law of wave movement, and reduce the workload of monitoring personnel. The system also includes: several wind power monitoring platforms and a system management console. The wind power monitoring platform is a monitoring platform set up at sea, and the system management console connects several wind power monitoring platforms to achieve comprehensive management. The system management console includes: a data processing module, a model training module, and a wind power platform management module. Among them, the wind power platform management module is used to configure and manage the parameters, basic information, and network camera information of each wind power monitoring platform. The data processing module is used to manually annotate the wave rectangular frame of the wave picture to generate a training set. The model training module includes a wave feature extraction model, which mainly trains, tests, and debugs the wave feature extraction model, and sends the trained wave feature extraction model to each wind power monitoring platform. The wave feature extraction model is mainly used to annotate the wave images with wave rectangles and calculate the wave height.

[0143] The wind power monitoring platform includes: an acquisition module and a calculation module. The video data acquisition module is connected to a network camera for video data acquisition. The wave height calculation module is used to calculate the real-time wave height. The wave information storage module is used to store wind power platform information and wave height data.

[0144] Specifically, the system management station communicates data with the wind power monitoring platform, so that the video data acquisition module of the wind power monitoring platform captures the wave video frames at uniform intervals, and collects a total of 1,000 pictures and transmits them back to the system management station. The data processing module selects 200 pictures to manually mark the wave rectangle frame to obtain a training set, and the 200 pictures in the training set are passed to the model training module to train the wave extraction model. The trained wave extraction model is used to automatically mark the remaining 800 pictures, and the marked pictures are used as new training sets to further train the wave extraction model, thereby improving the accuracy of the model and the accuracy of wave detection. The trained model is transmitted to the corresponding wind power monitoring platform again, and the wind power monitoring platform uses the wave extraction model to process subsequent real-time images, mark the wave rectangle frame and calculate the wave height, and store the height data, and send it to the system management station after packaging, thereby realizing automatic monitoring of wave height, with high accuracy and good effect, and it is also suitable for wind power management platforms in different locations.

[0145] The offshore booster platform wave height monitoring model builds an offshore booster platform wave height monitoring system, which can monitor the wave height in real time efficiently and accurately, reducing the workload of staff. The monitoring data provides a basis for subsequent analysis of the laws of wave movement.

[0146] It should be noted that it should be understood that the division of the various modules of the above system is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the x module can be a separately established processing element, or it can be integrated in a certain chip of the above system for implementation. In addition, it can also be stored in the memory of the above system in the form of program code, and called and executed by a certain processing element of the above system. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0147] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0148] See also Figure 8 , which is a schematic diagram showing the principle structure of an offshore booster platform wave height monitoring device in one embodiment of the present invention. Figure 8 As shown, this embodiment provides an offshore boost platform wave height monitoring device, the offshore boost platform wave height monitoring device comprising: a processor 81, a memory 82; the memory 82 is used to store computer programs; the processor 81 is connected to the memory 82, and is used to execute the computer program stored in the memory 82, so that the offshore boost platform wave height monitoring device performs the various steps of the offshore boost platform wave height monitoring method as described above.

[0149] Preferably, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0150] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0151] In summary, the method, system, medium and device for monitoring the wave height of an offshore booster platform provided by the present application have the following beneficial effects:

[0152] The method for monitoring the wave height of an offshore booster platform provided in the present application adopts an incremental learning algorithm based on deep learning, extracts a small number of pictures as training data sets for annotation, and then uses an incremental method to automatically add wave pictures to the training data set to achieve the purpose of expanding the data set, thereby improving the accuracy, generalization and training efficiency of the wave extraction model; in addition, by adding a channel attention mechanism and a spatial attention mechanism, the wave extraction model's attention to the core wave target is increased, effectively suppressing the interference of the surrounding background information, so that it can more accurately identify the position of the waves and calculate the wave height; and the wave extraction model has a strong processing capability, so that it can better handle the occlusion and background interference in the wave image, extract the wave crest position accurately, and thus the wave height has a high accuracy rate; the generalization ability of the wave extraction model is improved, making it suitable for different wave scenes. The offshore booster platform wave height monitoring model has a simple structure, and has the characteristics of fast detection speed, high detection accuracy, and flexible deployment on the mobile terminal. It can efficiently and accurately monitor the wave height in real time, reduce the workload of the staff, and the monitoring data provides a basis for the subsequent analysis of the law of wave movement.

[0153] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for monitoring sea wave height on an offshore booster platform, characterized in that: The following steps are involved: Acquire original images of target areas at sea; constructing an ocean wave image dataset based on the original image, and annotating the ocean wave image dataset; Acquire an ocean wave feature extraction model based on the ocean wave image dataset; Training the ocean wave feature extraction model to obtain an optimized ocean wave feature extraction model; Based on the optimized wave feature extraction model, the wave height is calculated and saved.

2. The method for monitoring sea wave height of an offshore booster platform according to claim 1, characterized in that: Acquiring the original image of the target area at sea includes the following steps: Acquire video data with scale columns of offshore platforms through monitoring probes; The original image of the video data is captured according to a preset time.

3. The method for monitoring sea wave height of an offshore booster platform according to claim 1, characterized in that: Constructing an ocean wave image dataset based on the original image and labeling the ocean wave image dataset comprises the following steps: Randomly select a number of samples from the original image as an ocean wave image dataset; wherein the ocean wave image dataset includes a training dataset and a test dataset; Manually annotate the wave rectangles as the training data set; use the unselected wave image rectangles as the test data set.

4. The method for monitoring sea wave height of an offshore booster platform according to claim 1, characterized in that: Acquiring an ocean wave feature extraction model based on the ocean wave image dataset comprises the following steps: Constructing a deep neural network model, and adding an attention module to the residual module in the deep neural network model to perform image processing on the wave image dataset; wherein the attention module includes: a channel attention module and a spatial attention module; Aggregating the ocean wave image dataset based on the residual module to obtain a first eigenvector of the ocean wave image; The first feature vectors are concatenated based on the residual module to obtain a second feature vector to construct a wave feature extraction model.

5. The method for monitoring sea wave height of an offshore booster platform according to claim 4, characterized in that: Aggregating the ocean wave image dataset based on the residual module to obtain a first feature vector of the ocean wave image comprises the following steps: Performing spatial pooling based on samples in the ocean wave image dataset to obtain two first channel vectors; Adding two first channel vectors based on a convolutional neural network to obtain a first weight coefficient; A first feature vector of the ocean wave image is obtained based on the first weight coefficient.

6. The method for monitoring sea wave height of an offshore booster platform according to claim 4, characterized in that: The first feature vectors are concatenated based on the residual module to obtain a second feature vector to construct a wave feature extraction model, including the following steps: Performing channel dimension pooling based on the first feature vector to obtain two second channel vectors; Concatenate the two second channel vectors to obtain a second weight coefficient; A second feature vector of the ocean wave image is obtained based on the second weight coefficient to construct an ocean wave feature extraction model.

7. The method for monitoring sea wave height of an offshore booster platform according to claim 1, characterized in that: Based on the optimized wave feature extraction model, calculating the wave height and saving it includes the following steps: Crop the newly collected wave image and mark its wave rectangular frame; Calculate a first pixel value of the cropped ocean wave image based on the ocean wave rectangular frame; Calculate the original pixel value of the wave height before cropping the wave image; The true height value of the wave is calculated based on the first pixel value and the original pixel value.

8. The method for monitoring sea wave height of an offshore booster platform according to claim 7, characterized in that: The calculation formula for the true height of the waves is: height=(h-mark) / scale The first pixel value calculation formula is: y=y1+(y2-y1)*0.5 The original pixel value calculation formula is: h=y+supple Among them, y1 and y2 represent the vertical axis coordinate values ​​of the pixel coordinates of the upper left vertex and the lower right vertex of the rectangular box respectively; supplement represents the pixel value clipped on the vertical axis; mark represents the pixel value of the wave at the 0-meter scale line; Scale is expressed as the pixel value per meter.

9. An offshore booster platform wave height monitoring system, characterized in that: include: An acquisition module, used to acquire original images of the target area at sea; A data processing module, used for constructing an ocean wave image dataset based on the original image; A feature extraction module, used for obtaining a wave feature extraction model based on the wave image dataset; A model training module, used for training the ocean wave feature extraction model to obtain an optimized ocean wave feature extraction model; The calculation module is used to calculate and save the wave height based on the optimized wave feature extraction model.

10. An offshore booster platform wave height monitoring device, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is connected to the memory and is used to execute the computer program stored in the memory so that the offshore boost platform wave height monitoring device executes the offshore boost platform wave height monitoring method according to any one of claims 1 to 8.

Images