A method, medium and system for detecting wave height based on semantic segmentation

Abstract

The present invention provides a wave height detection method, medium and system based on semantic segmentation, which belongs to the field of wave height detection technology, including: a nearshore camera obtains multiple wave image data, and forms a training data set after preprocessing. The GS‑SegFormer semantic segmentation model is used, which adopts an improved self-attention mechanism and a Ghost module, as well as a SimAM attention mechanism, to segment the wave image to be tested, and obtain the pixel-level boundary of the wave. The segmented image is subjected to Gaussian filtering and smoothing processing to reduce noise, and the pixel information is converted into an actual distance according to the camera parameters, and finally the actual height of each wave pixel is calculated. The method identifies the wave features through a deep learning model and calculates the actual height in combination with the camera parameters. The present invention solves the technical problem that the existing wave height detection method often cannot accurately segment the real contour of the wave, thereby affecting the accuracy of the subsequent height calculation.

Description

Technical Field

[0001] The present invention belongs to the technical field of ocean wave height detection, and in particular, relates to an ocean wave height detection method, medium and system based on semantic segmentation. Background Art

[0002] As an important natural phenomenon in the marine environment, ocean waves have always been the focus of human research and monitoring. Accurately measuring and predicting the height of ocean waves is of great significance to many fields such as marine environmental monitoring, ship safety, and maritime operations. Traditional wave monitoring methods usually rely on manual observation or simple measuring equipment, which have problems such as low observation frequency and unstable data quality. Therefore, the development of automated wave height monitoring methods based on advanced sensing technology has always been a research hotspot in the field of marine engineering.

[0003] In recent years, with the rapid development of computer vision and machine learning technologies, wave height detection methods based on image analysis have attracted widespread attention. This method usually uses a camera installed on the seashore to capture real-time wave image data, analyzes these images in combination with computer vision algorithms, and extracts the geometric features of the waves from them, thereby calculating the actual height of the waves. Compared with traditional methods, this image analysis-based method has the advantages of high automation, high monitoring frequency, and good data reliability.

[0004] However, existing wave height detection methods based on image analysis often cannot accurately segment the true contours of objects such as waves with irregular shapes and fuzzy boundaries, thus affecting the accuracy of subsequent height calculations. Summary of the invention

[0005] In view of this, the present invention provides a wave height detection method, medium and system based on semantic segmentation, which can solve the technical problem that the existing wave height detection method based on image analysis is often unable to accurately segment the true contour of objects such as waves with irregular shapes and fuzzy boundaries, thereby affecting the accuracy of subsequent height calculations.

[0006] The present invention is achieved in that:

[0007] A first aspect of the present invention provides a method for detecting wave height based on semantic segmentation, which comprises the following steps:

[0008] S10, acquiring a plurality of wave image data through a nearshore camera;

[0009] S20, preprocessing the acquired wave image, including denoising, correction, and cropping, to obtain a plurality of preprocessed images;

[0010] S30, obtaining the wave pixel-level label data of the multiple pre-processed images to form a training data set, and using the training data set to train a GS-SegFormer semantic segmentation model, wherein the GS-SegFormer semantic segmentation model uses SCConv to improve the self-attention mechanism, uses the Ghost module to process the multi-level characteristics of the MiT encoder, and introduces an improved multi-level SimAM attention mechanism;

[0011] S40, obtaining a wave image captured by a nearshore camera of the wave height to be measured, recording it as the image to be measured, and preprocessing the image to be measured to obtain a preprocessed image to be measured;

[0012] S50, using the trained GS-SegFormer model to segment the pre-processed image to be tested, to obtain a segmented image of the wave boundary at the pixel level of the wave;

[0013] S60, post-processing the segmented image, that is, using a Gaussian filtering method to smooth the wave boundaries in the pre-processed image to be tested to reduce noise;

[0014] S70, calibrating camera parameters, obtaining internal and external parameters of the camera, and calculating the relationship between the pixels of the image to be measured and the actual length;

[0015] S80, converting pixel information of the wave area in the segmented image into actual distance according to the post-processed segmented image and camera parameters;

[0016] S90: Based on the converted actual distance information, calculate and output the actual height of each wave pixel in the segmented image.

[0017] The use of SCConv to improve the self-attention mechanism is specifically expressed as follows:

[0018] For the lth layer (l=1,2,...,L):

[0019]

[0020] Q′ l =Conv2d(Q l ,k l ,s l ,p l ,d l );

[0021] K′ l =Conv2d(K l ,k l ,s l ,p l ,d l );

[0022] V′ l =Conv2d(V l ,k l ,s l ,p l ,d l );

[0023]

[0024] Z l =A l V′ l ;

[0025] Y l =LayerNorm(X l +Dropout(Z l )+FFN(Z l ));

[0026] X l+1 =Y l +ResNet(Y l );

[0027] In the formula, Represents the input feature map of the lth layer; n l represents the batch size, d l Indicates the number of channels of the input feature map, h l Represents the height of the input feature map, w l Indicates the width of the input feature map;

[0028] In the formula, represents a learnable weight matrix used to generate queries (Q), keys (K), and values ​​(V); Represents the scaling factor in the attention mechanism, which is used to determine the dimensions of the query / key / value;

[0029] In the formula, Represents a learnable bias vector, used in conjunction with a weight matrix; k l Indicates the convolution kernel size of the l-th layer SCConv operation; s l represents the step size of the l-th layer SCConv operation; p l Indicates the padding of the l-th layer SCConv operation; d l represents the expansion rate of the l-th layer SCConv operation; represents the learnable position encoding matrix; FFN is a feedforward neural network, defined as FFN(x) = W 2 ReLU(W 1 x+b 1 )+b 2; ResNet is a residual network block, defined as ResNet(x)=Conv2d(ReLU(BatchNorm(Conv2d(x)))).

[0030] Among them, the improved multi-level SimAM attention mechanism is specifically expressed as follows:

[0031] For layer l:

[0032]

[0033]

[0034] S l =X l ·w l ;

[0035]

[0036] In the formula, e l Represents the average value of the l-th layer feature, which is used to calculate the statistical information of the feature; Represents the variance of the l-th layer feature, which is used to measure the discreteness of the feature; λ l represents the mean absolute deviation of the l-th layer features; γ l represents the adaptive parameter of the lth layer, combining the information of variance and mean absolute deviation; α l Represents the learnable parameters of the lth layer, which are used to adjust γ l Flexibility; l Represents the attention weight of the lth layer, which is used to weight the features; S l Represents the features of the lth layer after SimAM attention weighting; Represents the final SimAM output of the lth layer, combining attention weighting and position encoding; PositionalEncoding (S l ) indicates application to S l The position encoding provides location information for the feature.

[0037] Among them, the multi-level attention mechanism combining SCConv and improved SimAM is specifically expressed as follows:

[0038] Step 1: Get Z through SCConv self-attention mechanism l

[0039] Step 2: Calculate Z l Statistics:

[0040]

[0041] Step 3: Calculate adaptive parameters:

[0042]

[0043] Step 4: Generate attention weights:

[0044]

[0045] Step 5. Apply attention weights:

[0046]

[0047] Step 6: Final fusion output:

[0048]

[0049] Step 7. Combine residual connection and Ghost module:

[0050]

[0051] X l+1 =Y l +NesNet(Y l )+GhostModule(Y l );

[0052] In the formula, Z l is the output of the SCConv self-attention mechanism, with the same dimension as the input feature map X l same; Z l The average value is used to calculate Z l Statistical information; Z l The variance of Z l The degree of dispersion; Z l The mean absolute deviation of For application to Z l The adaptive parameters of Z l Variance and mean absolute deviation information of β l is a learnable parameter used to adjust flexibility; For application to Z l The attention weight of Z′ l is the Z after improved SimAM attention weighting l ; The final fusion attention output combines attention weighting and position encoding; GhostModule (Y l ) is applied to Y l The Ghost module output is used to reduce the amount of calculation and maintain model performance.

[0053] The step of obtaining the label data of the wave pixel level of the plurality of pre-processed images specifically includes:

[0054] Step 1: Apply the initial segmentation algorithm to the preprocessed image;

[0055] Step 2: Use edge detection and active contour model to refine the boundary;

[0056] Step 3: Apply region growing and level set methods to perform region segmentation;

[0057] Step 4: Use the graph cut algorithm to obtain the global optimal segmentation;

[0058] Step 5: Apply the pre-trained deep learning model for auxiliary segmentation;

[0059] Step 6: Use interactive tools for manual correction;

[0060] Step 7: Apply CRF to refine labels;

[0061] Step 8: Fuse multi-scale results to obtain the final label.

[0062] The traditional image processing method is a method combining the Otsu threshold method and edge detection.

[0063] The step of preprocessing the image to be tested includes denoising, correction, cropping, enhancement and scaling.

[0064] Furthermore, the preprocessed images need to have uniform size and resolution.

[0065] Furthermore, the internal parameters include focal length and aperture, and the external parameters include position and angle.

[0066] Specifically, the specific steps of step S10, obtaining multiple wave image data through a near-shore camera, are: installing a high-definition camera at the seaside to continuously capture real-time wave image data in multiple time periods, covering wave characteristics under different sea conditions; performing necessary preprocessing on the obtained original wave image data, including adjusting the image size, color correction and other operations, so that it meets the input requirements of subsequent algorithms.

[0067] Among them, the specific steps of preprocessing the acquired wave image in step S20 are: using image denoising methods such as median filtering and Gaussian filtering to perform noise suppression processing on the original wave image; based on the image correction method of camera calibration, correcting the image according to the internal and external parameters of the camera to eliminate geometric distortion; appropriately cropping the preprocessed image to remove irrelevant areas and retain only the core part containing the waves to improve the processing efficiency and accuracy of subsequent algorithms.

[0068] Among them, the specific steps of step S30, obtaining the wave pixel-level label data of the multiple pre-processed images to form a training data set, and using the training data set to train a GS-SegFormer semantic segmentation model are: manually or semi-automatically adding pixel-level annotations to the wave area of ​​each pre-processed image to form a real wave segmentation label; combining the pre-processed image and the corresponding annotation data to form a training data set, and performing data enhancement operations to improve the generalization ability of the model; using the GS-SegFormer semantic segmentation model, which uses improved mechanisms such as SCConv, Ghost module, SimAM attention mechanism, etc., to learn the end-to-end mapping relationship from wave image to pixel-level wave segmentation on the training data set.

[0069] Among them, the step S40, obtains the wave image taken by the nearshore camera at the wave height to be measured, recorded as the image to be measured, and preprocesses the image to be measured to obtain the preprocessed image to be measured; uses the trained GS-SegFormer model to segment the preprocessed image to be measured to obtain the pixel-level wave boundary segmentation image of the wave. The specific steps are: obtain the wave image data to be measured, and use preprocessing operations similar to those in step S20, including denoising, correction, cropping, etc., so that it meets the input requirements of the GS-SegFormer model; use the aforementioned trained GS-SegFormer semantic segmentation model to perform pixel-level wave segmentation on these preprocessed images to generate segmented images containing wave boundary information.

[0070] Among them, the step S50, post-processing the segmented image, that is, using the Gaussian filtering method to smooth the wave boundaries in the pre-processed image to be tested, and the specific steps of reducing noise are: using the Gaussian filtering method to smooth the wave boundaries in the segmented image to reduce the irregularity of the waves themselves and the noise introduced in the pre-processing and segmentation process, so that the wave boundaries of the segmentation result are more continuous and smooth, and better reflect the actual contour of the waves. In the specific implementation, the kernel size and standard deviation of the Gaussian filter can be adjusted according to the actual situation to achieve the best smoothing effect.

[0071] The specific steps of step S60, calibrating the camera parameters, obtaining the internal and external parameters of the camera, and calculating the relationship between the pixels of the image to be measured and the actual length are: using the classic chessboard calibration method, by shooting a chessboard image of known size, estimating the internal parameters of the camera such as the focal length, principal point coordinates, and distortion coefficient; measuring the installation position and angle of the camera on site, and obtaining its external parameters in three-dimensional space; based on the camera model and calibration parameters, establishing the conversion relationship between the pixel coordinate system in the image to be measured and the actual physical space, and calculating the length unit of each pixel in the actual space.

[0072] Among them, the specific steps of converting the pixel information of the wave area in the segmented image into the actual distance according to the post-processed segmented image and camera parameters in step S70 are: for each pixel point identified as a wave in the segmented image, using the camera model and calibration parameters, calculating the three-dimensional coordinates of the pixel point in the actual space; combining the conversion relationship between the pixel and the actual length, obtaining the actual distance information corresponding to the pixel point; by performing such conversion calculations on all the wave pixels in the entire segmented image, finally obtaining a "height map" reflecting the actual wave distribution.

[0073] Wherein, the specific steps of calculating and outputting the actual height of each wave pixel in the segmented image based on the converted actual distance information in step S80 are: for each wave pixel point in the segmented image, extracting its vertical height z value in the actual space; performing statistical analysis on the height z of all wave pixel points in the entire segmented image to obtain the height distribution of the wave scene, such as the maximum height, average height and other indicators; and outputting the wave height information for further wave characteristic analysis and forecasting applications.

[0074] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are executed, they are used to execute the above-mentioned wave height detection method based on semantic segmentation.

[0075] A third aspect of the present invention provides a wave height detection system based on semantic segmentation, which includes the above-mentioned computer-readable storage medium.

[0076] Compared with the prior art, the beneficial effects of the wave height detection method, medium and system based on semantic segmentation provided by the present invention are:

[0077] 1. High degree of automation and fast monitoring frequency

[0078] This method uses near-shore cameras to capture real-time image data of waves, and combines it with advanced deep learning algorithms to achieve fully automatic measurement of wave height, greatly improving monitoring efficiency and frequency, and providing timely and reliable data support for applications such as marine environment monitoring and ship navigation.

[0079] 2. High wave segmentation accuracy and reliable detection results

[0080] This method uses the GS-SegFormer semantic segmentation model based on Transformers, which can accurately extract the pixel-level boundaries of waves from the wave image, greatly improving the recognition ability of the existing segmentation algorithm for complex wave contours. At the same time, it integrates SCConv, Ghost module, SimAM and other mechanisms to further improve the performance of the segmentation model.

[0081] 3. Use camera calibration to achieve accurate measurement

[0082] This method makes full use of the internal and external parameters obtained by camera calibration and establishes the conversion relationship between the pixel coordinate system and the actual physical space. It can reliably convert the wave information in the image into actual height data, and the detection accuracy is significantly improved.

[0083] In summary, the solution of the present invention solves the technical problem that the existing wave height detection method based on image analysis is often unable to accurately segment the true contour of objects such as waves with irregular shapes and blurred boundaries, thereby affecting the accuracy of subsequent height calculations. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 A flow chart of the method provided by the present invention;

[0085] Figure 2 Schematic diagram of GS-SegFormer algorithm;

[0086] Figure 3 This is a schematic diagram of the SCConv module;

[0087] Figure 4 It is a schematic diagram of the space reconstruction unit;

[0088] Figure 5 is a schematic diagram of a channel reconstruction unit;

[0089] Figure 6 This is a schematic diagram of the Ghost module;

[0090] Figure 7 Schematic diagram of SimAM’s attention mechanism. DETAILED DESCRIPTION

[0091] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0092] like Figure 1 FIG. 1 is a flow chart of a method for detecting wave height based on semantic segmentation provided by the present invention. The method comprises the following steps:

[0093] S10, acquiring a plurality of wave image data through a nearshore camera;

[0094] S20, preprocessing the acquired wave image, including denoising, correction, and cropping, to obtain a plurality of preprocessed images;

[0095] S30, obtaining the wave pixel-level label data of multiple pre-processed images to form a training data set, and using the training data set to train a GS-SegFormer semantic segmentation model. The GS-SegFormer semantic segmentation model uses SCConv to improve the self-attention mechanism, uses the Ghost module to process the multi-level characteristics of the MiT encoder, and introduces an improved multi-level SimAM attention mechanism;

[0096] S40, obtaining a wave image captured by a nearshore camera of the wave height to be measured, recording it as the image to be measured, and preprocessing the image to be measured to obtain a preprocessed image to be measured;

[0097] S50, using the trained GS-SegFormer model to segment the pre-processed image to be tested, to obtain a segmented image of the wave boundary at the pixel level of the wave;

[0098] S60, post-processing the segmented image, that is, using a Gaussian filtering method to smooth the wave boundaries in the pre-processed image to be tested to reduce noise;

[0099] S70, calibrating camera parameters, obtaining the camera's internal and external parameters, and calculating the relationship between the pixels of the image to be measured and the actual length;

[0100] S80, converting pixel information of the wave area in the segmented image into actual distance according to the post-processed segmented image and camera parameters;

[0101] S90: Based on the converted actual distance information, the actual height of each wave pixel in the segmented image is calculated and output.

[0102] The specific implementation methods of the above steps are described in detail below:

[0103] Step S10: Acquire multiple wave image data through a nearshore camera

[0104] In this step, we first need to install a high-definition camera on the beach to capture real-time image data of the waves. In order to obtain sufficient training data, it is usually necessary to continuously collect wave images for multiple time periods, such as collecting image data for several hours each on sunny and rainy days. These image data cover the characteristics of waves under different sea conditions, which is conducive to the training and generalization performance of subsequent algorithms. At the same time, it is also necessary to perform necessary preprocessing on these original images, such as adjusting the image size and color correction, so that they meet the input requirements of subsequent algorithms.

[0105] Step S20: pre-process the acquired wave image, including denoising, correction, and cropping, to obtain multiple pre-processed images.

[0106] In this step, a series of preprocessing operations are mainly performed on the original wave image obtained in step S10:

[0107] 1) Denoising: Due to the complexity of the camera shooting environment, the original image often contains some noise interference, such as light spots and water splashes caused by windy and rainy weather. In order to reduce the impact of these noises on subsequent processing, common image denoising methods such as median filtering and Gaussian filtering can be used to suppress the noise of the image.

[0108] 2) Image correction: Since the installation position and angle of the camera may have certain deviations, the original image may have some perspective distortion, rotation and other problems. In order to eliminate these geometric distortions, an image correction method based on camera calibration can be used to correct the image according to the internal and external parameters of the camera, so that the shape and position of the waves are more realistic and reliable.

[0109] 3) Image cropping: In order to further improve the processing efficiency of the subsequent algorithm, the pre-processed image can be appropriately cropped to remove irrelevant areas at the edge of the image and retain only the core part containing the waves. This can not only reduce the computational overhead, but also improve the focus and accuracy of the algorithm.

[0110] Through the above three preprocessing steps, the original wave image data is converted into multiple preprocessed images with uniform size, less noise and less geometric distortion, which lays a good foundation for subsequent training and segmentation processing.

[0111] Step S30: Obtain the wave pixel-level label data of the plurality of pre-processed images to form a training data set, and use the training data set to train a GS-SegFormer semantic segmentation model

[0112] In this step, you need to complete the following sub-steps:

[0113] 1) Labeling the wave images: For the multiple pre-processed images obtained in step S20, manually or semi-automatically add pixel-level annotations to the wave area of ​​each image to form a real wave segmentation label. These labeled data will be used as a training set to provide reliable supervision information for the machine learning model.

[0114] 2) Constructing a training data set: Combine the preprocessed images and the corresponding pixel-level annotation data to form a training data set. Usually, some data enhancement operations are also required, such as random flipping, scaling, adding noise, etc., to further expand the diversity of the training data and improve the generalization ability of the model.

[0115] 3) Training GS-SegFormer model: GS-SegFormer is a semantic segmentation model based on Transformers, which is used in this method to learn the mapping relationship from wave images to pixel-level wave segmentation. It adopts some improvement mechanisms, including:

[0116] -SCConv: A convolutional mechanism based on semantic correlation, which can better capture the correlation between different semantic regions in the image.

[0117] -Ghost module: A lightweight feature processing module that can effectively process the multi-level features extracted by the MiT encoder.

[0118] -SimAM Attention Mechanism: An adaptive attention mechanism that can automatically adjust the attention weights of different semantic regions.

[0119] By training this GS-SegFormer model on the training dataset, we can learn the end-to-end mapping relationship from wave images to pixel-level wave segmentation.

[0120] Step S40: Obtain a wave image taken by a nearshore camera at the wave height to be measured, record it as the image to be measured, and preprocess the image to be measured to obtain a preprocessed image to be measured; segment the preprocessed image to be measured using the trained GS-SegFormer model to obtain a segmented image of the wave boundary at the pixel level of the wave

[0121] In this step, we first need to obtain the wave image data to be tested. These images can come from nearshore cameras installed at the seaside to capture real-time wave conditions. Similar to the preprocessing operation in step S20, these images to be tested need to be preprocessed by denoising, correction, cropping, etc. to meet the input requirements of the GS-SegFormer model.

[0122] Step S50:

[0123] Next, the previously trained GS-SegFormer semantic segmentation model is used to perform pixel-level wave segmentation on these preprocessed images. Specifically, the model automatically identifies and extracts the boundary area of ​​the waves in the image and generates a segmented image, in which the wave part is marked as the foreground and the non-wave area is marked as the background. This segmentation result lays the foundation for the subsequent calculation of the wave height.

[0124] Step S60: Post-process the segmented image, that is, use the Gaussian filtering method to smooth the wave boundaries in the pre-processed image to reduce noise.

[0125] After obtaining the segmentation results of the GS-SegFormer model, some post-processing operations are required to further improve the quality of the segmentation results.

[0126] Among them, the most critical step is to use the Gaussian filtering method to smooth the wave boundaries in the segmented image. Due to the irregularity of the waves themselves, as well as some noise that may be introduced during the aforementioned preprocessing and segmentation process, the wave boundaries of the segmentation results may have some burrs or unevenness. Through Gaussian filtering, these boundaries can be effectively smoothed to make them more continuous and smooth, thereby better reflecting the actual contours of the waves.

[0127] This smoothing process can not only reduce noise, but also help the subsequent calculation of wave height, making the measurement results more accurate and reliable. In the specific implementation, the kernel size and standard deviation of the Gaussian filter can be adjusted according to the actual situation to achieve the best smoothing effect.

[0128] Step S70: calibrate the camera parameters, obtain the camera's internal and external parameters, and calculate the relationship between the pixels of the image to be measured and the actual length.

[0129] In order to convert the wave boundary information obtained from the segmented image into the actual wave height, it is necessary to first calibrate the camera used to shoot the image to be tested to obtain its internal parameters (focal length, aperture, etc.) and external parameters (position, angle, etc.).

[0130] Internal calibration can use the classic chessboard calibration method to estimate the camera's focal length, principal point coordinates, distortion coefficient and other internal parameters by shooting a chessboard image of known size. External calibration requires measuring the camera's installation position and angle on site to obtain its position and direction information in three-dimensional space.

[0131] With these calibration parameters, the conversion relationship between the pixel coordinate system in the image to be tested and the actual physical space can be established. Specifically, through the camera model and calibration parameters, the length unit corresponding to each pixel in the actual space can be calculated, providing scale information for subsequent height calculations.

[0132] Specifically:

[0133] For any pixel point (u, v) in the segmented image, its coordinates (X, Y, Z) in the actual three-dimensional space can be calculated through the camera's pinhole model. The mathematical expression of the pinhole model is as follows:

[0134]

[0135] Among them, R represents the rotation matrix of the camera (external parameter), T represents the translation vector of the camera (external parameter), represents the normalized pixel coordinates. It is the coordinate of the pixel point in the actual three-dimensional space.

[0136] The camera's internal parameters K include the focal length f, the principal point coordinates (c x ,c y ) and distortion coefficients, etc., can be obtained through chessboard calibration and other methods. With the internal and external parameters, the mapping relationship between the pixel coordinate system and the actual space coordinate system can be established, and then the three-dimensional coordinates (X, Y, Z) of each wave pixel in the actual space can be calculated.

[0137] Step S80: According to the post-processed segmented image and camera parameters, the pixel information of the wave area in the segmented image is converted into the actual distance

[0138] With the camera calibration parameters obtained in step S70, combined with the smoothed segmented image obtained in step S50, the conversion calculation from pixel coordinates to actual physical distance can be performed.

[0139] Specifically, for each pixel identified as a wave in the segmented image, the three-dimensional coordinates of the pixel in the actual space can be calculated using the camera model and calibration parameters. Combined with the conversion relationship between pixels and actual length, the actual distance information corresponding to the pixel can be obtained.

[0140] Specifically, with the coordinates (X, Y, Z) of each wave pixel point in the actual three-dimensional space calculated in step S70, it can be further converted into actual distance information.

[0141] First, the distance d from each wave pixel to the camera optical center can be calculated:

[0142]

[0143] Secondly, combined with the pixel-actual length conversion relationship obtained in step S60, this distance d can be converted into the actual physical distance D:

[0144]

[0145] Among them, k is the scaling factor from pixels to actual length, which can be obtained through camera calibration.

[0146] By performing such conversion calculations on all pixels in the wave area of ​​the entire segmented image, we can eventually obtain a "height map" that reflects the actual wave distribution, providing basic data for subsequent wave height analysis.

[0147] Step S90: Based on the converted actual distance information, calculate the actual height of each wave pixel in the segmented image and output

[0148] With the actual distance information obtained in step S80, the last step is to calculate the actual height of each wave pixel based on this data and output the result.

[0149] The specific calculation method is as follows: For each wave pixel in the segmented image, first obtain its three-dimensional coordinate information (x, y, z) in the actual space. Among them, x and y represent the horizontal distance, and z represents the vertical height. Since we are concerned about the height of the wave, we only need to extract the z value as the actual height of the pixel.

[0150] Specifically:

[0151] With the actual three-dimensional coordinates (X, Y, Z) of each wave pixel point obtained in step S80, the last step is to extract its vertical height Z as the actual wave height of the pixel point.

[0152] Specifically, for any wave pixel i in the segmented image, its actual height H i It can be expressed as:

[0153] H i =Z i

[0154] Among them, Z i It is the vertical height coordinate of the pixel point in the actual three-dimensional space.

[0155] By calculating the height H of all the wave pixels in the entire segmented image i Through statistical analysis, we can get the height distribution of the wave scene, such as the maximum height H max , average height H avg Other indicators:

[0156] H max =max{H i}

[0157]

[0158] Where N is the total number of ocean wave pixels in the segmented image.

[0159] By statistically analyzing the height z of all wave pixels in the entire segmented image, we can obtain the height distribution of the wave scene, such as the maximum height, average height, etc. This information can not only be directly output, but also used for further wave characteristic analysis and forecasting applications.

[0160] In summary, this wave height detection method based on semantic segmentation makes full use of computer vision and machine learning technology, and can realize the automation of the entire process from wave images to precise height measurement. It overcomes the limitations of traditional manual observation, improves the efficiency and accuracy of wave monitoring, and provides new technical support for the fields of marine environment monitoring, ship safety navigation, etc.

[0161] The following are the specific steps of SCConv's improved self-attention mechanism:

[0162] First, input feature X l Generate queries, keys, and values ​​through linear transformations:

[0163]

[0164] Then, apply the SCConv operation:

[0165] Q′ l =Conv2d(Q l ,k l ,s l ,p l ,d l )

[0166] K′ l =Conv2d(K l ,k l ,s l ,p l ,d l )

[0167] V′ l =Conv2d(V l ,k l ,s l ,p l ,d l )

[0168] Calculate attention weights:

[0169]

[0170] Finally, we get the attention-weighted output:

[0171] Z l =A l V′ l

[0172] The main advantages of SCConv are:

[0173] The capture of local spatial information is enhanced through convolution operations

[0174] The receptive field size can be flexibly adjusted (by k l and d l )

[0175] Maintains the global modeling capability of the self-attention mechanism

[0176] Introducing a learnable positional encoding M l , further enhancing spatial perception

[0177] This improvement enables the self-attention mechanism to better handle tasks with strong spatial correlation, such as image segmentation and object detection.

[0178] The calculation process in the multi-level SimAM attention mechanism is as follows:

[0179] Calculate feature statistics:

[0180]

[0181] Calculate the adaptive parameters:

[0182]

[0183] Generate attention weights:

[0184]

[0185] Apply attention weights:

[0186] S l =X l ·w l

[0187] Final output:

[0188]

[0189] The main features of the multi-level SimAM attention mechanism:

[0190] Adaptability: Through γ l Parameters can adaptively adjust the attention strength according to the statistical characteristics of the input features.

[0191] Non-linear activation: Use a sigmoid-type function to generate attention weights, introducing non-linearity.

[0192] Statistical feature utilization: The mean, variance and mean absolute deviation are combined to comprehensively consider the feature distribution.

[0193] Position awareness: By adding position encoding, the model's ability to perceive spatial position is enhanced.

[0194] Normalization: Use LayerNorm for normalization, which helps stabilize the training process.

[0195] This multi-level SimAM attention mechanism can capture the importance of features at different network levels, and through the learning of adaptive parameters, it can better adapt to different tasks and data distributions.

[0196] The following is the calculation process in the multi-level attention mechanism combining SCConv and improved SimAM:

[0197] First, we use the SCConv self-attention mechanism to obtain Z l

[0198] Calculate Z l Statistics:

[0199]

[0200] Calculate the adaptive parameters:

[0201]

[0202] Generate attention weights:

[0203]

[0204] Apply attention weights:

[0205]

[0206] Final fusion output:

[0207]

[0208] Combining residual connections and Ghost modules:

[0209]

[0210] X l+1 =Y l +ResNet(Y l )+GhostModule(Y l )

[0211] The main features of this multi-level attention mechanism that combines SCConv and improved SimAM are:

[0212] Dual Attention: Combining the spatial perception ability of SCConv and the adaptive nature of SimAM.

[0213] Adaptive Fusion: and β l Parameters can adaptively adjust the SimAM attention strength according to the feature distribution output by SCConv.

[0214] Multi-scale feature extraction: Use Ghost modules and residual connections to reduce computation while maintaining performance.

[0215] Flexibility: The contribution of SCConv and SimAM can be adjusted according to task requirements.

[0216] Enhanced feature expression: Through multiple attention mechanisms and nonlinear transformations, the model's ability to capture complex features is enhanced.

[0217] This fusion mechanism aims to comprehensively utilize the local perception ability of SCConv and the global adaptive characteristics of SimAM to obtain a more powerful and flexible feature representation. At the same time, by introducing the Ghost module, the computational complexity is reduced while maintaining the model performance.

[0218] The relevant parameters are explained as follows: l is the input feature map of the lth layer, with a dimension of n l ×d l ×h l × l ;n l is the batch size; d l is the number of channels; h l is the feature map height; w l is the feature map width; is a learnable weight matrix with dimension is a learnable bias vector with dimension is the dimension of query / key / value; M l is a learnable position encoding matrix with dimension n l ×n l ;k l is the convolution kernel size; s l is the convolution step size; p l Padding for convolution; d l is the convolution expansion rate; α l ,β l is a learnable scalar parameter; w l is the learnable weight of each layer feature; k o ,so ,p o is the kernel size, stride, and padding of the final output convolutional layer; Q l ,K l ,V l is the query, key, and value matrix; Q′ l ,K′ l ,V′ l is the convolved query, key, and value matrix; A l is the attention weight matrix; Z l is the value matrix after attention weighting; Y l is the layer output; e l , λ l ,γ l ,w l is the statistic and intermediate variable in SimAM; S l , It is the feature after SimAM attention; is the statistics and intermediate variables in the combined attention mechanism; Z′ l , is the combined attention feature; F is the feature after multi-scale fusion; O is the final output.

[0219] For the parameters and variables in these equations, their acquisition methods and calculation steps are as follows:

[0220] 1. All weight matrices (such as ) and the bias vector (such as ) are all learned through the back-propagation algorithm. Initialization can use He initialization or Xavier initialization.

[0221] 2. Parameters of the convolution operation Conv2d (k l ,s l ,p l ,d l ) needs to be determined according to the specific network structure design. Usually, the kernel size can be selected as 3x3 or 5x5, the step size can be 1 or 2, the padding is usually set to (kernel size-1) / 2 to keep the feature map size unchanged, and the expansion rate can be increased layer by layer starting from 1.

[0222] 3. Position encoding matrix M l It can be generated using sine-cosine functions or set as learnable parameters.

[0223] 4. The weight matrix and bias vector in FFN are also learned through back propagation. Usually, the dimension of the intermediate layer is set to 4 times the input dimension.

[0224] 5. The parameters of the convolutional layers in the ResNet block are also learned through back-propagation. The parameters of the batch normalization layer (scaling factor and offset) are also learnable.

[0225] 6. Alpha in SimAM Mechanism l and β l is a learnable scalar parameter that can be initialized to 0 or a small positive number.

[0226] 7. The convolutional layer parameters in GhostModule are also learned through back propagation. The number of channels c needs to be determined according to the network structure design.

[0227] 8. Weight w in multi-scale feature fusion l It is also a learnable parameter and can be initialized to a uniform weight 1 / L.

[0228] 9. The final output convolution layer parameter k o ,s o ,p o It needs to be set according to the task requirements to get the required output size.

[0229] This complex multi-level attention mechanism involves the interaction and cascading of multiple components, including self-attention, convolution operations, statistics calculations, residual connections, Ghost modules, etc. Its goal is to capture the relationship between features at different levels of abstraction while maintaining computational efficiency. In practical applications, it may be necessary to adjust and optimize according to specific tasks and computing resources.

[0230] Specifically, the principle of the present invention is: firstly, a near-shore camera is used to capture real-time image data of waves, then a deep learning semantic segmentation model is used to perform fine wave area recognition on these images, and finally, the segmentation results are converted into actual wave height information in combination with camera calibration parameters. The specific technical solution is as follows:

[0231] 1. Obtain ocean wave image data

[0232] This method first requires the installation of a high-definition camera on the beach to continuously capture real-time images of waves in multiple time periods. In order to obtain richer and more comprehensive training data, these image data should cover the characteristics of waves under different sea conditions, such as sunny days, rainy days, etc. At the same time, the original wave images obtained need to be preprocessed, such as resizing and color correction, so that they meet the input requirements of subsequent algorithms.

[0233] 2. Wave area recognition based on semantic segmentation

[0234] With the preprocessed wave image data, the next step is to use the semantic segmentation model of deep learning to identify the wave area in detail. This paper adopts a segmentation model called GS-SegFormer, which is based on the Transformers architecture and integrates some improved mechanisms, such as SCConv, Ghost module, SimAM attention mechanism, etc., which can better capture the complex and irregular visual features of waves.

[0235] First, we need to build a training dataset. We can manually or semi-automatically add pixel-level wave area annotations to the acquired wave images to form real segmentation labels. We can combine these preprocessed images with the corresponding annotated data and perform appropriate data augmentation to build the dataset required for training the GS-SegFormer model.

[0236] Then, the GS-SegFormer model is trained end-to-end on this training dataset to learn the mapping relationship from wave images to pixel-level wave segmentation. After the training is completed, the model can automatically identify and extract the boundary area of ​​the waves on the new wave image to be tested and generate a segmented image.

[0237] 3. Actual height calculation based on camera calibration

[0238] With the wave segmentation image generated by the GS-SegFormer model, we need to use the camera calibration information to convert the pixel data in the image into actual physical distance and height.

[0239] First, the camera used to shoot the image to be tested needs to be calibrated to obtain its internal parameters (focal length, aperture, etc.) and external parameters (position, angle, etc.). The internal parameter calibration can be done by the chessboard calibration method, while the external parameter calibration requires measuring the installation information of the camera on site. With these calibration parameters, the conversion relationship between the pixel coordinate system in the image to be tested and the actual physical space can be established, and the length unit of each pixel in the actual space can be calculated.

[0240] Then, using this pixel-actual length conversion relationship and combining it with the wave segmentation image generated by GS-SegFormer, the pixel information of the wave area in the image can be converted into actual three-dimensional space coordinates (x, y, z), where the z value represents the actual vertical height of the pixel.

[0241] Finally, by performing statistical analysis on the height z values ​​of all wave pixels in the entire segmented image, we can obtain the height distribution of the wave scene, such as the maximum height, average height and other indicators, and output these results.

[0242] In summary, the new wave height detection method based on semantic segmentation proposed in this invention fully integrates the technical advantages of computer vision and machine learning, and can realize the automation of the whole process from wave images to precise height measurement. Compared with the existing technology, it overcomes the limitations of traditional manual observation, improves the efficiency and accuracy of wave monitoring, and provides new technical support for the fields of marine environment monitoring, safe ship navigation, etc.

[0243] In order to better understand and implement the present invention, an embodiment 1 of the present invention is provided below. The specific implementation of this embodiment 1 is as follows:

[0244] Data acquisition: First, obtain wave image data through near-shore cameras. These data can include images of waves at different locations and times, covering various states and characteristics of waves.

[0245] Data preprocessing: The acquired wave image data is preprocessed, including denoising, resizing, contrast enhancement and other operations to improve image quality and analysis accuracy. Preprocessing operations can eliminate noise and interference in the image, making the subsequent segmentation and detection process more stable and reliable.

[0246] Semantic segmentation: Use the pre-trained semantic segmentation model to segment the pre-processed wave image. The semantic segmentation model can segment the image into regions with semantic information, waves, and background, thereby achieving pixel-level recognition and analysis of the waves.

[0247] Post-processing: Post-process the prediction results and use the Gaussian filtering method to smooth the detection results.

[0248] Height calculation: Calculate the height of the waves based on the predicted results. Convert the number of pixels into actual length units.

[0249] Result output: The calculated wave height information is output and the detection results are visualized so that users can intuitively understand and analyze the wave height data.

[0250] Among them, the pre-processed wave image is segmented using a pre-trained semantic segmentation model. The following are the detailed steps to obtain the model:

[0251] This embodiment 1 designs a novel GS-SegFormer semantic segmentation algorithm. The algorithm includes an improved self-attention mechanism using SCConv, which aims to improve the feature extraction ability of the model. At the same time, the Ghost module processes the multi-level features from the MiT encoder, reuses the existing features, reduces the amount of calculation, and enhances the richness of the features. The Simple Parameter-Free Attention Module (SimAM) is added after feature fusion to further improve the efficiency and accuracy of the model, making it more robust when dealing with various complex scenarios.

[0252] The GS-SegFormer algorithm is mainly divided into two parts, namely the encoder and the decoder. Figure 2 As shown in Figure 1, for a given image, it is first divided into multiple smaller patches and then input into the hierarchical Transformer encoder to obtain multi-level features of (1 / 4, 1 / 8, 1 / 16, 1 / 32) of the original image resolution. These multi-level features are then passed to the Ghost module and then to the All-MLP decoder to predict (H / 4)×(W / 4)×N cls The segmentation mask at resolution N cls Indicates the number of categories.

[0253] The specific implementation steps of the GS-SegFormer algorithm are as follows:

[0254] In the first step, the input image passes through a series of convolutional layers for feature extraction.

[0255] In the second step, the extracted features are fed into a Transformer encoder, which consists of multiple self-attention layers and feed-forward network layers modified using SCConv.

[0256] In the third step, multi-scale feature fusion is applied to the output of the Transformer encoder, by extracting features at multiple locations of different scales and fusing them together.

[0257] In the fourth step, the multi-level features are processed through the Ghost module, which effectively reduces the number of parameters and computational complexity.

[0258] In the fifth step, the features are further passed through an MLP layer to unify their channel dimensions.

[0259] In the sixth step, the features are upsampled to 1 / 4 of the original size, and these different levels of features are concatenated together to form a richer feature representation.

[0260] In the seventh step, features at different levels are fused together by using another MLP layer to produce a more powerful feature representation.

[0261] In the eighth step, the fused features are passed through a SimAM attention mechanism layer. The SimAM layer can adaptively reweight the feature map to enhance the focus on important features and suppress unimportant features.

[0262] In the ninth step, the fused (H / 4)×(W / 4)×N cls resolution features to predict the segmentation mask M. Where N cls Indicates the number of categories and determines the dimension of the segmentation mask. This step maps the fused features to the final segmentation result.

[0263] (1) Using SCConv to improve the self-attention mechanism

[0264] 1) Introduction to SCConv module

[0265] The spatial and channel reconstruction module (SCConv) is an effective architectural unit that can reduce computational costs and model storage while improving the performance of CNN models by reducing the spatial and channel redundancy that is widely present in standard convolutions. Through SRU and CRU, the redundancy of feature maps is reduced, while achieving significant performance improvements and greatly reducing the computational load.

[0266] SCConv utilizes two components: spatial reconstruction unit (SRU) and channel reconstruction unit (CRU), as Figure 3 As shown in the figure, SRU is used to suppress spatial redundancy and is implemented by the separation reconstruction method. CRU is used to reduce channel redundancy and is implemented by the separation transformation fusion strategy. The combination of these two reconstruction units enables SCConv to effectively compress redundant features in convolutional neural networks, reduce computational load and improve model performance.

[0267] The main function of SRU is to reduce spatial redundancy. It decomposes the input feature map in the spatial dimension by the separation and reconstruction method, then reconstructs the decomposed sub-bands, and finally merges the reconstructed sub-bands into the output feature map, such as Figure 4 shown.

[0268] The purpose of the separation operation is to separate the feature maps with large information content from the feature maps with small information content, corresponding to the spatial content. The scaling factor in the group normalization is used to evaluate the information content in different feature maps. The formulas are shown in 4-1 and 4-2.

[0269]

[0270] Among them, μ is the mean, σ is the standard deviation, γ and β are trainable variables, and ε is a very small constant used to ensure stability. The larger the γ, the greater the variation between pixels, and the richer the spatial information.

[0271] The reconstruction operation is to add features with more information to features with less information to generate features with more information and save space. The specific operation is cross reconstruction, which combines the two weighted features with different information to obtain X w1 and X w2 , after connecting them, we get the spatial refinement feature map X w , and its formula is shown in 4-3.

[0272]

[0273] in is element-wise multiplication, is element-wise addition, and ∪ is the union.

[0274] After SRU processing, features with large amounts of information are separated from features with small amounts of information, reducing redundant features in the spatial dimension.

[0275] The main function of CRU is to reduce channel redundancy. It decomposes the input feature map in the channel dimension by separating, transforming and fusing the sub-bands, and finally obtains the output feature map, such as Figure 5 shown.

[0276] The segmentation operation refines the input space into features X w It is divided into two parts: one part has a channel number of αC, and the other part has a channel number of (1-α)·C, where α is a hyperparameter and 0≤α≤1. Then the channel numbers of the two sets of features are compressed using a 1×1 convolution kernel to obtain X up and X low .

[0277] The conversion operation converts the input X up As the input of "Rich Feature Extraction", GWC and PWC are performed separately, and then added to get the output Y 1 . Enter X low As a supplement to "rich feature extraction", PWC is performed, and the result is combined with the original input to obtain Y 2 .

[0278] The fusion operation uses a simplified SKNet method to adaptively merge Y 1 and Y 2 Specifically, we first use the global average pooling technique to combine the global spatial information and channel statistical information to obtain the pooled S 1and S 2 Then for S 1 and S 2 Do Softmax to get the feature weight vector β 1 and β 2 Finally, the feature weight vector is used to get the output Y = β 1 Y 1 +β 2 Y 2 , Y is the feature extracted by the channel.

[0279] 2) Using SCConv module to improve self-attention mechanism

[0280] The specific process of extracting features from the self-attention mechanism after adding SCConv is as follows:

[0281] Step 1: Initialize the query vector Q, key vector K and value vector V linear transformation layer and SCConv module.

[0282] Step 2: Perform a linear transformation on the input feature x to obtain the query vector Q, and change its shape to adapt to the multi-head attention mechanism.

[0283] Step 3: Perform spatial dimensionality reduction and apply SCConv as required. If the spatial dimensionality reduction ratio is greater than 1, reshape x and send it to the spatial dimensionality reduction convolution layer for dimensionality reduction. Apply SCConv module on the reduced feature map to further extract or transform spatial features. Reshape and transpose the feature map processed by SCConv to adapt to subsequent operations, and apply layer normalization.

[0284] Step 4: Obtain key and value vectors through linear transformation. Regardless of whether spatial dimensionality reduction is performed, the feature x is sent to the linear layer to obtain the key and value vectors K and V, and their shapes are changed for subsequent attention calculations.

[0285] Step 5: Calculate the attention weights and weighted sum value vector. The attention weights are obtained based on the similarity between the query vector Q and the key vector K. These calculated weights are then reasonably assigned to the numerical vector V to obtain a weighted feature representation.

[0286] Step 6: Apply attention weights. Multiply the normalized attention weights with the numerical vector V to obtain the weighted feature representation and reshape it back to the original shape.

[0287] Step 7: Project the weighted features to the output space. Finally, the weighted features are transformed to the output feature space through the projection layer to obtain the final attention mechanism output.

[0288] (2) Introduction to Ghost module

[0289] The Ghost module is an innovative model compression method that aims to generate richer feature maps by reducing the number of parameters and achieve efficient use of computing resources. This method significantly reduces network parameters and computational complexity while ensuring that the model accuracy is not reduced, thereby improving the operation speed and reducing latency. The core principle of the Ghost module is based on a key observation that a large number of highly similar feature maps often appear in convolutional neural networks. These feature maps are called "Ghost pairs." Traditional network design ideas may tend to avoid the generation of such redundant feature maps, but the Ghost module takes a completely different strategy. It believes that these similar feature maps are actually a reflection of the powerful feature extraction capabilities of convolutional neural networks, so their generation should not be deliberately avoided. Based on this understanding, the Ghost module adopts a novel idea, that is, to generate more Ghost pairs through a series of simple and efficient linear operations. These operations are not only computationally inexpensive, but can also effectively enhance the feature extraction capabilities of the model. In this way, the Ghost module can significantly reduce the number of parameters and achieve lightweight models while maintaining model performance. Figure 6 As shown, the specific process is as follows:

[0290] The first step is to generate the original feature map. First, the Ghost Module performs a standard convolution operation to generate a part of the feature map, which is called the original feature map Y′ (intrinsic feature maps). These original feature maps contain the basic and core information of the input data. The amount of calculation for this part is shown in Formula 4-4.

[0291] Y′=X·f′ (4-4)

[0292] Among them, X is the input feature and f′ is the convolution kernel.

[0293] The second step is to generate the Ghost feature map through linear transformation. Then, for each original feature map, the Ghost Module applies a linear transformation Φ i,j , which is achieved through a small convolution kernel, generating s similar feature maps, called Ghost feature maps, as shown in Formula 4-5.

[0294]

[0295] Among them, y i ′ represents the i-th original feature map, which is transformed by linear transformation Φ i,j , get the j-th similar feature map y ij , m represents the number of original feature maps, and s represents the number of generated similar feature maps.

[0296] The third step is to merge feature maps. Finally, the Ghost Module merges the original feature map and the Ghost feature map generated by linear transformation to form the final output feature map set. The number of feature maps in this set is much larger than the number of feature maps obtained by only one standard convolution operation, but the computational cost does not increase significantly.

[0297] (3) Introduction to parameter-free attention mechanism

[0298] The SimAM attention mechanism is an effective spatial attention module. This mechanism is based on the attention mechanism of the human brain. By simulating the human brain's attention allocation method, the model can focus on different areas of the image and assign different weights. Specifically, SimAM determines the importance of each neuron by calculating the linear separability between feature maps, thereby generating an attention map, such as Figure 7 As shown. These weights can be viewed as attention maps to visualize the model's focus areas. At the same time, SimAM also considers attention in the spatial dimension and channel dimension, and can generate three-dimensional weights. This three-dimensional weight is superior to the traditional one-dimensional and two-dimensional weight attention because it can better simulate the attention mechanism of the human brain. Based on some well-known neuroscience theories, the researchers proposed to find the importance of each neuron by optimizing an energy function, which mainly relies on the solution of the defined energy function to select the corresponding operator, thereby greatly reducing the effort required in structural adjustment.

[0299] It is crucial to evaluate the importance of each neuron in the SimAM module. Neuroscience research has revealed that neurons with rich information exhibit unique discharge patterns, which makes them play an important role in neural networks. At the same time, when a neuron is activated, it tends to have an inhibitory effect on neighboring neurons. This phenomenon is called spatial inhibition in neuroscience. Therefore, when evaluating the importance of neurons, the SimAM module pays special attention to those neurons with spatial inhibition characteristics and gives them higher weights. In order to accurately identify these important neurons, the SimAM module adopts a simple and effective method: measuring the linear separability between neurons. Therefore, the energy function is defined as shown in 4-6:

[0300]

[0301] where w t is the weight, b t is the bias, t is the target neuron for the input feature, x i are other neurons, M is the number of neurons on the channel, and i is the index on the spatial dimension. Minimizing the above formula, adding regularization terms and using binary labels, we get its closed-form solution as shown in 4-7:

[0302]

[0303] The analytical solutions of the above equation are shown in 4-8 and 4-9:

[0304]

[0305] Among them, μ t is the mean of all neurons in the channel except the target neuron, is the variance of all neurons in the channel except the target neuron, and λ is the coefficient. Since all neurons on each channel follow the same distribution, the mean and variance of the input features can be calculated in two dimensions, H and W, to avoid repeated calculations. The minimum energy is calculated using the formula shown in 4-10:

[0306]

[0307] Where λ = 10 -4 , the weight of each neuron is Finally, the formula of the parameter-free attention mechanism is shown in 4-11:

[0308]

[0309] Among them, E is the grouping of all energy functions in channel and spatial dimensions, and X represents the feature map.

[0310] (4) Fine-tuning the pre-trained model: Finally, we use the ocean wave image dataset to fine-tune our improved GS-SegFormer model to adapt it to the ocean wave height detection task. By training on the ocean wave image dataset, the performance and generalization ability of the model are further improved.

[0311] During the training process, this chapter used a batch size of 8, and each iteration of the model processed 8 samples of data. The model was trained for a total of 200 epochs, and each epoch traversed the entire dataset once. As the number of training epochs increases, the model has the opportunity to gradually learn and optimize its parameters to better fit the training data. The optimizer uses AdamW, an optimization algorithm with adaptive learning rate. The momentum parameter is set to 0.9, which controls the average speed of exponential decay and helps stabilize the training process. Weight decay is set to 0.01, which is a regularization technique that prevents overfitting by penalizing the weights of the model. By adding the L2 norm of the weights as a regularization term in the loss function, weight decay can encourage the model to find simpler weight solutions, thereby improving the generalization ability of the model. The learning rate descent method uses a cosine annealing strategy. During the training process, the learning rate gradually decreases according to the shape of the cosine function. This strategy helps to converge quickly in the early stages of training, while making more detailed adjustments in the later stages to find better local optimal solutions.

[0312] S2. Calculate the wave height based on the pixel position. The following are the detailed steps of this method:

[0313] (1) Obtaining pixel position information: After semantic segmentation, the position information of each pixel in the wave image is obtained. The positions of these pixels are expressed as coordinates in a two-dimensional coordinate system, usually in pixels.

[0314] (2) Post-processing: The prediction results are post-processed and Gaussian filtering is used to smooth the wave boundaries and reduce pixel-level noise.

[0315] Gaussian filtering is a linear filtering method that uses a Gaussian kernel function to perform weighted averaging on an image. Gaussian filtering can effectively smooth images and reduce high-frequency noise and details. For smoothing the edge of ocean waves, Gaussian filtering can help remove subtle fluctuations and noise caused by the segmentation algorithm, making the edge smoother and more natural.

[0316] When applying Gaussian filtering, it is necessary to determine the standard deviation (σ) of the Gaussian kernel and the size of the filter. The standard deviation controls the smoothness of the filter, and a larger standard deviation leads to a stronger smoothing effect. Similarly, the choice of filter size also needs to be weighed according to the characteristics of the wave boundary. The function formula of the Gaussian filter is as follows:

[0317]

[0318] We select a Gaussian kernel of size 5×5 and set the standard deviation to 1. Applying Gaussian filtering can effectively smooth the wave boundaries and improve the accuracy of wave height calculation.

[0319] (3) Conversion between pixel position and actual position: The pixel position in the image is converted to the actual spatial position. This needs to take into account factors such as the image scale, camera parameters, and geographic coordinate system. Through accurate coordinate conversion, the pixel position can be mapped to the actual position in the ocean.

[0320] Calibrate the camera parameters and use a camera calibration board or other appropriate calibration methods to obtain the camera's intrinsic and extrinsic parameters. Intrinsic parameters include parameters such as focal length and optical center position, while extrinsic parameters include the camera's position and orientation. Based on the collected image data, use the calibration tool (calibrateCamera function in OpenCV) to calculate the camera's intrinsic and extrinsic parameters.

[0321] Calculate the relationship between pixels and actual length. Based on the camera's intrinsic parameters, the relationship between pixels and actual length can be calculated using the camera imaging principle. The camera imaging principle is used to calculate the pixel coordinates of the projection of a 3D point in the camera coordinate system onto the image plane. Use the camera's intrinsic and extrinsic parameters to convert the pixel coordinates into a 3D point in the camera coordinate system, and then calibrate according to the actual length to obtain the conversion relationship between pixels and actual length.

[0322] The pixel-to-actual distance conversion is performed to obtain the relationship between the pixel and the actual length, and the pixel information of the wave area predicted by the semantic segmentation model is converted into the actual distance. According to the pixel area of ​​the wave area predicted by the semantic segmentation model, the actual distance of the wave is calculated using the conversion relationship between the pixel and the actual length.

[0323] Result output: The calculated wave height information is output as the height value of a single pixel. At the same time, the detection results are visualized according to the wave level classification table, so that users can intuitively understand and analyze the wave height data, as shown in Table 1:

[0324] Table 1 Classification of sea wave levels

[0325]

[0326] In order to better understand and implement the present invention, Example 2 of a specific application scenario of the present invention is provided below: A maritime management department, facing the dual pressures of increasingly busy maritime traffic and the intensifying impact of climate change, urgently needs to establish a more advanced and accurate wave monitoring system to provide reliable support for ship navigation, maritime operations, etc.

[0327] Based on this, the maritime department decided to introduce the new wave height detection method based on semantic segmentation proposed in this invention, in order to realize the automation and intelligence of wave monitoring. The specific implementation process is as follows:

[0328] 1. Ocean wave image data collection

[0329] First, the department selected a suitable location on the seashore and installed a high-definition camera. The location and angle of the camera were carefully planned to cover a wide enough sea area while minimizing the impact of obstructions such as buildings and vegetation.

[0330] In order to obtain more abundant and comprehensive training data, the department arranged staff to collect wave images for 8 hours and 6 hours in sunny and rainy weather respectively. Every 10 minutes, the camera automatically took a wave image, and finally accumulated nearly 2,000 raw image data. These image data cover the characteristics of waves under different sea conditions, providing rich materials for subsequent algorithm training and testing.

[0331] 2. Image Preprocessing

[0332] Next, the department carried out necessary preprocessing operations on the acquired raw wave image data. First, the image was noise-reduced using methods such as median filtering and Gaussian filtering to remove the interference of light spots and water splashes caused by factors such as wind and rain.

[0333] Secondly, based on the camera calibration parameters, the image was geometrically corrected to eliminate the perspective distortion caused by the deviation of the camera installation angle. Specifically, the department used the chessboard calibration method to estimate the camera's focal length, principal point coordinates, distortion coefficient and other internal parameters by shooting a chessboard image of known size. At the same time, they also measured the camera's installation position and angle on site to obtain its external parameters in three-dimensional space. With these calibration parameters, the conversion relationship between the pixel coordinate system in the image to be tested and the actual physical space can be established.

[0334] Finally, in order to further improve the processing efficiency of the subsequent algorithms, the department appropriately cropped the preprocessed images, removed irrelevant edge areas, and retained only the core part containing the waves. Through the above preprocessing operations, the original wave image data was converted into multiple preprocessed images with uniform size, less noise, and less geometric distortion, laying a good foundation for subsequent training and segmentation processing.

[0335] 3. Semantic segmentation model training

[0336] With the preprocessed wave image dataset, the department then trained the GS-SegFormer semantic segmentation model. First, they added pixel-level annotations to the wave area of ​​each image through manual annotation to form a real wave segmentation label. These images and annotation data were combined into a training dataset, and some data enhancement operations were performed, such as random flipping and scaling, to improve the generalization ability of the model.

[0337] The department then used this training dataset to train the GS-SegFormer model end-to-end. GS-SegFormer is a Transformers-based semantic segmentation model, which is used in this method to learn the mapping relationship from wave images to pixel-level wave segmentation. It uses some improvement mechanisms, including:

[0338] 1) SCConv: A convolutional mechanism based on semantic correlation, which can better capture the correlation between different semantic regions in the image.

[0339] 2) Ghost module: a lightweight feature processing module that can effectively process the multi-level features extracted by the MiT encoder.

[0340] 3) SimAM attention mechanism: an adaptive attention mechanism that can automatically adjust the attention weights of different semantic regions.

[0341] By training the GS-SegFormer model on the training dataset, the department learned the end-to-end mapping relationship from wave images to pixel-level wave segmentation. During the training process, the segmentation accuracy of the model continued to improve, and the average IoU (intersection over union) index on the validation set finally reached 0.89, showing a strong ability to identify wave areas.

[0342] 4. Wave height detection

[0343] With the trained GS-SegFormer model, the department can start using it to perform actual wave height detection. First, they used a camera installed on the beach to obtain a set of wave image data to be tested. These images to be tested were collected at the same location as the training images, with similar backgrounds and shooting angles.

[0344] Next, the department performed the same preprocessing operations on the test images as the training images, including denoising, correction, cropping, etc. Then, the trained GS-SegFormer model was used to perform pixel-level wave segmentation on the preprocessed test images. The results showed that the model can accurately extract the boundary area of ​​the waves in the image and generate a high-quality segmented image.

[0345] In order to further improve the smoothness of the segmentation results, the department also performed Gaussian filtering on the segmented image, effectively reducing the noise caused by the irregularity of the waves themselves. So far, they have obtained a high-quality segmented image that reflects the true contours of the waves.

[0346] Next, the department used the internal and external parameters obtained from the aforementioned camera calibration to convert the pixel coordinates in the segmented image into actual three-dimensional space coordinates. Specifically, they first established a mapping relationship between the pixel coordinate system and the physical space coordinate system, and calculated the length unit of each pixel in the actual space. Then, combined with the pixel information of the wave area in the segmented image, the coordinates (x, y, z) of these pixel points in three-dimensional space were reversely calculated. Among them, the z value represents the actual vertical height of the point.

[0347] By statistically analyzing the height z values ​​of all wave pixels in the entire segmented image, the department obtained the height distribution of the wave scene. The data showed that in this set of images to be tested, the maximum height of the waves was 4.6 meters and the average height was 2.8 meters. This information can not only be directly output, but also used for further analysis and forecasting of wave characteristics.

[0348] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for detecting sea wave height based on semantic segmentation, characterized in that: The following steps are involved: S10, acquiring a plurality of wave image data through a nearshore camera; S20, preprocessing the acquired wave image, including denoising, correction, and cropping, to obtain a plurality of preprocessed images; S30, obtaining the wave pixel-level label data of the multiple pre-processed images to form a training data set, and using the training data set to train a GS-SegFormer semantic segmentation model, wherein the GS-SegFormer semantic segmentation model uses SCConv to improve the self-attention mechanism, uses the Ghost module to process the multi-level characteristics of the MiT encoder, and introduces an improved multi-level SimAM attention mechanism; S40, obtaining a wave image captured by a nearshore camera of the wave height to be measured, recording it as the image to be measured, and preprocessing the image to be measured to obtain a preprocessed image to be measured; S50, using the trained GS-SegFormer model to segment the pre-processed image to be tested, to obtain a segmented image of the wave boundary at the pixel level of the wave; S60, post-processing the segmented image, that is, using a Gaussian filtering method to smooth the wave boundaries in the pre-processed image to be tested to reduce noise; S70, calibrating camera parameters, obtaining internal and external parameters of the camera, and calculating the relationship between the pixels of the image to be measured and the actual length; S80, converting pixel information of the wave area in the segmented image into actual distance according to the post-processed segmented image and camera parameters; S90, based on the converted actual distance information, calculating and outputting the actual height of each wave pixel in the segmented image; The use of SCConv to improve the self-attention mechanism is specifically expressed as follows: For the lth layer (l=1,2,...,L): Q′ l =Conv2d(Q l ,k l ,s l ,p l ,d l ); K′ l =Conv2d(K l ,k l ,s l ,p l ,d l ); V l ′=Conv2d(V l ,k l ,s l ,p l ,d l ); From l =A l In l ′; Y l =LayerNorm(X l +Dropout(Z l )+FFN(Z l )); X l+1 =And l +ResNet(Y l ); In the formula, Represents the input feature map of the lth layer; n l represents the batch size, d l Indicates the number of channels of the input feature map, h l Represents the height of the input feature map, w l Indicates the width of the input feature map; In the formula, represents a learnable weight matrix used to generate queries (Q), keys (K), and values ​​(V); Represents the scaling factor in the attention mechanism, which is used to determine the dimensions of the query / key / value; In the formula, Represents a learnable bias vector, used in conjunction with a weight matrix; k l Indicates the convolution kernel size of the l-th layer SCConv operation; s l represents the step size of the l-th layer SCConv operation; p l Indicates the padding of the l-th layer SCConv operation; d l represents the expansion rate of the l-th layer SCConv operation; represents a learnable position encoding matrix; FFN is a feedforward neural network, defined as FFN(x)=W2ReLU(W1x+b1)+b2; ResNet is a residual network block, defined as ResNet(x)=Conv2d(ReLU(BatchNorm(Conv2d(x)))); Among them, the improved multi-level SimAM attention mechanism is specifically expressed as follows: For layer l: S l =X l ·w l ; In the formula, e l Represents the average value of the l-th layer feature, which is used to calculate the statistical information of the feature; Represents the variance of the l-th layer feature, which is used to measure the discreteness of the feature; λ l represents the mean absolute deviation of the l-th layer features; γ l represents the adaptive parameter of the lth layer, combining the information of variance and mean absolute deviation; α l Represents the learnable parameters of the lth layer, used to adjust γ l Flexibility; l Represents the attention weight of the lth layer, which is used to weight the features; S l Represents the features of the lth layer after SimAM attention weighting; Represents the final SimAM output of the lth layer, combining attention weighting and position encoding; PositionalEncoding (S l ) indicates application to S l The position encoding provides location information for the feature.

2. The method for detecting wave height based on semantic segmentation according to claim 1, characterized in that: Combining SCConv and improving SimAM’s multi-level attention mechanism, the specific expression is as follows: Step 1: Get Z through SCConv self-attention mechanism l Step 2: Calculate Z l Statistics: Step 3: Calculate adaptive parameters: Step 4: Generate attention weights: Step 5. Apply attention weights: Step 6: Final fusion output: Step 7. Combine residual connection and Ghost module: X l+1 =And l +ResNet(Y l )+GhostModule(Y l ); In the formula, Z l is the output of the SCConv self-attention mechanism, with the same dimension as the input feature map X l same; Z l The average value is used to calculate Z l Statistical information; Z l The variance of Z l The degree of dispersion; Z l The mean absolute deviation of For application to Z l The adaptive parameters of Z l Variance and mean absolute deviation information of β l is a learnable parameter used to adjust flexibility; For application to Z l The attention weight of Z′ l is the Z after improved SimAM attention weighting l ; The final fusion attention output combines attention weighting and position encoding; GhostModule (Y l ) is applied to Y l The Ghost module output is used to reduce the amount of calculation and maintain model performance.

3. The method for detecting wave height based on semantic segmentation according to claim 2, characterized in that: The step of obtaining the label data of the wave pixel level of the plurality of pre-processed images specifically comprises: Step 1: Apply the initial segmentation algorithm to the preprocessed image; Step 2: Use edge detection and active contour model to refine the boundary; Step 3: Apply region growing and level set methods to perform region segmentation; Step 4: Use the graph cut algorithm to obtain the global optimal segmentation; Step 5: Apply the pre-trained deep learning model for auxiliary segmentation; Step 6: Use interactive tools for manual correction; Step 7: Apply CRF to refine labels; Step 8: Fuse multi-scale results to obtain the final label.

4. The method for detecting wave height based on semantic segmentation according to claim 3, characterized in that: The step of preprocessing the image to be tested includes denoising, correction, cropping, enhancement, and scaling.

5. The method for detecting wave height based on semantic segmentation according to claim 4, characterized in that: The preprocessed images need to have uniform size and resolution.

6. The method for detecting wave height based on semantic segmentation according to claim 5, characterized in that: The internal parameters include focal length and aperture, and the external parameters include position and angle.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed, they are used to execute the wave height detection method based on semantic segmentation according to any one of claims 1 to 6.

8. A wave height detection system based on semantic segmentation, characterized in that: A computer-readable storage medium comprising the method of claim 7.

Images