Ocean internal wave extraction method and device

Abstract

The invention discloses an ocean internal wave extraction method and device, and the method comprises the steps: obtaining an SAR remote sensing image, and intercepting a local image based on the internal wave features in the SAR remote sensing image; the internal wave features comprise internal wave positions, internal wave lengths and internal wave directions; internal wave positions in the local images are labeled, and obtained labeled images serve as a training set; training the convolutional neural network by using the training set until the model converges to obtain an internal wave detection model; and inputting a to-be-detected image into the internal wave detection model to obtain internal wave features of the to-be-detected image. According to the method, the efficiency of internal wave discrimination is improved by training the internal wave detection model, and batch identification of ocean internal waves is realized. The method is high in recognition speed, high in recognition efficiency and high in recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of marine internal wave extraction, and in particular to a method and apparatus for marine internal wave extraction. Background Technology

[0002] Because internal ocean waves can affect ship navigation, oil extraction, and sonar propagation, their extraction has become a pressing problem. With the rapid development of satellite remote sensing and deep learning technologies, image processing and feature extraction have become hot topics in the study of internal ocean waves. The development of deep learning and Synthetic Aperture Radar (SAR) has brought new opportunities for the detection of internal ocean waves. By constructing network structures, complex functions can be simulated, thereby better extracting the essential features of the original data.

[0003] However, existing methods for quickly and effectively extracting internal wave features, such as propagation direction and wavelength, from complex SAR images are often insufficient. Because internal waves exhibit diverse and complex characteristics in SAR images, traditional methods struggle to guarantee high accuracy in feature extraction and pattern recognition, and are easily affected by noise and background interference during the recognition process. Furthermore, traditional methods are insufficient for quickly and accurately extracting features from a large volume of satellite remote sensing imagery, failing to meet the demands for rapid response. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, and storage medium for extracting internal ocean waves, thereby at least solving the above-mentioned technical problems existing in the prior art.

[0005] According to a first aspect of this application, a method for extracting internal ocean waves is provided, the method comprising:

[0006] Acquire SAR remote sensing images, and extract local images based on the internal wave characteristics in the SAR remote sensing images; the internal wave characteristics include internal wave position, internal wave wavelength, and internal wave direction;

[0007] The positions of the internal waves in the local image are labeled, and the labeled images are used as the training set.

[0008] The convolutional neural network is trained using the training set until the model converges, thus obtaining the internal wave detection model;

[0009] The internal wave characteristics of the image to be detected are obtained by inputting the image to be detected into the internal wave detection model.

[0010] In one possible implementation, after acquiring the SAR remote sensing image, the method further includes:

[0011] The SAR remote sensing image is preprocessed.

[0012] In one possible implementation, the preprocessing of the SAR remote sensing image includes:

[0013] The SAR remote sensing image is sequentially subjected to orbit correction, noise removal, radiometric calibration, filtering, and terrain correction.

[0014] In one possible implementation, the step of extracting a local image based on the internal wave characteristics in the SAR remote sensing image includes:

[0015] Edge detection is performed on the SAR remote sensing image to obtain an image containing edge information;

[0016] The Hough transform method is used to perform line detection on the image containing edge information to obtain the Hough transform detection result; the Hough transform detection result is a binary image containing at least one line segment composed of white pixels, where the white pixel line segment refers to a line segment composed of white pixels.

[0017] Morphological operations are performed on the Hough transform detection results to extract the internal wave position, internal wave wavelength, and internal wave direction;

[0018] Based on the internal wave position, internal wave wavelength, and internal wave direction, the SAR remote sensing image is cropped to obtain a local image.

[0019] In one possible implementation, extracting the internal wave wavelength includes:

[0020] Extract all sub-signals from the Hough transform detection result;

[0021] The Hilbert transform is used to calculate the analytic signal corresponding to each sub-signal, thereby obtaining the frequency domain single-sided spectrum.

[0022] The frequency domain single-side spectrum is demodulated to the baseband to obtain the baseband signal;

[0023] The L2 norm of the baseband signal gradient is calculated as an estimate of the bandwidth;

[0024] The sum of the minimum bandwidths is taken as the internal wavelength.

[0025] In one possible implementation, extracting the internal wave direction includes:

[0026] The following method is used to detect linear features in the Hough transform detection results. The expression for the binary image A(x,y) is as follows:

[0027]

[0028] Where θ is the angle, ρ€R is the distance to the origin, and the δ(.) function is used to select the data of the line where ρ-xcosθ-ysinθ=0 is located from the input image. After integration, it is mapped to the extreme points in the Radon domain.

[0029] The binary image is divided into multiple regions, and the internal wave in each region is regarded as a straight line. An elliptic normalized Radon transform is performed on each region. The extreme points in the Radon domain correspond to the bright and dark fringes of the internal wave, and the angle θ corresponding to the maximum point is determined.

[0030] Calculate the average value of angle θ in each region to obtain the propagation direction of the internal wave.

[0031] In one possible implementation, the convolutional neural network is a Mask R-CNN network;

[0032] The Mask R-CNN network includes a multi-layer structure, each layer of which consists of multiple two-dimensional planes. Each two-dimensional plane is a multi-layer neural network composed of multiple independent neurons. The output of each neuron is obtained by multiplying the neuron node features by the input value of the features of the previous layer with preset weights, adding the bias value, and then performing a non-linear transformation.

[0033] The input layer of the Mask R-CNN network performs convolution calculations using convolution operators to form a feature map. It then uses pooling layers to improve the sensitivity of data features. After three convolutions and three poolings, the network is flattened and then passes through fully connected layers and classification layers before reaching the output layer.

[0034] In one possible implementation, the step of marking the positions of internal waves in the local image includes:

[0035] Determine the position of the internal wave in the local image;

[0036] The position of the inner wave is marked by recording the rectangles (x1, y1, x2, y2) at the top left and bottom right corners.

[0037] According to a second aspect of this application, a marine internal wave extraction apparatus is provided, the apparatus comprising:

[0038] The acquisition module is used to acquire SAR remote sensing images and extract local images based on the internal wave characteristics in the SAR remote sensing images; the internal wave characteristics include the internal wave position, internal wave wavelength, and internal wave direction;

[0039] The annotation module is used to annotate the positions of internal waves in the local image, and the annotated image is used as a training set.

[0040] The training module is used to train the convolutional neural network using the training set until the model converges, thereby obtaining the internal wave detection model.

[0041] The detection module is used to input the image to be detected into the internal wave detection model to obtain the internal wave characteristics of the image to be detected.

[0042] According to a third aspect of this application, an electronic device is provided, comprising:

[0043] At least one processor; and

[0044] A memory communicatively connected to the at least one processor; wherein,

[0045] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0046] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.

[0047] According to a fifth aspect of this application, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in this application.

[0048] The technical solution of this application can improve the efficiency of internal wave discrimination, realize batch identification of ocean internal waves, and improve the identification speed, efficiency and accuracy.

[0049] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0050] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, wherein:

[0051] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0052] Figure 1 A schematic diagram illustrating the implementation process of the ocean internal wave extraction method in an embodiment of this application is shown;

[0053] Figure 2 A schematic diagram of the structure of the marine internal wave extraction device in the embodiments of this application is shown;

[0054] Figure 3 A schematic diagram of the composition structure of the electronic device in an embodiment of this application is shown. Detailed Implementation

[0055] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0058] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0060] It should be understood that in the various embodiments of this application, the sequence number of each implementation process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0061] like Figure 1 As shown in the figure, this application provides a method for extracting internal ocean waves, the method comprising:

[0062] S101, acquire a SAR remote sensing image, and extract a local image based on the internal wave characteristics in the SAR remote sensing image; the internal wave characteristics include the internal wave position, internal wave wavelength, and internal wave direction;

[0063] This application utilizes satellite-acquired SAR remote sensing images. It should be noted that the shape of internal waves in SAR images—whether they appear brighter initially and then darker, or vice versa—is influenced by the presence of flares in the imaging area and the polarity of the internal waves themselves. In SAR images, internal waves in flare areas primarily undergo specular reflection, while diffuse reflection plays a much greater role than specular reflection in non-flare areas. Due to the modulation effect of internal waves on the surface flow field, convergent areas precede divergent areas. Consequently, rougher sea surfaces appear in front of smoother sea surfaces. Therefore, smoother flare areas exhibit stronger specular reflection, while rougher flare areas show weaker specular reflection, resulting in internal waves appearing as darker initially and then brighter in the remote sensing image. Conversely, the stronger reflection in non-flare areas is diffuse reflection; rougher non-flare areas exhibit stronger diffuse reflection, while smoother non-flare areas show weaker diffuse reflection, resulting in internal waves appearing as brighter initially and then darker in the remote sensing image.

[0064] S102, the positions of the internal waves in the local image are labeled, and the labeled image is used as the training set;

[0065] Understandably, the width and height of the local images in this application range from several hundred to several thousand pixels, which is beneficial for training data and learning the morphological characteristics of internal waves. In remote sensing images, ocean areas are generally dark, making it difficult to pinpoint the exact location of internal waves in some extracted local images, which is detrimental to the subsequent annotation task. Furthermore, CNN networks perform worse in detecting internal waves in darker images than in images with clearer target regions. Therefore, for these darker images, the image brightness is manually adjusted to achieve clearer observation of internal waves.

[0066] S103, The convolutional neural network is trained using the training set until the model converges to obtain the internal wave detection model;

[0067] This application uses a training set to train a convolutional neural network, and obtains an internal wave detection model when the model converges.

[0068] S104, input the image to be detected into the internal wave detection model to obtain the internal wave characteristics of the image to be detected.

[0069] After obtaining the internal wave detection model, the image to be detected can be directly input into the internal wave detection model to obtain the internal wave characteristics of the image to be detected.

[0070] This application trains a convolutional neural network based on the analysis of a large amount of SAR ocean internal wave data. The network automatically detects regions containing ocean internal waves based on the texture features of these waves in SAR images. The advantage of the convolutional neural network lies in its ability to provide more refined object segmentation results, making it very useful for applications requiring high precision. The average detection speed for a single image is approximately 2 seconds, improving the efficiency of internal wave discrimination and enabling batch recognition of ocean internal waves. It boasts advantages in recognition speed, efficiency, and accuracy.

[0071] In some embodiments, after acquiring SAR remote sensing images, the method further includes:

[0072] The SAR remote sensing image is preprocessed.

[0073] In some embodiments, the preprocessing of the SAR remote sensing image includes:

[0074] The SAR remote sensing image is sequentially subjected to orbit correction, noise removal, radiometric calibration, filtering, and terrain correction.

[0075] Specifically, this application utilizes snappy (a Python API for snap) to automatically preprocess SAR remote sensing images. After orbit correction, noise removal, radiometric calibration, filtering, and terrain correction, based on the detected internal wave crest lines, Hough transform, line detection, and morphological methods are used to intelligently extract characteristic parameters such as internal wave position, time, wavelength, and wave direction.

[0076] In some embodiments, the step of cropping a local image based on the internal wave features in the SAR remote sensing image includes:

[0077] Edge detection is performed on the SAR remote sensing image to obtain an image containing edge information;

[0078] The Hough transform method is used to perform line detection on the image containing edge information to obtain the Hough transform detection result; the Hough transform detection result is a binary image containing at least one line segment composed of white pixels, where the white pixel line segment refers to a line segment composed of white pixels.

[0079] Morphological operations are performed on the Hough transform detection results to extract the internal wave position, internal wave wavelength, and internal wave direction;

[0080] Based on the internal wave position, internal wave wavelength, and internal wave direction, the SAR remote sensing image is cropped to obtain a local image.

[0081] Specifically, in this application, edge detection is performed on SAR remote sensing images to obtain images containing edge information. The Hough transform method is used to perform line detection on the images containing edge information to obtain a binary image containing at least one line segment composed of white pixels, where the white pixel line segment refers to a line segment composed of white pixels. Morphological operations are performed on the binary image to extract the inner wave position, inner wave wavelength, and inner wave direction.

[0082] Specifically, this application employs an edge detector for image preprocessing, transforming the original SAR remote sensing image into an image containing only edges. Due to imperfections in the image or edge detection, some points or pixels may be missing, or noise may cause the boundaries obtained by the edge detector to deviate from the actual boundaries. Therefore, it is impossible to intuitively categorize the detected edges into sets of straight lines, circles, and ellipses. The Hough transform solves this problem by using a voting step in the Hough transform algorithm to find the parameters of the graphic in a complex parameter space, from which the shape of the edge can be determined.

[0083] Line detection represents a line as its midpoint and the directions and distances to its two endpoints, allowing for the simultaneous regression of the midpoint position, line direction, and distances from the midpoint to both sides. This avoids the process of predicting endpoints and then combining them pairwise to predict scores. A general feature extractor is used to extract multi-scale features, which are then processed through different decoding layers for line pixel semantic segmentation, midpoint detection, and displacement vector prediction from the midpoint to the two endpoints. Line semantic segmentation serves as auxiliary information to aid in midpoint localization, displacement vector calculation, and final fusion. Morphological image processing primarily extracts image components that are meaningful for expressing and depicting the shape of regions, enabling subsequent recognition work to capture the most essential shape features of the target object, such as boundaries and connected regions. Techniques like thinning, pixelation, and deburring are also frequently used in image preprocessing and post-processing, providing a powerful supplement to image enhancement techniques. The basic idea of ​​morphology is to use a special structuring element to measure or extract corresponding shapes or features from the input image for further image analysis and target recognition.

[0084] After processing by the above-mentioned intelligent image extraction method, the nonlinear signals contained in the extracted internal wave image—additive noise and multiplicative noise—are removed, and the internal wave is extracted from the ocean background, thereby completing the intelligent extraction of the internal wave position, time, wavelength, and wave direction.

[0085] In some embodiments, the extraction of the internal wave wavelength includes:

[0086] Extract all sub-signals from the Hough transform detection result;

[0087] The Hilbert transform is used to calculate the analytic signal corresponding to each sub-signal, thereby obtaining the frequency domain single-sided spectrum.

[0088] The frequency domain single-side spectrum is demodulated to the baseband to obtain the baseband signal;

[0089] The L2 norm of the baseband signal gradient is calculated as an estimate of the bandwidth;

[0090] The sum of the minimum bandwidths is taken as the internal wavelength.

[0091] Specifically, this application extracts the internal wave wavelength based on the variational mode decomposition method. The core idea of ​​this algorithm is to minimize the sum of the estimated bandwidths of each mode under the constraint of reconstructing the original signal. The bandwidth of each mode is estimated through the following steps:

[0092] The Hilbert transform is used to calculate the analytic signal corresponding to each sub-signal ku, thereby obtaining the frequency domain single-edge spectrum. This step eliminates the influence of the edge. Then, the analytic signal is demodulated to the baseband by multiplying by a complex exponent, and the L2 norm of the baseband signal gradient is calculated as an estimate of the bandwidth.

[0093] After the above process, the constrained variational optimization problem can be expressed as follows:

[0094]

[0095] in, This represents the set of modes u to be searched. k and a set of center frequencies ω k This minimizes the objective function. Construct an analytic signal. Through convolution operation, the real signal u k (t) is converted into its corresponding analytic signal. It is the convolution kernel of the Hilbert transform in the time domain. This indicates that the analytical signal is shifted to baseband to calculate the bandwidth. This involves taking the time derivative of the demodulated baseband signal and then calculating its energy (the square of the L2 norm). According to the properties of the Fourier transform, the time-domain derivative equals the frequency-weighted derivative in the frequency domain. The ultimate goal is to ensure that each mode is as "smooth" as possible after demodulation (i.e., with the narrowest possible bandwidth), while their sum perfectly reconstructs the original signal. To solve this optimization problem, we first construct an augmented Lagrangian function that transforms the above equation into an unconstrained optimization problem, containing two penalty terms: a quadratic term to enhance reconstruction fidelity and a Lagrange multiplier term to ensure the reconstruction condition is met. The resulting augmented Lagrangian function is expressed as follows:

[0096]

[0097] in, This represents the bandwidth constraint term, and the goal of this part is to minimize u in each mode. k The bandwidth of (t). This is a reconstruction error term, representing the goal of ensuring that the sum of all decomposed modes can reconstruct the original signal x(t) as accurately as possible. The optimization objective is to minimize the energy difference between the original signal and the reconstructed signal. <λ(t),x(t)-∑ k u kThe term λ(t) is a Lagrange multiplier term used to impose a strict constraint. It enforces the constraint x(t) - ∑ through the Lagrange multiplier λ(t). k u k (t). This is a standard optimization technique used to transform a constrained problem into an unconstrained problem.

[0098] Solving the above equation can be decomposed into a series of iterative sub-optimization problems, namely, iteratively minimizing the Lagrangian function. In the nth iteration, the mode uk(t) is updated by the following equation:

[0099]

[0100] in, This indicates the target mode currently being solved. This means that when updating the k-th mode, the modes that have already been updated in this iteration (i.e., the 1st, 2nd, ..., k-1th modes) are used. This means that for modalities that have not yet been updated (i.e., the (k+1),...,Kth modalities), their old values ​​from the previous iteration are temporarily used. λ n In this step of updating all modes, the center frequency and Lagrange multipliers remain unchanged from the previous round. They will remain unchanged in all u... k After all updates are complete, perform individual updates. Transform the above minimization problem into the frequency domain and solve it using Wiener filtering to obtain the frequency domain expression of uk(t):

[0101]

[0102] in, Represents mode u k Fourier transform, The power spectral density of this mode describes the distribution of signal energy at different frequencies ω. The denominator represents the total energy of this mode, and the numerator represents the first moment of the power spectrum, which can be understood as a weighted sum of energy densities over frequencies ω.

[0103] In some embodiments, extracting the internal wave direction includes: detecting linear features in the Hough transform detection results using the following method, where the expression for the binary image A(x,y) is:

[0104]

[0105] Where r(θ,ρ) is the result of the Radon transform, representing the sum of the intensities of image A(x,y) along the straight line defined by a given direction θ and distance ρ. The δ(.) function is used to select data from the input image along the straight line where ρ-xcosθ-ysinθ=0, which is then integrated and mapped to the extreme points in the Radon domain.

[0106] The binary image is divided into multiple regions, and the internal wave in each region is regarded as a straight line. An elliptic normalized Radon transform is performed on each region. The extreme points in the Radon domain correspond to the bright and dark fringes of the internal wave, and the angle θ corresponding to the maximum point is determined.

[0107] Calculate the average value of angle θ in each region to obtain the propagation direction of the internal wave.

[0108] Understandably, this application uses Radon transform to extract the propagation direction of the internal wave. Since the internal wave has a curved characteristic, the internal wave image is divided into multiple regions. The internal wave in each region can be approximated as a straight line. An elliptic normalized Radon transform is performed on each region. The extreme points in the Radon domain correspond to the bright and dark fringes of the internal wave. The θ angle corresponding to the maximum point is found, and the average value of the θ angle in each region is calculated to obtain the propagation direction of the internal wave.

[0109] In some embodiments, the convolutional neural network employs a Mask R-CNN network;

[0110] The Mask R-CNN network includes a multi-layer structure, each layer of which consists of multiple two-dimensional planes. Each two-dimensional plane is a multi-layer neural network composed of multiple independent neurons. The output of each neuron is obtained by multiplying the neuron node features by the input value of the features of the previous layer with preset weights, adding the bias value, and then performing a non-linear transformation.

[0111] The input layer of the Mask R-CNN network performs convolution calculations using convolution operators to form a feature map. It then uses pooling layers to improve the sensitivity of data features. After three convolutions and three poolings, the network is flattened and then passes through fully connected layers and classification layers before reaching the output layer.

[0112] It should be noted that the Mask R-CNN network is a multi-layer neural network where each layer consists of multiple two-dimensional planes, and each plane consists of multiple independent neurons. The output of each neuron is the neuron node feature multiplied by the input value of the previous layer's features with certain weights, plus the bias value, and then subjected to a non-linear transformation. The input layer performs convolution calculations to form a feature map, and then performs pooling calculations to improve the sensitivity of data features. After three convolutions and three poolings, it is flattened, and finally passes through a fully connected layer and a classification layer to form the output layer. The model construction process uses internal wave features as labels for training: first, the gradient images are labeled using LabelMe software to generate a labeled dataset, and then the Mask R-CNN model completes feature extraction and machine learning to achieve an intelligent internal wave feature extraction model. Feature element calculations are performed on the internal waves to calculate the wavelength and direction of the internal waves at the corresponding latitude and longitude, thereby completing the intelligent extraction of the internal wave position, wavelength, and direction.

[0113] Mask R-CNN is a deep learning model with convolutional neural network characteristics, demonstrating good performance in image feature extraction and object classification. During convolutional neural network training, the learning rate is a crucial network parameter; a suitable learning rate allows the network model to converge quickly. If the learning rate is set too high, it may accelerate training but also cause the network to oscillate around its optimal state, or even prevent convergence. Conversely, if the learning rate is set too low, while the network model may achieve the optimal value, it may experience local overfitting and slow convergence.

[0114] In some embodiments, the annotation of the inner wave positions in the local image includes:

[0115] Determine the position of the internal wave in the local image;

[0116] The position of the inner wave is marked by recording the rectangles (x1, y1, x2, y2) at the top left and bottom right corners.

[0117] After obtaining a local image that clearly shows the region where the internal wave occurs, the specific location of the internal wave in the image is labeled. Labeling is done by recording the rectangular frames (x1, y1, x2, y2) at the top left and bottom right corners to define the exact location of the internal wave. Since the goal of this study is internal wave detection, no specific classification was performed when labeling the internal waves; all internal waves were labeled with "iw" to generate sample data.

[0118] For individual solitary internal waves and wave packets, this paper can determine the region where the internal wave is located and label the target box based on the edges. However, when internal waves propagate in wave packet groups in SAR remote sensing images, due to the intersecting and overlapping characteristics, it is impossible to label them based on the edges of each wave packet. In addition, there is no previous literature documenting a method for creating wave packet group samples. Therefore, for wave packet groups, we label several intersecting wave packets as a single sample training data and evaluate the effectiveness of wave packet group labeling based on the detection results.

[0119] After labeling all local images, the generated file records the location and category ("iw") of each internal wave sample in the image. Finally, a file is generated to store the data for each local image. At this point, the ocean internal wave sample library is complete, and the internal wave image data can be fed into the Mask R-CNN network model for training to learn internal wave morphological characteristics.

[0120] The ocean internal wave extraction method provided in this application, based on the analysis of a large amount of SAR ocean internal wave data, utilizes a convolutional neural network (Mask R-CNN) to automatically detect regions containing ocean internal waves based on the texture features of ocean internal waves in SAR images. The advantage of Mask R-CNN lies in its ability to provide more refined object segmentation results, which is very useful for applications requiring high accuracy. On average, it detects a single image in approximately 2 seconds on a GPU, improving the efficiency of internal wave discrimination to a certain extent. This enables batch recognition of ocean internal waves, achieving high recognition speed, efficiency, and accuracy.

[0121] like Figure 2 As shown in the figure, this application provides an internal ocean wave extraction device, the device comprising:

[0122] The acquisition module 201 is used to acquire SAR remote sensing images and extract local images based on the internal wave features in the SAR remote sensing images; the internal wave features include the internal wave position, internal wave wavelength, and internal wave direction;

[0123] The annotation module 202 is used to annotate the positions of internal waves in the local image and use the annotated image as a training set;

[0124] Training module 203 is used to train the convolutional neural network using the training set until the model converges, thereby obtaining the internal wave detection model;

[0125] The detection module 204 is used to input the image to be detected into the internal wave detection model to obtain the internal wave characteristics of the image to be detected.

[0126] The working principle of the ocean internal wave extraction device provided in this application is as follows: the acquisition module 201 acquires SAR remote sensing images and extracts local images based on the internal wave features in the SAR remote sensing images; the internal wave features include the internal wave position, internal wave wavelength, and internal wave direction; the annotation module 202 annotates the internal wave positions in the local images and uses the annotated images as a training set; the training module 203 uses the training set to train the convolutional neural network until the model converges to obtain an internal wave detection model; the detection module 204 inputs the image to be detected into the internal wave detection model to obtain the internal wave features of the image to be detected.

[0127] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0128] The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to perform the ocean internal wave extraction method described in this application. The computer instructions are used to cause the computer to perform the ocean internal wave extraction method described in this application.

[0129] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the ocean internal wave extraction method of this application.

[0130] Figure 3 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0131] like Figure 3As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0132] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0133] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the ocean internal wave extraction method. For example, in some embodiments, the ocean internal wave extraction method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the ocean internal wave extraction method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the ocean internal wave extraction method by any other suitable means (e.g., by means of firmware).

[0134] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0136] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0137] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0138] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0139] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0140] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for extracting internal ocean waves, characterized in that, The method includes: Acquire SAR remote sensing images, and extract local images based on the internal wave characteristics in the SAR remote sensing images; the internal wave characteristics include internal wave position, internal wave wavelength, and internal wave direction; The positions of the internal waves in the local image are labeled, and the labeled images are used as the training set. The convolutional neural network is trained using the training set until the model converges, thus obtaining the internal wave detection model; The internal wave characteristics of the image to be detected are obtained by inputting the image to be detected into the internal wave detection model.

2. The method according to claim 1, characterized in that, After acquiring SAR remote sensing images, the following is also included: The SAR remote sensing image is preprocessed.

3. The method according to claim 2, characterized in that, The preprocessing of the SAR remote sensing image includes: The SAR remote sensing image is sequentially subjected to orbit correction, noise removal, radiometric calibration, filtering, and terrain correction.

4. The method according to claim 1, characterized in that, The step of extracting a local image based on the internal wave features in the SAR remote sensing image includes: Edge detection is performed on the SAR remote sensing image to obtain an image containing edge information; The Hough transform method is used to perform line detection on the image containing edge information to obtain the Hough transform detection result; the Hough transform detection result is a binary image containing at least one line segment composed of white pixels, where the white pixel line segment refers to a line segment composed of white pixels. Morphological operations are performed on the Hough transform detection results to extract the internal wave position, internal wave wavelength, and internal wave direction; Based on the internal wave position, internal wave wavelength, and internal wave direction, the SAR remote sensing image is cropped to obtain a local image.

5. The method according to claim 4, characterized in that, Extracting the internal wave wavelength, including: Extract all sub-signals from the Hough transform detection result; The Hilbert transform is used to calculate the analytic signal corresponding to each sub-signal, thereby obtaining the frequency domain single-sided spectrum. The frequency domain single-side spectrum is demodulated to the baseband to obtain the baseband signal; The L2 norm of the baseband signal gradient is calculated as an estimate of the bandwidth; The sum of the minimum bandwidths is taken as the internal wavelength.

6. The method according to claim 4, characterized in that, Extracting the internal wave direction, including: The following method is used to detect linear features in the Hough transform detection results. The expression for the binary image A(x,y) is as follows: Where r(θ,ρ) is the result of Radon transform, representing the sum of the intensity of image A(x,y) on the straight line defined by the given direction θ and distance ρ; the δ(.) function is used to select the data of the straight line where ρ-xcosθ-ysinθ=0 from the input image, and after integration, it is mapped to the extreme points in the Radon domain; The binary image is divided into multiple regions, and the internal wave in each region is regarded as a straight line. An elliptic normalized Radon transform is performed on each region. The extreme points in the Radon domain correspond to the bright and dark fringes of the internal wave, and the angle θ corresponding to the maximum point is determined. Calculate the average value of angle θ in each region to obtain the propagation direction of the internal wave.

7. The method according to claim 1, characterized in that, The convolutional neural network used is the Mask R-CNN network; The Mask R-CNN network includes a multi-layer structure, each layer of which consists of multiple two-dimensional planes. Each two-dimensional plane is a multi-layer neural network composed of multiple independent neurons. The output of each neuron is obtained by multiplying the neuron node features by the input value of the features of the previous layer with preset weights, adding the bias value, and then performing a non-linear transformation. The input layer of the Mask R-CNN network performs convolution calculations using convolution operators to form a feature map. It then uses pooling layers to improve the sensitivity of data features. After three convolutions and three poolings, the network is flattened and then passes through fully connected layers and classification layers before reaching the output layer.

8. The method according to claim 1, characterized in that, The annotation of the internal wave positions in the local image includes: Determine the position of the internal wave in the local image; The position of the inner wave is marked by recording the rectangles (x1, y1, x2, y2) at the top left and bottom right corners.

9. A marine internal wave extraction device, characterized in that, The device includes: The acquisition module is used to acquire SAR remote sensing images and extract local images based on the internal wave characteristics in the SAR remote sensing images; the internal wave characteristics include the internal wave position, internal wave wavelength, and internal wave direction; The annotation module is used to annotate the positions of internal waves in the local image, and the annotated image is used as a training set. The training module is used to train the convolutional neural network using the training set until the model converges, thereby obtaining the internal wave detection model. The detection module is used to input the image to be detected into the internal wave detection model to obtain the internal wave characteristics of the image to be detected.

10. An electronic device, characterized in that, At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images