Internal wave automatic detection method and system based on neural network

Abstract

The invention belongs to the technical field of ocean internal wave research, and discloses an internal wave automatic detection method and system based on a neural network. The method comprises the following steps: selecting a research sea area and collecting ocean internal wave SAR image data based on frequencies of internal waves generated in different sea areas and an available quantity of disclosed SAR radar image data; extracting an internal wave original image based on the obtained ocean internal wave SAR radar image, and cutting the original image into a gray level image; based on each cut grayscale image, automatically marking the ocean internal wave bright and dark fringes by using an improved Canny edge detection algorithm, and outputting a result as an image; and constructing a training data set on the basis of an edge detection result and the cut internal wave SAR image, and then realizing intelligent identification of internal wave stripes on the basis of a CycleGAN model. According to the invention, accurate and efficient identification of internal wave bright and dark stripes can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of ocean internal wave research, and particularly relates to an automatic internal wave detection method and system based on a neural network. Background Art

[0002] Ocean internal waves are non-linear waves widely existing in the entire ocean, which play a decisive role in the horizontal and vertical exchange processes of seawater substances, momentum, and heat; at the same time, the existence of internal waves has a very large destructive force. Due to resonance, internal waves can affect the normal navigation of ships and underwater vehicles at sea, and the associated ocean currents and spikes can damage coastal facilities. Currently, with the development of remote sensing technology, SAR radar remote sensing images have become one of the main methods for observing ocean internal waves. However, the vast majority of research relies on manual methods to identify and extract internal waves, resulting in a large workload and very low work efficiency.

[0003] With the gradual development of deep learning, object detection models represented by Fast-RCNN have been applied to the field of internal wave recognition. However, these models use manually marked labels as training data, still suffering from human omission problems that lead to a reduction in the recognition accuracy of the model; models that generate candidate regions through selective search methods and perform regression classification on the candidate regions have the disadvantages of slow running speed and high computational cost.

[0004] Currently, the existing invention of an automatic ocean internal wave recognition method under high-definition images (publication number CN117975294A, publication date 20240503) constructs an automatic ocean internal wave recognition model under high-definition images and performs ocean internal wave recognition in the target sea area based on satellite remote sensing images. However, this method obtains an ocean internal wave image dataset by label marking the internal wave target area using data marking software, and there is interference from human factors in the recognition of internal waves.

[0005] The existing invention of an ocean internal wave extraction method, system, and program product for complex environmental backgrounds (publication number CN118587476A, publication date 20240903) designs an ocean internal wave extraction method, system, and program product for complex environmental backgrounds. For the polarimetric SAR image data of the ocean, an internal wave automatic extraction network using the U-Net model is used to extract internal waves from the target ocean polarimetric SAR image. However, this method is trained based on manual annotation by users, and there is still interference from human factors.

[0006] The existing invention, a method and device for automatically identifying internal waves in satellite low-light cloud images (publication number CN115393736A, publication date November 25, 2022), proposed a method for detecting internal waves in marine SAR images. A rotating box is used to label the internal waves in the marine SAR images. The rotating box fits better with the arc-shaped marine internal waves. However, there is still a problem that the redundant information in the selected area is too much, resulting in an increase in training costs.

[0007] Through the above analysis, the problems and defects of the existing technology are as follows: Most studies apply supervised models such as Fast-RCNN, U-Net, etc. However, supervised models rely on labeled data and may require domain experts to label, which is time-consuming and expensive; in the data for model training, the box selection method is used to label the internal wave SAR images, there is a lot of redundant information, increasing the model training cost, and there are a large number of internal wave stripes in the selected internal wave area, and the features of a single stripe cannot be extracted. Summary of the Invention

[0008] To overcome the problems in the related technology, the disclosed embodiments of the present invention provide an automatic internal wave detection method and system based on a neural network.

[0009] The technical solution is as follows: An automatic internal wave detection method based on a neural network, the method includes the steps:

[0010] S1, based on the frequency of internal waves occurring in different sea areas and the acquisition volume of the publicly available SAR radar image data, select the research sea area and collect the marine internal wave SAR image data;

[0011] S2, based on the obtained marine internal wave SAR radar image, extract the original internal wave image and crop the original image into a grayscale image;

[0012] S3, based on each cropped grayscale image, use an improved Canny edge detection algorithm to automatically mark the bright and dark stripes of the marine internal waves, and output the result as an image;

[0013] S4, based on the edge detection result and the cropped internal wave SAR image, construct a training dataset, and realize the intelligent recognition of internal wave stripes based on the CycleGAN model; use the original internal wave image of the sea area as a test set to test the CycleGAN model, test the generalization of the CycleGAN model, and use the object detection evaluation index to evaluate the accuracy of internal wave recognition.

[0014] In step S2, extracting the original internal wave image and cropping the original image into a grayscale image includes: cropping each picture into a square png file with a size of 256×256.

[0015] In step S3, the improved Canny edge detection algorithm includes:

[0016] Dividing the image into multiple local small regions, dynamically setting local thresholds according to the statistical characteristics including mean and standard deviation in each region, setting regions of size 4×4, and calculating the mean and standard deviation of the local region in each region:

[0017]

[0018] In the formula, μ is the mean, σ is the standard deviation, k is the region size, and I(x+i,y+j) is the pixel value of the pixel point (x+i,y+j) on the region;

[0019] For each region, dynamically generate a local high threshold T high and a low threshold T low :

[0020] T low = μ - d·σ

[0021] T high = μ + d·σ

[0022] In the formula, d is an adjustable scale factor, and the edge detection result is optimized by adjusting the scale factor to control the output of a complete internal wave edge image.

[0023] In step S4, the training data set is an internal wave intelligent recognition data set, including source domain data composed of SAR radar image data with obvious internal wave stripes, and target domain data composed of edge detection result images;

[0024] Based on the CycleGAN model, in the intelligent recognition of internal wave stripes, add two zero matrices to the gray scale matrix of each source domain image data to form a three-channel image:

[0025]

[0026] In the formula, M expended (x,y,c) is the processed three-channel image, M() is the gray scale matrix, c is the channel index, x is the abscissa corresponding to a single pixel on a certain channel, and y is the ordinate corresponding to a single pixel on a certain channel.

[0027] In step S4, the intelligent recognition of internal wave stripes based on the CycleGAN model includes:

[0028] The CycleGAN model network sets two generators, f and g, which are used to generate internal wave fringes and restore the ocean background respectively. Generator f first learns the feature of the input image data through a three-layer convolutional structure, reconstructs the converted feature through a transposed convolutional layer, and then connects a convolutional layer to restore the output image to a size of 256×256, obtaining the internal wave fringe image from the original radar image. At the same time, two discriminators D m and D n are set to guide the two generators f and g to learn the mapping relationship. Discriminator D m and discriminator D n learn the data features of the source domain M and the target domain N respectively, and determine whether the data generated by the generator conforms to the two data domains. Feature maps of the data in the source domain and the target domain are extracted through four convolutional layers, and the fifth convolutional layer converts the extracted feature maps into discriminant outputs.

[0029] Furthermore, the training of the CycleGAN model includes the optimization of the loss function. Two losses are set for the CycleGAN model, namely the adversarial loss and the cycle consistency loss. Through the adversarial loss, it is ensured that the generator can generate real target domain data:

[0030]

[0031] In the formula, L cyclegan () is the adversarial loss, D N is the discrimination of n and f(m), is the mathematical expectation (Expection) for the data n, specifically the expected value sampled from the real data distribution P data ; similarly is the mathematical expectation for the data m, D N () is the discriminator, f(m) is the fringe result generated by the generator f for m, and m and n are radar images from M and N respectively.

[0032] Furthermore, the generator g and the discriminator D m are set with corresponding adversarial losses, and the cycle consistency loss is introduced to ensure that the data from the source domain M to the target domain N can be as consistent as the original data after returning to the source domain. The expression is:

[0033]

[0034] In the formula, L cycle () is the cycle consistency loss, g(f(m)) is the result after the generator g reconstructs f(m), f(g(n)) is the result after the generator f reconstructs g(n), and ‖‖1 is the 1-norm.

[0035] Furthermore, when the input data m is generated by generator f to obtain f(m) and then further generated by generator g to obtain g(f(m)), similarly, when the input data n is generated by generator g to obtain g(n) and then further generated by generator f to obtain f(g(n)), by training the CycleGAN model, we obtain:

[0036] m≈g(f(m))

[0037] n≈f(g(n)).

[0038] Another object of the present invention is to provide an internal wave automatic detection system based on a neural network. This system implements the above-mentioned internal wave automatic detection method based on a neural network. The system includes:

[0039] An SAR radar image data acquisition module, which is used to select a research sea area and collect SAR images of internal waves in the ocean based on the frequency of internal waves occurring in different sea areas and the acquisition volume of publicly available SAR radar image data

[0040] A raw SAR radar image processing module, which is used to extract the internal wave raw image based on the acquired SAR images of internal waves in the ocean and crop the raw image into a grayscale image;

[0041] An internal wave light and dark stripe detection module, which is used to automatically mark the light and dark stripes of internal waves in the ocean based on each cropped grayscale image using an improved Canny edge detection algorithm and output the result as an image;

[0042] An internal wave stripe intelligent recognition module, which is used to construct a training data set based on the edge detection results and the cropped internal wave SAR images, and implement intelligent recognition of internal wave stripes based on the CycleGAN model; use the original internal wave images of the sea area as a test set to test the CycleGAN model, verify the generalization of the CycleGAN model, and use object detection evaluation metrics to evaluate the accuracy of internal wave recognition.

[0043] Furthermore, this system is carried on a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the functions in the above-mentioned internal wave automatic detection system based on a neural network can be realized.

[0044] Combining all the above technical solutions, the beneficial effects of the present invention are:

[0045] (1) Based on the imaging mechanism characteristics of internal waves on SAR images, the present invention uses an improved Canny edge detection algorithm to detect internal wave light and dark stripes for marine internal wave SAR radar images of four different sea areas, and optimizes the detection results by adjusting the edge strength pixel threshold. Then, an improved CycleGAN neural network is used to train the model based on the internal wave edge detection images and the original internal wave images of the four sea areas as training sets, and the original internal wave images of some sea areas are used as test sets, so that the model can realize accurate and efficient recognition of internal wave light and dark stripes.

[0046] (2) The present invention proposes to achieve accurate conversion and feature extraction from satellite images to internal wave streak images through unsupervised learning of generative adversarial networks. This method can greatly improve the efficiency and accuracy of internal wave streak recognition, reduce the reliance on manual annotation, and significantly reduce the recognition cost. At the same time, the present invention provides data support for the fields of ocean dynamics research, internal wave propagation monitoring, marine environmental assessment, etc., and has important commercial value, especially in the market potential in ocean monitoring, energy development and military applications.

[0047] Domestic and foreign research on internal wave streak recognition mostly uses traditional image processing methods or deep learning technology based on supervised learning, but these methods have problems such as limited feature extraction ability, large data requirements, and poor applicability. The present invention introduces the CycleGAN network to achieve cross-domain unsupervised learning recognition of internal wave streaks, which can perform efficient conversion and recognition without paired data. The present invention breaks through and fills the technical gap in the field of internal wave streak recognition under the condition of missing cross-domain data.

[0048] (3) The identification of internal wave streaks has always been a technical problem in the field of marine remote sensing, which is mainly limited by data scarcity, feature complexity and the lack of adaptability of traditional algorithms. The present invention successfully solves the problem of multi-source image feature domain conversion that traditional methods cannot handle by introducing unsupervised generative adversarial networks, providing a new solution for the identification and analysis of internal wave streaks, and solving this long-standing technical bottleneck. Traditional concepts believe that internal wave streak identification relies on high-quality paired data sets or complex feature engineering, while the present invention proves the feasibility of generative adversarial networks that do not require paired data in the identification of internal wave streaks through the bidirectional mapping capability of the CycleGAN model. The present invention overcomes the industry's technical prejudice against the applicability of unsupervised learning in the field of complex natural phenomenon identification, and significantly expands the scope of application of generative adversarial networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure;

[0050] Figure 1 It is a flowchart of the automatic internal wave detection method based on neural network provided by an embodiment of the present invention;

[0051] Figure 2 It is a schematic diagram of the original SAR radar image provided by an embodiment of the present invention;

[0052] Figure 3 It is a schematic diagram of SAR image cropping provided by an embodiment of the present invention;

[0053] Figure 4 It is an input image in edge detection before optimization provided by an embodiment of the present invention;

[0054] Figure 5 It is an edge detection output image in edge detection before optimization provided by an embodiment of the present invention;

[0055] Figure 6 It is a recognition result image of the original image after the edge detection algorithm is optimized in the edge detection of internal wave stripes provided by an embodiment of the present invention;

[0056] Figure 7 It is an internal wave recognition image at a certain location in a certain sea provided by an embodiment of the present invention;

[0057] Figure 8 It is an internal wave recognition image at a certain location in a certain other sea provided by an embodiment of the present invention;

[0058] Figure 9 It is an internal wave recognition image at a certain location in a certain third sea provided by an embodiment of the present invention;

[0059] Figure 10 It is an internal wave recognition image at a certain location in a certain fourth sea provided by an embodiment of the present invention;

[0060] Figure 11 It is a source domain data image in an example of the automatic detection image dataset provided by an embodiment of the present invention;

[0061] Figure 12 It is a target domain data image in an example of the automatic detection image dataset provided by an embodiment of the present invention;

[0062] Figure 13 It is a schematic diagram of the principle of the CycleGAN model and the structures of the generator and discriminator provided by an embodiment of the present invention;

[0063] Figure 14 It is a comparison diagram of the recognition results of the internal wave SAR radar image at a certain location in a certain sea provided by an embodiment of the present invention. Among them, (a) is the internal wave SAR radar image at a certain location, (b) is the edge detection result image of the existing technology, and (c) is the intelligent recognition result image of the present invention;

[0064] Figure 15It is a comparison chart of the recognition results of the internal wave SAR radar images at a certain location in a certain sea provided by an embodiment of the present invention. Among them, (a) is the internal wave SAR radar image at a certain location, (b) is the edge detection result chart of the prior art, and (c) is the intelligent recognition result chart of the present invention;

[0065] Figure 16 It is a comparison chart of the recognition results of the internal wave SAR radar images at a certain location in a certain sea provided by an embodiment of the present invention. Among them, (a) is the internal wave SAR radar image at a certain location, (b) is the edge detection result chart of the prior art, and (c) is the intelligent recognition result chart of the present invention;

[0066] Figure 17 It is a comparison chart of the recognition results of the internal wave SAR radar images at a certain location in a certain sea provided by an embodiment of the present invention. Among them, (a) is the internal wave SAR radar image at a certain location, (b) is the edge detection result chart of the prior art, and (c) is the intelligent recognition result chart of the present invention. Specific embodiments

[0067] To make the above objects, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0068] The innovation of the present invention lies in: the present invention proposes a method for automatically detecting internal waves in SAR radar images of internal waves in different sea areas based on a neural network; this method uses an improved edge detection algorithm to detect each cropped SAR original image, directly selects the internal waves with obvious bright and dark stripe features, and eliminates redundant background field information, avoiding human factor interference and reducing the model training cost; at the same time, the unsupervised model CycleGAN model is applied to train the edge detection results and the original SAR image dataset, without relying on the quantity of manually marked and annotated data, and automatically mines the potential connections in the two datasets.

[0069] The present invention proposes a method for automatically detecting internal waves in SAR radar images of internal waves in different sea areas based on a neural network. This method uses an improved edge detection algorithm to detect each cropped SAR original image, directly selects the internal waves with obvious bright and dark stripe features, and eliminates redundant background field information, avoiding human factor interference and reducing the model training cost;

[0070] This method applies the unsupervised model CycleGAN model to train the edge detection results and the original SAR image dataset, without relying on the quantity of labeled data, and automatically mines the potential connections between the two datasets.

[0071] Example 1, as Figure 1 shown, the automatic internal wave detection method based on neural network provided by the embodiment of the present invention includes:

[0072] S1. Based on the frequency of internal waves occurring in different sea areas and the acquisition volume of publicly available SAR radar image data, select the research sea area and collect SAR image data of ocean internal waves;

[0073] S2. Based on the obtained SAR radar images of ocean internal waves, extract the original internal wave images and crop the original images into grayscale images;

[0074] S3. Based on each cropped grayscale image, use the improved Canny edge detection algorithm to automatically mark the bright and dark stripes of ocean internal waves and output the results as images;

[0075] S4. Based on the edge detection results and the cropped internal wave SAR images, construct a training dataset, and realize the intelligent recognition of internal wave stripes based on the CycleGAN model; use the original internal wave images of the sea area as the test set to test the CycleGAN model, verify the generalization of the CycleGAN model, and use the object detection evaluation index to evaluate the accuracy of internal wave recognition.

[0076] Exemplarily, step S1 includes: SAR radar image data acquisition, retrieving and collecting the sea areas where internal waves often occur and the available internal wave image data.

[0077] Exemplarily, step S2 includes: original SAR radar image processing. Based on the obtained original SAR radar images (mostly in satellite coding format), use SARSpace professional software to extract the original internal wave images and crop the original images into png pictures with a size of 256×256 pixels, so as to facilitate the construction of the neural network dataset; that is, it is convenient for reading and training with the neural network model.

[0078] Exemplarily, step S3 includes: detection of bright and dark stripes of ocean internal waves. For each cropped grayscale image, due to the special imaging mechanism of internal waves on the SAR radar, the internal waves are shown as bright and dark striped features. Specifically, based on the image data preprocessed in step S2, use the improved Canny edge detection algorithm to automatically mark the bright and dark stripes of ocean internal waves. That is, detect the bright and dark stripes of internal waves in the image and output the results as images.

[0079] Exemplarily, step S4 includes: Based on the original internal wave image processing results and edge detection results of steps S2 and S3, an intelligent recognition dataset for ocean internal waves is constructed. The dataset requires two sets of data, one is the source domain data M, and the other is the target domain data N. Corresponding to the present invention, the source domain data consists of SAR radar image data with obvious internal wave stripes, and the target domain data consists of the internal wave feature images in the edge detection results. Based on the CycleGAN model, using the SAR radar image data and edge detection data as training inputs, it is expected that the model can learn the mapping relationship between the two datasets, so that the network can more deeply learn the characteristics of ocean internal waves, realize the mutual conversion of the two data domains, thereby realizing the automatic detection of ocean internal waves, and using object detection metrics for model evaluation.

[0080] As can be seen from the above embodiments, the present invention trains a SAR radar image dataset containing internal waves in multiple sea areas based on a neural network, and can accurately identify the internal wave phenomenon in the sea area. Compared with the existing methods, it can avoid the interference of human factors through algorithms, thereby reducing the research costs of manpower and material resources. At the same time, for the internal wave stripes, line selection rather than box selection is applied, which reduces the model training operation cost and makes the identification of internal waves more straightforward. For high-incidence areas of internal waves, it can provide key information for ship navigation, help ships avoid high-incidence areas of internal waves, and reduce accident risks; at the same time, for the field of internal wave research, it helps to study the ocean dynamic process, deepen the understanding of the internal structure and dynamic mechanism of the ocean, and promote the development of ocean science.

[0081] Exemplarily, step S1 is specifically as follows: For sea areas with a relatively high occurrence frequency of internal waves, collect the publicly available SAR radar data in Sea Area One, Sea Area Two, Sea Area Three, and Sea Area Four of a certain sea, and use special Sarspace software to extract the images. The following are the approximate imaging areas of the collected internal wave SAR images and the original internal wave SAR images, as Figure 2 shown.

[0082] Exemplarily, step S2 is specifically as follows: Since the resolution of the obtained original SAR image is very large, and many areas with internal waves only account for a part of the original image. Considering the computer computing power and image processing efficiency, before carrying out internal wave recognition, the following preprocessing is performed on the image: Each picture is cropped into a square png file with a size of 256×256 (pixels). Figure 3 It is a schematic diagram of SAR image cropping.

[0083] Exemplarily, step S3 is specifically as follows: Based on the Canny edge detection algorithm, taking the processed image in step S2 as the input, perform edge detection on the internal wave stripes. The input image in the edge detection before optimization is as Figure 4As shown, the edge detection output in the edge detection before optimization is as Figure 5 shown.

[0084] Examples of results are as follows: Figure 5 The edge curve of internal waves can be detected in [the figure], but there are still a large number of cluttered curves on the image that are not the edges of internal wave stripes. The detection of these non-edge curves is caused by inappropriate threshold selection for non-maximum suppression processing. In this regard, the traditional algorithm filters the local maximum pixel points x generated in non-maximum suppression by setting two thresholds X1 and X2 (X1 > X2) to achieve the purpose of noise reduction and maintaining edge continuity. During this process, the pixel points are filtered into three sets, the strong edge set S strong and the weak edge set S weak and the discarded edge set S discarded , which are defined as follows:

[0085] {S weak | X1 > x > X2}

[0086] {S strong | x > X1}

[0087] {S discarded | X2 > x}

[0088] Exemplarily, the final strong edge set is the final image edge. However, in the present invention, a large amount of image data is generated starting from the cropped image, and selecting thresholds for each image will consume a large amount of time cost. Therefore, the canny edge detection algorithm is improved herein. The image is divided into multiple local small regions, and local thresholds are dynamically set according to the statistical characteristics (mean, standard deviation) within each region to ensure the adaptability of different regions. Specifically, the present invention sets regions of size 4×4, and calculates the mean and standard deviation of the local region in each region:

[0089]

[0090] In the formula, μ is the mean, σ is the standard deviation, k is the region size, and I(x + i, y + j) is the pixel value of the pixel point (x + i, y + j) on the region;

[0091] For each region, a local high threshold T high and a low threshold T low are dynamically generated:

[0092] T low = μ - d·σ

[0093] T high = μ + d·σ

[0094] Wherein, d is an adjustable scaling factor, and the edge detection result is optimized by adjusting the scaling factor, so as to control the output of a complete internal wave edge image.

[0095] In the edge detection of internal wave fringes, the original image before optimizing the edge detection algorithm is as Figure 4 , and the recognition result of the original image before optimizing the edge detection algorithm in the edge detection of internal wave fringes is as Figure 5 , and the recognition result of the original image after optimizing the edge detection algorithm in the edge detection of internal wave fringes is as Figure 6 .

[0096] Exemplarily, based on the above optimized edge detection algorithm, for the SAR radar image, edge detection is performed on the processed internal wave image in step S2. Some of the internal wave fringe detection results are as follows, as Figure 7 The internal wave recognition map at a certain location in a certain sea, as Figure 8 The internal wave recognition map at a certain location in a certain other sea, as Figure 9 The internal wave recognition map at a certain location in a certain third sea, as Figure 10 The internal wave recognition map at a certain location in a certain fourth sea.

[0097] Exemplarily, the specific step S4 is: according to the SAR radar image data in step S2 above and the internal wave edge detection result in step S3, an internal wave intelligent recognition data set is constructed. The training data required by the CycleGAN model includes two groups: one group is source domain data, and the other group is target domain data. Corresponding to this study, the source domain data consists of the SAR radar image data with obvious internal wave fringes in step S2, and the target domain data is composed of the edge detection result images. As Figure 11 The source domain data in the example of the automatic detection image data set, Figure 12 The target domain data in the example of the automatic detection image data set.

[0098] Furthermore, due to the need for multi-channel input of the model, the two-dimensional gray matrix corresponding to the source domain data is extended to a three-channel data to form an image as shown in Figure 11 - Figure 12 the figure. The present invention innovatively proposes that the specific method is to add two zero matrices to the gray matrix of each source domain image data to form a three-channel image:

[0099]

[0100] Wherein, M expended (x, y, c) is the processed three-channel image, M() is the gray matrix, c is the channel index, x is the abscissa corresponding to a single pixel on a certain channel, and y is the ordinate corresponding to a single pixel on a certain channel.

[0101] Finally, a total of 180 source domain data, 157 target domain data, and some test set data were produced. Since the training input images in the CycleGAN model do not need to be matched, the image data in the target domain are 157 images with good edge detection results, and there are 180 internal wave SAR images as source domain data. In subsequent research, after updating the SAR image data in the invention sea area, the processed image data can be directly added to the source domain data set to increase the number of images in the training set, so as to increase the accuracy of internal wave intelligent recognition.

[0102] For example, by building a CycleGAN model, the internal wave is intelligently identified. The principle of the CycleGAN model and the structure diagram of the generator and discriminator are as follows: Figure 13 shown.

[0103] Among them, the CycleGAN model network sets two generators f and g, which are used to generate internal wave stripes and restore the ocean background respectively. In order to achieve effective learning of the characteristics of ocean internal wave stripes, the generator f first learns the input image data features through a three-layer convolution structure, reconstructs the converted features through a deconvolution layer, and restores the output image to a size of 256×256 through a convolution layer, so as to obtain the internal wave stripe image from the original radar image. At the same time, two discriminators Dm and Dn are set to guide the two generators f and g to learn the mapping relationship. The discriminators Dm and Dn learn the data features of the source domain M and the target domain N respectively, and judge whether the data generated by the generator conforms to the two data domains. The feature maps of the source domain and the target domain are extracted through four convolution layers, and the fifth convolution layer converts the extracted feature maps into discriminant outputs.

[0104] For example, for the CycleGAN model network, the continuous optimization process of the generator and the discriminator is realized. The process of training the network is actually the optimization process of the loss function. Two losses are set for the network, namely, adversarial loss and cycle consistency loss. Through adversarial loss, it is ensured that the generator can generate "real-looking" target domain data:

[0105]

[0106] Where, L cyclegan () To counter the loss, D N For identification of n and f(m), is the mathematical expectation for data n, specifically from the real data distribution P data The expected value of the sample in is the mathematical expectation for data m, D N() is the discriminator, f(m) is the fringe result generated by the generator f for m, and m and n are radar images from M and N respectively.

[0107] Furthermore, the generator g and the discriminator D m also have corresponding adversarial losses. To avoid inaccurate mapping relationships caused by unsupervised training, a cycle consistency loss is introduced to ensure that the data from the source domain M to the target domain N can be as consistent as possible with the original data after returning to the source domain:

[0108]

[0109] In the formula, L cycle () is the cycle consistency loss, g(f(m)) is the result after the generator g reconstructs f(m), f(g(n)) is the result after the generator g reconstructs g(n), and ‖‖1 is the 1-norm.

[0110] Thus, when the input data m passes through the generator f to generate f(m) and then through the generator g to generate g(f(m)), g(f(m)) represents the result of the generator g restoring the fringe detection result f(m) to a radar image result containing the background field; similarly, when the input data n passes through the generator g to generate g(n) and then through the generator f to generate f(g(n)), f(g(n)) represents the result of the generator f restoring the result g(n) of restoring the background field to an internal wave fringe result; through training the model, the following effects are expected to be achieved:

[0111] m≈g(f(m))

[0112] n≈f(g(n))

[0113] Exemplarily, for the constructed image datasets of the source domain M and the target domain N, the training of this model is carried out;

[0114] Exemplarily, the pth weight file after training is exported;

[0115] Exemplarily, for the constructed test set images, the model is tested to achieve accurate identification of internal wave fringes;

[0116] Exemplarily, the present invention builds and trains and tests the model based on the Pytorch framework.

[0117] Example 2, the embodiment of the present invention provides an automatic internal wave detection system based on a neural network, including:

[0118] An SAR radar image data acquisition module, which is used to select a research sea area and collect SAR image data of internal waves in the ocean based on the frequency of internal waves occurring in different sea areas and the acquisition volume of publicly available SAR radar image data

[0119] The original SAR radar image processing module is used to extract the original internal wave image based on the acquired SAR radar image of ocean internal waves, and crop the original image into a grayscale image;

[0120] The ocean internal wave bright and dark stripe detection module is used to automatically mark the bright and dark stripes of ocean internal waves based on each cropped grayscale image using an improved Canny edge detection algorithm, and output the result as an image;

[0121] The ocean internal wave stripe intelligent recognition module is used to construct a training dataset based on the edge detection results and the cropped internal wave SAR image, and realize the intelligent recognition of internal wave stripes based on the CycleGAN model; use the original internal wave image of the sea area as the test set to test the CycleGAN model, verify the generalization of the CycleGAN model, and use the object detection evaluation index to evaluate the accuracy of internal wave recognition.

[0122] After the model training for the above steps, through the input of grayscale images, the model can output an internal wave stripe image without the ocean background, thus realizing the recognition of internal wave stripes in the target sea area. For example Figure 7 - Figure 10 as shown.

[0123] As Figure 7 - Figure 10 shown, the model can well detect and extract the internal wave stripe information in the SAR radar image.

[0124] And conduct accuracy analysis on the results of the above intelligent recognition. In the field of image processing, accuracy is a widely used model evaluation index, which is used to measure the proportion of the overall correct predictions of the model. It is the ratio of the number of all correct predictions of the model (including true positives and true negatives) to the total number of predictions. The accuracy is defined as follows:

[0125]

[0126] In the formula, TP is the number of true positives, TN is the number of false negatives, FP is the number of false positives, TN is the number of true negatives, and the higher the accuracy Accuracy, the better the performance. Specifically in the internal wave recognition problem, the concepts of each parameter are as follows:

[0127] Table 1 Index concepts in internal wave recognition

[0128]

[0129] The following compares the edge detection results of the existing technologies for ocean internal waves with the intelligent recognition results of the present invention, compares the detection results and calculates the recognition accuracy.

[0130] As Figure 14 the comparison diagram of the recognition results of the internal wave SAR radar image at a certain location in a certain sea;Figure 14 Figure (a) in Figure 14 is the SAR radar image of internal waves at a certain position, and figure (b) in Figure 14 is the edge detection result diagram of the prior art. Figure (c) in Figure 15 is the intelligent recognition result diagram of the present invention; such as Figure 15 Figure (a) in Figure 15 is the SAR radar image of internal waves at a certain position, and figure (b) in Figure 15 is the edge detection result diagram of the prior art. Figure (c) in Figure 16 is the intelligent recognition result diagram of the present invention; such as Figure 16 Figure (a) in Figure 16 is the SAR radar image of internal waves at a certain position, and figure (b) in Figure 16 is the edge detection result diagram of the prior art. Figure (c) in Figure 17 is the intelligent recognition result diagram of the present invention; such as Figure 17 Figure (a) in Figure 17 is the SAR radar image of internal waves at a certain position, and figure (b) in Figure 17 is the intelligent recognition result diagram of the present invention.

[0131] Therefore, the recognition accuracy rate of internal waves in the four sea areas is:

[0132] Table 2 Intelligent Recognition Accuracy Rate of Internal Waves

[0133]

[0134]

[0135] By comparing the above figures, it can be seen that the present invention can well detect the light and dark stripe features of internal waves. From the recognition accuracy rate calculated for the edge detection results, it can be known that the accuracy of intelligent recognition is not less than 93%, which can better directly calibrate the ocean internal wave stripes in different sea areas and reduce the interference of human factors in the process of internal wave recognition research.

[0136] The above is only a relatively preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. An automatic internal wave detection method based on a neural network, characterized in that, The method includes the steps of: S1. Select a research sea area and collect SAR image data of internal waves based on the frequencies of internal waves occurring in different sea areas and the acquisition volume of publicly available SAR radar image data; S2. Extract the original internal wave image based on the acquired SAR radar image of internal waves, and crop the original image into a grayscale image; S3. Based on each cropped grayscale image, use an improved Canny edge detection algorithm to automatically mark the bright and dark stripes of internal waves in the ocean, and output the result as an image; S4. Based on the edge detection results and the cropped internal wave SAR image, construct a training dataset, and realize the intelligent recognition of internal wave stripes based on the CycleGAN model; use the original internal wave image of the sea area as a test set to test the CycleGAN model, verify the generalization of the CycleGAN model, and use object detection evaluation metrics to evaluate the accuracy of internal wave recognition.

2. The automatic internal wave detection method based on a neural network according to claim 1, wherein In step S2, when extracting the original internal wave image and cropping the original image into a grayscale image, it includes: cropping each picture into a square png file with a size of 256×256.

3. The automatic internal wave detection method based on a neural network according to claim 1, characterized in that, In step S3, the improved Canny edge detection algorithm includes: Dividing the image into multiple local small regions, dynamically setting local thresholds according to the statistical characteristics including the mean and standard deviation in each region, setting a region with a size of 4×4, and calculating the mean and standard deviation of the local region in each region: where μ is the mean, σ is the standard deviation, k is the region size, and I(x+i,y+j) is the pixel value of the pixel point (x+i,y+j) on the region; For each region, a local high threshold T is dynamically generated high and a low threshold T low : T low = μ - d·σ T high = μ + d·σ where d is an adjustable scale factor, and the edge detection result is optimized by adjusting the scale factor, so as to control the output of a complete internal wave edge image.

4. The automatic internal wave detection method based on neural network according to claim 1, characterized in that, In step S4, the training dataset is an internal wave intelligent recognition dataset, including source domain data composed of SAR radar image data with obvious internal wave stripes, and target domain data composed of edge detection result images; Based on the CycleGAN model, when realizing the intelligent recognition of internal wave stripes, add two layers of zero matrices to the grayscale matrix of each source domain image data to form a three-channel image; Where, M expended (x, y, c) is the processed three-channel image, M() is the grayscale matrix, c is the channel index, x is the abscissa corresponding to a single pixel on a certain channel, and y is the ordinate corresponding to a single pixel on a certain channel.

5. The automatic internal wave detection method based on a neural network according to claim 1, wherein In step S4, realizing the intelligent recognition of internal wave stripes based on the CycleGAN model includes: The CycleGAN model network sets two generators, generator f and generator g, which are used to generate internal wave fringes and restore the ocean background respectively. Generator f first learns the feature of the input image data through a three-layer convolutional structure, reconstructs the transformed feature through a transposed convolutional layer, and then connects a convolutional layer to restore the output image to a size of 256×256, obtaining the internal wave fringe image from the original radar image. At the same time, two discriminators D m and D n are set to guide the two generators, generator f and generator g, to learn the mapping relationship. Discriminator D m and discriminator D n learn the data features of the source domain M and the target domain N respectively, and determine whether the data generated by the generator conforms to the two data domains. The feature maps of the data in the source domain and the target domain are extracted through four convolutional layers, and the fifth convolutional layer converts the extracted feature maps into discriminant outputs.

6. The automatic internal wave detection method based on a neural network according to claim 5, characterized in that The training of the CycleGAN model includes the optimization of the loss function. For the CycleGAN model, two losses are set, namely the adversarial loss and the cycle consistency loss. Through the adversarial loss, ensure that the generator can generate real target domain data: Where L cyclegan () is the adversarial loss, and D N is the discrimination of n and f(m), is the mathematical expectation for data n, is the mathematical expectation for data m, and D N () is the discriminator, f(m) is the stripe result generated by the m generator f, and m and n are radar images from M and N respectively.

7. The automatic internal wave detection method based on neural network according to claim 6, characterized in that Generator g and discriminator D m A corresponding adversarial loss is set, and a cycle consistency loss is introduced to ensure that the data from the source domain M to the target domain N can be as consistent as possible with the original data after returning to the source domain. The expression is as follows: Where L cycle () is the cyclic consistency loss, g(f(m)) is the result after the generator g reconstructs f(m), f(g(n)) is the result after the generator g reconstructs g(n), and ‖‖1 is the 1-norm.

8. The automatic internal wave detection method based on a neural network according to claim 7, characterized in that When the input data m passes through the generator f to generate f(m) and then passes through the generator g to generate g(f(m)), similarly, when the input data n passes through the generator g to generate g(n) and then passes through the generator f to generate f(g(n)), by training the CycleGAN model, obtain: m≈g(f(m)) n≈f(g(n)).

9. An automatic internal wave detection system based on a neural network, characterized in that, The system implements the neural network-based internal wave automatic detection method described in any one of claims 1-8. The system includes: The SAR radar image data acquisition module is used to select the research sea area and collect the original SAR radar image data of ocean internal waves based on the frequency of internal waves occurring in different sea areas and the acquisition volume of the publicly available SAR radar image data. The original SAR radar image processing module is used to extract the internal wave original image based on the acquired ocean internal wave SAR radar image and crop the original image into a grayscale image; The ocean internal wave light and dark stripe detection module is used to automatically mark the ocean internal wave light and dark stripes using an improved Canny edge detection algorithm based on each cropped grayscale image and output the result as an image; The ocean internal wave stripe intelligent recognition module is used to construct a training data set based on the edge detection result and the internal wave SAR image after cropping processing, and realize the intelligent recognition of internal wave stripes based on the CycleGAN model; use the original internal wave image of the sea area as a test set to test the CycleGAN model, test the generalization of the CycleGAN model, and use the object detection evaluation index to evaluate the accuracy of internal wave recognition.

10. The automatic internal wave detection system based on a neural network according to claim 9, wherein This system is carried on a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the functions in the above-mentioned internal wave automatic detection system based on a neural network can be realized.

Images