Method and device for detecting internal waves in marine sar images, and electronic device

Abstract

The application provides a method and device for detecting oceanic internal waves in a marine SAR image and electronic equipment, wherein the marine SAR image sample is used to label the oceanic internal waves in the marine SAR image by using a rotating box, the rotating box is more fitted to the arc-shaped oceanic internal waves, and the rotating box contains less redundant information; when the oceanic internal wave detection model is trained, less marine SAR image samples are required, and the training time is short; in addition, the updated training data set is used to iteratively train the oceanic internal wave detection model in the training process, so that the training time is further shortened; and since the rotating box contains less redundant information, the accuracy of the finally trained oceanic internal wave detection model is good.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus and electronic device for detecting internal ocean waves in marine SAR images. Background Technology

[0002] Internal ocean waves are an important form of ocean currents, playing a crucial role in transferring energy across large and medium scales, and are a significant cause of ocean mixing and the formation of fine structures. They transmit energy from the upper ocean layer to deeper layers, causing fluctuations in isodense surfaces and altering both the magnitude and direction of sound speed. Satellite remote sensing, represented by SAR (Synthetic Aperture Radar), is an effective means of observing internal ocean waves over large areas.

[0003] In marine SAR images, internal ocean waves exhibit alternating bright and dark stripes. Detecting internal ocean waves using deep learning-based models is a common approach. Horizontal bounding box-based detection labels identified in the image with horizontal rectangles. However, when applied to internal ocean wave detection, this method suffers from drawbacks. Because internal ocean waves are irregularly curved, labeling samples with horizontal bounding boxes introduces a large amount of interference or erroneous information. Therefore, training requires a large amount of training data, is time-consuming, and the model may also misinterpret extraneous information from the image as features of internal ocean waves, thus reducing its accuracy.

[0004] In summary, traditional methods for detecting internal ocean waves in marine SAR images suffer from technical problems such as long model training time and poor model accuracy. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method, apparatus and electronic device for detecting ocean internal waves in ocean SAR images, so as to alleviate the technical problems of long model training time and poor model accuracy in existing methods for detecting ocean internal waves in ocean SAR images.

[0006] In a first aspect, embodiments of the present invention provide a method for detecting internal ocean waves in ocean SAR images, comprising:

[0007] Ocean SAR image samples are acquired and divided into training dataset, validation dataset and test dataset. The ocean SAR image samples are obtained by annotating ocean internal waves in ocean SAR images using a rotated bounding box.

[0008] The ocean internal wave detection model is trained using the ocean SAR image samples to obtain the trained ocean internal wave detection model. When the number of training iterations using the training dataset reaches a preset number, the current ocean internal wave detection model is tested using the validation dataset every first preset number of training iterations. The training dataset is updated based on the first test result and a preset training dataset update strategy, and the updated training dataset is used as the training dataset to continue training the current ocean internal wave detection model. Simultaneously, during the training process using the training dataset, the current ocean internal wave detection model is tested using the test dataset every second preset number of training iterations. The optimal weights of the current ocean internal wave detection model are saved based on the second test result until the number of training iterations reaches a preset total number, thus obtaining the target optimal weights. This completes one full training iteration of the ocean internal wave detection model.

[0009] The trained ocean internal wave detection model is used to detect ocean internal waves in the ocean SAR image to be detected, and the ocean internal wave detection results in the ocean SAR image to be detected are obtained.

[0010] Furthermore, acquiring marine SAR image samples includes:

[0011] Acquire initial marine SAR image samples, wherein the initial marine SAR image samples are obtained manually by labeling the internal ocean waves in each initial marine SAR image using a rotating frame based on the initial marine SAR images under different polarization modes, and the proportion of the internal ocean waves in the rotating frame to the rotating frame is a preset proportion;

[0012] Each initial marine SAR image sample is cropped according to a preset step size and a preset size, and the rotation frame in each cropped sub-initial marine SAR image sample is adjusted to obtain the marine SAR image sample.

[0013] Furthermore, the training dataset is updated based on the obtained first test results and the preset training dataset update strategy, including:

[0014] If the first test result is that the false detection rate is lower than the first preset threshold, then the current weight of the current ocean internal wave detection model is retained, and a first preset number of negative samples and a first preset number of positive samples are added to the training dataset to obtain the updated training dataset.

[0015] If the first test result is that the false detection rate is higher than the second preset threshold, then the current weight of the current ocean internal wave detection model is changed to the weight before the first preset number of training times, and the number of negative samples in the training dataset is reduced by the second preset number, thereby obtaining the updated training dataset.

[0016] If the first test result shows that the false detection rate is between the first preset threshold and the second preset threshold, then the current weights of the current ocean internal wave detection model are retained, and the training dataset is used as the updated training dataset.

[0017] Furthermore, based on the obtained second test results, the optimal weights of the current ocean internal wave detection model are saved, including:

[0018] If the second test result shows that the false detection rate is the lowest compared to the previous false detection rate, then the optimal weight of the current ocean internal wave detection model is saved.

[0019] Furthermore, the trained ocean internal wave detection model is obtained by performing at least two complete training sessions on the ocean internal wave detection model, and the two test results corresponding to the two sets of target optimal weights obtained from the last two complete training sessions are the same.

[0020] Furthermore, the ocean internal wave detection model includes: ResNet50, a feature pyramid module connected to the ResNet50, a RiRoI Align module connected to the ResNet50, a depthwise separable convolution module connected to the feature pyramid module and the RiRoI Align module, a channel spatial attention mechanism module connected to the depthwise separable convolution module, and a softmax module connected to the channel spatial attention mechanism module.

[0021] The input features are used to generate a correlation coefficient matrix through the 3D convolution module of the channel spatial attention mechanism module. The correlation coefficient matrix is ​​then multiplied element-wise with the input features. Finally, the input features and the multiplied features are added together to obtain the filtered features after being filtered by the channel spatial attention mechanism module.

[0022] Furthermore, the trained ocean internal wave detection model is used to detect ocean internal waves in the ocean SAR image to be detected, including:

[0023] The ResNet50 extracts the deep features of internal ocean waves in the ocean SAR image to be detected, and obtains the deep features.

[0024] The feature pyramid module performs multi-scale feature extraction on the deep features to obtain rotated features;

[0025] The RiRoI Align module extracts features from the deep features to obtain rotation-invariant features;

[0026] The depthwise separable convolution module processes the sum of the rotation features and the rotation-invariant features to obtain the processed features;

[0027] The channel spatial attention mechanism module filters the processed features to obtain filtered features.

[0028] The softmax module normalizes the filtered features to obtain the category score of each pixel, and then obtains the bounding box and confidence score of the ocean internal wave in the ocean SAR image to be detected. The bounding box and confidence score of the ocean internal wave in the ocean SAR image to be detected are the detection results of the ocean internal wave in the ocean SAR image to be detected.

[0029] Secondly, embodiments of the present invention also provide a device for detecting internal ocean waves in ocean SAR images, comprising:

[0030] An acquisition unit is used to acquire marine SAR image samples and divide the marine SAR image samples into a training dataset, a validation dataset, and a test dataset. The marine SAR image samples are obtained by annotating the internal ocean waves in the marine SAR image using a rotating bounding box.

[0031] The training unit is used to train the ocean internal wave detection model using the ocean SAR image samples to obtain the trained ocean internal wave detection model. When the number of training iterations using the training dataset reaches a preset number, the current ocean internal wave detection model is tested once using the validation dataset every first preset number of training iterations. The training dataset is updated based on the first test result and a preset training dataset update strategy, and the updated training dataset is used as the training dataset to continue training the current ocean internal wave detection model. Simultaneously, during the training process using the training dataset, the current ocean internal wave detection model is tested once using the test dataset every second preset number of training iterations. The optimal weight of the current ocean internal wave detection model is saved based on the second test result until the number of training iterations reaches a preset total number, thus obtaining the target optimal weight. This completes one full training iteration of the ocean internal wave detection model.

[0032] The ocean internal wave detection unit is used to perform ocean internal wave detection on the ocean SAR image to be detected using the trained ocean internal wave detection model, and obtain the ocean internal wave detection result in the ocean SAR image to be detected.

[0033] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0034] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to perform the method described in any of the first aspects above.

[0035] In this embodiment of the invention, a method for detecting internal ocean waves in marine SAR images is provided, comprising: acquiring marine SAR image samples and dividing the marine SAR image samples into a training dataset, a validation dataset, and a test dataset, wherein the marine SAR image samples are obtained by annotating the internal ocean waves in the marine SAR images using a rotated bounding box; training an internal ocean wave detection model using the marine SAR image samples to obtain a trained internal ocean wave detection model, wherein when the number of training iterations of the internal ocean wave detection model using the training dataset reaches a preset number, the current internal ocean wave detection model is tested once using the validation dataset every first preset number of training iterations, and the model is updated based on the obtained first test result and a preset training dataset update strategy. The training dataset is updated, and the updated training dataset is used to continue training the current ocean internal wave detection model. Simultaneously, during the training process using the training dataset, the current ocean internal wave detection model is tested using the test dataset every second preset training iteration. The optimal weights of the current ocean internal wave detection model are saved based on the second test results. This process continues until the preset total number of training iterations is reached, yielding the target optimal weights. This completes one full training iteration of the ocean internal wave detection model. The trained ocean internal wave detection model is then used to detect ocean internal waves in the ocean SAR image to be detected, yielding the ocean internal wave detection results in the ocean SAR image. As described above, in the method for detecting internal ocean waves in marine SAR images of the present invention, the internal ocean waves in the marine SAR images are labeled using a rotating frame. The rotating frame fits the arc-shaped internal ocean waves better and contains less redundant information. When training the internal ocean wave detection model, fewer marine SAR image samples are required, and the training time is short. In addition, during the training process, an updated training dataset is used to iteratively train the internal ocean wave detection model, which further shortens the training time. Because the rotating frame contains less redundant information, the accuracy of the final trained internal ocean wave detection model is good, thus alleviating the technical problems of long model training time and poor model accuracy in existing methods for detecting internal ocean waves in marine SAR images. Attached Figure Description

[0036] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1A flowchart illustrating a method for detecting internal ocean waves in marine SAR images, provided in an embodiment of the present invention;

[0038] Figure 2 A flowchart illustrating the training of the ocean internal wave detection model provided in this embodiment of the invention;

[0039] Figure 3 This is a schematic diagram of the structure of the ocean internal wave detection model provided in an embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram of the channel spatial attention mechanism module provided in an embodiment of the present invention;

[0041] Figure 5 This is a schematic diagram comparing the test results of two models provided in an embodiment of the present invention;

[0042] Figure 6 A schematic diagram of a device for detecting internal ocean waves in marine SAR images provided in an embodiment of the present invention;

[0043] Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0044] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Existing methods for detecting internal ocean waves in marine SAR images suffer from problems such as long model training time and poor model accuracy.

[0046] Based on this, in the ocean internal wave detection method of the present invention, the ocean SAR image samples are labeled with rotating frames to annotate the ocean internal waves in the ocean SAR image. The rotating frames fit the arc-shaped ocean internal waves better and contain less redundant information. When training the ocean internal wave detection model, fewer ocean SAR image samples are required and the training time is short. In addition, during the training process, the updated training dataset is used to iteratively train the ocean internal wave detection model, which further shortens the training time. Since the rotating frames contain less redundant information, the accuracy of the ocean internal wave detection model after training is good.

[0047] To facilitate understanding of this embodiment, a method for detecting internal ocean waves in marine SAR images disclosed in this embodiment of the invention will first be described in detail.

[0048] Example 1:

[0049] According to an embodiment of the present invention, an embodiment of a method for detecting internal ocean waves in an ocean SAR image is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0050] Figure 1 This is a flowchart of a method for detecting internal ocean waves in marine SAR images according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes the following steps:

[0051] Step S102: Obtain marine SAR image samples and divide the marine SAR image samples into training dataset, validation dataset and test dataset. The marine SAR image samples are obtained by annotating the internal ocean waves in the marine SAR image using a rotating bounding box.

[0052] Specifically, when dividing marine SAR image samples into training datasets, validation datasets, and test datasets, they are divided into training datasets, validation datasets, and test datasets in a ratio of 8:1:1.

[0053] Step S104: The ocean internal wave detection model is trained using ocean SAR image samples to obtain the trained ocean internal wave detection model. When the number of training iterations using the training dataset reaches a preset number, the current ocean internal wave detection model is tested once using the validation dataset every first preset number of training iterations. The training dataset is updated based on the first test result and the preset training dataset update strategy, and the updated training dataset is used as the training dataset to continue training the current ocean internal wave detection model. Simultaneously, during the training process using the training dataset for the ocean internal wave detection model / current ocean internal wave detection model, the current ocean internal wave detection model is tested once using the test dataset every second preset number of training iterations. The optimal weight of the current ocean internal wave detection model is saved based on the second test result until the preset total number of training iterations is reached, thus obtaining the target optimal weight. This completes one full training iteration of the ocean internal wave detection model.

[0054] Specifically, before training, the necessary runtime environment for the ocean internal wave detection model is configured, including PyTorch 1.8, mmdet 2.25, and mmcv-full 1.6.0. Then, the ocean internal wave detection model is deployed. Next, the training parameters of the ocean internal wave detection model are set, with Bachsize set to 8, the initial learning rate set to 0.003, and the network parameters updated using the Adam optimization algorithm. The total number of training epochs is 1000, and the model weights are saved every 10 epochs. The model is then trained using ocean SAR image samples based on the set training parameters.

[0055] The inventors considered that due to the complexity and variability of ocean phenomena, the ocean internal wave detection model is easily affected by other similar ocean phenomena (such as ocean fronts, mesoscale eddies, etc.), leading to fitting errors. Therefore, during training, after 200 training rounds (i.e., when the training dataset has been used to train the ocean internal wave detection model to a preset number of times), every 50 rounds (i.e., the first preset number of training rounds), the current ocean internal wave detection model is tested using the validation dataset. The training dataset is then updated based on the results of the first test (e.g., ...). Figure 2 As shown, the updated training dataset is used as the training dataset to continue training the current ocean internal wave detection model. At the same time, in order to obtain the target optimal weights, during the training process of the ocean internal wave detection model / current ocean internal wave detection model using the training dataset, the model effect is verified by using the test dataset every 10 training rounds (i.e., every second preset training interval, the current ocean internal wave detection model is tested once using the test dataset). The optimal weights of the current ocean internal wave detection model are saved based on the obtained second test results until the preset total number of training rounds is reached, and the target optimal weights are obtained. In this way, a complete training of the ocean internal wave detection model is completed.

[0056] Step S106: The trained ocean internal wave detection model is used to detect ocean internal waves in the ocean SAR image to be detected, and the ocean internal wave detection results in the ocean SAR image to be detected are obtained.

[0057] In this embodiment of the invention, a method for detecting internal ocean waves in marine SAR images is provided, comprising: acquiring marine SAR image samples and dividing the marine SAR image samples into a training dataset, a validation dataset, and a test dataset, wherein the marine SAR image samples are obtained by annotating the internal ocean waves in the marine SAR images using a rotated bounding box; training an internal ocean wave detection model using the marine SAR image samples to obtain a trained internal ocean wave detection model, wherein when the number of training iterations of the internal ocean wave detection model using the training dataset reaches a preset number, the current internal ocean wave detection model is tested once using the validation dataset every first preset number of training iterations, and the model is updated based on the obtained first test result and a preset training dataset update strategy. The training dataset is updated, and the updated training dataset is used to continue training the current ocean internal wave detection model. Simultaneously, during the training process using the training dataset, the current ocean internal wave detection model is tested using the test dataset every second preset training iteration. The optimal weights of the current ocean internal wave detection model are saved based on the second test results. This process continues until the preset total number of training iterations is reached, yielding the target optimal weights. This completes one full training iteration of the ocean internal wave detection model. The trained ocean internal wave detection model is then used to detect ocean internal waves in the ocean SAR image to be detected, yielding the ocean internal wave detection results in the ocean SAR image. As described above, in the method for detecting internal ocean waves in marine SAR images of the present invention, the internal ocean waves in the marine SAR images are labeled using a rotating frame. The rotating frame fits the arc-shaped internal ocean waves better and contains less redundant information. When training the internal ocean wave detection model, fewer marine SAR image samples are required, and the training time is short. In addition, during the training process, an updated training dataset is used to iteratively train the internal ocean wave detection model, which further shortens the training time. Because the rotating frame contains less redundant information, the accuracy of the final trained internal ocean wave detection model is good, thus alleviating the technical problems of long model training time and poor model accuracy in existing methods for detecting internal ocean waves in marine SAR images.

[0058] The above provides a brief overview of the method for detecting internal ocean waves in marine SAR images according to the present invention. The specific details involved are described in detail below.

[0059] In an optional embodiment of the present invention, acquiring marine SAR image samples specifically includes the following steps:

[0060] (1) Obtain initial marine SAR image samples, wherein the initial marine SAR image samples are obtained by manually annotating the internal waves of the ocean in each initial marine SAR image using a rotating frame based on the initial marine SAR images under different polarization modes, and the proportion of the internal waves of the ocean in the rotating frame to the rotating frame is a preset proportion.

[0061] Specifically, the precise location, shape, and angle of internal ocean waves are determined manually based on initial ocean SAR images under different polarization modes (HH, VV, VH). Then, based on the determined location, shape, and angle, rotating bounding boxes are manually used to label the internal ocean waves in all initial ocean SAR images. Since internal ocean waves share extremely similar characteristics with ocean vortices and ocean fronts, to improve the accuracy of model detection, some surrounding environmental information needs to be retained when labeling internal ocean waves. The ratio of internal ocean waves to the environment can be 7:3, which allows the use of some environmental features surrounding the internal ocean waves to highlight their distinction from other ocean phenomena (such as ocean vortices and ocean fronts). Each initial ocean SAR image and its corresponding rotating bounding box label are called an initial ocean SAR image sample.

[0062] (2) Each initial marine SAR image sample is cropped according to a preset step size and a preset size, and the rotation frame in each sub-initial marine SAR image sample obtained by cropping is adjusted to obtain a marine SAR image sample.

[0063] Specifically, each initial marine SAR image sample is cropped into multiple images at a size of 1024 pixels × 1024 pixels (i.e., the preset size). To increase the number of samples, the step size is set to 256 pixels (i.e., the preset step size). At the same time, the corresponding labels are also processed and adjusted to obtain marine SAR image samples.

[0064] In an optional embodiment of the present invention, the training dataset is updated according to the obtained first test result and a preset training dataset update strategy, specifically including the following steps:

[0065] (1) If the first test result is that the false detection rate is lower than the first preset threshold, the current weight of the current ocean internal wave detection model is retained, and the first preset number of negative samples and the first preset number of positive samples are added to the training dataset to obtain the updated training dataset.

[0066] (2) If the first test result is that the false detection rate is higher than the second preset threshold, the current weight of the current ocean internal wave detection model is changed to the weight before the first preset number of training times, and the number of negative samples in the training dataset is reduced by the second preset number, so as to obtain the updated training dataset.

[0067] (3) If the false detection rate of the first test result is between the first preset threshold and the second preset threshold, then the current weight of the current ocean internal wave detection model is retained, and the training dataset is used as the updated training dataset.

[0068] Specifically, the model's fit to internal ocean waves is calculated based on the first test results. If the current internal ocean wave detection model's fit is biased towards internal waves, meaning the false positive and false negative rate (i.e., the false detection rate) is below 30% (i.e., the first preset threshold), then the current weights are retained, and 10% of other ocean phenomena (i.e., the first preset number of negative samples) and 10% of internal ocean wave data (the first preset number of positive samples) are added to further improve the model's ability to avoid errors and its detection accuracy. If the current internal ocean wave detection model's fit is biased towards non-internal waves, meaning the false positive and false negative rate (i.e., the false detection rate) is above 50%, then the weights from 50 rounds ago are loaded to retrain the model, while 20% of other ocean phenomena data (i.e., the second preset number of negative samples) are reduced to correct the model's fit to internal ocean wave features. If the current internal ocean wave detection model's false positive and false negative rate (i.e., the false detection rate) is between 30% and 50%, then the training dataset remains unchanged, and the model continues to be trained.

[0069] In an optional embodiment of the present invention, the optimal weights of the current ocean internal wave detection model are saved based on the obtained second test results, specifically including the following steps:

[0070] If the second test result shows the lowest false detection rate compared to the previous one, then the optimal weights of the current ocean internal wave detection model are saved.

[0071] Specifically, if the false detection rate of the second test is not the lowest compared to the previous false detection rate, then the weight of the current ocean internal wave detection model is not the optimal weight, and the previously saved optimal weight remains the optimal weight.

[0072] In an optional embodiment of the present invention, the trained ocean internal wave detection model is obtained by performing at least two complete training sessions on the ocean internal wave detection model, and the two test results corresponding to the two sets of target optimal weights obtained from the last two complete training sessions are the same.

[0073] Specifically, after completing a full training cycle, the final output model weights are loaded, and the model is tested using a test dataset. The training parameters and model structure are adjusted based on the test results. Then, the model is retrained according to the above steps (i.e., a full training cycle is performed again). The training is iterated until the test results on the test dataset no longer improve. The final model structure and weights are then output, which is the trained ocean internal wave detection model.

[0074] In an optional embodiment of the present invention, such as Figure 3As shown, the ocean internal wave detection model includes: ResNet50, a feature pyramid module connected to ResNet50, a RiRoI Align module connected to ResNet50, a depthwise separable convolution module connected to the feature pyramid module and the RiRoI Align module, a channel space attention mechanism module connected to the depthwise separable convolution module, and a softmax module connected to the channel space attention mechanism module.

[0075] The input features are used to generate a correlation coefficient matrix through the 3D convolution module of the channel space attention mechanism module. The correlation coefficient matrix is ​​then multiplied element-wise with the input features. Finally, the input features and the multiplied features are added together to obtain the filtered features after being filtered by the channel space attention mechanism module.

[0076] Specifically, the ocean internal wave detection model includes: a feature extraction backbone, a rotation feature extraction component, and a feature refinement and classification component. The overall structure is as follows: Figure 3 As shown, the core feature extraction part of the model uses ResNet50 to extract deep features of internal ocean waves in ocean SAR images. The rotation feature extraction part employs the feature pyramid module and the RiRoIAlign module from ReDet, which are used to extract rotation features and ensure rotation invariance, respectively. When processing rotation features, the ordinary RRoI Align can only align features in the spatial dimension, resulting in misalignment in the orientation dimension. RiRoI Align, however, can extract completely rotation-invariant features, mitigating the adverse effects of the diverse shapes and angles of internal ocean waves, allowing the model to learn the characteristics of internal ocean waves more accurately. The features extracted by the feature pyramid module and the RiRoI Align module are summed element-wise to generate new features, which are then input into the feature refinement and classification part. The feature refinement and classification part uses depthwise separable convolution and channel spatial attention mechanisms to refine and filter the features, then normalizes them using the softmax module, and finally performs class scoring and rotation bounding box prediction.

[0077] The structure of the channel spatial attention mechanism module is as follows: Figure 4 As shown, the input features are processed by a 3D convolution module to generate a correlation coefficient matrix. Then, the correlation coefficient matrix is ​​multiplied element-wise with the input features. Finally, the input features and the multiplied features are summed to obtain the filtered features. The entire process is described in the following formula:

[0078] F cs =H cs (F N )☉F N +F N

[0079] Among them, F cs H represents the filtering feature.cs (·) indicates a 3D convolutional module, F N The input features are represented by ⊙, and element-wise multiplication is represented by ⊙. This attention mechanism can filter features in both channel and spatial dimensions with fewer parameters and less computation, thereby reducing erroneous ocean internal wave features, such as ocean fronts and eddies, and thus improving the accuracy of model detection.

[0080] The table below compares the test results of the ocean internal wave detection model (which can be called the rotation detection model) and the horizontal detection model of this invention under the same training strategy. ReDet is the rotation detection model, and YOLOv8l is the horizontal detection model. The comparison shows that the accuracy of the ReDet model is much higher than that of YOLOv8l. For ocean internal wave detection, the training efficiency of the rotation detection model is higher than that of the horizontal detection model. Under the condition of limited data, the rotation detection model is more suitable for ocean internal wave detection.

[0081] Model Correct number False detection count Number of missed detections Real number accuracy False detection rate ReDet 29 3 9 39 0.74 0.08 YOLOv8l 16 4 23 39 0.41 0.10

[0082] Figure 5 The image shows the visual performance of two models (rotation detection model and horizontal detection model) on the test dataset (left image shows YOLOv8l test results, right image shows ReDet test results). It can be seen that the rotation detection model can identify most ocean internal waves more accurately than the horizontal detection model.

[0083] This invention presents a novel method for detecting internal ocean waves in marine SAR images, utilizing a rotating bounding box for the first time. Using a rotating bounding box to label internal ocean waves minimizes the influence of extraneous information, allowing the model to learn the characteristics of internal ocean waves more accurately during training. This results in a rotating detection model (i.e., an internal ocean wave detection model) achieving higher learning efficiency and detection accuracy compared to a horizontal detection model. Furthermore, this invention employs a novel training strategy that modifies the training dataset based on the model's fit to features, preventing incorrect fitting of other ocean phenomena and simultaneously improving training accuracy and efficiency.

[0084] Example 2:

[0085] This invention also provides a device for detecting internal ocean waves in marine SAR images. This device is mainly used to execute the method for detecting internal ocean waves in marine SAR images provided in Embodiment 1 of this invention. The following is a detailed description of the device for detecting internal ocean waves in marine SAR images provided in this invention.

[0086] Figure 6 This is a schematic diagram of a device for detecting internal ocean waves in ocean SAR images according to an embodiment of the present invention, as shown below. Figure 6 As shown, the device mainly includes: an acquisition unit 10, a training unit 20, and an ocean internal wave detection unit 30, wherein:

[0087] The acquisition unit is used to acquire marine SAR image samples and divide the marine SAR image samples into training datasets, validation datasets and test datasets. The marine SAR image samples are obtained by annotating the internal ocean waves in the marine SAR images using a rotated bounding box.

[0088] The training unit is used to train the ocean internal wave detection model using ocean SAR image samples to obtain the trained ocean internal wave detection model. When the number of training iterations using the training dataset reaches a preset number, the current ocean internal wave detection model is tested once using the validation dataset every first preset number of training iterations. The training dataset is updated based on the first test result and a preset training dataset update strategy, and the updated training dataset is used as the training dataset to continue training the current ocean internal wave detection model. Simultaneously, during the training process using the training dataset for the current ocean internal wave detection model, the current ocean internal wave detection model is tested once using the test dataset every second preset number of training iterations. The optimal weights of the current ocean internal wave detection model are saved based on the second test result until the preset total number of training iterations is reached, thus obtaining the target optimal weights and completing one full training iteration of the ocean internal wave detection model.

[0089] The ocean internal wave detection unit is used to perform ocean internal wave detection on the ocean SAR image to be detected using the trained ocean internal wave detection model, and obtain the ocean internal wave detection results in the ocean SAR image to be detected.

[0090] In this embodiment of the invention, a device for detecting internal ocean waves in marine SAR images is provided, comprising: acquiring marine SAR image samples and dividing the marine SAR image samples into a training dataset, a validation dataset, and a test dataset, wherein the marine SAR image samples are obtained by annotating the internal ocean waves in the marine SAR images using a rotated bounding box; training an internal ocean wave detection model using the marine SAR image samples to obtain a trained internal ocean wave detection model, wherein when the number of training iterations of the internal ocean wave detection model using the training dataset reaches a preset number, the current internal ocean wave detection model is tested once using the validation dataset every first preset number of training iterations, and the model is updated based on the obtained first test result and a preset training dataset update strategy. The training dataset is updated, and the updated training dataset is used to continue training the current ocean internal wave detection model. Simultaneously, during the training process using the training dataset, the current ocean internal wave detection model is tested using the test dataset every second preset training iteration. The optimal weights of the current ocean internal wave detection model are saved based on the second test results. This process continues until the preset total number of training iterations is reached, yielding the target optimal weights. This completes one full training iteration of the ocean internal wave detection model. The trained ocean internal wave detection model is then used to detect ocean internal waves in the ocean SAR image to be detected, yielding the ocean internal wave detection results in the ocean SAR image. As described above, in the ocean internal wave detection device of the present invention, the ocean SAR image samples are labeled with rotating frames to represent the ocean internal waves. The rotating frames fit the arc-shaped ocean internal waves better and contain less redundant information. When training the ocean internal wave detection model, fewer ocean SAR image samples are required, and the training time is short. In addition, during the training process, an updated training dataset is used to iteratively train the ocean internal wave detection model, which further shortens the training time. Because the rotating frames contain less redundant information, the accuracy of the finally trained ocean internal wave detection model is good, which alleviates the technical problems of long model training time and poor model accuracy in existing ocean internal wave detection methods in ocean SAR images.

[0091] Optionally, the acquisition unit is further configured to: acquire initial marine SAR image samples, wherein the initial marine SAR image samples are obtained manually by labeling the internal ocean waves in each initial marine SAR image using a rotating frame based on the initial marine SAR images under different polarization modes, and the proportion of the internal ocean waves in the rotating frame to the rotating frame is a preset proportion; crop each initial marine SAR image sample according to a preset step size and a preset size, and adjust the rotating frames in each cropped sub-initial marine SAR image sample to obtain marine SAR image samples.

[0092] Optionally, the training unit is further configured to: if the first test result is that the false detection rate is lower than a first preset threshold, retain the current weights of the current ocean internal wave detection model, and add a first preset number of negative samples and a first preset number of positive samples to the training dataset to obtain an updated training dataset; if the first test result is that the false detection rate is higher than a second preset threshold, change the current weights of the current ocean internal wave detection model to the weights before a first preset number of training iterations, and reduce the number of negative samples in the training dataset by a second preset number to obtain an updated training dataset; if the first test result is that the false detection rate is between the first preset threshold and the second preset threshold, retain the current weights of the current ocean internal wave detection model, and use the training dataset as the updated training dataset.

[0093] Optionally, the training unit is also used to: if the second test result shows the lowest false detection rate compared to the previous false detection rate, then save the optimal weights of the current ocean internal wave detection model.

[0094] Optionally, the trained ocean internal wave detection model is obtained by performing at least two complete training sessions on the ocean internal wave detection model, and the two test results corresponding to the two sets of target optimal weights obtained from the last two complete training sessions are the same.

[0095] Optionally, the ocean internal wave detection model includes: ResNet50, a feature pyramid module connected to ResNet50, a RiRoI Align module connected to ResNet50, a depthwise separable convolution module connected to the feature pyramid module and the RiRoI Align module, a channel spatial attention mechanism module connected to the depthwise separable convolution module, and a softmax module connected to the channel spatial attention mechanism module; wherein, the input features are used to generate a correlation coefficient matrix through the 3D convolution module of the channel spatial attention mechanism module, and the correlation coefficient matrix is ​​multiplied element-wise with the input features, and then the input features and the multiplied features are added together to obtain the filtered features after being filtered by the channel spatial attention mechanism module.

[0096] Optionally, the ocean internal wave detection unit is also used for: extracting deep features of ocean internal waves in the ocean SAR image to be detected using ResNet50 to obtain deep features; performing multi-scale feature extraction on the deep features to obtain rotation features; extracting features from the deep features using the RiRoI Align module to obtain rotation-invariant features; processing the features obtained by summing the rotation features and rotation-invariant features using the depth-separable convolution module to obtain processed features; filtering the processed features using the channel spatial attention mechanism module to obtain filtered features; and normalizing the filtered features using the softmax module to obtain the category score of each pixel, thereby obtaining the rotation bounding box and confidence score of the ocean internal waves in the ocean SAR image to be detected. The rotation bounding box and confidence score of the ocean internal waves in the ocean SAR image to be detected are the detection results of the ocean internal waves in the ocean SAR image to be detected.

[0097] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0098] like Figure 7 As shown in the embodiment of this application, an electronic device 600 includes a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions that can be executed by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 via the bus. The processor 601 executes the machine-readable instructions to perform the steps of the method for detecting internal ocean waves in ocean SAR images as described above.

[0099] Specifically, the memory 602 and processor 601 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 601 runs the computer program stored in the memory 602, it can execute the above-mentioned method for detecting internal ocean waves in ocean SAR images.

[0100] The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 601 or by instructions in software form. The processor 601 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 602, and processor 601 reads the information from memory 602 and, in conjunction with its hardware, completes the steps of the above method.

[0101] Corresponding to the above-described method for detecting internal ocean waves in marine SAR images, this application also provides a computer-readable storage medium storing machine-executable instructions. When these machine-executable instructions are invoked and executed by a processor, they cause the processor to perform the steps of the above-described method for detecting internal ocean waves in marine SAR images.

[0102] The device for detecting internal ocean waves in marine SAR images provided in this application embodiment can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this application embodiment are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0103] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0104] For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0107] If the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the ocean internal wave detection method in the ocean SAR images described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0109] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for detecting internal ocean waves in marine SAR images, characterized in that, include: Ocean SAR image samples are acquired and divided into training dataset, validation dataset and test dataset. The ocean SAR image samples are obtained by annotating ocean internal waves in ocean SAR images using a rotated bounding box. The ocean internal wave detection model is trained using the ocean SAR image samples to obtain the trained ocean internal wave detection model. When the number of training iterations using the training dataset reaches a preset number, the current ocean internal wave detection model is tested using the validation dataset every first preset number of training iterations. The training dataset is updated based on the first test result and a preset training dataset update strategy, and the updated training dataset is used as the training dataset to continue training the current ocean internal wave detection model. Simultaneously, during the training process using the training dataset, the current ocean internal wave detection model is tested using the test dataset every second preset number of training iterations. The optimal weights of the current ocean internal wave detection model are saved based on the second test result until the number of training iterations reaches a preset total number, thus obtaining the target optimal weights. This completes one full training iteration of the ocean internal wave detection model. The trained ocean internal wave detection model is used to detect ocean internal waves in the ocean SAR image to be detected, and the ocean internal wave detection results in the ocean SAR image to be detected are obtained. The process of updating the training dataset based on the obtained first test result and a preset training dataset update strategy includes: If the first test result is that the false detection rate is lower than the first preset threshold, then the current weight of the current ocean internal wave detection model is retained, and a first preset number of negative samples and a first preset number of positive samples are added to the training dataset to obtain the updated training dataset. If the first test result is that the false detection rate is higher than the second preset threshold, then the current weight of the current ocean internal wave detection model is changed to the weight before the first preset number of training times, and the number of negative samples in the training dataset is reduced by the second preset number, thereby obtaining the updated training dataset. If the first test result shows that the false detection rate is between the first preset threshold and the second preset threshold, then the current weight of the current ocean internal wave detection model is retained, and the training dataset is used as the updated training dataset. The ocean internal wave detection model includes: ResNet50, a feature pyramid module connected to ResNet50, a RiRoI Align module connected to ResNet50, a depthwise separable convolution module connected to the feature pyramid module and the RiRoI Align module, a channel spatial attention mechanism module connected to the depthwise separable convolution module, and a softmax module connected to the channel spatial attention mechanism module. The input features are used to generate a correlation coefficient matrix through the 3D convolution module of the channel space attention mechanism module. The correlation coefficient matrix is ​​then multiplied element-wise with the input features. Finally, the input features and the multiplied features are added together to obtain the filtered features after being filtered by the channel space attention mechanism module. The training-based ocean internal wave detection model is used to detect ocean internal waves in the ocean SAR image to be detected, including: The ResNet50 extracts the deep features of internal ocean waves in the ocean SAR image to be detected, and obtains the deep features. The feature pyramid module performs multi-scale feature extraction on the deep features to obtain rotated features; The RiRoI Align module extracts features from the deep features to obtain rotation-invariant features; The depthwise separable convolution module processes the sum of the rotation features and the rotation-invariant features to obtain the processed features; The channel spatial attention mechanism module filters the processed features to obtain filtered features. The softmax module normalizes the filtered features to obtain the category score of each pixel, and then obtains the bounding box and confidence score of the ocean internal wave in the ocean SAR image to be detected. The bounding box and confidence score of the ocean internal wave in the ocean SAR image to be detected are the detection results of the ocean internal wave in the ocean SAR image to be detected.

2. The method according to claim 1, characterized in that, Acquire marine SAR image samples, including: Acquire initial marine SAR image samples, wherein the initial marine SAR image samples are obtained manually by labeling the internal ocean waves in each initial marine SAR image using a rotating frame based on the initial marine SAR images under different polarization modes, and the proportion of the internal ocean waves in the rotating frame to the rotating frame is a preset proportion; Each initial marine SAR image sample is cropped according to a preset step size and a preset size, and the rotation frame in each cropped sub-initial marine SAR image sample is adjusted to obtain the marine SAR image sample.

3. The method according to claim 1, characterized in that, Based on the obtained second test results, save the optimal weights of the current ocean internal wave detection model, including: If the second test result shows that the false detection rate is the lowest compared to the previous false detection rate, then the optimal weight of the current ocean internal wave detection model is saved.

4. The method according to claim 1, characterized in that, The trained ocean internal wave detection model is obtained by performing at least two complete training sessions on the ocean internal wave detection model, and the two test results corresponding to the two sets of target optimal weights obtained from the last two complete training sessions are the same.

5. A device for detecting internal ocean waves in marine SAR images, characterized in that, include: An acquisition unit is used to acquire marine SAR image samples and divide the marine SAR image samples into a training dataset, a validation dataset, and a test dataset. The marine SAR image samples are obtained by annotating the internal ocean waves in the marine SAR image using a rotating bounding box. The training unit is used to train the ocean internal wave detection model using the ocean SAR image samples to obtain the trained ocean internal wave detection model. When the number of training iterations using the training dataset reaches a preset number, the current ocean internal wave detection model is tested once using the validation dataset every first preset number of training iterations. The training dataset is updated based on the first test result and a preset training dataset update strategy, and the updated training dataset is used as the training dataset to continue training the current ocean internal wave detection model. Simultaneously, during the training process using the training dataset, the current ocean internal wave detection model is tested once using the test dataset every second preset number of training iterations. The optimal weight of the current ocean internal wave detection model is saved based on the second test result until the number of training iterations reaches a preset total number, thus obtaining the target optimal weight. This completes one full training iteration of the ocean internal wave detection model. The ocean internal wave detection unit is used to perform ocean internal wave detection on the ocean SAR image to be detected using the trained ocean internal wave detection model, and to obtain the ocean internal wave detection result in the ocean SAR image to be detected. The training unit is further configured to: if the first test result is a false detection rate lower than a first preset threshold, retain the current weights of the current ocean internal wave detection model, and add a first preset number of negative samples and a first preset number of positive samples to the training dataset to obtain the updated training dataset; if the first test result is a false detection rate higher than a second preset threshold, change the current weights of the current ocean internal wave detection model to the weights before a first preset number of training iterations, and reduce the number of negative samples in the training dataset by a second preset number to obtain the updated training dataset; if the first test result is a false detection rate between the first preset threshold and the second preset threshold, retain the current weights of the current ocean internal wave detection model, and use the training dataset as the updated training dataset. The ocean internal wave detection model includes: a ResNet50, a feature pyramid module connected to the ResNet50, a RiRoI Align module connected to the ResNet50, a depthwise separable convolution module connected to the feature pyramid module and the RiRoI Align module, a channel spatial attention mechanism module connected to the depthwise separable convolution module, and a softmax module connected to the channel spatial attention mechanism module. The input features are processed by the 3D convolution module of the channel spatial attention mechanism module to generate a correlation coefficient matrix. This correlation coefficient matrix is ​​then multiplied element-wise with the input features. Finally, the input features and the multiplied features are added together to obtain the filtered features after being filtered by the channel spatial attention mechanism module. The ocean internal wave detection unit is further configured to: extract deep features of ocean internal waves in the ocean SAR image to be detected using ResNet50; extract multi-scale features of the deep features using the feature pyramid module; extract rotation features using the RiRoI Align module; extract rotation-invariant features using the deep features; process the sum of the rotation features and the rotation-invariant features using the depth-separable convolution module; filter the processed features using the channel spatial attention mechanism module; and normalize the filtered features using the softmax module to obtain the category score for each pixel, thereby obtaining the rotation bounding box and confidence score of the ocean internal waves in the ocean SAR image to be detected. The rotation bounding box and confidence score of the ocean internal waves in the ocean SAR image to be detected are the detection results of the ocean internal waves in the ocean SAR image to be detected.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 4.

Images