A mesoscale vortex intelligent identification method and system based on K8s
By adopting the improved method of K8s and deep learning models in mesoscale vortex recognition, the problems of low recognition efficiency and poor generalization are solved, and high-precision and efficient vortex recognition are achieved, which is suitable for big data scenarios in the marine field. And provide domestic open source easy-to-use systems.
Patent Information
- Application Number
- CN202310551874.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-15
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2043-05-15
AI Technical Summary
The existing mesoscale vortex recognition methods have problems such as low recognition efficiency, poor generalization and relying on professional knowledge. They lack domestic open source and efficient and easy-to-use intelligent recognition systems, making it difficult to adapt to big data scenarios in the marine field.
The mesoscale vortex intelligent recognition method based on K8s is adopted to improve the backbone network and attention mechanism of the deep learning model, combined with distributed parallel training, to improve the recognition accuracy and the elastic scaling and load balancing capabilities of the system.
It realizes high-precision recognition of mesoscale vortexes, improves recognition efficiency and system scalability, is suitable for big data scenarios in the marine field, and provides a domestic open source and easy-to-use system.
Smart Images

Figure CN116681960B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of vortex identification, and in particular relates to a mesoscale vortex intelligent identification method and system based on K8s. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] Mesoscale vortices are a common phenomenon in the ocean and an important part of mesoscale phenomena in the ocean. Mesoscale vortices were first discovered in the 1970s. Mesoscale vortices are much larger than ordinary small vortices, and much smaller than large circulations in the ocean, so they are called mesoscale vortices. The movement of mesoscale vortices will continuously affect the horizontal or vertical distribution of indicators such as temperature, salinity and chlorophyll concentration in the ocean. Therefore, the intelligent identification of mesoscale vortices is of great significance to the development of research in various fields such as national marine environmental monitoring, marine energy cycle, national defense and military, and marine underwater navigation safety. It is crucial to enhance our understanding of national marine ecological environment detection, marine energy cycle and ocean dynamics.
[0004] From the 1990s to the present, the identification methods of mesoscale vortices have undergone three major technical iterations, namely: the traditional physical identification method, the support vector machine identification method based on machine learning, and the identification method based on deep learning. The traditional physical identification methods can be roughly divided into three categories: methods based on flow field geometric characteristics, methods based on physical parameters, and hybrid methods based on flow field geometric characteristics and physical parameters:
[0005] (1) The method based on the geometric characteristics of the flow field assumes that the characteristic shape of the vortex is roughly an ellipse, and then uses the contour closing technology for further processing. If the geometric shape of the vortex in the extracted data is irregular or the geometric characteristics of the vortex are not obvious, this method cannot detect the existence of the vortex from the data to be detected, and the recognition efficiency of this method is not high;
[0006] (2) The method based on physical parameters is highly dependent on the parameters adjusted by experts in the marine field. By selecting a set of appropriate thresholds for the region of interest, the size of the threshold selection is directly linked to the final vortex identification accuracy, and the generalization ability of this method is also limited;
[0007] (3) The hybrid method based on flow field geometric characteristics and physical parameters can have the advantages of the first two methods, but its disadvantage is that it requires researchers to have rich ocean-related professional knowledge. In addition, the data to be identified must also be very clean and complete. Ashkezari et al. later proposed using a support vector machine method based on machine learning for vortex identification, but the disadvantage of this method is that it will produce overfitting, which will cause the model to fail to fully learn the vortex characteristics in the data set.
[0008] Non-traditional vortex identification methods mainly rely on machine learning methods, mainly including: support vector machine-based methods. At present, deep learning has performed well in the field of computer vision. Using neural network models in deep learning for vortex identification can effectively improve the accuracy of mesoscale vortex identification and vortex boundary segmentation. For example: Lguensat et al. proposed a model with an encoder-decoder structure for the field of vortex identification. Franz et al. used a combination of deep learning methods and traditional methods to identify vortices. However, the feature extraction capabilities of the above two deep learning-based methods need to be improved, especially for some vortices with smaller radii. The recognition effect is not very good, and they cannot fully utilize the contextual semantic information of the vortex to accurately segment the vortex boundary.
[0009] The inventors found that all of the above methods have some problems. For example, the traditional mesoscale vortex identification method requires researchers to have rich experience in parameter adjustment in the ocean field. The machine learning-based method not only requires manual design of feature extractors, which is time-consuming and labor-intensive, but also the final mesoscale vortex identification effect also has the problems of poor generalization and low recognition efficiency.
[0010] In addition, with regard to the research and development of the mesoscale vortex identification system, after market research, it was found that there is currently no truly domestically produced open source mesoscale vortex intelligent identification system in China. Most of the system function designs are uneven, and the system is based on a single-machine architecture. The system's elasticity, load balancing, and fault tolerance are not high, especially for processing scenes with large amounts of data related to the ocean field. Most systems are only used by internal personnel of research units. For researchers in non-marine fields, if they are involved in research content related to marine mesoscale vortices, it is very difficult to carry out marine-related scientific research without rich professional knowledge in the marine field. Summary of the invention
[0011] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a method and system for intelligent identification of mesoscale vortices based on K8s. The method can not only accurately identify cyclonic vortices and anticyclonic vortices in sea surface height images, but also introduce attention mechanism technology to enable the model to more accurately segment the boundary information of the vortex, effectively solving the problem of poor generalization of mesoscale vortex identification effect in previous methods; based on this method, a mesoscale vortex intelligent identification system based on K8s (Kubernetes) is developed. The system is built based on a K8s cluster. Using K8s not only makes it easier to perform distributed parallel training, but also enables the system running in the container to have stronger elastic scaling, load balancing and fault tolerance, effectively solving the problems of difficult model training and low system identification efficiency caused by the huge amount of data in the ocean field, and at the same time can greatly improve the work efficiency of mesoscale vortex identification of experts or researchers in the ocean field.
[0012] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:
[0013] The first aspect of the present invention provides a mesoscale vortex intelligent identification method based on K8s.
[0014] A mesoscale vortex intelligent identification method based on K8s includes the following steps:
[0015] Obtain an image to be recognized;
[0016] The image to be identified is input into the trained mesoscale vortex identification model based on deep learning. The feature of the image to be identified is extracted through the deep separable convolution in the backbone network Xception of the encoder. The feature map output by the backbone network is weighted based on the CBAM hybrid attention mechanism to obtain the output feature map. The output feature map is input into ASPP, the expansion rate of the dilated convolution is set, and the pooling branch in ASPP is replaced with the deep separable convolution. The feature maps of the dilated convolution and the deep separable convolution are spliced and compressed according to the channel dimension to obtain the feature map output by the encoder.
[0017] The feature map output by the encoder is input into the decoder, and the weights of receptive fields of different sizes are calculated based on the SKNet attention mechanism. The weights are aggregated with feature maps of different scales to obtain the output of SKNet and upsampled. The upsampled feature map is then concatenated with the low-level feature map from the backbone network Xception according to the channel dimension, and the convolution layer is applied to refine the features and restore the size through upsampling to achieve intelligent recognition of mesoscale vortices.
[0018] The second aspect of the present invention provides a mesoscale vortex intelligent identification system based on K8s.
[0019] A mesoscale vortex intelligent identification system based on K8s, comprising:
[0020] The image acquisition module is configured to: acquire an image to be identified;
[0021] The encoder feature extraction module is configured to: input the image to be identified into the trained mesoscale vortex recognition model based on deep learning, extract features of the image to be identified through the deep separable convolution in the backbone network Xception of the encoder, and weight the feature map output by the backbone network based on the CBAM hybrid attention mechanism to obtain the output feature map; input the output feature map into ASPP, set the expansion rate of the dilated convolution, replace the pooling branch in ASPP with the deep separable convolution, splice and compress the feature maps of the dilated convolution and the deep separable convolution according to the channel dimension, and obtain the feature map output by the encoder;
[0022] The decoder intelligent recognition module is configured as follows: input the feature map output by the encoder into the decoder, calculate the weights of receptive fields of different sizes based on the SKNet attention mechanism, aggregate the weights of feature maps of different scales to obtain the output of SKNet and upsample; then concatenate the upsampled feature map with the low-level feature map from the backbone network Xception according to the channel dimension, apply the convolution layer to refine the features and restore the size through upsampling, so as to realize the intelligent recognition of mesoscale vortices.
[0023] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the K8s-based mesoscale vortex intelligent identification method as described in the first aspect of the present invention.
[0024] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the mesoscale vortex intelligent identification method based on K8s as described in the first aspect of the present invention are implemented.
[0025] One or more of the above technical solutions have the following beneficial effects:
[0026] (1) The method designed in the present invention improves the structures of backbone networks such as Xception and ASPP, and applies the advantages of deep separable convolution and dilated convolution to vortex identification, so that the method can identify cyclonic vortices and anticyclonic vortices in sea surface height images with high accuracy, effectively solving the problem of poor generalization of mesoscale vortex identification in previous methods.
[0027] (2) The method designed in the present invention improves the model's feature extraction capability for mesoscale vortices by introducing the CBAM hybrid attention mechanism and the SKNet attention mechanism. Compared with previous methods, the model makes full use of the contextual semantic information and features of the vortex to accurately segment the boundary information of the vortex.
[0028] (3) The K8s-based mesoscale vortex intelligent identification system developed in the present invention is built based on a K8s cluster. The use of K8s not only makes it easier to perform distributed parallel training, but also enables the running system to have stronger elastic scaling, load balancing and fault tolerance capabilities, effectively solving the problems of training difficulties and low system recognition efficiency caused by the huge amount of relevant data in the ocean field.
[0029] (4) The mesoscale vortex intelligent identification system based on K8s developed in the present invention has greatly improved the work efficiency of experts and researchers in the marine field. It is also more user-friendly and more practical for researchers in non-marine fields.
[0030] (5) The mesoscale vortex intelligent identification system based on K8s developed in the present invention is domestically produced, open source and easy to use. Compared with some foreign non-open source systems, the system in the present invention has higher security. It is of great significance to apply it to various fields such as national marine environment monitoring, marine energy cycle, national defense and military, and marine underwater navigation safety in the future.
[0031] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0033] Figure 1 This is a flow chart of the method of the first embodiment.
[0034] Figure 2 This is the modified Xception data processing flow chart.
[0035] Figure 3 This is the modified ASPP data processing flow chart.
[0036] Figure 4 This is the overall architecture diagram of CBAM.
[0037] Figure 5 This is the data processing flow chart of the channel attention module.
[0038] Figure 6 Figure 2 is the data processing flow chart of the spatial attention module.
[0039] Figure 7 This is the data processing flow chart of the SKNet module.
[0040] Figure 8 This is a schematic diagram of the working principle of the system of the second embodiment.
[0041] Fig. 9 The interface diagram of the SSH image vortex online recognition function is automatically selected for the second embodiment.
[0042] Fig.10 This is a diagram of the interface for customizing the vortex online recognition function of uploading an SSH image in the second embodiment. DETAILED DESCRIPTION
[0043] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0044] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.
[0045] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.
[0046] The overall idea proposed by the present invention is:
[0047] The following shortcomings of the prior art:
[0048] (1) The method based on flow field geometric characteristics relies on the fact that the extracted vortices must have regular shapes and obvious geometric characteristics.
[0049] (2) Physical parameter-based methods rely on experts’ extensive oceanographic experience to adjust parameters.
[0050] (3) The hybrid method based on flow field geometric characteristics and physical parameters requires researchers to have rich ocean-related professional knowledge, and the identified data must also be very clean and complete.
[0051] (4) The support vector machine method based on machine learning also requires very complex feature engineering to preprocess the data, and the results of model prediction may lead to overfitting.
[0052] (5) The feature extraction capability of existing deep learning methods needs to be improved, especially for some vortices with smaller radius. Moreover, they cannot fully utilize the contextual semantic information of the vortex to accurately segment the boundary of the vortex.
[0053] (6) After market research, it was found that there is currently no domestically produced open source, efficient and easy-to-use mesoscale vortex intelligent identification system in China, and most systems have low elasticity, load balancing and fault tolerance capabilities, especially for processing large amounts of ocean-related data. In addition, most mesoscale vortex identification systems are not suitable for researchers in non-marine fields who lack experience.
[0054] The present invention proposes a mesoscale vortex intelligent identification method and system based on K8s.
[0055] Firstly, based on the dataset constructed from the remote sensing images of the South China Sea surface height provided by the Copernicus Marine Environment Monitoring Service (CMEMS), a deep learning-based intelligent recognition method for marine mesoscale eddies is designed. By improving the structures of the backbone network Xception and ASPP modules, the advantages of deep separable convolution and dilated convolution are fully applied to eddy recognition, so that the model can identify cyclonic eddies and anticyclonic eddies in SSH images with high accuracy, effectively solving the problem of poor generalization of mesoscale eddy recognition in previous methods. Secondly, the CBAM hybrid attention mechanism and SKNet attention mechanism are introduced to improve the model's feature extraction ability for mesoscale eddies. Compared with previous methods, the model makes full use of the contextual semantic information and features of the eddies to accurately segment the boundary information of the eddies. This method can not only accurately identify cyclonic eddies and anticyclonic eddies in sea surface height images, but also fully utilize the contextual semantic information and features of the eddies to accurately segment the boundary information of the eddies by introducing the attention mechanism technology, effectively solving the problem of poor generalization of mesoscale eddy recognition in previous methods.
[0056] Secondly, based on this method, the present invention develops a mesoscale vortex intelligent identification system based on K8s (Kubernetes), and the system has three main functions: first, the system is built based on the K8s cluster. Using K8s is not only easier to perform distributed parallel training, but also enables the system running in the container to have stronger elastic scaling, load balancing and fault tolerance, effectively solving the problems of model training difficulties and low system recognition efficiency caused by the huge amount of data in the ocean field. Second, the system can greatly improve the work efficiency of mesoscale vortex identification of experts or researchers in the ocean field. Third, the system is of great significance for the future application of research in various fields such as national marine environment monitoring, marine energy cycle, national defense and military, and marine underwater navigation safety.
[0057] DeepLabV3 Plus is a deep learning model for image semantic segmentation proposed by the Google Brain team in 2018. It can classify each pixel in the input image and assign it to the corresponding object category. The DeepLabV3 Plus model mainly includes: encoder, ASPP module, decoder:
[0058] Among them, encoder (Encoder): using pre-trained ResNet, MobileNet or Xception and other classic CNN, the input image is mapped into a low-dimensional feature space. In the present invention, Xception is mainly used as the backbone network. Xception is a deep learning convolutional neural network model. Unlike traditional CNN, it uses depthwise separable convolution to replace traditional convolutional layers to reduce the number of model parameters and the amount of calculation, thereby improving the efficiency and performance of the model. Due to the reduction in computational cost without increasing the number of parameters, the Xception model can be trained and reasoned faster, and has achieved high performance that is comparable to or better than traditional CNN models on multiple data sets. At the same time, it is also widely used in the field of transfer learning for visual tasks, because it has achieved good results in common computer vision data sets.
[0059] ASPP (Atrous Spatial Pyramid Pooling) module: Parallel convolution processing is performed on feature maps of different sampling rates to capture multi-scale feature information and context information to further improve segmentation accuracy.
[0060] Decoder: Combines the high-level semantic information generated by the encoder with the spatial information and upsamples the high-resolution feature map to the corresponding size of the input image, ultimately producing a segmentation mask.
[0061] The DeepLabV3 Plus model performs well on many datasets and has become one of the benchmark methods for many computer vision tasks.
[0062] Embodiment 1
[0063] This embodiment discloses a mesoscale vortex intelligent identification method based on K8s.
[0064] like Figure 1 As shown, a mesoscale vortex intelligent identification method based on K8s includes the following steps:
[0065] Obtain an image to be recognized;
[0066] The image to be identified is input into the trained mesoscale vortex identification model based on deep learning. The feature of the image to be identified is extracted through the deep separable convolution in the backbone network Xception of the encoder. The feature map output by the backbone network is weighted based on the CBAM hybrid attention mechanism to obtain the output feature map. The output feature map is input into ASPP, the expansion rate of the dilated convolution is set, and the pooling branch in ASPP is replaced with the deep separable convolution. The feature maps of the dilated convolution and the deep separable convolution are spliced and compressed according to the channel dimension to obtain the feature map output by the encoder.
[0067] The feature map output by the encoder is input into the decoder, and the weights of receptive fields of different sizes are calculated based on the SKNet attention mechanism. The weights are aggregated with feature maps of different scales to obtain the output of SKNet and upsampled. The upsampled feature map is then concatenated with the low-level feature map from the backbone network Xception according to the channel dimension, and the convolution layer is applied to refine the features and restore the size through upsampling to achieve intelligent recognition of mesoscale vortices.
[0068] The specific technical solutions are as follows:
[0069] (1) Overall model architecture
[0070] The overall architecture of the model is improved based on the architecture style of the semantic segmentation network DeepLabV3 Plus. Figure 1As shown in the figure. The model mainly consists of two parts: encoder and decoder. The encoder module is mainly a feature extractor based on convolutional neural network, which can extract the main features of vortices in the input sea surface height data. The encoder module is mainly composed of a modified Xception as the backbone network and a modified ASPP (Atrous Spatial Pyramid Pooling) structure. The convolutional layers used in the backbone network Xception are mainly depth-separable convolution and dilated convolution. Depth-separable convolution can greatly reduce the number of model parameters while realizing channel and region separation. Dilated convolution can not only expand the receptive field to include larger contextual semantic information, but also enhance the model's ability to extract small-scale vortex features. The decoder module mainly performs a series of upsampling operations on the feature maps output from the encoder. The purpose of the upsampling operation is to restore the edge information of the mesoscale vortex and make the boundary of the identified vortex more refined. In addition, the present invention also introduces the attention mechanism technology to improve the performance of the model. The present invention introduces a lightweight CBAM (Convolutional Block Attention Module) hybrid attention mechanism at the exit of the backbone network and introduces the SKNet (Selective Kernel Networks) attention mechanism at the entrance of the decoder, because the attention mechanism technology can obtain more eddy flow detail information and higher-level features by applying weights, suppressing other useless information. The above is an overview of the overall architecture of the recognition model in the present invention. The following content will introduce the specific implementation of each module in detail.
[0071] (2) Encoder module
[0072] The encoder module is mainly composed of a modified Xception as the backbone network and a modified ASPP (Atrous Spatial Pyramid Pooling) structure. Xception is a structural improvement of Inception V3. The main content of the improvement is to use depthwise separable convolution to replace the multi-scale convolution operation in the original Inception V3. The function of the traditional convolution layer is to learn cross-channel correlation and spatial correlation at the same time, while the Inception structure focuses on decomposing the learning process of cross-channel correlation and spatial correlation, and to a certain extent realizes the decoupling of the learning process of cross-channel correlation and spatial correlation. The depthwise separable convolution not only continues to inherit the decoupling idea of Inception, but also compared with the traditional convolution operation, the depthwise separable convolution can have stronger feature extraction capabilities while greatly reducing the number of parameters and computing costs of the network model.
[0073] The modified backbone network Xception mainly consists of three parts: Entry flow, Middle flow and Exit flow. It contains 18 Xception blocks in total, including 4 Entry flow parts, 12 Middle flow parts and 2 Exit flow parts. The main structure inside each Xception block is realized by residual network combined with depthwise separable convolution.
[0074] The modified Xception is as follows Figure 2 As shown. Compared with the original Xception, the improvements made by the present invention are mainly the following three points: First, the number of layers of depth-separable convolution in the Middle flow is reduced. From the original three layers to the current two layers, the main reason is that for vortices with smaller radius in the input image, too many depth-separable convolution layers in one Flow will cause the model to be unable to detect vortices with smaller radius. Second, the number of repetitions of the Middle Flow part is increased from the original 8 to 12, which enhances the overall feature extraction capability of the backbone network. Third, all maximum pooling operations in the original Xecption are replaced by depth-separable convolutions with a step size of 2, because the maximum pooling operation will cause more serious pixel loss, which is not conducive to segmenting those small-scale vortices, and the depth-separable convolution with a step size of 2 can achieve better results without losing pixels.
[0075] In addition, the present invention also makes some improvements to the ASPP structure in the original encoder. The modified ASPP is as follows: Figure 3 As shown, the improvements made by the present invention mainly include the following two points: first, the expansion rate of the dilated convolution is modified from the original 6, 12, 18 to 6, 12, 12. It is found through experiments that the modified expansion rate makes the recognition effect of vortex better. Second, the pooling branch in the original ASPP structure is replaced by a depth-separable convolution with a step size of 2, which is the same as in the backbone network, because the pooling operation is very unfavorable for segmenting small-scale vortices in the present invention, and the depth-separable convolution with a step size of 2 can be maintained without losing pixels to achieve better results. Then the feature map is spliced according to the channel dimension, and finally the 1x1 convolution is used to reduce the number of channels to 1 to obtain the output of this module.
[0076] (3) Decoder module
[0077] First, the feature map output in the ASPP in the encoder is bilinearly upsampled by 4 times. The present invention also introduces the SKNet attention mechanism before upsampling, and then the feature map output by the attention mechanism is spliced with the low-level feature map from the backbone network according to the channel dimension. Then several 3x3 convolutional layers are applied to refine the features. Finally, a 4-fold bilinear upsampling is performed to restore the original input image size, and finally the intelligent recognition of mesoscale vortices is achieved.
[0078] (4) CBAM Hybrid Attention Mechanism
[0079] As we all know, attention plays an important role in human vision. The obvious characteristic of the human visual system is that people do not try to Figure 1 Instead of processing the entire observed scene, the brain is accustomed to selectively focusing on the part that the brain wants to pay attention to. Similarly, in deep learning, the role of the attention mechanism is to hope that the neural network can automatically learn the features that need to be paid attention to in the input image or text sequence. The main principle of the attention mechanism is to generate a mask through a series of operations of the neural network, and the value in the mask is the score of the point that currently needs attention. From different implementation perspectives, the attention mechanism is divided into: channel attention mechanism, spatial attention mechanism and hybrid attention mechanism. The goal of the present invention is to improve the learning ability of the network model by using the attention mechanism, so that the model can learn the necessary features in the mesoscale vortex and suppress unnecessary features.
[0080] CBAM is a hybrid attention mechanism. It integrates the channel attention mechanism and the spatial attention mechanism. For an input feature map, it first passes through a channel attention module to obtain a weighted result, and then passes through a spatial attention module to obtain the final weighted result. CBAM is a lightweight universal module that can be integrated into any model based on the CNN architecture, and its computational overhead is very low.
[0081] The overall architecture of CBAM is as follows Figure 4 As shown. For a given intermediate feature map F∈R C×H×W As input, CBAM calculates the channel attention map M in turn. c ∈R C×1×1 and the spatial attention map M s ∈R 1×H×W , the entire attention calculation process can be summarized as the following formula:
[0082]
[0083]
[0084] in represents element-wise multiplication. F is the input feature map, and F' is the final output feature map. Next, the specific implementation of each attention module is described separately.
[0085] ① Channel attention module: such as Figure 5 As shown in the figure, first, for a given input feature map, the spatial information of the feature map is aggregated by average pooling and maximum pooling operations, and the average pooling features are generated respectively. And the maximum pooling feature Then, the average pooling feature map and the maximum pooling feature map generated in the previous step are input into a multi-layer perceptron (MLP) with shared parameters. In order to reduce the number of model parameters, the size of the hidden layer is set to R c / r×1×1 , where r is the reduction rate. Finally, the output feature vectors are merged using an element-wise sum operation. The channel attention calculation formula is as follows:
[0086]
[0087] Where σ represents the sigmoid function, W 0 ∈R C / r×C , W 1 ∈R C×C / r The two inputs share the weight W of the MLP 0 and W 1 , and the ReLU activation function is followed by W 0 .
[0088] ②Spatial attention module: such as Figure 6 As shown, first, for a given input feature map, two 2D feature maps are generated by average pooling operation and maximum pooling operation respectively: and Represent the average pooling feature and the maximum pooling feature respectively. Then, the two 2D feature maps are spliced according to the channel dimension, and the spliced feature maps are convolved through a standard convolution operation. Finally, the final spatial attention feature map is activated by the sigmoid function. The spatial attention calculation formula is as follows:
[0089]
[0090] Where σ represents the sigmoid function, f 7×7 Represents a convolution operation with a convolution kernel size of 7×7.
[0091] (5)SKNet module
[0092] SKNet introduces the attention mechanism into the receptive field of convolutional neural networks. The biggest innovation in SKNet is the introduction of SK convolution, which applies the attention mechanism to the receptive field, allowing objects of different sizes to extract features through adaptive weighted combinations of receptive fields of different sizes. SK convolution consists of three parts: Spilt, Fuse, and Select. The Spilt part uses multi-scale receptive fields to extract features. The Fuse part fuses the features extracted from multi-scale receptive fields, and then reduces the dimension through downsampling and fully connected layers to facilitate the calculation of the weights of feature maps of each layer. The Select part mainly aggregates the feature maps of convolution kernels of different scales according to the selection weights to obtain the final output.
[0093] like Figure 7 As shown in the Split part: For any given input feature map, 3*3 and 5*5 convolution operations are performed respectively to obtain feature maps and In order to reduce the number of parameters, the 5*5 convolution operation is implemented by using a 3*3 dilated convolution. Fuse part: First, the results of multiple branches are fused through element-wise summation operation: Then, the fused feature map U is subjected to global average pooling to obtain a one-dimensional vector S, which is then input into the fully connected layer network for dimensionality reduction to obtain a one-dimensional vector Z. Select part: First, the one-dimensional vector Z is subjected to the softmax function to calculate the weights a and b of the receptive fields of different sizes. Then, the feature maps extracted from different receptive fields are respectively and Perform element-wise product operations with a and b respectively. Finally, perform element-wise summation operation on the feature maps extracted from the receptive fields of each scale to obtain the feature map V.
[0094] (6) Experimental process
[0095] The model designed in the present invention is trained in a K8s cluster built on a GPU cloud server provided by the Shanhe Supercomputing Cloud Platform of the National Supercomputing Jinan Center. The operating system version is CentOS 8.2 64bit, the K8s version is 1.24.4, the GPU used is NVIDIA A100, the data set is a data set constructed based on the remote sensing images of the South China Sea surface height provided by the Copernicus Marine Environment Monitoring Service (CMEMS), and the framework used for code writing is TensorFlow. During the model training process, the optimizer selected Adam, and the loss function selected Soft Dice Loss based on the Dice coefficient. The Dice coefficient is particularly suitable as an evaluation indicator for semantic segmentation, which is accurate to the pixel level. The calculation formula of the Dice coefficient is as follows:
[0096]
[0097] Among them, |M| represents the area where the real result appears, |N| represents the area where the model predicts the result, and the value range of dice is 0 to 1. The closer the value is to 1, the better the model segmentation effect is, and the closer the value is to 0, the worse the model segmentation effect is. The corresponding loss function Soft Dice Loss can be extended from the Dice coefficient. The loss function calculation formula is as follows:
[0098]
[0099] After multiple training and parameter adjustments, the epoch was finally set to 100 and the batch size was set to 16. As for the final training results, the precision and recall in the training set both reached 93.68%, and the precision and recall in the validation set both reached 86.48%.
[0100] (7) Introduction to system architecture
[0101] The present invention develops a mesoscale vortex intelligent identification system based on K8s based on the model designed above. The system is a front-end and back-end separation project based on the B / S (Browser / Server) architecture. The front-end interface of the system is developed based on Vue. The back-end API of the system is developed based on the Flask framework, because Flask is a lightweight Web framework written in Python language, and the framework has the advantages of flexibility, lightness, simplicity and ease of use. The functions of this system related to mesoscale vortex identification are mainly provided to users in the form of services. The user groups mainly include researchers who have scientific research needs for mesoscale vortex identification.
[0102] The main functional modules of the system include: user registration and login module, service overview module, case display module, and vortex online identification module. The user registration and login module mainly provides users with login and registration functions to facilitate backend user management. The service overview module is mainly for people in the field of non-marine mesoscale vortex research to quickly understand and get started in the field of vortex identification. The case display module mainly demonstrates the algorithm identification principle and effect of the present invention. The vortex online identification module is the core functional module of this system, and the vortex online identification module will be introduced in detail later.
[0103] Model and system deployment process: This invention deploys both the model and the system in the same K8s cluster. Compared with the traditional stand-alone deployment method, the containerized deployment method can make the system more elastic, load-balanced and fault-tolerant. The model and system deployment process can refer to the working principle of the system Figure 8, 4 servers were used to build the K8s cluster in the experiment, where the operating system version is: CentOS 8.2 64bit, the Docker version is: 23.0.1, the K8s version is 1.24.4, and the nodes in the K8s cluster include: 1 Master Node and 3 Worker Nodes. First, the present invention packages the Vue project of the system front end into the nginx image through commands, and the image name is: nginx-vue:1.0, and then creates a Deployment with a label of app=ngixn based on the image. The Deployment will create 3 Pods with a label of app=nginx at one time based on the nginx image. These 3 Pods are scheduled to different Worker Nodes as much as possible through the default scheduling strategy of K8s to provide users with front-end Web services. Since the IP addresses of different Pods in the K8s cluster are not fixed, in order to unify the IP addresses of different Pod copies, it is also necessary to create a Service with the label app=nginx. Service can not only expose a group of Pods as the same network service, but also meet the load balancing requirements in the same group of Pods. Secondly, deploy the backend model code to the basic image based on TensorFlow provided by DockerHub, the image name is: my-model:1.0, and then create a Deployment with the label app=model based on the image. The Deployment will create three Pods with the label app=model based on the my-model:1.0 image at one time. These three Pods are scheduled to different Worker Nodes as much as possible through the default scheduling strategy of K8s to provide backend business processing services for the front-end page. It is also necessary to create a Servie with the label app=model to unify the IP address. Finally, as the user scale continues to expand in the later stage, in order to improve the scalability of the system, facilitate the later expansion of different Service services in the system, and meet the load balancing requirements of different Services, you can create an Ingress to proxy different Services. The YMAL configuration file information for the above operation is as follows:
[0104]
[0105]
[0106]
[0107] The working principle of the vortex identification module in the system: Figure 8As shown in the figure, first, the user accesses the Web project through a browser. The user can customize the upload of the SSH image to be identified. The request first passes through Ingress, and then the request is randomly hit by the load balancing function of the Service labeled app=nginx to the Pod labeled app=nginx in three different Worker Nodes. Then one of the Pods labeled app=nginx will give priority to accessing the Pod labeled app=model in the same Worker Node, because the requests between different Pods in the same Worker Node will directly go through the traffic within the cluster, and will not go through the public network traffic outside the node, because this can greatly reduce the waste of traffic resources. Only when the Pod you want to access in the same node hangs up, the request will cross the node to access other Pod copies. At this time, the request comes to the Pod of app=model, and then the Flask deployed in the Pod receives the image uploaded by the user in the request body, first cuts the size of the image to a fixed size that the model can input, and then inputs it into the trained deep learning-based mesoscale vortex recognition model for recognition. After the recognition is completed, Flask will gradually respond to the browser with the recognition results in the form of a response. Finally, Vue renders the recognition results to the browser through two-way data binding for users to view.
[0108] The vortex online recognition module in the system mainly includes two sub-functions: automatic selection of SSH image vortex online recognition function and custom upload of SSH image vortex online recognition function.
[0109] ① Automatically select the SSH image vortex online recognition function: This function system will use the image data in the data set provided by the backend to perform
[0110] Vortex online recognition is used to evaluate the performance of the previously trained model, allowing users to more intuitively experience the recognition effect of the model online. The SSH image input to this function is randomly selected from the validation set images provided by the system background. Fig. 9 As shown. When the user clicks the "Auto Select" button, the system background will automatically select a vortex image of SHH at random. The user only needs to wait for a few seconds, and the system background will automatically return the result after model recognition. The returned results contain a total of three pictures. The first one is the original sea surface height image after rendering, the second one is the result after model recognition and segmentation, and the third one is the real result corresponding to the original image. In the recognized results: yellow represents cyclonic vortex and green represents anticyclonic vortex.
[0111] ② Custom upload SSH image vortex online recognition function: such as Fig.10 As shown, users can click the "Custom Upload" button to
[0112] The user can upload the vortex image in the local device for online vortex recognition by clicking the button, but the image uploaded by the user must be a SHH type remote sensing image. After the user uploads the image, he only needs to wait for a few seconds, and the system background will automatically return the recognition result. The returned result contains a total of two images. The first one is the original SSH image uploaded by the user, and the second one is the result of model recognition and segmentation. In the recognition result, yellow represents cyclonic vortex and green represents anticyclonic vortex.
[0113] Embodiment 2
[0114] This embodiment discloses a mesoscale vortex intelligent identification system based on K8s.
[0115] A mesoscale vortex intelligent identification system based on K8s, comprising:
[0116] The image acquisition module is configured to: acquire an image to be identified;
[0117] The encoder feature extraction module is configured to: input the image to be identified into the trained mesoscale vortex recognition model based on deep learning, extract features of the image to be identified through the deep separable convolution in the backbone network Xception of the encoder, and weight the feature map output by the backbone network based on the CBAM hybrid attention mechanism to obtain the output feature map; input the output feature map into ASPP, set the expansion rate of the dilated convolution, replace the pooling branch in ASPP with the deep separable convolution, splice and compress the feature maps of the dilated convolution and the deep separable convolution according to the channel dimension, and obtain the feature map output by the encoder;
[0118] The decoder intelligent recognition module is configured as follows: input the feature map output by the encoder into the decoder, calculate the weights of receptive fields of different sizes based on the SKNet attention mechanism, aggregate the weights of feature maps of different scales to obtain the output of SKNet and upsample; then concatenate the upsampled feature map with the low-level feature map from the backbone network Xception according to the channel dimension, apply the convolution layer to refine the features and restore the size through upsampling, so as to realize the intelligent recognition of mesoscale vortices.
[0119] Embodiment 3
[0120] The purpose of this embodiment is to provide a computer-readable storage medium.
[0121] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the mesoscale vortex intelligent identification method based on K8s as described in Example 1 of the present disclosure.
[0122] Embodiment 4
[0123] The purpose of this embodiment is to provide an electronic device.
[0124] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the mesoscale vortex intelligent identification method based on K8s as described in Example 1 of the present disclosure are implemented.
[0125] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.
[0126] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.
[0127] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A mesoscale vortex intelligent identification method based on K8s, It is characterized in that The following steps are involved: Obtain an image to be recognized; The image to be identified is input into the trained mesoscale vortex identification model based on deep learning. The feature of the image to be identified is extracted through the deep separable convolution in the backbone network Xception of the encoder. The feature map output by the backbone network is weighted based on the CBAM hybrid attention mechanism to obtain the output feature map. The output feature map is input into ASPP, the expansion rate of the dilated convolution is set, and the pooling branch in ASPP is replaced with the deep separable convolution. The feature maps of the dilated convolution and the deep separable convolution are spliced and compressed according to the channel dimension to obtain the feature map output by the encoder. The feature map output by the encoder is input into the decoder, and the weights of receptive fields of different sizes are calculated based on the SKNet attention mechanism. The weights are aggregated to obtain the output of SKNet and upsampled. Then the upsampled feature map is concatenated with the low-level feature map from the backbone network Xception according to the channel dimension, the convolution layer is applied to refine the features and the size is restored by upsampling to achieve intelligent recognition of mesoscale vortices; The mesoscale vortex recognition model based on deep learning is obtained by improving the architecture of the semantic segmentation network DeepLabV3 Plus. The specific improvements are: The number of layers of depthwise separable convolution in the Middle flow of the backbone network Xception was changed from three to two, the number of repetitions of the Middle Flow part was increased from 8 to 12, and all the maximum pooling operations in the original Xecption were replaced by depthwise separable convolution with a stride of 2; For the ASPP in the original encoder, the dilation rate of the dilated convolution is changed from 6, 12, 18 to 6, 12, 12, and the pooling branch in the original ASPP is replaced with a depthwise separable convolution with a stride of 2; A lightweight CBAM hybrid attention mechanism is introduced at the exit of the backbone network, and an SKNet attention mechanism is introduced at the entrance of the decoder; SKNet introduces the attention mechanism into the receptive field of convolutional neural networks. SKNet introduces SK convolution, which consists of three parts: Spilt, Fuse, and Select. The Spilt part uses multi-scale receptive fields to extract features. The Fuse part fuses the features extracted from multi-scale receptive fields, and then reduces the dimension through downsampling and fully connected layers to calculate the weights of feature maps of each layer. The Select part aggregates the feature maps of convolution kernels of different scales according to the weights to obtain the final output. The method also includes building a K8s cluster in a GPU cloud server, training a mesoscale vortex recognition model based on deep learning in the built K8s cluster, and obtaining a trained mesoscale vortex recognition model based on deep learning; During the model training process, the loss function is selected as Soft Dice Loss, and the calculation formula is: Among them, |M| represents the area where the true result appears, and |N| represents the area where the model predicted result appears.
2. The mesoscale vortex intelligent identification method based on K8s as claimed in claim 1, It is characterized in that The backbone network Xception includes Entry flow, Middle flow and Exit flow, with a total of 18 Xception blocks, including 4 Entry flow parts, 12 Middle flow parts and 2 Exit flow parts; each Xception block includes a residual network combined with depthwise separable convolution.
3. The mesoscale vortex intelligent identification method based on K8s as claimed in claim 1, It is characterized in that The CBAM hybrid attention mechanism includes a channel attention mechanism and a spatial attention mechanism. For the feature map output by the backbone network, it first passes through a channel attention module to obtain a weighted result, and then passes through a spatial attention module to obtain the final weighted result.
4. The mesoscale vortex intelligent identification method based on K8s as claimed in claim 3, It is characterized in that For a given feature map As input, CBAM calculates the channel attention map and spatial attention map , the entire attention calculation process can be summarized as the following formula: in represents element-wise multiplication; is the input feature map; is the final output feature map.
5. A mesoscale vortex intelligent identification system based on K8s, Features: include: The image acquisition module is configured to: acquire an image to be identified; The encoder feature extraction module is configured to: input the image to be identified into the trained mesoscale vortex recognition model based on deep learning, extract features of the image to be identified through the deep separable convolution in the backbone network Xception of the encoder, and weight the feature map output by the backbone network based on the CBAM hybrid attention mechanism to obtain the output feature map; input the output feature map into ASPP, set the expansion rate of the dilated convolution, replace the pooling branch in ASPP with the deep separable convolution, splice and compress the feature maps of the dilated convolution and the deep separable convolution according to the channel dimension, and obtain the feature map output by the encoder; The decoder intelligent recognition module is configured to: input the feature map output by the encoder into the decoder, calculate the weights of receptive fields of different sizes based on the SKNet attention mechanism, aggregate the weights of feature maps of different scales to obtain the output of SKNet and perform upsampling; Then the upsampled feature map is concatenated with the low-level feature map from the backbone network Xception according to the channel dimension, the convolution layer is applied to refine the features and the size is restored by upsampling to achieve intelligent recognition of mesoscale vortices; The mesoscale vortex recognition model based on deep learning is obtained by improving the architecture of the semantic segmentation network DeepLabV3 Plus. The specific improvements are: The number of layers of depthwise separable convolution in the Middle flow of the backbone network Xception was changed from three to two, the number of repetitions of the Middle Flow part was increased from 8 to 12, and all the maximum pooling operations in the original Xecption were replaced by depthwise separable convolution with a stride of 2; For the ASPP in the original encoder, the dilation rate of the dilated convolution is changed from 6, 12, 18 to 6, 12, 12, and the pooling branch in the original ASPP is replaced with a depthwise separable convolution with a stride of 2; A lightweight CBAM hybrid attention mechanism is introduced at the exit of the backbone network, and an SKNet attention mechanism is introduced at the entrance of the decoder; SKNet introduces the attention mechanism into the receptive field of convolutional neural networks. SKNet introduces SK convolution, which consists of three parts: Spilt, Fuse, and Select. The Spilt part uses multi-scale receptive fields to extract features. The Fuse part fuses the features extracted from multi-scale receptive fields, and then reduces the dimension through downsampling and fully connected layers to calculate the weights of feature maps of each layer. The Select part aggregates the feature maps of convolution kernels of different scales according to the weights to obtain the final output. The method also includes building a K8s cluster in a GPU cloud server, training a mesoscale vortex recognition model based on deep learning in the built K8s cluster, and obtaining a trained mesoscale vortex recognition model based on deep learning; During the model training process, the loss function is selected as Soft Dice Loss, and the calculation formula is: Among them, |M| represents the area where the true result appears, and |N| represents the area where the model predicted result appears.
6. A computer-readable storage medium having a program stored thereon, It is characterized in that When the program is executed by a processor, the steps in the K8s-based mesoscale vortex intelligent identification method as described in any one of claims 1 to 4 are implemented.
7. An electronic device comprising a memory, a processor and a program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps in the K8s-based mesoscale vortex intelligent identification method as described in any one of claims 1 to 4 are implemented.
