Model deployment method and device, equipment and storage medium

Abstract

The invention discloses a model deployment method and device, equipment and a storage medium. The method comprises the steps of obtaining a sample set for sea ice detection; performing model training based on the sample set, and obtaining a first detection model after training; the first detection model is compressed, a second detection model is obtained, the network structure of the second detection model is the same as that of the first detection model, and the parameter quantity of the second detection model is smaller than that of the first detection model; training a second detection model based on the first detection model and the sample set, and obtaining a third detection model after training; format conversion is carried out on the third detection model to obtain a fourth detection model, and the format of the fourth detection model is an open neural network switching format; and deploying the fourth detection model in the open neural network exchange format in a satellite hardware environment for sea ice detection. According to the method, a high-precision on-satellite sea ice detection effect can be quickly obtained in a low-computing-power on-satellite hardware environment.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model deployment method, apparatus, device and storage medium. Background Art

[0002] Sea ice refers to the ice layer covering the surface of polar waters. The presence of sea ice poses a serious threat to polar marine operations. For example, a ship with insufficient structural strength collides with sea ice, which can cause damage to the hull and threaten the lives of operators. Therefore, it is particularly important for ships at sea to respond quickly to changes in sea ice.

[0003] In practical applications, deep learning-based on-board detection methods can be used to determine the sea ice coverage, that is, the satellite processes and analyzes sea ice images based on deep learning to detect the sea ice coverage and transmit the sea ice coverage to the ship. However, since the computing power of on-board hardware is much lower than that of ground equipment, and deep learning algorithms usually require higher computing power, the on-board detection effect is not good after the deep learning model is deployed on the satellite. Summary of the invention

[0004] The embodiments of the present application provide a model deployment method, apparatus, device and storage medium to improve on-board detection effects.

[0005] In a first aspect, an embodiment of the present application provides a model deployment method, including:

[0006] Obtain sample sets for sea ice detection;

[0007] Performing model training based on the sample set, and obtaining a first detection model after the training is completed;

[0008] Compressing the first detection model to obtain a second detection model, wherein the second detection model has the same network structure as the first detection model, and the number of parameters of the second detection model is smaller than the number of parameters of the first detection model;

[0009] Training the second detection model based on the first detection model and the sample set, and obtaining a third detection model after the training is completed;

[0010] Performing format conversion on the third detection model to obtain a fourth detection model, wherein the format of the fourth detection model is an open neural network exchange format;

[0011] The fourth detection model in the open neural network exchange format is deployed in a satellite hardware environment for sea ice detection.

[0012] Optionally, the training of the second detection model based on the first detection model and the sample set, and obtaining a third detection model after the training, comprises:

[0013] Inputting the sample set into the first detection model and the second detection model respectively, obtaining a first detection result and a second detection result output by the second detection model, and a third detection result output by the first detection model, wherein the first detection result and the third detection result are both related to a preset temperature parameter, and the second detection result is independent of the temperature parameter;

[0014] Determine a distillation loss between the first detection model and the second detection model based on a preset loss function, the first detection result, and the third detection result, and determine a training loss of the second detection model based on the loss function and the second detection result;

[0015] The second detection model is trained based on the distillation loss and the training loss, and the third detection model is obtained after the training is completed.

[0016] Optionally, the output layers of the first detection model and the second detection model are implemented using an S-shaped growth curve Sigmoid function, and the Sigmoid function includes the temperature parameter, and the temperature parameter is used to adjust the smoothness of the Sigmoid function.

[0017] Optionally, the first detection model is obtained by training an improved DeepLabv3+ model, wherein the improved DeepLabv3+ model includes an encoder and a decoder;

[0018] The encoder is used to extract features of sample images in the sample set to obtain feature maps corresponding to the sample images;

[0019] The decoder is used to restore the resolution of the feature map to obtain the detection result of the sample image.

[0020] Optionally, the encoder comprises a convolutional neural network ConvNeXt, wherein the ConvNeXt comprises a first downsampling layer, a layer normalization module, a second downsampling layer and a ConvNeXt block;

[0021] The first downsampling layer includes a first convolutional layer, the size of the first convolutional layer is 4×4, and the stride of the first convolutional layer is 2;

[0022] The second downsampling layer includes a layer normalization module and a second convolutional layer, the size of the second convolutional layer is 2×2, and the stride of the second convolutional layer is 2;

[0023] The ConvNeXt block includes a depth-wise convolution layer, a layer normalization module, a third convolution layer and a layer scaling module, the size of the depth-wise convolution layer is 7×7, the step size of the depth-wise convolution layer is 1, the size of the third convolution layer is 1×1, and the step size of the third convolution layer is 1.

[0024] Optionally, the encoder further includes a dilated space pooling pyramid network, the dilated space pooling pyramid network is connected to the ConvNeXt, and the dilated space pooling pyramid network includes a third downsampling layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, and a seventh convolutional layer;

[0025] The third downsampling layer, the fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer and the seventh convolutional layer are respectively connected to the output layer of the ConvNeXt;

[0026] The fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer and the seventh convolutional layer are connected in a cascade manner using a splicing module;

[0027] The third downsampling layer is a mean downsampling layer, the size of the fourth convolutional layer is 3×3, and the expansion rate of the fourth convolutional layer is 6, the size of the fifth convolutional layer is 3×3, and the expansion rate of the fifth convolutional layer is 12, the size of the sixth convolutional layer is 3×3, and the expansion rate of the sixth convolutional layer is 18, and the size of the seventh convolutional layer is 1×1.

[0028] Optionally, obtaining a sample set for sea ice detection includes:

[0029] Acquire a dual-polarization synthetic aperture radar (SAR) sample set corresponding to the satellite, wherein the dual-polarization SAR sample set includes a first dual-polarization SAR sample set and a second dual-polarization SAR sample set, wherein the first dual-polarization SAR sample set includes an image label, and the second dual-polarization SAR sample set does not have an image label;

[0030] Performing model training based on the first dual-polarization SAR sample set and the image labels of the first dual-polarization SAR sample set to obtain a fourth detection model;

[0031] Performing sea ice detection on the second dual-polarization SAR sample set based on the fourth detection model to obtain a detection result of the second dual-polarization SAR sample set;

[0032] The sample set is constructed by using the first dual-polarization SAR sample set, the image label of the first dual-polarization SAR sample set, the second dual-polarization SAR sample set, and the detection result of the second polarization SAR sample set.

[0033] In a second aspect, an embodiment of the present application provides a model deployment device, including:

[0034] A sample acquisition module, used to obtain a sample set for sea ice detection;

[0035] A first model training module, used for performing model training based on the sample set, and obtaining a first detection model after the training is completed;

[0036] A model compression module, used for compressing the first detection model to obtain a second detection model, wherein the second detection model has the same network structure as the first detection model, and the number of parameters of the second detection model is smaller than the number of parameters of the first detection model;

[0037] A second model training module is used to train the second detection model based on the first detection model and the sample set, and obtain a third detection model after the training;

[0038] a format conversion module, used for performing format conversion on the third detection model to obtain a fourth detection model, wherein the format of the fourth detection model is an open neural network exchange format;

[0039] A model deployment module is used to deploy the fourth detection model in the open neural network exchange format in a satellite hardware environment for sea ice detection.

[0040] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor, a memory, and a system bus;

[0041] The processor and the memory are connected via the system bus;

[0042] The memory is used to store a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes any implementation step of the above-mentioned model deployment method based on weak measurement technology.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on an electronic device, the electronic device executes any implementation step of the above-mentioned model deployment method based on weak measurement technology.

[0044] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0045] In an embodiment of the present application, after obtaining a sample set for sea ice detection, model training can be performed based on the sample set, and a first detection model can be obtained after the training is completed. Then, the first detection model is compressed to obtain a second detection model. Among them, the second detection model has the same network structure as the first detection model, and the parameter amount of the second detection model is less than the parameter amount of the first detection model. Then, the second detection model can be trained based on the first detection model and the sample set, and a third detection model can be obtained after the training is completed. Accordingly, the third detection model can be format-converted to obtain a fourth detection model, the format of the fourth detection model is open neural network exchange, and the fourth model in the open neural network exchange format is deployed in a satellite hardware environment for sea ice detection. Since the first detection model is a large parameter model obtained by normal training, the detection effect of the first detection model is better. In this way, the trained first detection model is compressed and the model training is continued, so that a third detection model with unchanged detection effect and reduced parameter amount can be obtained. Finally, the fourth detection model in the open neural network exchange format is obtained through format conversion, and the fourth detection model can be matched with the satellite hardware environment for sea ice detection with the help of the open neural network exchange format, so that a high-precision and lightweight sea ice cover detection method can be implemented on board through the fourth detection model. This method can quickly achieve high-precision on-board sea ice detection effects in a low-computing power on-board hardware environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of a model deployment method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a dual-polarization SAR image and a binary mask of the dual-polarization SAR image provided in an embodiment of the present application;

[0048] Figure 3a A schematic diagram of the structure of an improved DeepLabv3+ model provided in an embodiment of the present application;

[0049] Figure 3b A schematic diagram of the structure of a ConvNeXt provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of determining distillation loss and training loss provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of the structure of a model deployment module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] As mentioned above, in practical applications, deep learning-based on-board detection methods can be used to determine sea ice coverage, that is, the satellite processes and analyzes sea ice images based on deep learning to detect sea ice coverage and transmit the sea ice coverage to the ship. However, since the computing power of on-board hardware is much lower than that of ground equipment, and deep learning algorithms usually require higher computing power, the on-board detection effect is not good after the deep learning model is deployed on the satellite.

[0053] Based on this, in order to solve the above problems, the embodiment of the present application provides a model deployment method, which may include: after obtaining a sample set for sea ice detection, a model training may be performed based on the sample set, and a first detection model may be obtained after the training is completed. Then, the first detection model is compressed to obtain a second detection model. Among them, the second detection model has the same network structure as the first detection model, and the parameter amount of the second detection model is less than the parameter amount of the first detection model. Then, the second detection model may be trained based on the first detection model and the sample set, and a third detection model may be obtained after the training is completed. Accordingly, the third detection model may be format converted to obtain a fourth detection model, the format of the fourth detection model is an open neural network exchange, and the fourth model in the open neural network exchange format is deployed in a satellite hardware environment for sea ice detection. Since the first detection model is a large parameter model obtained by normal training, the detection effect of the first detection model is better. In this way, the first detection model that has been trained is compressed and the model training is continued, so that a third detection model with the detection effect remaining unchanged and the parameter amount reduced can be obtained. Finally, the fourth detection model in the open neural network exchange format is obtained through format conversion, and the fourth detection model can be matched with the satellite hardware environment for sea ice detection with the help of the open neural network exchange format, so that a high-precision and lightweight sea ice cover detection method can be implemented on board through the fourth detection model. This method can quickly achieve high-precision on-board sea ice detection effects in a low-computing power on-board hardware environment.

[0054] It should be noted that the embodiments of the present application may not limit the execution subject of the model deployment method. For example, the model deployment method of the embodiments of the present application may be applied to data processing devices such as terminal devices or servers. Among them, the terminal device may be an electronic device such as a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc. The server may be an independent server, a cluster server, or a cloud server.

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] Figure 1 A flow chart of a model deployment method provided in an embodiment of the present application. Figure 1 As shown, the model deployment method provided in the embodiment of the present application uses the corresponding model deployment device as the execution subject to describe the specific implementation of the solution. The model deployment method may include the following steps S101-S106.

[0057] S101: Obtain a sample set for sea ice detection.

[0058] Regarding the process of obtaining the sample, in the specific implementation, first, the dual-polarization synthetic aperture radar (SAR) sample set corresponding to the satellite can be obtained. In practical applications, taking the Sentinel-1 satellite in the Arctic region as an example, the HH / HV (HH represents horizontal polarization, HV represents cross polarization) dual-polarization SAR data can be collected through the EW mode (an ultra-wide mode) of the satellite; then, the HH / HV dual-polarization SAR data can be preprocessed to obtain the above-mentioned dual-polarization SAR image. Among them, the preprocessing means are, for example, radiation correction, thermal noise removal and land mask.

[0059] Radiometric correction involves converting SAR data from raw radar echo signals into backscatter coefficients with physical units. Radiometric correction allows SAR data to reflect true surface characteristics rather than errors caused by changes in sensor or environmental conditions. When receiving weak echo signals from the earth's surface, the SAR system also receives thermal noise from the system itself. Therefore, thermal noise removal helps to reduce the impact of this internal noise on the data, thereby improving data quality. Land masking can reduce interference from land echoes by identifying and excluding land areas and retaining only data from ocean or water areas.

[0060] Furthermore, in the above dual-polarization SAR sample set, some samples may have labels and belong to publicly available sea ice detection samples, while another part of the samples do not have labels and can be pseudo-labeled through subsequent semi-supervised learning. That is to say, the above dual-polarization SAR sample set includes a first dual-polarization SAR sample set and a second dual-polarization SAR sample set, wherein the first dual-polarization SAR sample set includes image labels, and the second dual-polarization SAR sample set has no image labels.

[0061] Next, model training can be performed based on the first dual-polarization SAR sample set and the image labels of the first dual-polarization SAR sample set to obtain a fourth detection model. In other words, a basic sea ice detection model can be obtained by using the first dual-polarization SAR sample set with image labels for model training in a semi-supervised learning manner.

[0062] Then, sea ice detection can be performed on the second dual-polarization SAR sample set based on the fourth detection model to obtain a detection result of the second dual-polarization SAR sample set.

[0063] In practical applications, dual-polarization SAR images are Figure 2 As shown in (a) in FIG. 1 , the detection result or image label can be a binary mask of a dual-polarization SAR image, that is, Figure 2 As shown in (b) in .

[0064] In this way, the sample set can be constructed by the first dual-polarization SAR sample set, the image label of the first dual-polarization SAR sample set, the second dual-polarization SAR sample set, and the detection result of the second polarization SAR sample set. It can be seen that by using the trained fourth detection model to detect the unlabeled second dual-polarization SAR sample set, the detection result can be obtained as the image label of the second dual-polarization SAR sample set, thereby constructing a sample set, so as to enrich the sample set when the labeled data is scarce, and use the data-rich sample set to improve the performance of the model.

[0065] S102: Perform model training based on the sample set, and obtain a first detection model after the training is completed.

[0066] In practical applications, the fourth detection model and the first detection model can be obtained by training the improved DeepLabv3+ model (a semantic segmentation model). For ease of understanding, the structure of the improved DeepLabv3+ model is first introduced below.

[0067] Specifically, combined Figure 3aAs shown, the improved DeepLabv3+ model may include an encoder and a decoder. The encoder may be used to extract features of sample images in a sample set to obtain feature maps corresponding to the sample images, while the decoder may be used to restore the resolution of the feature maps to obtain detection results of the sample images.

[0068] For the encoder, it may specifically include a backbone network and a dilated spatial pooling pyramid network.

[0069] The backbone network can be implemented using a convolutional neural network ConvNeXt. That is, the encoder includes a convolutional neural network ConvNeXt. Figure 3b As shown, ConvNeXt includes a first downsampling layer, a layer normalization module, a second downsampling layer and a ConvNeXt block.

[0070] The first downsampling layer includes a first convolution layer, the size of the first convolution layer is 4×4, and the step size of the first convolution layer is 2. In this way, the first downsampling layer can downsample the input dual-polarization SAR image, that is, the sample image, by two times.

[0071] The layer normalization module can be used to accelerate model convergence, achieve stable training, and improve the generalization ability of the model.

[0072] The second downsampling layer includes a layer normalization module and a second convolution layer, the size of the second convolution layer is 2×2, and the stride of the second convolution layer is 2. In this way, the second downsampling layer can further downsample the sample image by a factor of two.

[0073] The ConvNeXt block includes a depth-wise convolution layer, a layer normalization module, a third convolution layer, and a layer scaling module, the size of the depth-wise convolution layer is 7×7, the step size of the depth-wise convolution layer is 1, the size of the third convolution layer is 1×1, and the step size of the third convolution layer is 1. In addition, the ConvNeXt block also includes an activation function layer, which is implemented using Gaussian Error Linear Units (GELU), and the activation function layer is located between two third convolution layers.

[0074] For the dilated spatial pooling pyramid network, the dilated spatial pooling pyramid network is connected with ConvNeXt. Figure 3a As shown, the atrous spatial pooling pyramid network includes a third downsampling layer 1, a fourth convolutional layer 2, a fifth convolutional layer 3, a sixth convolutional layer 4 and a seventh convolutional layer 5.

[0075] Among them, the third downsampling layer 1, the fourth convolution layer 2, the fifth convolution layer 3, the sixth convolution layer 4 and the seventh convolution layer 5 are connected to the output layer of ConvNeXt respectively. The third downsampling layer 1 is a mean downsampling layer, the size of the fourth convolution layer 2 is 3×3, and the expansion rate of the fourth convolution layer 2 is 6, the size of the fifth convolution layer 3 is 3×3, and the expansion rate of the fifth convolution layer 3 is 12, the size of the sixth convolution layer 4 is 3×3, and the expansion rate of the sixth convolution layer 4 is 18, and the size of the seventh convolution layer 5 is 1×1. The fourth convolution layer 2, the fifth convolution layer 3, the sixth convolution layer 4 and the seventh convolution layer 5 are connected by multiple splicing modules in a cascade manner, so that the convolution results obtained in each layer can be spliced ​​and input into the next layer, thereby improving the richness of the final feature map.

[0076] Specifically, combined Figure 3a As shown, the fourth convolutional layer 2 and the fifth convolutional layer 3 are connected via a splicing module 23. The splicing module 23 is used to splice the output result of ConvNeXt and the convolution result of the fourth convolutional layer 2 to obtain a first splicing result, and output the first splicing result to the fifth convolutional layer 3 and the splicing module 33.

[0077] The fifth convolutional layer 3 and the sixth convolutional layer 4 are connected through a splicing module 34, and the splicing module 34 and the splicing module 23 are connected through a splicing module 34. The splicing module 33 is used to splice the output result of ConvNeXt and the first splicing result to obtain a second splicing result, and output the second splicing result to the splicing module 34 and the splicing module 35. The splicing module 34 is used to splice the second splicing result and the convolution result of the fifth convolutional layer 3 to obtain a third splicing result, and output the third splicing result to the sixth convolutional layer 4 and the splicing module 54.

[0078] The sixth convolutional layer 4 and the seventh convolutional layer 5 are connected through a splicing module 45, and the splicing module 33 and the splicing module 45 are connected in turn through the splicing module 35 and the splicing module 54. The splicing module 35 is used to splice the output result of ConvNeXt and the second splicing result to obtain a fourth splicing result, and output the fourth splicing result to the splicing module 54. The splicing module 54 is used to splice the third splicing result and the fourth splicing result to obtain a fifth splicing result, and output the fifth splicing result to the splicing module 45. The splicing module 45 is used to splice the fifth splicing result with the convolution result of the sixth convolutional layer to obtain a sixth splicing result, and output the sixth splicing result to the seventh convolutional layer 5.

[0079] In addition, combined Figure 3aAs shown, the atrous spatial pooling pyramid network also includes an eighth convolutional layer 6. The eighth convolutional layer 6 is used to process the sampling result output by the third downsampling layer 1, the convolution result output by the fourth convolutional layer 2, the convolution result output by the fifth convolutional layer 3, the convolution result output by the sixth convolutional layer 4, and the convolution result output by the seventh convolutional layer 5, and input the processing result into the upsampling layer 7 of the decoder.

[0080] For the decoder, the embodiment of the present application can be implemented based on the structure of the decoder in the improved DeepLabv3+ model. In order to obtain more accurate detection results, combined with Figure 3a As shown, in addition to the ninth convolutional layer 8, the tenth convolutional layer 9, the upsampling layer 7 and the splicing module 10 included in the original decoder, the output end of the decoder may further include a correction module. The correction module may include an eleventh convolutional layer 11, and the size of the eleventh convolutional layer 11 is 3×3.

[0081] S103: compress the first detection model to obtain a second detection model.

[0082] The second detection model has the same network structure as the first detection model, and the number of parameters of the second detection model is smaller than that of the first detection model. That is to say, in a specific implementation, the number of neurons in each layer of the network structure of the first detection model can be reduced so that the number of parameters of the reduced model is smaller than that of the first detection model, thereby obtaining the second detection model.

[0083] S104: Train a second detection model based on the first detection model and the sample set, and obtain a third detection model after the training is completed.

[0084] In the embodiment of the present application, the training process for the second detection model may specifically include the following steps 41 to 43.

[0085] Step 41: Input the sample set into the first detection model and the second detection model respectively, and obtain the first detection result and the second detection result output by the second detection model, and the third detection result output by the first detection model.

[0086] In practical applications, the output ends of the first detection model and the second detection model can use an S-shaped growth curve Sigmoid function as an activation function. The Sigmoid function may include a preset temperature parameter, which is used to adjust the smoothness of the Sigmoid function. Among them, the preset temperature parameter can be adjusted according to human experience, and its value range is not limited here.

[0087] Among them, the first detection result and the third detection result are both related to the preset temperature parameter, and the second detection result is independent of the temperature parameter. Therefore, for ease of understanding, combined with the following formula (1), the first detection result and the third detection result can be calculated using the following Sigmoid function that introduces the temperature parameter:

[0088]

[0089] Wherein, Sigmoid(x) is the first detection result or the third detection result, x is the input result input to the activation function, and T is a preset temperature parameter.

[0090] The second detection result can be calculated using the Sigmoid function that does not include the temperature parameter, that is, the original Sigmoid function, as shown in the following formula (2):

[0091]

[0092] Among them, Sigmoid(x)′ is the second detection result, and x is the input result input to the activation function.

[0093] Step 42: Based on a preset loss function, the first detection result and the third detection result, determine the distillation loss between the first detection model and the second detection model, and based on the loss function and the second detection result, determine the training loss of the second detection model.

[0094] In practical applications, the pre-set loss function can be implemented using the cross entropy loss function, as shown in the following formula (3):

[0095]

[0096] in, is the loss value, y is the label of the sample image, For the test results.

[0097] Furthermore, combined with Figure 4 As shown, since the first detection result and the third detection result are both related to the preset temperature parameter, based on the loss function embodied in the above formula (3), the third detection result is used as y in the above formula (3), and the first detection result is used as The distillation loss between the first detection model and the second detection model can be obtained. In addition, since the second detection result is independent of the above temperature parameter, based on the loss function embodied in the above formula (3), the true value of the sample is used as y in the above formula (3), that is, the image mark of the first dual-polarization SAR image in the above sample set and the detection result of the second polarization SAR image are used as y in the above formula (3), and the second detection result is used as The training loss of the second detection model can be obtained.

[0098] Step 43: Perform model training on the second detection model based on the distillation loss and the training loss, and obtain the third detection model after the training is completed.

[0099] In practical applications, after obtaining the distillation loss and the training loss, the training of the first detection model and the second detection model can be achieved by minimizing the above loss function and back-propagating its gradient to adjust the network parameters during the training process.

[0100] S105: Convert the format of the third detection model to obtain a fourth detection model, where the format of the fourth detection model is an open neural network exchange format.

[0101] In practical applications, the trained low-parameter detection model can be converted into an open neural network exchange format, that is, the third detection model is converted into an open neural network exchange format to obtain a fourth detection model. The open neural network exchange format is an intermediate file format with good compatibility, which can realize the migration and deployment of neural networks between different frameworks and different hardware platforms. Based on this, in an embodiment of the present application, the input end of the fourth detection model can maintain dynamic dimensional input, so as to adapt to input images of different sizes.

[0102] Furthermore, in the process of format conversion, the following steps may be specifically included: integrating the addition operation immediately following the convolution layer in the third detection model into the convolution layer, merging the continuous transposition operations in the third detection model, merging the addition operation after a matrix multiplication in the third detection model into a general matrix multiplication (General MatrixMultiply, referred to as GEMM), removing unused nodes, and removing identity operations. The above format conversion process can be implemented in an existing open neural network exchange format database, which will not be repeated here.

[0103] S106: Deploy the fourth detection model in the open neural network exchange format in a satellite hardware environment for sea ice detection.

[0104] After the fourth detection model is obtained, it can be uploaded to the hardware environment for transmission to the on-board detection.

[0105] Accordingly, when performing detection tasks on board, the fourth detection model can be executed through the existing open neural network exchange format database. Specifically, first, the library can initialize the parameters for sea ice detection in the fourth detection model to the GPU on board; then, the GPU on board can detect the sea ice image to be detected and obtain the corresponding detection results.

[0106] Based on the relevant contents of the above steps S101-S106, it can be known that in an embodiment of the present application, after obtaining a sample set for sea ice detection, a model training can be performed based on the sample set, and a first detection model can be obtained after the training is completed. Then, the first detection model is compressed to obtain a second detection model. Among them, the second detection model has the same network structure as the first detection model, and the parameter amount of the second detection model is less than the parameter amount of the first detection model. Then, the second detection model can be trained based on the first detection model and the sample set, and a third detection model can be obtained after the training is completed. Accordingly, the third detection model can be format-converted to obtain a fourth detection model, the format of the fourth detection model is open neural network exchange, and the fourth model in the open neural network exchange format is deployed in a satellite hardware environment for sea ice detection. Since the first detection model is a large parameter model obtained by normal training, the detection effect of the first detection model is better. In this way, the first detection model that has been trained is compressed and the model training is continued, so that a third detection model with unchanged detection effect and reduced parameter amount can be obtained. Finally, the fourth detection model in the open neural network exchange format is obtained through format conversion, and the fourth detection model can be matched with the satellite hardware environment for sea ice detection with the help of the open neural network exchange format, so that a high-precision and lightweight sea ice cover detection method can be implemented on board through the fourth detection model. This method can quickly achieve high-precision on-board sea ice detection effects in a low-computing power on-board hardware environment.

[0107] Further, based on the model deployment method provided in the above embodiment, the embodiment of the present application can also provide a model deployment device. The model deployment device is described below in conjunction with the embodiments and drawings.

[0108] Figure 5 A schematic diagram of the structure of a model deployment device provided in an embodiment of the present application. Figure 5 As shown, the model deployment device 500 provided in the embodiment of the present application may include:

[0109] A sample acquisition module 501 is used to acquire a sample set for sea ice detection;

[0110] A first model training module 502, used to perform model training based on the sample set, and obtain a first detection model after the training;

[0111] A model compression module 503 is used to compress the first detection model to obtain a second detection model, wherein the second detection model has the same network structure as the first detection model, and the number of parameters of the second detection model is smaller than the number of parameters of the first detection model;

[0112] A second model training module 504 is used to train the second detection model based on the first detection model and the sample set, and obtain a third detection model after the training;

[0113] A format conversion module 505 is used to perform format conversion on the third detection model to obtain a fourth detection model, wherein the format of the fourth detection model is an open neural network exchange format;

[0114] The model deployment module 506 is used to deploy the fourth detection model in the open neural network exchange format in a satellite hardware environment for sea ice detection.

[0115] Optionally, the second model training module 504 is specifically used to:

[0116] Inputting the sample set into the first detection model and the second detection model respectively, obtaining a first detection result and a second detection result output by the second detection model, and a third detection result output by the first detection model, wherein the first detection result and the third detection result are both related to a preset temperature parameter, and the second detection result is independent of the temperature parameter;

[0117] Determine a distillation loss between the first detection model and the second detection model based on a preset loss function, the first detection result, and the third detection result, and determine a training loss of the second detection model based on the loss function and the second detection result;

[0118] The second detection model is trained based on the distillation loss and the training loss, and the third detection model is obtained after the training is completed.

[0119] Optionally, the sample acquisition module 501 is specifically configured to:

[0120] Acquire a dual-polarization synthetic aperture radar (SAR) sample set corresponding to the satellite, wherein the dual-polarization SAR sample set includes a first dual-polarization SAR sample set and a second dual-polarization SAR sample set, wherein the first dual-polarization SAR sample set includes an image label, and the second dual-polarization SAR sample set does not have an image label;

[0121] Performing model training based on the first dual-polarization SAR sample set and the image labels of the first dual-polarization SAR sample set to obtain a fourth detection model;

[0122] Performing sea ice detection on the second dual-polarization SAR sample set based on the fourth detection model to obtain a detection result of the second dual-polarization SAR sample set;

[0123] The sample set is constructed by using the first dual-polarization SAR sample set, the image label of the first dual-polarization SAR sample set, the second dual-polarization SAR sample set, and the detection result of the second polarization SAR sample set.

[0124] Furthermore, an embodiment of the present application also provides an electronic device, comprising: a processor, a memory, and a system bus;

[0125] The processor and the memory are connected via the system bus;

[0126] The memory is used to store one or more programs, and the one or more programs include instructions, which, when executed by the processor, enable the processor to perform any implementation step of the above-mentioned model deployment method.

[0127] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on an electronic device, any implementation step of the above-mentioned model deployment method is implemented.

[0128] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in each embodiment of the present application or certain parts of the embodiments. It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other.

[0129] As for the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0130] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0131] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model deployment method, characterized in that: include: Obtain sample sets for sea ice detection; Performing model training based on the sample set, and obtaining a first detection model after the training is completed; Compressing the first detection model to obtain a second detection model, wherein the second detection model has the same network structure as the first detection model, and the number of parameters of the second detection model is smaller than the number of parameters of the first detection model; Training the second detection model based on the first detection model and the sample set, and obtaining a third detection model after the training is completed; Performing format conversion on the third detection model to obtain a fourth detection model, wherein the format of the fourth detection model is an open neural network exchange format; The fourth detection model in the open neural network exchange format is deployed in a satellite hardware environment for sea ice detection.

2. The model deployment method according to claim 1, characterized in that: The step of training the second detection model based on the first detection model and the sample set, and obtaining a third detection model after the training, comprises: Inputting the sample set into the first detection model and the second detection model respectively, obtaining a first detection result and a second detection result output by the second detection model, and a third detection result output by the first detection model, wherein the first detection result and the third detection result are both related to a preset temperature parameter, and the second detection result is independent of the temperature parameter; Determine a distillation loss between the first detection model and the second detection model based on a preset loss function, the first detection result, and the third detection result, and determine a training loss of the second detection model based on the loss function and the second detection result; The second detection model is trained based on the distillation loss and the training loss, and the third detection model is obtained after the training is completed.

3. The model deployment method according to claim 2, characterized in that: The output layers of the first detection model and the second detection model are implemented using an S-shaped growth curve Sigmoid function, and the Sigmoid function includes the temperature parameter, and the temperature parameter is used to adjust the smoothness of the Sigmoid function.

4. The model deployment method according to claim 1, characterized in that: The first detection model is obtained by training an improved DeepLabv3+ model, wherein the improved DeepLabv3+ model includes an encoder and a decoder; The encoder is used to extract features of sample images in the sample set to obtain feature maps corresponding to the sample images; The decoder is used to restore the resolution of the feature map to obtain the detection result of the sample image.

5. The model deployment method according to claim 4, characterized in that: The encoder comprises a convolutional neural network ConvNeXt, wherein the ConvNeXt comprises a first downsampling layer, a layer normalization module, a second downsampling layer and a ConvNeXt block; The first downsampling layer includes a first convolutional layer, the size of the first convolutional layer is 4×4, and the stride of the first convolutional layer is 2; The second downsampling layer includes a layer normalization module and a second convolutional layer, the size of the second convolutional layer is 2×2, and the stride of the second convolutional layer is 2; The ConvNeXt block includes a depth-wise convolution layer, a layer normalization module, a third convolution layer and a layer scaling module, the size of the depth-wise convolution layer is 7×7, the step size of the depth-wise convolution layer is 1, the size of the third convolution layer is 1×1, and the step size of the third convolution layer is 1.

6. The model deployment method according to claim 5, characterized in that: The encoder further includes a dilated space pooling pyramid network, the dilated space pooling pyramid network is connected to the ConvNeXt, and the dilated space pooling pyramid network includes a third downsampling layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, and a seventh convolutional layer; The third downsampling layer, the fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer and the seventh convolutional layer are respectively connected to the output layer of the ConvNeXt; The fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer and the seventh convolutional layer are connected in a cascade manner using a splicing module; The third downsampling layer is a mean downsampling layer, the size of the fourth convolutional layer is 3×3, and the expansion rate of the fourth convolutional layer is 6, the size of the fifth convolutional layer is 3×3, and the expansion rate of the fifth convolutional layer is 12, the size of the sixth convolutional layer is 3×3, and the expansion rate of the sixth convolutional layer is 18, and the size of the seventh convolutional layer is 1×1.

7. The model deployment method according to claim 1, characterized in that: The obtaining of a sample set for sea ice detection comprises: Acquire a dual-polarization synthetic aperture radar (SAR) sample set corresponding to the satellite, wherein the dual-polarization SAR sample set includes a first dual-polarization SAR sample set and a second dual-polarization SAR sample set, wherein the first dual-polarization SAR sample set includes an image label, and the second dual-polarization SAR sample set does not have an image label; Performing model training based on the first dual-polarization SAR sample set and the image labels of the first dual-polarization SAR sample set to obtain a fourth detection model; Performing sea ice detection on the second dual-polarization SAR sample set based on the fourth detection model to obtain a detection result of the second dual-polarization SAR sample set; The sample set is constructed by using the first dual-polarization SAR sample set, the image label of the first dual-polarization SAR sample set, the second dual-polarization SAR sample set, and the detection result of the second polarization SAR sample set.

8. A model deployment device, characterized in that: include: A sample acquisition module, used to obtain a sample set for sea ice detection; A first model training module, used for performing model training based on the sample set, and obtaining a first detection model after the training is completed; A model compression module, used for compressing the first detection model to obtain a second detection model, wherein the second detection model has the same network structure as the first detection model, and the number of parameters of the second detection model is smaller than the number of parameters of the first detection model; A second model training module is used to train the second detection model based on the first detection model and the sample set, and obtain a third detection model after the training; a format conversion module, used for performing format conversion on the third detection model to obtain a fourth detection model, wherein the format of the fourth detection model is an open neural network exchange format; A model deployment module is used to deploy the fourth detection model in the open neural network exchange format in a satellite hardware environment for sea ice detection.

9. An electronic device, characterized in that: The device includes: a processor, a memory, and a system bus; The processor and the memory are connected via the system bus; The memory is used to store a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor performs the steps of the model deployment method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a terminal device, the steps of the model deployment method described in any one of claims 1 to 7 are implemented.

Images