Satellite cloud picture segmentation method and device, equipment and storage medium
By utilizing the dynamic routing mechanism and spatial relationship learning of convolutional capsule neural networks, the problem of poor adaptability to multi-scale characteristics in satellite cloud image segmentation is solved, achieving higher segmentation accuracy and applicability.
Patent Information
- Application Number
- CN202510771063.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-10
- Publication Date
- 2025-11-18
AI Technical Summary
Existing satellite cloud image segmentation methods rely on manually designed features and rules, which are difficult to adapt to the diversity and complexity of clouds. Traditional convolutional neural networks have difficulty handling the multi-scale characteristics of clouds when the receptive field is fixed, resulting in poor segmentation accuracy and applicability.
Convolutional capsule neural networks are used for satellite cloud image segmentation. Through the dynamic routing mechanism and powerful spatial relationship learning ability of convolutional capsule neural networks, the hierarchical relationship between different cloud clusters in satellite cloud images can be identified and captured, adapting to the multi-scale characteristics of clouds.
It improves the accuracy and applicability of satellite cloud image segmentation, enabling better identification of changes in the spatial location of clouds and solving the problem of scale changes when the receptive field is fixed in traditional methods.
Smart Images
Figure CN120976235A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present disclosure relates to the technical field of remote sensing image processing, and particularly relates to a satellite cloud image segmentation method and device, equipment and a storage medium. BACKGROUND
[0002] Satellite cloud images can be obtained through satellite remote sensing technology. Satellite cloud images play a crucial role in meteorology, climate research, and natural disaster monitoring. Among them, cloud coverage information is of great significance for weather forecasting, climate simulation, and natural disaster monitoring. However, due to the complexity and variability of clouds, accurate segmentation of satellite cloud images has always been a challenging problem.
[0003] Currently, traditional satellite cloud image segmentation methods usually rely on manually designed features and rules, which are not sufficient in the face of the diversity and complexity of clouds. Due to the great changes in the shape and size of clouds, traditional Convolutional Neural Networks (CNN) are difficult to adapt to the multi-scale characteristics of clouds under the condition of fixed receptive field, which leads to low accuracy and poor applicability of current cloud image segmentation. SUMMARY
[0004] To solve the above technical problems, the present disclosure provides a satellite cloud image segmentation method, device, equipment and storage medium.
[0005] The first aspect of the present disclosure provides a satellite cloud image segmentation method, comprising:
[0006] obtaining a target satellite cloud image;
[0007] inputting the target satellite cloud image into a target convolutional capsule neural network, identifying and segmenting each cloud cluster region in the target satellite cloud image based on the target convolutional capsule neural network, and obtaining a segmentation result of the target satellite cloud image.
[0008] The second aspect of the present disclosure provides a satellite cloud image segmentation device, comprising:
[0009] a first obtaining module configured to obtain a target satellite cloud image;
[0010] a segmentation module configured to input the target satellite cloud image into a target convolutional capsule neural network, identify and segment each cloud cluster region in the target satellite cloud image based on the target convolutional capsule neural network, and obtain a segmentation result of the target satellite cloud image.
[0011] The third aspect of the present disclosure provides a computer device comprising a memory and a processor, wherein the memory stores a computer program which, when executed by the processor, can implement the satellite cloud image segmentation method of the first aspect.
[0012] A fourth aspect of the present disclosure provides a computer-readable storage medium, and the computer program is stored in the storage medium and can implement the satellite cloud image segmentation method of the first aspect when the computer program is executed by a processor.
[0013] Compared with the prior art, the technical solutions provided by the present disclosure have the following advantages:
[0014] The present disclosure obtains a target satellite cloud image, inputs the target satellite cloud image into a target convolution capsule neural network, identifies and segments each cloud cluster region in the target satellite cloud image based on the target convolution capsule neural network, and obtains a segmentation result of the target satellite cloud image. Due to the strong spatial relationship learning and identification capability of the convolution capsule neural network, through the dynamic routing mechanism, the convolution capsule neural network can accurately identify and capture the hierarchical relationship between different cloud cluster regions in the satellite cloud image, can better adapt to the multi-scale characteristics of clouds, can accurately identify the spatial position change of clouds, can effectively solve the problem that the traditional convolution neural network is difficult to handle the scale change under the condition of fixed receptive field, and can improve the accuracy and applicability of satellite cloud image segmentation. BRIEF DESCRIPTION OF DRAWINGS
[0015] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.
[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings required to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.
[0017] Figure 1 is a flowchart of a satellite cloud image segmentation method provided by an embodiment of the present disclosure;
[0018] Figure 2 is a flowchart of a target convolution capsule neural network training method provided by an embodiment of the present disclosure;
[0019] Figure 3 is a structural schematic diagram of a satellite cloud image segmentation device provided by an embodiment of the present disclosure;
[0020] Figure 4 is a structural schematic diagram of a computer device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0021] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the schemes of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0022] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other different manners from those described herein; obviously, the embodiments described in the specification are only a part of the embodiments of the present disclosure, and not all the embodiments.
[0023] It should be understood that each step recorded in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.
[0024] It should be noted that, in this document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between or among the entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a... " does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.
[0025] It should be noted that the modification of "one" and "multiple" mentioned in the present disclosure is illustrative rather than limiting, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".
[0026] In order to better understand the inventive concept of the embodiments of the present disclosure, the technical solutions of the embodiments of the present disclosure will be described below in conjunction with exemplary embodiments.
[0027] Figure 1 is a flowchart of a satellite cloud image segmentation method provided by the embodiments of the present disclosure. The method can be executed by a computer device, which can be understood as any device with computing function and processing capability. As shown in Figure 1 The satellite cloud image segmentation method provided by the embodiments of the present disclosure includes the following steps:
[0028] Step 110, acquiring a target satellite cloud image.
[0029] In the embodiments of the present disclosure, the computer device can obtain a target satellite cloud image to be segmented.
[0030] In step 120, the target satellite cloud image is input into the target convolutional capsule neural network, each cloud cluster region in the target satellite cloud image is identified and segmented based on the target convolutional capsule neural network, and a segmentation result of the target satellite cloud image is obtained.
[0031] In the embodiments of the present disclosure, after obtaining the target satellite cloud image, the computer device can input the target satellite cloud image into the target convolutional capsule neural network, identify and segment each cloud cluster region in the target satellite cloud image based on the target convolutional capsule neural network, and obtain a segmentation result of the target satellite cloud image.
[0032] The target convolutional capsule neural network is pre-set in the computer device. The convolutional capsule neural network (Capsule Neural Network, CapsNet) can be understood as a hybrid architecture that combines the advantages of convolutional neural network (CNN) and capsule network (Capsule Network). Compared with the traditional convolutional neural network, the convolutional capsule neural network has strong spatial relationship learning ability and can accurately capture the spatial relationship between different entities in an image.
[0033] The target convolutional capsule neural network in the embodiments of the present disclosure can identify the shape and structural changes of each cloud in the target satellite cloud image, and identify the hierarchical relationship between different cloud layers in the target satellite cloud image, thereby identifying and segmenting each cloud cluster region in the target satellite cloud image, and obtaining a segmentation result of the target satellite cloud image.
[0034] The segmentation result of the target satellite cloud image can include at least one of the position of each cloud, the boundary of the cloud, the cloud layer region where the cloud is located, each cloud layer region contained in the target satellite cloud image, and the positional relationship between different cloud layer regions, and other data, which is not limited here.
[0035] In some embodiments, the above inputting the target satellite cloud image into the target convolutional capsule neural network, identifying and segmenting each cloud cluster region in the target satellite cloud image based on the target convolutional capsule neural network, and obtaining a segmentation result of the target satellite cloud image can include steps 1201-1205:
[0036] In step 1201, the target satellite cloud image is feature extracted based on a convolutional layer in the target convolutional capsule neural network, and a plurality of basic cloud cluster features are obtained.
[0037] In the embodiments of the present disclosure, the target convolutional capsule neural network comprises a convolutional layer, and the computer device can perform feature extraction (convolution operation) on the target satellite cloud image based on the convolutional layer in the target convolutional capsule neural network to obtain a plurality of basic cloud cluster features of the target satellite cloud image.
[0038] The basic cloud cluster features can comprise edge, texture and other features of the cloud.
[0039] In step 1202, the computer device converts the plurality of basic cloud cluster features into a plurality of primary capsule vectors based on the plurality of primary capsules in the target convolutional capsule neural network, each primary capsule vector corresponding to a primary cloud cluster feature, the length of the primary capsule vector being used to represent the existence probability of the primary cloud cluster feature corresponding to the primary capsule vector, and the direction of the primary capsule vector being used to represent the position of the primary cloud cluster feature corresponding to the primary capsule vector.
[0040] In the embodiments of the present disclosure, the target convolutional capsule neural network comprises a primary capsule layer, and the primary capsule layer comprises a plurality of primary capsules.
[0041] The computer device can convert the plurality of basic cloud cluster features into a plurality of primary capsule vectors based on the plurality of primary capsules in the target convolutional capsule neural network, each primary capsule vector corresponding to a primary cloud cluster feature, the length of the primary capsule vector being used to represent the existence probability of the primary cloud cluster feature corresponding to the primary capsule vector, and the direction of the primary capsule vector being used to represent the position of the primary cloud cluster feature corresponding to the primary capsule vector.
[0042] The position of the primary cloud cluster feature can be understood as the spatial position of the primary cloud cluster feature.
[0043] For example, the primary capsule vector can be represented by the following formula (1):
[0044]
[0045] wherein v ij represents the output primary capsule vector; Squash represents a Squash function for performing nonlinear activation on data and compressing the data to a certain range, for example, the range of (0, 1); u ijk represents the input basic cloud cluster feature; c ijk represents the weight (initial coupling coefficient) of the primary capsule.
[0046] In some embodiments, for each primary capsule, a multi-scale convolutional processing can be performed on the plurality of basic cloud cluster features based on the plurality of different size convolutional kernels contained in the primary capsule to obtain the primary capsule vector corresponding to the primary capsule. In this way, the network can effectively extract features of clouds of different scales, and different size convolutional kernels or multi-resolution inputs can be used.
[0047] For example, the multi-scale convolution processing can be represented by the following formula (2):
[0048] X multiscale = Concatenate(Conv kernel_size 3(X), Conv kernel_size 5(X),...) (2);
[0050] wherein X multiscale represents the output of the multi-scale convolution, i.e., the primary capsule vector; C oncatenate represents a connection function; Conv kernel_size 3(X) represents a convolution operation using a 3x3 convolution kernel.
[0051] Step 1203, based on the dynamic routing mechanism, determine the connection weight between each primary capsule and each high-level capsule.
[0052] In the embodiments of the present disclosure, the computer device can determine the connection weight between each primary capsule and each high-level capsule based on the dynamic routing mechanism (Dynamic Routing),
[0053] The dynamic routing mechanism can transmit the output of the low-level capsule to the high-level capsule according to the coupling coefficient between the low-level capsule and the high-level capsule. This mechanism allows the network to adaptively learn and adjust the feature representation, so as to better capture the spatial relationship and entity attribute in the image, and explicitly model the "part-whole" relationship.
[0054] Step 1204, based on the connection weight, aggregate the plurality of primary capsule vectors in each high-level capsule to obtain a plurality of high-level capsule vectors.
[0055] In the embodiments of the present disclosure, the computer device can aggregate the plurality of primary capsule vectors in each high-level capsule based on the connection weight between each primary capsule and each high-level capsule, to obtain a plurality of high-level capsule vectors.
[0056] Step 1205, based on the plurality of high-level capsule vectors, identify and segment each cloud cluster region in the target satellite cloud image to obtain a segmentation result of the target satellite cloud image.
[0057] The present disclosure obtains a target satellite cloud image, inputs the target satellite cloud image into a target convolutional capsule neural network, identifies and segments each cloud cluster region in the target satellite cloud image based on the target convolutional capsule neural network, and obtains a segmentation result of the target satellite cloud image. Due to the strong spatial relationship learning and identification capability of the convolutional capsule neural network, through the dynamic routing mechanism, the convolutional capsule neural network can accurately identify and capture the hierarchical relationship between different cloud cluster regions in the satellite cloud image, can better adapt to the multi-scale characteristics of clouds, can accurately identify the spatial position change of clouds, can effectively solve the problem that the traditional convolutional neural network is difficult to process scale changes in the case of fixed receptive field, and can improve the accuracy and applicability of satellite cloud image segmentation.
[0058] In some embodiments, the capsule structure in the target convolutional capsule neural network can be a nested capsule (Nested Capsules) structure to better model the hierarchical relationship between different cloud layers.
[0059] For example, the nested capsule can be represented by the following formula (3):
[0060]
[0061] wherein c nested represents the weight in the dynamic routing of the nested capsule, represents the importance of the first sub-capsule in the nested capsule, and these weights are calculated by a dynamic routing algorithm; u nested represents a vector input to the first sub-capsule of the nested capsule; v nested represents an output vector of the nested capsule, which contains the encoding of the input information by the nested capsule.
[0062] In some embodiments of the present disclosure, before the target satellite cloud image is input into the target convolutional capsule neural network, the computer device can perform Figure 2 The flowchart of the target convolutional capsule neural network training method provided in the present disclosure is shown in FIG. 2, and the target convolutional capsule neural network training method provided in the present embodiment includes the following steps: Figure 2
[0063] Step 210, obtaining a plurality of satellite cloud images and cloud cluster position annotation data of each satellite cloud image.
[0064] In the present embodiment of the present disclosure, the computer device can obtain a plurality of satellite cloud images and cloud cluster position annotation data of each satellite cloud image from a satellite remote sensing platform. The plurality of satellite cloud images can include satellite cloud images of different regions, different seasons, and different cloud coverage conditions.
[0065] The cloud cluster position annotation data can include at least one of cloud position data, cloud boundary data, cloud layer position data, and position relationship data between different cloud layers in the satellite cloud image.
[0066] Specifically, the cloud cluster position annotation data can be converted into a format required by the convolutional capsule neural network, which is usually a pixel-level label map including the annotation data of each pixel in the satellite cloud image.
[0067] In some embodiments, the computer device can pre-process each satellite cloud image based on a preset pre-processing manner to obtain a pre-processed satellite cloud image, and then construct a satellite cloud image dataset based on the pre-processed satellite cloud image and the cloud cluster position annotation data of each satellite cloud image.
[0068] The preset pre-processing manner can include at least one of image normalization processing, image cropping processing, and image enhancement processing.
[0069] Through pre-processing, more training samples can be obtained to ensure the quality and consistency of the images input into the convolutional capsule neural network.
[0070] Specifically, image normalization processing of the satellite cloud image can convert the pixel value of the satellite cloud image to between 0 and 1, which helps the convolutional capsule neural network to better learn and generalize. For example, the image normalization processing expression can be as follows (4):
[0071]
[0072] wherein X normalized represents the satellite cloud image after image normalization processing; X represents the initial satellite cloud image; mean(X) represents the mean value of the initial satellite cloud image; and std(X) represents the standard deviation of the initial satellite cloud image.
[0073] Image enhancement processing can be understood as a function of a series of random transformations or deformations on the image. The purpose of image enhancement is to make the model more robust and improve the generalization ability by introducing changes. Image enhancement processing can include at least one of rotation, scaling, flipping, translation, and brightness adjustment. For example, the image enhancement processing expression can be as follows (5):
[0074] X augmented = augmentation_function(X) (5);
[0075] wherein augmentation_function represents an image enhancement processing function, which is a function of a series of random transformations or deformations on the input image during training; and Xaugmented represents the satellite cloud image after image enhancement processing.
[0076] Step 220, constructing a satellite cloud image dataset based on the plurality of satellite cloud images and the cloud cluster position annotation data of each satellite cloud image.
[0077] Step 230, input the satellite cloud image dataset into the convolution capsule neural network, and iteratively train the convolution capsule neural network based on the satellite cloud image dataset until the accuracy of the cloud image segmentation result output by the convolution capsule neural network meets the preset requirement, and the training of the convolution capsule neural network is completed.
[0078] In the embodiments of the present disclosure, the computer device can input the satellite cloud image dataset into the convolution capsule neural network, iteratively train the convolution capsule neural network based on the satellite cloud image dataset, until the accuracy of the cloud image segmentation result output by the convolution capsule neural network meets the preset accuracy requirement, and the training of the convolution capsule neural network is completed.
[0079] The preset accuracy requirement can be set as needed, which is not limited here.
[0080] In some embodiments, the computer device can calculate at least one of the reconstruction loss, the learning rate and the regularization coefficient in the training process of the convolution capsule neural network; based on at least one of the reconstruction loss, the learning rate and the regularization coefficient, adjust the parameters of the convolution capsule neural network until the accuracy of the cloud image segmentation result output by the convolution capsule neural network meets the preset accuracy requirement, and the training of the convolution capsule neural network is completed.
[0081] Among them, the reconstruction loss is introduced into the convolution capsule neural network, which is used to encourage the network to learn important features, and is helpful to improve the generalization performance of the model.
[0082] For example, the reconstruction loss L reconstruction It can be calculated by the following formula (6):
[0083] L reconstruction =||X-X reconstructead || 2 | (6);
[0084] Among them, X reconstructed represents the reconstructed image obtained by reconstructing through the convolution capsule neural network model; X represents the original image, that is, the original satellite image.
[0085] The reconstruction process is as follows:
[0086] (1) The capsule vector (high-level capsule vector) output by the capsule is operated in reverse to obtain a reconstructed image, which usually includes:
[0087] Reconstruction layer (Reconstruction Layer): a special reconstruction layer, whose goal is to generate a reconstructed image with the same size as the input image;
[0088] Reconstruction Weights: Learning how to map the capsule output back to the input image;
[0089] Reconstruction Activation Function: Converting the output of the reconstruction layer to pixel values.
[0090] (2) The goal of the reconstruction process is to minimize the reconstruction loss: The reconstruction loss is usually measured using Mean Squared Error (MSE) or other appropriate loss functions to measure the difference between the reconstructed image and the original input image.
[0091] Use dynamic learning rate adjustment or other optimization strategies to ensure the stability and convergence of the training process, such as (7):
[0092]
[0093] where LR represents the learning rate; ILR represents the initial learning rate; decay represents the learning rate decay coefficient; epoch represents the current training round.
[0094] Introduce appropriate regularization methods, such as Dropout, in the network to avoid overfitting, regularization can be understood as each neuron in the training phase is retained with probability p, the output of the discarded neuron is zero. For example, regularization can be represented by the following formula (8):
[0095] u dropout =u original ⊙Bernoulli(dropout_probability) (8);
[0096] where Bernoulli(dropout_probability) represents Bernoulli distribution sampling with a given probability, used to simulate Dropout, u orginal represents the original input vector.
[0097] In some embodiments, the computer device can also evaluate the trained convolutional capsule neural network, use an independent test set to verify its performance on unseen data, investigate the robustness and generalization ability of the model under different scenarios, seasons and cloud conditions, and then optimize and optimize the convolutional capsule neural network according to the evaluation results to further improve the segmentation accuracy and applicability.
[0098] Specifically, the model fusion technology can be used to combine the prediction results of multiple models to further improve the overall performance. If there is a large amount of pre-training data, transfer learning can be considered to speed up the convergence of the model and improve the generalization performance. The hyperparameters of the network, such as the learning rate, the number of capsules, and the capsule dimension, are carefully adjusted to obtain the best performance.
[0099] Specifically, appropriate evaluation indicators such as cross-entropy loss, intersection over union (IoU), etc. can be selected to comprehensively evaluate the performance of the convolutional capsule neural network on the test set. For example, the cross-entropy loss can be represented by the following formula (9):
[0100]
[0101] where L represents the cross-entropy loss; y represents the actual label (ground truth); represents the prediction output of the convolutional capsule neural network; and C represents the number of classes.
[0102] Step 240, determining the trained convolutional capsule neural network as a target convolutional capsule neural network.
[0103] Thus, the target convolutional capsule neural network can be trained by the satellite cloud image dataset, so that the target convolutional capsule neural network has strong spatial relationship learning and recognition capabilities. Through the dynamic routing mechanism, the target convolutional capsule neural network can accurately model the hierarchical relationship between different cloud cluster regions in the satellite cloud image, better adapt to the multi-scale characteristics of the cloud, accurately recognize the spatial position changes of the cloud, effectively solve the problem that the traditional convolutional neural network is difficult to handle the scale changes under the fixed receptive field, and improve the accuracy and applicability of the satellite cloud image segmentation.
[0104] In some embodiments of the present disclosure, after obtaining the segmentation result of the target satellite cloud image, the computer device can display the segmentation result of the target satellite cloud image based on a preset visualization manner, so as to visually display the segmentation result of the target satellite cloud image and improve the display effect.
[0105] The preset visualization manner can be set as needed, for example, it can be an image, a video, etc., which is not limited here.
[0106] Figure 3 is a structural schematic diagram of a satellite cloud image segmentation device provided by an embodiment of the present disclosure. The device can be understood as the above-mentioned computer device or part of the function modules in the above-mentioned computer device. As shown in Figure 3 The satellite cloud image segmentation device 300 includes:
[0107] The first obtaining module 310 is configured to obtain a target satellite cloud image.
[0108] The segmentation module 320 is configured to input the target satellite cloud image into a target convolutional capsule neural network, identify and segment each cloud cluster region in the target satellite cloud image based on the target convolutional capsule neural network, and obtain a segmentation result of the target satellite cloud image.
[0109] Optionally, the satellite cloud image segmentation apparatus includes:
[0110] The second obtaining module is configured to obtain a plurality of satellite cloud images and cloud cluster position labeling data of each satellite cloud image.
[0111] The construction module is configured to construct a satellite cloud image dataset based on the plurality of satellite cloud images and the cloud cluster position labeling data of each satellite cloud image.
[0112] The training module is configured to input the satellite cloud image dataset into the convolutional capsule neural network, iteratively train the convolutional capsule neural network based on the satellite cloud image dataset, and complete the training of the convolutional capsule neural network until the accuracy of the cloud image segmentation result output by the convolutional capsule neural network meets a preset accuracy requirement.
[0113] The determination module is configured to determine the trained convolutional capsule neural network as the target convolutional capsule neural network.
[0114] Optionally, the construction module includes:
[0115] The preprocessing submodule is configured to pre-process each satellite cloud image based on a preset preprocessing manner to obtain a pre-processed satellite cloud image.
[0116] The construction submodule is configured to construct the satellite cloud image dataset based on the pre-processed satellite cloud image and the cloud cluster position labeling data of each satellite cloud image.
[0117] The preset preprocessing manner includes at least one of image normalization processing, image cropping processing, and image enhancement processing.
[0118] The image enhancement processing includes at least one of rotation, scaling, flipping, translation, and brightness adjustment.
[0119] Optionally, the training module includes:
[0120] The calculation submodule is configured to calculate at least one of reconstruction loss, learning rate, and a regularization coefficient in the training process of the convolutional capsule neural network.
[0121] The adjusting sub-module is configured to adjust parameters of the convolutional capsule neural network based on at least one of the reconstruction loss, the learning rate, and the regularization coefficient until the accuracy of the cloud image segmentation result output by the convolutional capsule neural network meets a preset accuracy requirement, and the training of the convolutional capsule neural network is completed.
[0122] Optionally, the segmentation module comprises:
[0123] The feature extraction sub-module is configured to perform feature extraction on the target satellite cloud image based on a convolutional layer in the target convolutional capsule neural network, and obtain a plurality of basic cloud cluster features of the target satellite cloud image.
[0124] The feature conversion sub-module is configured to convert the plurality of basic cloud cluster features into a plurality of primary capsule vectors based on a plurality of primary capsules in the target convolutional capsule neural network, each primary capsule vector corresponding to a primary cloud cluster feature, a length of the primary capsule vector being used to represent an existence probability of the primary cloud cluster feature corresponding to the primary capsule vector, and a direction of the primary capsule vector being used to represent a position of the primary cloud cluster feature corresponding to the primary capsule vector.
[0125] The weight determination sub-module is configured to determine a connection weight between each primary capsule and each high-level capsule based on a dynamic routing mechanism.
[0126] The aggregation sub-module is configured to aggregate the plurality of primary capsule vectors in each high-level capsule based on the connection weight, and obtain a plurality of high-level capsule vectors.
[0127] The segmentation sub-module is configured to identify and segment each cloud cluster region in the target satellite cloud image based on the plurality of high-level capsule vectors, and obtain a segmentation result of the target satellite cloud image.
[0128] Optionally, the feature conversion sub-module comprises:
[0129] The multi-scale convolution unit is configured to perform multi-scale convolution processing on the plurality of basic cloud cluster features based on a plurality of different sizes of convolution kernels contained in each primary capsule, and obtain a primary capsule vector corresponding to the primary capsule.
[0130] Optionally, the capsule structure in the target convolutional capsule neural network is a nested capsule structure.
[0131] The satellite cloud image segmentation device provided by the embodiments of the present disclosure can implement the method of any one of the above-mentioned embodiments, and has similar implementation manners and beneficial effects, which will not be described here again.
[0132] The embodiments of the present disclosure further provide a computer device, comprising a processor and a memory, wherein the memory stores a computer program which, when executed by the processor, can implement the method of any of the above embodiments, and has similar implementation manners and beneficial effects, which will not be described herein again.
[0133] The computer device in the embodiments of the present disclosure can be understood as any device with processing and computing capabilities, which can include but is not limited to mobile terminals such as smart phones, notebook computers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), vehicle-mounted terminals, etc., and fixed electronic devices such as digital TVs, desktop computers, etc.
[0134] Figure 4 is a structural schematic diagram of a computer device provided by the embodiments of the present disclosure, as Figure 4 shown, the computer device 400 can include a processor 410 and a memory 420, wherein the memory 420 stores a computer program 421, which, when executed by the processor 410, can implement the method provided by any of the above embodiments, and has similar implementation manners and beneficial effects, which will not be described herein again.
[0135] Of course, in order to simplify, Figure 4 only some of the components of the computer device 400 related to the present disclosure are shown in the figure, and components such as buses, input / output interfaces, input devices and output devices are omitted. In addition, according to specific application circumstances, the computer device 400 can also include any other appropriate components.
[0136] The embodiments of the present disclosure provide a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the method of any of the above embodiments can be implemented, and has similar implementation manners and beneficial effects, which will not be described herein again.
[0137] The above computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more conductive wires, portable disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any appropriate combination of the above.
[0138] The computer program above can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer device, partly on the user's computer device, as a stand-alone software package, partly on the user's computer device and partly on a remote computer device or entirely on the remote computer device or server.
[0139] The above description is only preferred embodiments of the present disclosure and a description of the principles of the technology used. Those skilled in the art should understand that the disclosed range of the present disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) having similar functions.
[0140] In addition, although each process is described in a specific order, this should not be understood as requiring the processes to be performed in the specific order shown or in a sequential order. In certain circumstances, multi-tasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments.
[0141] The above description is only a specific implementation of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A satellite cloud image segmentation method, characterized in that, include: Obtain target satellite cloud imagery; The target satellite cloud image is input into a target convolutional capsule neural network. Based on the target convolutional capsule neural network, each cloud region in the target satellite cloud image is identified and segmented to obtain the segmentation result of the target satellite cloud image.
2. The method according to claim 1, characterized in that, Before inputting the target satellite cloud image into the target convolutional capsule neural network, the method further includes: Acquire multiple satellite cloud images and cloud cluster location annotation data for each satellite cloud image; Based on the multiple satellite cloud images and the cloud cluster location annotation data of each satellite cloud image, a satellite cloud image dataset is constructed; The satellite cloud image dataset is input into the convolutional capsule neural network. The convolutional capsule neural network is iteratively trained based on the satellite cloud image dataset until the accuracy of the cloud image segmentation result output by the convolutional capsule neural network meets the preset accuracy requirement, thus completing the training of the convolutional capsule neural network. The trained convolutional capsule neural network is identified as the target convolutional capsule neural network.
3. The method according to claim 2, characterized in that, The satellite cloud image dataset is constructed based on the multiple satellite cloud images and the cloud cluster location annotation data of each satellite cloud image, including: The satellite cloud images are preprocessed according to a preset preprocessing method to obtain preprocessed satellite cloud images; Based on the preprocessed satellite cloud images and the cloud cluster location annotation data of each satellite cloud image, a satellite cloud image dataset is constructed; The preset preprocessing method includes at least one of image normalization processing, image cropping processing, and image enhancement processing; The image enhancement processing includes at least one of rotation, scaling, flipping, translation, and brightness adjustment.
4. The method according to claim 2, characterized in that, The iterative training of the convolutional capsule neural network based on the satellite cloud image dataset until the accuracy of the cloud image segmentation result output by the convolutional capsule neural network meets the preset accuracy requirement, thereby completing the training of the convolutional capsule neural network, includes: Calculate at least one of the following data during the training process of the convolutional capsule neural network: reconstruction loss, learning rate, and regularization coefficient; Based on at least one of the reconstruction loss, the learning rate, and the regularization coefficient, the parameters of the convolutional capsule neural network are adjusted until the accuracy of the cloud map segmentation result output by the convolutional capsule neural network meets the preset accuracy requirement, thereby completing the training of the convolutional capsule neural network.
5. The method according to claim 1, characterized in that, The process of identifying and segmenting various cloud regions in the target satellite cloud image based on the target convolutional capsule neural network to obtain the segmentation result of the target satellite cloud image includes: Based on the convolutional layers in the target convolutional capsule neural network, feature extraction is performed on the target satellite cloud image to obtain multiple basic cloud cluster features of the target satellite cloud image; Based on the multiple primary capsules in the target convolutional capsule neural network, the multiple basic cloud features are converted into multiple primary capsule vectors. Each primary capsule vector corresponds to a primary cloud feature. The length of the primary capsule vector is used to characterize the existence probability of the primary cloud feature corresponding to the primary capsule vector, and the direction of the primary capsule vector is used to characterize the position of the primary cloud feature corresponding to the primary capsule vector. Based on a dynamic routing mechanism, the connection weights between each primary capsule and each advanced capsule are determined; Based on the connection weights, multiple primary capsule vectors are aggregated in each advanced capsule to obtain multiple advanced capsule vectors; Based on multiple advanced capsule vectors, each cloud region in the target satellite cloud image is identified and segmented to obtain the segmentation result of the target satellite cloud image.
6. The method according to claim 5, characterized in that, The process of converting multiple basic cloud features into multiple primary capsule vectors based on multiple primary capsules in the target convolutional capsule neural network includes: For each primary capsule, based on the multiple convolution kernels of different sizes contained in the primary capsule, multi-scale convolution processing is performed on the multiple basic cloud features to obtain the primary capsule vector corresponding to the primary capsule.
7. The method according to any one of claims 1-6, characterized in that, The capsule structure in the target convolutional capsule neural network is a nested capsule structure.
8. A satellite cloud image segmentation device, characterized in that, include: The first acquisition module is used to acquire target satellite cloud images; The segmentation module is used to input the target satellite cloud image into the target convolutional capsule neural network, and to identify and segment each cloud region in the target satellite cloud image based on the target convolutional capsule neural network to obtain the segmentation result of the target satellite cloud image.
9. A computer device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the satellite cloud image segmentation method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the satellite cloud image segmentation method as described in any one of claims 1-7.