Ground-based cloud classification method and system based on deep learning and cloud field knowledge fusion

Abstract

The application discloses a ground cloud classification method and system based on deep learning and cloud field knowledge fusion, and the method comprises the following steps: obtaining a ground cloud image to be classified and performing pretreatment; inputting the ground cloud image after the pretreatment into a ground cloud deep learning model for classification, and outputting the category of the ground cloud; wherein the ground cloud deep learning model comprises a double-branch structure, one branch outputs a cloud genus classification result, and the other branch generates a multi-dimensional rule vector as intermediate supervision information in a training stage, and the multi-dimensional rule vector corresponds to a multi-dimensional binary label; the training of the ground cloud deep learning model is completed under the guidance of a loss function; and the loss function fuses a cross-entropy classification loss of a cloud genus label, a supervision loss of a rule label and a structural constraint loss between rule vector dimensions. The application integrates cloud field knowledge into the training of the deep learning model, and improves the accuracy and reliability of cloud genus classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a ground cloud classification method based on deep learning and cloud field knowledge fusion. BACKGROUND

[0002] Meteorological observation is the basis of meteorological services, and meteorological observation technology is a significant symbol of modernization. Clouds, as an important phenomenon of atmospheric hydrological cycle, have a huge impact on global climate and local weather, and their observation is a key task of meteorological services.

[0003] With the development of technology, cloud observation has gradually shifted from manual to automatic. Automatic cloud observation mainly includes two categories: physical property measurement based on sensors (such as laser cloud height meters, etc.) and image analysis of visual imaging, including space-based and ground-based observations. Although satellite cloud images play an important role in large-scale observation, they have limited capabilities in local fine observation, and ground-based cloud images effectively complement this deficiency. Cloud recognition is one of the core tasks of ground-based cloud image analysis, and although deep learning has made progress in image analysis, existing algorithms still cannot meet the demand in terms of accuracy and detail.

[0004] Ground-based cloud images have strong professional and special characteristics, and their fine-grained classification and recognition are difficult. Compared with ordinary scene images, ground-based cloud images have limited color, contour, and semantic information, so deep learning networks pre-trained based on ordinary images do not work well. Ye's research also shows that increasing the depth of the network does not effectively improve the accuracy of ground-based cloud image cloud shape classification, and may even result in a decrease in accuracy. Research on how to integrate the physical characteristics and professional knowledge of ground-based cloud images into the construction and training of deep learning networks has important theoretical and application value.

[0005] Secondly, some cloud genera have high similarity in ground-based cloud images and are difficult to distinguish. Therefore, many studies do not classify according to the ten cloud genera defined by the World Meteorological Organization, but instead merge some difficult-to-distinguish cloud genera or redefine them into 6 categories or fewer based on color, texture, thickness, and other significant differences. For example, Liu defined clouds into 7 categories by adding multi-modal information of clouds and using graph convolution to utilize the correlation between different clouds to achieve cloud classification. Ye et al. tried to enhance the distinguishability of each cloud genus in the feature space through an improved measure learning method, although some progress was made, but it was still not ideal.

[0006] Currently, most cloud shape classification and recognition algorithms for ground-based cloud images rely on supervised learning data-driven models, which simply analyze and transform features from the perspective of classification of a large number of samples, ignoring professional knowledge such as the definition of cloud genera and their basic visual attributes, thus limiting the further improvement of cloud shape classification accuracy and detail. SUMMARY

[0007] The present application mainly aims to provide a ground cloud classification method and system based on deep learning and cloud field knowledge, which can improve the accuracy and reliability of cloud classification.

[0008] The technical solution adopted by the present application is:

[0009] The present application provides a ground cloud classification method based on deep learning and cloud field knowledge, comprising the following steps:

[0010] S1, obtaining a ground cloud image to be classified;

[0011] S2, preprocessing the ground cloud image to be classified, including cropping, scaling, flipping and normalization operations;

[0012] S3, inputting the preprocessed ground cloud image into a ground cloud deep learning model for classification, and outputting the category of the ground cloud;

[0013] The construction and training process of the ground cloud deep learning model is as follows:

[0014] A ground cloud training set is constructed, and after the ground cloud images in the ground cloud training set are preprocessed, cloud genus labels representing cloud genus categories are marked, and structured coding is performed according to a preset expert ground cloud discrimination process to construct corresponding rule labels, which are multi-dimensional binary labels, each dimension corresponding to a discrimination node and the corresponding discrimination result in the expert ground cloud discrimination process;

[0015] The ground cloud deep learning model is constructed, including a double-branch structure, one branch for outputting cloud genus classification results, and the other branch for generating a multi-dimensional rule vector as intermediate supervision information in the training stage, the multi-dimensional rule vector corresponding to the multi-dimensional binary label;

[0016] A loss function is constructed, and the training and testing of the ground cloud deep learning model are completed under the guidance of the loss function; the loss function fuses the cross-entropy classification loss of the cloud genus label, the supervision loss of the rule label and the structural constraint loss between the dimensions of the rule vector; wherein the structural constraint between the dimensions of the rule vector is realized by limiting the difference between the predicted values of a certain dimension subset and another dimension subset in the multi-dimensional rule vector.

[0017] According to the above technical solution, the preprocessing is specifically:

[0018] A region is randomly cropped from the ground cloud image, and the area and the aspect ratio of the cropped region are randomly selected within a specified range;

[0019] The cropped ground cloud image is scaled to an appropriate size;

[0020] The randomly cropped and scaled image is randomly horizontally flipped with a preset probability;

[0021] The normalized ground cloud image is obtained by performing normalization on the random cropped, scaled and flipped ground cloud image.

[0022] According to the above technical solution, the cloud category includes cumulonimbus, cumulus, stratus, altostratus, nimbostratus, stratocumulus, cirrus, cumulus, altocumulus and stratocumulus.

[0023] According to the above technical solution, the determination nodes in the expert ground cloud determination process include:

[0024] The first node: determine whether there is a clear and visible single pile cloud structure;

[0025] The second node: determine whether the upper end of the cloud is profiled;

[0026] The third node: determine whether the cloud layer has the same consistent, continuous or intermittent characteristics, and does not have cloud rolls or cloud elements;

[0027] The fourth node: determine whether the sun or moon appears as a bright piece;

[0028] The fifth node: determine whether there is a gray or light blue to dark gray cloud layer rising continuously;

[0029] The sixth node: determine whether there is a dense, extensive and low cloud layer, which is divergent or "wet";

[0030] The seventh node: determine whether the cloud layer has the characteristics of white filaments or long filaments;

[0031] The eighth node: determine whether the size of each cloud element is less than a preset width;

[0032] The ninth node: determine whether the size of each circular cloud element is within a preset width range.

[0033] According to the above technical solution, when the determination result of the determination node is yes, the code is "1", otherwise the code is "0".

[0034] According to the above technical solution, the ground cloud deep learning model is a ResNet18 network, and the backbone network of the ResNet18 network includes an initial convolutional layer, a maximum pooling layer and four residual layers.

[0035] According to the above technical solution, the multiple losses in the loss function are fused according to preset weight coefficients.

[0036] The application also provides a ground cloud classification system based on deep learning and fusion of cloud field knowledge, comprising:

[0037] An image acquisition module is configured to acquire a ground cloud image to be classified.

[0038] a preprocessing module configured to preprocess the ground-based cloud image to be classified, including cropping, scaling, flipping and normalization operations;

[0039] a classification module configured to input the preprocessed ground-based cloud image into a ground-based cloud deep learning model for classification, and output a category of the ground-based cloud;

[0040] The system further comprises a model construction and training module, which is specifically configured to:

[0041] construct a ground-based cloud training set, wherein the ground-based cloud images in the ground-based cloud training set are preprocessed, cloud genus labels representing cloud genus categories are marked, and structured coding is performed according to a preset expert ground-based cloud discrimination process to construct corresponding rule labels, the rule labels being multi-dimensional binary labels, each dimension corresponding to a discrimination node in the expert ground-based cloud discrimination process and a corresponding discrimination result;

[0042] construct a ground-based cloud deep learning model comprising a double-branch structure, one branch being configured to output a cloud genus classification result, and the other branch being configured to generate a multi-dimensional rule vector as intermediate supervision information in a training stage, the multi-dimensional rule vector corresponding to the multi-dimensional binary label;

[0043] construct a loss function, and complete training and testing of the ground-based cloud deep learning model under the guidance of the loss function; the loss function fuses a cross-entropy classification loss of the cloud genus label, a supervision loss of the rule label and a structural constraint loss between dimensions of the rule vector; wherein the structural constraint between dimensions of the rule vector is specifically realized by limiting the difference between the predicted value of a certain dimension subset in the multi-dimensional rule vector and the predicted value of another dimension subset.

[0044] According to the above technical solution, the ground-based cloud deep learning model is a ResNet18 network, and the backbone network of the ResNet18 network comprises an initial convolutional layer, a max-pooling layer and four residual layers.

[0045] The application further provides a computer storage medium, which stores a computer program executable by a processor, and the computer program executes the ground-based cloud classification method based on deep learning fusion of cloud field knowledge according to the above technical solution.

[0046] The beneficial effects generated by the present application are: the present application sets the deep learning model as two branches, one of which is dedicated to the model training stage, which will generate a multi-dimensional rule vector as intermediate supervision information, which corresponds to a multi-dimensional binary label, and the multi-dimensional binary label is structured and coded according to the preset expert ground cloud discrimination process to build, each dimension corresponds to a discrimination node and the corresponding discrimination result in the expert ground cloud discrimination process; then the model is iteratively trained according to the loss function which combines the cross-entropy classification loss of the cloud genus label, the supervision loss of the rule label and the structural constraint loss between the rule vector dimensions. It can be seen that the present application integrates cloud domain knowledge into the training of the deep learning model, guides the model learning, and makes the classification of the model more in line with the discrimination of the expert ground cloud, thereby improving the accuracy and reliability of the cloud genus classification.

[0047] Further, the present application divides the cloud genus into ten categories, and sets the discrimination nodes in the expert ground cloud discrimination process to nine, which is more conducive to integrating the classification knowledge of the ground cloud in the model training according to different discrimination nodes.

[0048] Of course, implementing any product of the present application does not necessarily need to achieve all the advantages described above at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, brief introductions will be given to the drawings needed to be used in the embodiments or prior art descriptions. Obviously, the drawings described below are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0050] Figure 1 is the flow chart of the ground cloud classification method of the present application based on deep learning and fusion of cloud domain knowledge;

[0051] Figure 2 is the ground cloud discrimination process chart prepared by the expert in the present application embodiment;

[0052] Figure 3 is the deep learning model structure diagram in the present application embodiment;

[0053] Figure 4 is the loss value change diagram of each part loss in the deep learning model training in the present application embodiment;

[0054] Figure 5 is the comparative experiment result diagram in the present application embodiment. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0056] It should be noted that the diagrams provided in the embodiments of the present application only schematically illustrate the basic concepts of the present application, and therefore only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The shapes, number and proportions of the components in actual implementation can be arbitrarily changed, and the layout pattern of the components can be more complex.

[0057] In the present application, it should also be noted that, when terms such as "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like appear, the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, when the terms "first" and "second" appear, they are only for description and distinction purposes, and cannot be understood as indicating or implying relative importance.

[0058] In addition, it should also be noted that the features of various embodiments of the present application can be combined or integrated partially or entirely, and can interact and operate in different ways as understood by those skilled in the art. Each embodiment can be implemented independently of each other, or in an associated relationship.

[0059] As shown in Figure 1 The ground-based cloud classification method based on deep learning and fusion of cloud field knowledge according to the embodiments of the present application includes the following steps:

[0060] S1, obtaining a ground-based cloud image to be classified;

[0061] S2, pre-processing the ground-based cloud image to be classified, including cropping, scaling, flipping and normalization operations;

[0062] S3, inputting the pre-processed ground-based cloud image into a ground-based cloud deep learning model for classification, and outputting the category of the ground-based cloud;

[0063] The construction and training process of the ground-based cloud deep learning model is as follows:

[0064] The ground cloud training set is constructed, the ground cloud image in the ground cloud training set is marked after pre-processing, a cloud genus label indicating a cloud genus category is marked, and a corresponding rule label is constructed according to a preset expert ground cloud discrimination process and is structured coding, the rule label is a multi-dimensional binary label, each dimension corresponds to a discrimination node in the expert ground cloud discrimination process and a corresponding discrimination result;

[0065] The ground cloud deep learning model is constructed, including a double-branch structure, one branch is used for outputting a cloud genus classification result, and the other branch is used for generating a multi-dimensional rule vector as intermediate supervision information in a training stage, the multi-dimensional rule vector corresponds to the multi-dimensional binary label;

[0066] The loss function is constructed, and the training and testing of the ground cloud deep learning model are completed under the guidance of the loss function; the loss function fuses a cross-entropy classification loss of the cloud genus label, a supervision loss of the rule label and a structural constraint loss between rule vector dimensions; wherein the structural constraint between rule vector dimensions is specifically realized by limiting the difference between the predicted value of a certain dimension subset in the multi-dimensional rule vector and the predicted value of another dimension subset.

[0067] The deep learning model is set as two branches, one of which is used for the model training stage and generates a multi-dimensional rule vector as intermediate supervision information, which corresponds to the multi-dimensional binary label, and the multi-dimensional binary label is constructed according to the preset expert ground cloud discrimination process and is structured coding, each dimension corresponds to a discrimination node in the expert ground cloud discrimination process and a corresponding discrimination result; and the model is iteratively trained according to the loss function fusing the cross-entropy classification loss of the cloud genus label, the supervision loss of the rule label and the structural constraint loss between rule vector dimensions. It can be seen that the cloud field knowledge is introduced into the training of the deep learning model to guide the model learning, so that the classification of the model is more in line with the discrimination of the expert ground cloud, and the accuracy and reliability of the cloud genus classification are improved.

[0068] In an embodiment, the preset expert ground cloud discrimination process is as shown in Figure 2 The following discriminations are sequentially performed at each node:

[0069] Node 1: observing whether a single pile cloud structure is clearly visible in the sky.

[0070] If yes, go to node 2;

[0071] Otherwise, go to node 3.

[0072] Node 2: observing whether the upper end of the cloud is outline blurred.

[0073] If yes, it is determined to be cumulonimbus;

[0074] Otherwise, it is determined to be cumulus.

[0075] Node 3: Observe whether the cloud layer has the same uniform, continuous or broken features, without cloud rolls or cloud elements.

[0076] If yes, go to Node 4;

[0077] Otherwise, go to Node 7.

[0078] Node 4: Observe whether the sun or moon looks like a bright piece of debris.

[0079] If yes, determine that it is a cirrostratus cloud;

[0080] Otherwise, go to Node 5.

[0081] Node 5: Observe whether there are gray or light blue to dark gray cloud layers continuously rising.

[0082] If yes, determine that it is a cirrus cloud;

[0083] Otherwise, go to Node 6.

[0084] Node 6: Observe whether there are dense, extensive and lower cloud layers, in a divergent or "wet" shape.

[0085] If yes, determine that it is a nimbostratus cloud;

[0086] Otherwise, determine that it is a stratus cloud.

[0087] Node 7: Observe whether the cloud layer has the features of white filaments or long filaments.

[0088] If yes, determine that it is a cirrus cloud;

[0089] Otherwise, go to Node 8.

[0090] Node 8: Observe whether the size of each cloud element is less than one finger wide.

[0091] If yes, determine that it is a cirrocumulus cloud;

[0092] Otherwise, go to Node 9.

[0093] Node 9: Observe whether the size of each round cloud element is 1 to 3 finger wide.

[0094] If yes, determine that it is a alto cumulus cloud;

[0095] Otherwise, determine that it is a stratus cumulus cloud.

[0096] In this embodiment, the specific structured coding method is: according to Figure 2The ground cloud recognition flowchart can obtain 9 discrimination nodes, and each cloud genus corresponds to a discrimination path. The discrimination nodes in the discrimination path of a cloud genus are coded as "1" if the selection is "yes", and the remaining nodes (in the discrimination path, and the discrimination nodes selected as "no", and the discrimination nodes not in the discrimination path) are coded as "0". According to the order of the discrimination nodes in the middle, the rule label composed of 9 discrimination nodes in order is shown in Table 1 as follows. Figure 2

[0097] Table 1 Cloud genus and rule label

[0098]

[0099] In this embodiment, according to the expert ground cloud discrimination flowchart, a multi-dimensional binary rule label is constructed for each cloud genus, wherein each dimension corresponds to a discrimination node in the expert ground cloud discrimination flowchart. When a certain type of cloud genus satisfies the discrimination condition at the node, the corresponding dimension of the label is assigned a value of 1, otherwise 0.

[0100] In an embodiment, a deep learning model is constructed with the rule label as the training supervision information. Taking ResNet18 as an example, the network architecture is as shown in Figure 3 , and specifically includes the following steps:

[0101] (1) Construct a forward structure: the input is a ground cloud image with a size of 224x224x3. The image first passes through the forward propagation structure of the ResNet18 backbone network. The backbone network is composed of an initial convolutional layer (Conv1), a maximum pooling layer (MaxPool), and four residual modules (Layer1, Layer2, Layer3, Layer4). The feature maps Fm1 (56x56x64), Fm2 (28x28x128), Fm3 (14x14x256), and Fm4 (7x7x512) generated by the four residual modules Layer1, Layer2, Layer3, and Layer4, respectively;

[0102] (2) Branch 1: Fm1, Fm2, and Fm3 are input and pass through 1x1 convolution for channel compression, respectively, and are uniformly compressed to 32 channels to obtain compressed feature maps Fm1' (56x56x32), Fm2' (28x28x32), and Fm3' (14x14x32). Global average pooling (GAP) is performed on the three groups of compressed feature maps to obtain three 1x32 feature vectors. The three vectors are concatenated in the channel direction to form a feature vector Ft (1x96). In the training stage, Ft is input into fully connected layers FC1 (input dimension 96, output dimension 64, activation function ReLU) and FC2 (input dimension 64, output dimension 9) to output a 9-dimensional rule vector​ , used for alignment with rule tags.

[0103] (3) Branch 2: Take Fm4 (7×7×512) as input and perform global average pooling (GAP) to obtain the feature vector Fd (1×512). Concatenate Fd with Ft to form the feature vector Ff (1×608), and input it into the fully connected layers FC3 (input dimension 608, output dimension 128, activation function is ReLU) and FC4 (input dimension 128, output dimension is the number of cloud genus categories C, C=10 in this example), and output the cloud genus prediction classification vector. Then, the cloud classification results are output after passing through the softmax normalization function.

[0104] In one embodiment, the constructed knowledge-guided loss function The specific formula is as follows:

[0105]

[0106] in This represents the cross-entropy classification loss based on cloud genus labels. This represents the rule label supervision loss, used to guide the model to output rule vectors. This represents the structural constraint loss between the dimensions of the rule vector. The weight coefficients of the loss function (in this example) =0.5, =0.1).

[0107]

[0108] , Indicates the first c The cloud attribute tag of the class (1 if it is the target category, 0 otherwise). The model predicts that it is the first c The probability value of the class.

[0109] Furthermore, The specific formula is as follows:

[0110]

[0111] Where N represents the dimension of the regular vector (N=9 in this example). For the first rule label dimension, The first rule vector output by the model dimension, .

[0112] Furthermore, The specific formula is as follows:

[0113]

[0114] wherein, represents the maximum value difference between two dimension subsets of a rule vector involved in the ith rule vector dimension structural constraint, represents the maximum value difference between two dimension subsets of a rule vector involved in the ith rule vector dimension structural constraint, represents the expected value of the maximum value difference between two dimension subsets of a rule vector involved in the ith rule vector dimension structural constraint. Further,

[0115] The specific formula of is as follows:

[0116]

[0117] wherein, is a rule vector output by the model, are two groups of dimension subsets in the rule vector, each group can contain one or more dimensions.

[0118] Further, the rule vector dimension structural constraint is constructed based on the logical relationship between each decision node in the foundation cloud expert decision flowchart. The constraint limits the difference between the predicted values of a certain dimension subset and another dimension subset in the rule vector, so that the two groups of dimension subsets maintain a significant difference or a close relationship.

[0119] In this embodiment, nodes 1 to 9 correspond to dimensions 0 to 8 of the rule label respectively (in actual implementation, the mapping relationship between each dimension of the rule label and the decision node can be adjusted according to actual conditions), and the following two rule vector dimension structural constraints are constructed, as follows:

[0120] Constraint one: except for the category of stratocumulus, when the decision results of decision nodes 1 and 2 are “yes” (label 1) at least one, the decision results of decision nodes 3 to 9 should all be “no” (label 0); when the decision results of decision nodes 1 and 2 are both “no” (label 0), at least one of the decision results of decision nodes 3 to 9 is “yes” (label 1);

[0121] converted into mathematical expressions: the maximum value difference between the dimension subset composed of and the dimension subset composed of tends to 1, corresponding to ;

[0122] ​Constraint 2: Except for the stratus category, when the discrimination result of discrimination node 3 is "yes" (label 1), one of the discrimination results of discrimination nodes 4, 5 and 6 should also be "yes" (label 1); when the discrimination result of discrimination node 3 is "no" (label 0), the discrimination results of discrimination nodes 4, 5 and 6 are all "no" (label 0);

[0123] The conversion into a mathematical expression is as follows: The maximum value difference between the dimension subsets composed of tends to 0, corresponding to , .

[0124] Since the structural constraints between the rule vector dimensions in the method essentially participate in model training as auxiliary references, although the rule labels of stratus and stratocumulus do not completely conform to the node relationships in the expert discrimination process, they will not have a negative impact on the overall model performance. The auxiliary constraints guide the model to learn the feature rules of most cloud categories through the loss function, and for special categories such as stratus and stratocumulus, the model can adjust the prediction results according to the sample features and classification accuracy during the training process to ensure that the overall recognition effect is not affected.

[0125] In this example, the construction of two constraints is based on the node relationships in the expert discrimination process that have the most clear logical relationships and cover a wide range, which can effectively guide the model to learn the structural features with rule labels. At the same time, in order to avoid excessive rules causing training interference or model performance degradation, the above two constraints are constructed on the premise of ensuring the guiding effect.

[0126] In this embodiment, the classification loss serves as the main supervision signal without the need for additional weight control, the rule label supervision loss is used to guide the model to learn the classification logic that conforms to the expert process, the weight is set to 0.5, and the structural constraint loss between the dimensions of the rule vector is used to strengthen the logical relationships between the dimensions of the rule label, and the weight is set to 0.1. These two fixed weight values reflect the strength distribution of the three in the overall learning goal, which is a structural determination setting.

[0127] In an embodiment, the ground-based cloud images used are all-sky photographic images from multiple geographical regions and different meteorological conditions, and the image resolution is uniform at 224x224 pixels. Figure 4 The loss value change graph of each part of the loss value of the model training stage in an embodiment of the present application shows the trend of the loss value gradually decreasing in the training process, which reflects the convergence performance and optimization effect of the model. The training process includes a verification stage, which is used to evaluate the performance of the model on unseen samples and assist in tuning the model parameters to ensure the generalization ability of the model. Figure 4 The total loss change also includes the verification stage, which further reflects the stability and generalization performance of the model.Figure 5 The contrast experiment result graph of an embodiment of the present application shows that the model finally realizes a classification accuracy of 85.52% under a small sample condition, which is significantly better than the contrast method, verifying the effectiveness of the present method.

[0128] In order to realize the above method embodiment, the present application further provides a ground cloud classification system based on deep learning and fusion of cloud field knowledge, specifically comprising:

[0129] An image acquisition module is configured to acquire a ground cloud image to be classified.

[0130] A preprocessing module is configured to perform preprocessing on the ground cloud image to be classified, including cropping, scaling, flipping and normalization operations.

[0131] A classification module is configured to input the preprocessed ground cloud image into a ground cloud deep learning model for classification, and output the category of the ground cloud.

[0132] The system further comprises a model construction and training module, which is specifically configured to:

[0133] Construct a ground cloud training set. After the ground cloud images in the ground cloud training set are preprocessed, cloud genus labels representing cloud genus categories are marked, and structured coding is performed according to a preset expert ground cloud discrimination process to construct corresponding rule labels. The rule labels are multi-dimensional binary labels, each dimension corresponding to a discrimination node and the corresponding discrimination result in the expert ground cloud discrimination process.

[0134] Construct a ground cloud deep learning model comprising a double-branch structure. One branch is configured to output cloud genus classification results, and the other branch is configured to generate a multi-dimensional rule vector as intermediate supervision information in the training stage. The multi-dimensional rule vector corresponds to the multi-dimensional binary label.

[0135] Construct a loss function, and complete the training and testing of the ground cloud deep learning model under the guidance of the loss function. The loss function fuses the cross-entropy classification loss of the cloud genus label, the supervision loss of the rule label and the structural constraint loss between the dimensions of the rule vector. The structural constraint between the dimensions of the rule vector is specifically realized by limiting the difference between the predicted values of a subset of a certain dimension and the predicted values of another subset of a certain dimension in the multi-dimensional rule vector.

[0136] Further, the ground cloud deep learning model is a ResNet18 network. The backbone network of the ResNet18 network comprises an initial convolutional layer, a maximum pooling layer and four residual layers.

[0137] It can be understood that each module is mainly used to realize each step of the method embodiment, which will not be described here.

[0138] The application further provides a computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application store, and the like, which stores a computer program, and the program is executed by a processor to realize corresponding functions. The computer readable storage medium of the embodiment is executed by the processor to realize the ground cloud classification method based on deep learning fusion cloud field knowledge of the method embodiment.

[0139] It should be noted that, according to the needs of implementation, each step / component described in the application can be split into more steps / components, or two or more steps / components or part of the operation of the steps / components can be combined into a new step / component, to achieve the purpose of the application.

[0140] The size of the serial number of each step in the above embodiment does not mean the order of execution, the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0141] It should be understood that those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes should belong to the protection scope of the appended claims of the application.

Claims

1. A ground-based cloud classification method based on deep learning and fusion of cloud domain knowledge, characterized in that, The method comprises the following steps: S1, obtaining a ground-based cloud image to be classified; S2, preprocessing the ground-based cloud image to be classified, including cropping, scaling, flipping and normalization operations; S3, inputting the preprocessed ground-based cloud image into a ground-based cloud deep learning model for classification, and outputting the category of the ground-based cloud; The construction and training process of the ground-based cloud deep learning model is as follows: A ground-based cloud training set is constructed, and after the ground-based cloud images in the ground-based cloud training set are preprocessed, cloud genus labels representing cloud genus categories are marked, and structured coding is performed according to a pre-set expert ground-based cloud identification process to construct corresponding rule labels, the rule labels being multi-dimensional binary labels, each dimension corresponding to a judgment node and the corresponding judgment result in the expert ground-based cloud identification process; The ground-based cloud deep learning model is constructed, including a double-branch structure, one branch being used for outputting cloud genus classification results, and the other branch being used for generating a multi-dimensional rule vector as intermediate supervision information in the training stage, the multi-dimensional rule vector corresponding to the multi-dimensional binary label; A loss function is constructed, and the training and testing of the ground-based cloud deep learning model are completed under the guidance of the loss function; the loss function fuses the cross-entropy classification loss of the cloud genus label, the supervision loss of the rule label and the structural constraint loss between the dimensions of the rule vector; wherein the structural constraint between the dimensions of the rule vector is specifically realized by limiting the difference between the predicted values of a certain dimension subset and another dimension subset in the multi-dimensional rule vector. 2.The ground-based cloud classification method based on deep learning and cloud field knowledge fusion according to claim 1, characterized in that, The preprocessing is specifically as follows: A region is randomly cropped from the ground-based cloud image, and the area and the width-height ratio of the cropped region are randomly selected within a specified range; The cropped ground-based cloud image is scaled to a proper size; The scaled and cropped image is randomly horizontally flipped with a pre-set probability; The normalized ground-based cloud image is obtained by normalizing the randomly cropped, scaled and flipped ground-based cloud image. 3.The ground cloud classification method based on deep learning and cloud field knowledge fusion according to claim 1, characterized in that, The cloud genus categories include cumulonimbus, cumulus, stratus, altostratus, nimbostratus, stratocumulus, cirrus, cirrostratus, altocumulus, stratocumulus.

4. The method of claim 1-3, wherein, The judgment nodes in the expert ground-based cloud identification process include: A first node: judging whether there is a clear and visible separate pile cloud structure; A second node: judging whether the upper end of the cloud is profile blurred; A third node: judging whether the cloud layer has the same consistent, continuous or intermittent characteristics and does not have cloud rolls or cloud elements; A fourth node: judging whether the sun or the moon looks like a bright piece of debris; A fifth node: judging whether there is a gray or light blue to dark gray cloud layer continuously rising; A sixth node: judging whether there is a dense, extensive and relatively low cloud layer, which is divergent or "wet"; A seventh node: judging whether the cloud layer has the characteristics of white filaments or long filaments; An eighth node: judging whether the size of each cloud element is less than a pre-set width; A ninth node: judging whether the size of each circular cloud element is within a pre-set width range. 5.The ground-based cloud classification method based on deep learning and cloud field knowledge fusion according to claim 1, characterized in that, When structured coding, if the judgment result of the judgment node is yes, the code is "1", otherwise, the code is "0". 6.The ground-based cloud classification method based on deep learning and cloud field knowledge fusion according to claim 1, characterized in that, The ground-based cloud deep learning model is a ResNet18 network, and the backbone network of the ResNet18 network includes an initial convolutional layer, a maximum pooling layer and four residual layers. 7.The ground-based cloud classification method based on deep learning and cloud field knowledge fusion according to claim 1, characterized in that, The multiple losses in the loss function are fused according to pre-set weight coefficients. 8.A ground-based cloud classification system based on deep learning and fusion of cloud domain knowledge, characterized in that, The system comprises: an image acquisition module configured to acquire ground-based cloud images to be classified; a preprocessing module configured to preprocess the ground-based cloud images to be classified, including cropping, scaling, flipping and normalization operations; a classification module configured to input the preprocessed ground-based cloud images into a ground-based cloud deep learning model for classification, and output the category of the ground-based cloud; The system further comprises a model construction and training module, which is specifically configured to: construct a ground-based cloud training set, wherein the ground-based cloud images in the ground-based cloud training set are preprocessed, and a cloud genus label representing the cloud genus category is marked, and a corresponding rule label is constructed according to a pre-set expert ground-based cloud discrimination process, the rule label being a multi-dimensional binary label, each dimension corresponding to a discrimination node and a corresponding discrimination result in the expert ground-based cloud discrimination process; construct a ground-based cloud deep learning model comprising a double-branch structure, one branch being configured to output a cloud genus classification result, and the other branch being configured to generate a multi-dimensional rule vector as intermediate supervision information in the training stage, the multi-dimensional rule vector corresponding to the multi-dimensional binary label; construct a loss function, and complete the training and testing of the ground-based cloud deep learning model under the guidance of the loss function; the loss function fuses a cross-entropy classification loss of the cloud genus label, a supervision loss of the rule label and a structural constraint loss between dimensions of the rule vector; wherein the structural constraint between dimensions of the rule vector is specifically realized by limiting the difference between the predicted values of a certain dimension subset and another dimension subset in the multi-dimensional rule vector. 9.The ground-based cloud classification system based on deep learning and cloud domain knowledge fusion according to claim 8, characterized in that, The ground-based cloud deep learning model is a ResNet18 network, and the backbone network of the ResNet18 network comprises an initial convolutional layer, a max-pooling layer and four residual layers.

10. A computer storage medium, characterized in that It has a computer program stored therein, which can be executed by the processor, and the computer program executes the ground-based cloud classification method based on deep learning fusion of cloud domain knowledge according to any one of claims 1-7.

Images