A satellite cloud image recognition method, device and medium based on generalized nonlinear convolutional neural network

Abstract

The present invention designs a satellite cloud image recognition method, device, and medium based on a generalized nonlinear convolutional neural network, capable of real-time and efficient identification of cloud conditions in satellite cloud images. This method inputs labeled satellite cloud images into the network, employs generalized nonlinear convolution to learn a complex solution space, then feeds the output labels and the actual labels of the satellite cloud images into a loss function for gradient feedback, training the entire network model. Finally, the real-time satellite cloud images are preprocessed and input into the network for recognition. Generalized nonlinear convolution can adaptively find the optimal solution, avoiding the limitations of a monotonous solution space for specific operators. This method can accurately identify cloud conditions in satellite cloud images with high precision and timeliness.

Description

Technical Field

[0001] The present invention relates to a satellite cloud image recognition method, device and medium based on a generalized nonlinear convolutional neural network, and belongs to the field of meteorological analysis research. Background Art

[0002] Remote sensing satellite cloud images provide all-weather information on atmospheric, oceanic, and cloud conditions. Accurate satellite cloud image recognition can classify various cloud clusters, providing a crucial source of information for subsequent weather forecasts and disaster weather monitoring. In recent years, with the advancement of machine learning and deep learning technologies, meteorological cloud image recognition research has made significant progress.

[0003] However, existing recognition often lacks accuracy. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, device and medium for satellite cloud image recognition based on a generalized nonlinear convolutional neural network. By utilizing a generalized nonlinear neuron operator, the method can efficiently perform satellite cloud image recognition and meteorological analysis, and timely prevent and control meteorological disasters.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a satellite cloud image recognition method based on a generalized nonlinear convolutional neural network, comprising:

[0007] Obtain satellite cloud images;

[0008] Preprocessing the satellite cloud image;

[0009] Inputting the satellite cloud image into a trained generalized nonlinear convolutional neural network;

[0010] Output the cloud category of the satellite cloud image after identification and classification; the cloud category includes cirrus cloud, mesocloud, cumulus cloud, and stratus cloud.

[0011] Furthermore, the method of obtaining satellite cloud images includes shooting through meteorological satellites.

[0012] Furthermore, the method for preprocessing the satellite cloud image includes:

[0013] Perform grayscale processing;

[0014] Perform normalization processing without changing the effective information of the original shape outline, remove redundancy, and retain the core information;

[0015] Furthermore, the generalized nonlinear convolutional neural network includes:

[0016] Two hidden layers: used to abstract the features of the input satellite cloud image data to another spatial dimension to display its more abstract features;

[0017] Nonlinear convolution layer: used to adaptively calculate the nonlinear activation function after convolution using Taylor formula, converting the original features of satellite cloud images into a more complex feature space through nonlinear convolution;

[0018] Pooling layer: used to process the feature space to obtain recognition results.

[0019] The generalized nonlinear convolutional neural network uses the Q-order Taylor formula to approximate the node operator, and the composite node function is represented by the Q-order Taylor formula:

[0020] Ψ(w,y)=w0+w1(ya)+w2(ya) 2 +…+w Q (ya) Q

[0021] In the formula, w is the expansion coefficient, y is the independent variable, and a is the expansion point.

[0022] Furthermore, the training method of the generalized nonlinear convolutional neural network includes:

[0023] Obtain meteorological satellite cloud atlas as training set;

[0024] preprocessing the training set;

[0025] The generalized nonlinear convolutional neural network is trained based on the preprocessed training set to identify and classify satellite cloud images. The loss function of the training is:

[0026]

[0027] Where K represents the number of labels, y i，k is the true label value, k is the label number, and the probability that the i-th sample is predicted to be the k-th label value is p i，k , N represents the number of samples, and the final average is taken as the final loss function.

[0028] Furthermore, the method of preprocessing the training set includes:

[0029] Satellite cloud images are batch processed, and all cloud images are of the same size. The images are shuffled and randomly rotated in each round of training to enhance the robustness of the model.

[0030] Furthermore, a generalized nonlinear convolutional neural network is trained based on the preprocessed training set to identify and classify satellite cloud images, including:

[0031] The image is input into the neural network, and the nonlinear convolution layer is used to extract features. The generative neurons are used to realize nonlinear connection. The generalized nonlinear convolutional neural network is equipped with node operators to maximize learning ability. The composite node function is expressed by the Q-order Taylor formula:

[0032] Ψ(w,y)=w0+w1(ya)+w2(ya) 2 +…+w Q (ya) Q

[0033] Where the value of a does not affect the accuracy of the approximation, because the node operator acts on the output of the neurons in the previous layer, and the output of the neurons in each layer is bounded based on the activation function;

[0034] Forward propagation from the input layer to the output layer, the generative neuron of the generalized nonlinear convolutional neural network has a 3D kernel matrix, where the qth weight of the kernel element is represented by . Each node operator function with a single kernel element will be approximated by a composite node operator, represented by the Maclaurin function:

[0035]

[0036] Where m and n are variables, r and t are the corresponding error terms, and l is the number of layers.

[0037] It's the kernel. It is the output of the previous layer, which uses sum pooling and tanh activation function.

[0038] Calculate the loss function and use the cross entropy function commonly used in image recognition and classification:

[0039]

[0040] Where, y i,k is the true label value, K represents the number of labels, and the probability that the i-th sample is predicted to be the k-th label value is p i，k , N represents the number of samples, and the final average is taken as the final loss function;

[0041] The loss function is used to guide back propagation and update the iterative weight kernel. Because the node function operator is approximated by the Maclaurin formula, back propagation is more convenient and accurate:

[0042]

[0043]

[0044] Among them, ssx and ssy are existence factors, It is the downsampling loss of the return, and then upsampling is used to calculate the error of the neuron, using average pooling. Then the weight (kernel) and bias sensitivity are calculated:

[0045]

[0046]

[0047] is the qth two-dimensional sub-nucleus, consisting of the nuclear elements Composition, during each back-propagation iteration t, the kernel parameters of each neuron of the generalized nonlinear convolutional neural network and deviation will be updated until the stopping criterion is met, and ε(t) is the learning factor during iteration.

[0048] The network is iteratively updated through the above steps, and the entire neural network is trained with the loss function as a guide.

[0049] In a second aspect, the present invention provides a satellite cloud image recognition device based on a generalized nonlinear convolutional neural network, characterized in that the device comprises:

[0050] Acquisition module: used to obtain satellite cloud images;

[0051] Preprocessing module: used for preprocessing the satellite cloud image;

[0052] Output module: used to input the satellite cloud image into the trained generalized nonlinear convolutional neural network, and output the cloud category of the satellite cloud image after identification and classification; the cloud category includes cirrus cloud, mesocloud, cumulus cloud, and stratus cloud.

[0053] In a third aspect, the present invention provides a satellite cloud image recognition device based on a generalized nonlinear convolutional neural network, comprising a processor and a storage medium;

[0054] The storage medium is used to store instructions;

[0055] The processor is configured to operate according to the instructions to execute the steps of the method according to the first aspect.

[0056] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. The present invention designs a satellite cloud image recognition method, device and medium based on a generalized nonlinear convolutional neural network, which effectively solves the difficulties of complex and changeable cloud image morphological features and highly complex solution space. The generalized nonlinear convolution can learn more complex feature spaces through Taylor formula approximation node operators, overcoming the difficulty that traditional cloud image recognition methods need to build very complex network structures to approximate the optimal solution. The designed method has a small amount of computational complexity and high timeliness in the interpretation process, meeting the high timeliness business needs of meteorological analysis.

[0059] 2. The generalized nonlinear convolutional neural network of the present invention introduces a generalized nonlinear neuron operator and uses the Taylor formula to approximate the operator of the neuron connection. In this way, the kernel weight parameters can be iteratively updated in feedforward propagation and backpropagation. It is applicable to linear and nonlinear convolution situations, greatly improving the learning ability of complex nonlinear features, and can overcome the limitation that traditional networks can only be applied to simple linear convolution. In the face of highly complex nonlinear solution space, this method can also effectively handle it, accurately improve the recognition and classification accuracy, extract key cloud map information, and lay a good foundation for subsequent meteorological analysis work.

[0060] 3. For highly complex tasks, generalized nonlinear convolutional neural networks can achieve results similar to those of highly complex convolutional neural network models, significantly reducing the amount of computation and improving timeliness. This invention utilizes a generalized nonlinear convolutional neural network model to identify and classify meteorological satellite cloud images. This method can process highly complex satellite imagery, improving recognition accuracy and speed, and significantly increasing recognition efficiency. Furthermore, this invention can also perform real-time satellite cloud image recognition, improving timeliness. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a flow chart of meteorological satellite cloud image recognition of the present invention;

[0062] Figure 2 This is a generalized nonlinear convolutional neural network model framework diagram of the present invention;

[0063] Figure 3 It is the activation map of the nonlinear convolutional layer neurons in the generalized nonlinear convolutional neural network of the present invention. DETAILED DESCRIPTION

[0064] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0065] Example 1:

[0066] This embodiment provides a satellite cloud image recognition method based on a generalized nonlinear convolutional neural network. This embodiment utilizes a generalized nonlinear convolutional neural network model implemented on the PyTorch platform, considered one of the most popular deep learning platforms in recent years, due to its simplicity and efficiency. The model is suitable for identifying and classifying captured meteorological satellite cloud images. The network is first trained on labeled satellite cloud images. The high precision and time-efficiency of generalized nonlinear convolution improves the model's accuracy and robustness, enabling accurate identification of cloud conditions within captured meteorological satellite cloud images.

[0067] The method of this embodiment includes the following steps:

[0068] (1) Take satellite cloud images through meteorological satellites;

[0069] (2) Satellite cloud images are identified by a method based on generalized nonlinear convolutional neural networks, and the cloud condition categories (cirrus, mesoscopic, hydroid, and stratus) of the cloud images are output.

[0070] Generalized nonlinear convolutional neural networks include:

[0071] Two hidden layers: The hidden layer abstracts the features of the input satellite cloud image data to another spatial dimension, presenting its more abstract features. Theoretically, a two-layer neural network can fit any function. When propagating in the gradient direction, it will search for the optimal solution that meets the target requirements in this complex solution space.

[0072] Nonlinear convolution layer: The Taylor formula is used to adaptively calculate the nonlinear activation function after convolution, converting the original features of the satellite cloud image into a more complex feature space through nonlinear convolution, expanding the possibilities of the feature space. When encountering satellite cloud images that the model has never seen before, it can also accurately identify cloud conditions.

[0073] Pooling layer: Satellite cloud image features are invariant. The pooling layer does not lose the features of the image, which can greatly reduce the time of convolution operations. By reducing the channel dimension, it solves the fitting problem in the recognition process.

[0074] The implementation process of step (2) is as follows:

[0075] (21) A generalized nonlinear convolutional neural network is trained to identify and classify satellite cloud images. The loss function of the training is:

[0076]

[0077] Where K represents the number of labels, and the probability that the i-th sample is predicted to be the k-th label value is p i，k , N represents the number of samples, and the final average is taken as the final loss function;

[0078] (22) Traditional node operators can be multiplication, exponential, sine, quadratic function, Gaussian, derivative of Gaussian (DoG), Laplace of Gaussian (LoG) or Hermitian operator. The present invention innovatively uses the Q-order Taylor formula to approximate the node operator, avoiding the limitations of specific operators and greatly improving the learning ability of the model. The composite node function is represented by the Q-order Taylor formula:

[0079] Ψ(w,y)=w0+w1(ya)+w2(ya) 2 +…+w Q (ya) Q

[0080] (3) Output the satellite cloud image after identification and classification;

[0081] The training process of the generalized nonlinear convolutional neural network for satellite cloud image recognition is as follows:

[0082] The picture is input into the neural network. First, a nonlinear convolution layer is used to extract features. Nonlinear connections are mainly realized by generating neurons. It has a "composite node operator" that is iteratively created during backpropagation training without restrictions. In this way, each generating neuron in the generalized nonlinear convolutional neural network can perform self-optimized node operators for each kernel element and each connection (to each neuron in the previous layer) through backpropagation training to maximize learning ability. The composite node function is expressed by the Q-order Taylor formula:

[0083] Ψ(w,y)=w0+w1(ya)+w2(ya) 2 +…+w Q (ya) Q The value of a does not affect the accuracy of the approximation, because the node operator acts on the output of the neurons in the previous layer. The output of neurons in each layer is bounded based on the activation function. For example, sigmoid activation has an output in the range of [0, 1], and tanh activation has an output in the range of [-1, 1]. a takes an intermediate value. The network model uses tanh activation, so a is 0. The activation function is the Q-order Maclaurin formula, and the model row uses the 7th order (Q=7):

[0084] Ψ(w,y)=w0+w1y+w2y 2 +…+w Q y Q

[0085] Then, forward propagation from the input layer to the output layer, the generative neuron of the generalized nonlinear convolutional neural network has a 3D kernel matrix, where the qth weight of the kernel element is represented by . Each node operator function with a single kernel element will be approximated by a composite node operator, represented by the Maclaurin function:

[0086]

[0087] In the formula It's the kernel. It is the output of the previous layer, which uses sum pooling and tanh activation function.

[0088] Calculate the loss function and use the cross entropy function commonly used in image recognition and classification:

[0089]

[0090] Where K represents the number of labels, and the probability that the i-th sample is predicted to be the k-th label value is p i，k , N represents the number of samples, and the final average is taken as the final loss function;

[0091] The loss function is used to guide back propagation and update the iterative weight kernel. Because the node function operator is approximated by the Maclaurin formula, back propagation is more convenient and accurate:

[0092]

[0093]

[0094] Among them, ssx and ssy are existence factors, It is the downsampling loss of the return, and then upsampling is used to calculate the error of the neuron, using average pooling. Then the weight (kernel) and bias sensitivity are calculated:

[0095]

[0096]

[0097] is the qth two-dimensional sub-nucleus, consisting of the nuclear elements Composition, during each back-propagation iteration t, the kernel parameters (weights) of each neuron of the generalized nonlinear convolutional neural network and deviation will be updated until the stopping criterion is met, and ε(t) is the learning factor during iteration.

[0098] By iteratively updating the network through the above steps, the entire neural network is trained using the loss function as a guide. The labeled satellite cloud images are input into the trained network to classify the specific cloud image information.

[0099] Testing phase Figure 1 As shown, the specific steps include:

[0100] Step 1: Take meteorological satellite cloud images through meteorological satellites;

[0101] Step 2: Identify satellite cloud images using a method based on generalized nonlinear convolutional neural networks. Take the captured satellite cloud images as input and pass them through two hidden layers: a nonlinear convolution layer and a pooling layer. The nonlinear convolution layer is a trained MacLaughlin neuron connection node operator. The sum pooling layer is selected. After nonlinear mapping through two hidden layers, the classified cloud image information is output and the classification accuracy is calculated.

[0102] Step 3: Output the satellite cloud image after identification and classification, and obtain the categories of the satellite cloud image, including cirrus clouds, mesic clouds, cumulus clouds, and stratus clouds.

[0103] Example 2:

[0104] This embodiment provides a satellite cloud image recognition device based on a generalized nonlinear convolutional neural network, the device comprising:

[0105] Acquisition module: used to obtain satellite cloud images;

[0106] Preprocessing module: used for preprocessing the satellite cloud image;

[0107] Output module: used to input the satellite cloud image into the trained generalized nonlinear convolutional neural network, and output the cloud category of the satellite cloud image after identification and classification; the cloud category includes cirrus cloud, mesocloud, cumulus cloud, and stratus cloud.

[0108] The device of this embodiment can be used to execute the method described in the first embodiment.

[0109] Example 3:

[0110] The present invention provides a satellite cloud image recognition device based on a generalized nonlinear convolutional neural network, comprising a processor and a storage medium;

[0111] The storage medium is used to store instructions;

[0112] The processor is configured to operate according to the instructions to execute the steps of the method according to the first embodiment.

[0113] Example 4:

[0114] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method described in the first embodiment are implemented.

[0115] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0116] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0119] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A satellite cloud image recognition method based on a generalized nonlinear convolutional neural network, characterized in that: The following steps are involved: Obtain satellite cloud images; Preprocessing the satellite cloud image; Inputting the satellite cloud image into a trained generalized nonlinear convolutional neural network; Output the cloud category of the satellite cloud image after identification and classification; The cloud condition categories include cirrus, mesoscopic, cumulonimbus, and stratus; The generalized nonlinear convolutional neural network includes: Two hidden layers: used to abstract the features of the input satellite cloud image data to another spatial dimension to display its more abstract features; Nonlinear convolution layer: used to adaptively calculate the nonlinear activation function after convolution using Taylor formula, converting the original features of satellite cloud images into a more complex feature space through nonlinear convolution; Pooling layer: used to process the feature space to obtain recognition results; The generalized nonlinear convolutional neural network uses the Q-order Taylor formula to approximate the node operator, and the composite node function is represented by the Q-order Taylor formula: Where w is the expansion coefficient, y is the independent variable, and a is the expansion point.

2. The satellite cloud image recognition method based on generalized nonlinear convolutional neural network according to claim 1, characterized in that: Methods for obtaining satellite cloud images include taking them through meteorological satellites.

3. The satellite cloud image recognition method based on generalized nonlinear convolutional neural network according to claim 1, characterized in that: The method for preprocessing the satellite cloud image includes: Perform grayscale processing; Normalization is performed without changing the effective information of the original shape contour, removing redundancy and retaining the core information.

4. The satellite cloud image recognition method based on generalized nonlinear convolutional neural network according to claim 1, characterized in that: The training method of the generalized nonlinear convolutional neural network includes: Obtain meteorological satellite cloud atlas as training set; preprocessing the training set; The generalized nonlinear convolutional neural network is trained based on the preprocessed training set to identify and classify satellite cloud images. The loss function of the training is: Where K represents the number of labels, y i,k is the true label value, k is the label number, and the probability that the i-th sample is predicted to be the k-th label value is , N represents the number of samples, and the final average is taken as the final loss function.

5. The satellite cloud image recognition method based on generalized nonlinear convolutional neural network according to claim 4 is characterized in that: The method of preprocessing the training set includes: Satellite cloud images are batch processed, and all cloud images are of the same size. The images are shuffled and randomly rotated in each round of training to enhance the robustness of the model.

6. The satellite cloud image recognition method based on generalized nonlinear convolutional neural network according to claim 4, characterized in that: The generalized nonlinear convolutional neural network is trained based on the preprocessed training set to identify and classify satellite cloud images, including: The image is input into the neural network, and the nonlinear convolution layer is used to extract features. The generative neurons are used to realize nonlinear connection. The generalized nonlinear convolutional neural network is equipped with node operators to maximize learning ability. The composite node function is expressed by the Q-order Taylor formula: Where the value of a does not affect the accuracy of the approximation, because the node operator acts on the output of the neurons in the previous layer, and the output of the neurons in each layer is bounded based on the activation function; Forward propagation from the input layer to the output layer, the generative neuron of the generalized nonlinear convolutional neural network has a 3D kernel matrix, where the qth weight of the kernel element is represented by; each node operator function with a single kernel element will be approximated by a composite node operator, represented by the Maclaurin function: Where m and n are variables, r and t are the corresponding error terms, and l is the number of layers; It's the kernel. It is the output of the previous layer, specifically using sum pooling and tanh activation function; Calculate the loss function and use the cross entropy function commonly used in image recognition and classification: Where, y i,k is the true label value, K represents the number of labels, and the probability that the i-th sample is predicted to be the k-th label value is , N represents the number of samples, and the final average is taken as the final loss function; The loss function is used to guide back propagation and update the iterative weight kernel. Because the node function operator is approximated by the Maclaurin formula, back propagation is more convenient and accurate: Among them, ssx and ssy are existence factors, It is the downsampling loss of the return, and then upsampling is used to calculate the error of the neuron, using average pooling; followed by calculating the weight (kernel) and bias sensitivity: is the qth two-dimensional sub-nucleus, consisting of the nuclear elements Composition, during each back-propagation iteration t, the kernel parameters of each neuron of the generalized nonlinear convolutional neural network and deviation Will be updated until the stopping criteria is met. is the learning factor during iteration; The network is iteratively updated through the above steps, and the entire neural network is trained with the loss function as a guide.

7. A satellite cloud image recognition device based on a generalized nonlinear convolutional neural network, characterized in that: The device comprises: Acquisition module: used to obtain satellite cloud images; Preprocessing module: used for preprocessing the satellite cloud image; Output module: used to input the satellite cloud image into the trained generalized nonlinear convolutional neural network and output the cloud category of the satellite cloud image after identification and classification; the cloud category includes cirrus, mesic, cumulonimbus, and stratus; The generalized nonlinear convolutional neural network includes: Two hidden layers: used to abstract the features of the input satellite cloud image data to another spatial dimension to display its more abstract features; Nonlinear convolution layer: used to adaptively calculate the nonlinear activation function after convolution using Taylor formula, converting the original features of satellite cloud images into a more complex feature space through nonlinear convolution; Pooling layer: used to process the feature space to obtain recognition results; The generalized nonlinear convolutional neural network uses the Q-order Taylor formula to approximate the node operator, and the composite node function is represented by the Q-order Taylor formula: Where w is the expansion coefficient, y is the independent variable, and a is the expansion point.

8. A satellite cloud image recognition device based on a generalized nonlinear convolutional neural network, characterized in that: including processors and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images