Ground-based cloud image-based weather recognition method, system, and sky imager

Abstract

This invention provides a weather recognition method based on ground-based cloud imagery, a weather recognition system based on ground-based cloud imagery, a computer-readable storage medium, and a sky imager. The weather recognition method based on ground-based cloud imagery includes: acquiring ground-based cloud imagery; and inputting the ground-based cloud imagery into a weather recognition model to identify the weather in the imagery. The weather recognition model is constructed based on a residual network and a multi-head attention mechanism, and is trained using a combination of knowledge distillation and adversarial training. The weather recognition method based on ground-based cloud imagery enhances the model's parallel processing capability for different image regions during weather recognition, improves the ability to identify key areas of different weather types, enhances the model's robustness to noisy data or abnormal weather conditions, and further lightweights the inference stage, reducing training resource consumption while improving model performance.

Description

Technical Field

[0001] This invention relates to the field of weather recognition, and more particularly to a weather recognition method based on ground-based cloud images, a weather recognition system based on ground-based cloud images, a computer-readable storage medium, and a sky imager. Background Technology

[0002] Weather recognition technology is widely used in fields such as photovoltaic power generation and meteorological observation. It is achieved by combining computer image processing and machine vision technology to improve the accuracy, efficiency and generalization ability of weather recognition.

[0003] In a prior art (such as patent CN1180720715A), a ground-based cloud image classification method based on attention convolution is provided. This method uses an improved VGG as the backbone network to extract cloud image features. Then, a channel attention module is used to improve the feature representation effect, and finally, a ground-based cloud image training model is built. However, while the VGG-style single-branch topology can improve inference speed, it may lead to a decrease in the model's generalization ability; furthermore, the model is biased towards extracting local features and cannot comprehensively capture global information in the image, which may lead to misclassification under complex weather conditions; additionally, it may lead to overfitting when processing test data that differs significantly from the training data.

[0004] Another existing technology (such as patent CN111695640A) provides a method for training a ground-based cloud image recognition model. This method utilizes a fine-tuned AlexNet network to implement a ground-based cloud image recognition model, enabling intelligent recognition of various ground-based cloud images and improving the range and accuracy of cloud recognition. However, in practice, the performance improvement of the AlexNet network is not significant, and it may suffer from overfitting when faced with large datasets.

[0005] To overcome the aforementioned shortcomings of existing technologies, there is an urgent need in this field for a weather recognition technology based on ground-based cloud images and a sky imager that can enhance the parallel processing capability of the weather recognition model for different image regions during the weather recognition process, improve the recognition capability of key areas of different weather types, and enhance the robustness of the weather recognition model in the face of noisy data or abnormal weather conditions. Furthermore, the weather recognition model can achieve further lightweighting in the inference stage, reducing the consumption of training resources while improving the performance of the weather recognition model. Summary of the Invention

[0006] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0007] To overcome the aforementioned deficiencies in existing technologies, this invention provides a weather recognition method based on ground-based cloud images, a weather recognition system based on ground-based cloud images, a computer-readable storage medium, and a sky imager. These technologies enhance the parallel processing capability of the weather recognition model for different image regions during the weather recognition process, improve the ability to identify key areas of different weather types, and increase the robustness of the weather recognition model in the face of noisy data or abnormal weather conditions. Furthermore, the weather recognition model can achieve further lightweighting during the inference phase, reducing training resource consumption while improving the performance of the weather recognition model.

[0008] Specifically, the weather recognition method based on ground-based cloud images provided by the first aspect of the present invention includes: acquiring a ground-based cloud image; and inputting the ground-based cloud image into a weather recognition model to identify the weather of the ground-based cloud image, wherein the weather recognition model is constructed based on a residual network and a multi-head attention mechanism, and the weather recognition model is trained by a combination of knowledge distillation and adversarial training.

[0009] Preferably, in one embodiment of the present invention, the step of training the weather recognition model using a combination of knowledge distillation and adversarial training includes: using a ground-based cloud image dataset and adversarial samples as input data, and dividing them into a training set, a validation set, and a test set; training the weather recognition model using the training set based on the combination of knowledge distillation and adversarial training; adjusting the parameters of the trained weather recognition model using the validation set; and evaluating the performance of the weather recognition model using the test set.

[0010] Preferably, in one embodiment of the present invention, the step of training the weather recognition model by a combination of knowledge distillation and adversarial training includes: defining a learning rate using a dynamic learning rate adjustment mechanism.

[0011] Preferably, in one embodiment of the present invention, the step of training the weather recognition model includes: inputting ground-based cloud images and adversarial examples from the training set into a student model and a teacher model, determining the output of the student model and the output of the teacher model; calculating the KL divergence between the output of the student model and the output of the teacher model; and performing backpropagation based on the KL divergence, so that the student model learns the knowledge of the teacher model, thereby achieving the training of the weather recognition model.

[0012] Preferably, in one embodiment of the present invention, the step of training the weather recognition model further includes: determining the cross-entropy loss based on the output of the student model and the output of the teacher model; and performing backpropagation based on the KL divergence and the cross-entropy loss to train the weather recognition model.

[0013] Preferably, in one embodiment of the present invention, the method further includes the steps of: storing the acquired ground-based cloud image into a weather dataset; and optimizing the weather recognition model based on an online learning optimization algorithm according to the weather dataset.

[0014] Preferably, in one embodiment of the present invention, the ground cloud map includes a ground cloud map formed by data preprocessing operations, the data preprocessing operations including data augmentation, normalization processing and / or tensor transformation.

[0015] Furthermore, the weather identification system based on ground-based cloud images provided according to the second aspect of the present invention includes a memory and a processor. The memory stores computer instructions. The processor is connected to the memory and configured to execute the computer instructions stored in the memory to implement the weather identification method based on ground-based cloud images provided in any of the above embodiments.

[0016] Furthermore, the computer-readable storage medium provided according to the third aspect of the present invention stores computer instructions. When the computer instructions are executed by a processor, the weather identification method based on ground-based cloud images provided in any of the above embodiments is implemented.

[0017] Furthermore, the sky imager provided according to the fourth aspect of the present invention includes a camera module for capturing ground-based cloud images; and a microcomputer configured with a weather recognition system based on ground-based cloud images provided in the second aspect of the present invention. Attached Figure Description

[0018] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0019] Figure 1 A schematic diagram of a weather recognition system based on ground-based cloud images provided according to some embodiments of the present invention is shown;

[0020] Figure 2 A schematic diagram of a sky imager provided according to some embodiments of the present invention is shown;

[0021] Figure 3A schematic diagram of a ground-based cloud image acquired by a sky imager according to some embodiments of the present invention is shown;

[0022] Figure 4 A flowchart of a weather identification method based on ground-based cloud images according to some embodiments of the present invention is shown;

[0023] Figure 5 A schematic diagram of a weather recognition model provided according to some embodiments of the present invention is shown;

[0024] Figure 6 A schematic diagram illustrating the training of a weather recognition model provided according to some embodiments of the present invention is shown;

[0025] Figure 7 The graphs show the accuracy and loss rate of the weather recognition model with multi-head attention mechanism and the residual network model without multi-head attention mechanism during the training phase.

[0026] Figure 8 The graphs showing the accuracy and loss rate of the weather recognition model with and without multi-head attention mechanisms during the testing phase are illustrated; and

[0027] Figure 9 A flowchart of a weather identification method based on ground-based cloud images, according to some embodiments of the present invention, is shown.

[0028] Figure label:

[0029] 100: Weather recognition system based on ground-based cloud images;

[0030] 110: Memory;

[0031] 111: Computer-readable storage medium;

[0032] 120: Processor;

[0033] 200: Sky Imager;

[0034] 201: Supporting structure;

[0035] 210: Raspberry Pi Module;

[0036] 220: Camera module;

[0037] 230: Communication module;

[0038] 240: Temperature control module;

[0039] 250: Power supply module;

[0040] 301: First column;

[0041] 302: Second column;

[0042] 303: Third column;

[0043] 304: Fourth column;

[0044] 305: Column 5;

[0045] 306: Column 6;

[0046] 307: Column 7;

[0047] 400: Weather recognition method based on ground-based cloud images;

[0048] 500: Weather Recognition Model;

[0049] 510: Residual block;

[0050] 520: Attention Mechanism;

[0051] 530: Residual block;

[0052] 540: Output;

[0053] 711: Accuracy curve;

[0054] 712: Accuracy curve;

[0055] 721: Loss rate curve;

[0056] 722: Loss rate curve;

[0057] 811: Accuracy curve;

[0058] 812: Accuracy curve;

[0059] 821: Loss rate curve;

[0060] 822: Loss rate curve;

[0061] S410~S420: Steps. Detailed Implementation

[0062] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the aspects described below with reference to the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0063] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0064] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood as the orientations shown in the relevant paragraphs and accompanying drawings. These relative terms are for illustrative purposes only and do not imply that the described apparatus must be manufactured or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0065] It is understood that although terms such as "first," "second," and "third" may be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first components, regions, layers, and / or parts discussed below may be referred to as second components, regions, layers, and / or parts without departing from some embodiments of the present invention.

[0066] As mentioned above, in the field of weather recognition, the performance of existing models is low, and they cannot effectively cope with complex, highly variable, or massive amounts of data.

[0067] To overcome the aforementioned deficiencies in existing technologies, this invention provides a weather recognition method based on ground-based cloud images, a weather recognition system based on ground-based cloud images, a computer-readable storage medium, and a sky imager. These technologies enhance the parallel processing capability of the weather recognition model for different image regions during the weather recognition process, improve the ability to identify key areas of different weather types, and increase the robustness of the weather recognition model in the face of noisy data or abnormal weather conditions. Furthermore, the weather recognition model can achieve further lightweighting during the inference phase, reducing training resource consumption while improving the performance of the weather recognition model.

[0068] In some non-limiting embodiments, the weather identification method based on ground-based cloud images provided in the first aspect of the present invention can be implemented via the weather identification system based on ground-based cloud images provided in the second aspect of the present invention.

[0069] Please refer to Figure 1 , Figure 1A schematic diagram of a weather recognition system based on ground-based cloud maps provided according to some embodiments of the present invention is shown.

[0070] like Figure 1 As shown, the ground-based cloud image-based weather identification system 100 may be configured with a memory 110 and a processor 120. The memory 110 includes, but is not limited to, the computer-readable storage medium 111 described in the third aspect of the present invention, which stores computer instructions thereon. The processor 120 is connected to the memory 110 and configured to execute the computer instructions stored in the memory 110 to implement the ground-based cloud image-based weather identification method provided in the first aspect of the present invention.

[0071] The weather recognition system based on ground-based cloud images provided in the second aspect of the present invention can be configured in the microcomputer of the sky imager provided in the fourth aspect of the present invention.

[0072] Please refer to Figure 2 and Figure 3 , Figure 2 A schematic diagram of a sky imager according to some embodiments of the present invention is shown. Figure 3 A schematic diagram of a ground-based cloud image acquired by a sky imager according to some embodiments of the present invention is shown.

[0073] Sky imagers are built using microcomputers. For example... Figure 2 As shown, the sky imager 200 may include a Raspberry Pi module 210. The Raspberry Pi module 210 is a microcomputer built on a Raspberry Pi. The Raspberry Pi (RPi) is an expandable microcomputer with advantages such as small size, low cost, and strong expandability. It can be remotely connected and controlled and can be applied to various industrial scenarios. The weather recognition system based on ground-based cloud images provided by this invention can be configured in the Raspberry Pi module 210.

[0074] Ground-based cloud images can be obtained using sky imagers. For example... Figure 2 As shown, the sky imager 200 may include a camera module 220 for capturing ground-based cloud images, and the camera module 220 may be equipped with a high-resolution camera. The camera module 220 can continuously capture ground-based cloud images under various weather conditions, and the original images captured by the camera module 220 can be used as follows: Figure 3 As shown in the first column 301. Thus, the self-built Raspberry Pi Sky Imager 200 can capture and collect ground-based cloud images. The collected ground-based cloud images can be transmitted and stored in the Raspberry Pi module 210 (e.g., in a configured database) for weather recognition. The stored ground-based cloud images can be used to train or optimize the weather recognition model deployed in the Raspberry Pi module 210.

[0075] exist Figure 2 In the illustrated embodiment, a communication module 230, a temperature control module 240, and a power supply module 250 may also be arranged on the support structure 201 of the sky imager 200. The communication module 230 can transmit the ground-based cloud image data collected by the camera module 220 to the Raspberry Pi module 210 for processing in real time, thereby achieving instant data transmission and ensuring data timeliness. In addition, through the temperature control module 240, the sky imager 200 can operate stably in different climatic environments.

[0076] Thus, the Sky Imager 200 built using Raspberry Pi is not only inexpensive, but also easy to expand and customize, making it suitable for a wide range of weather monitoring applications.

[0077] The following will first describe the working principle of the aforementioned ground-based cloud image-based weather identification system and the aforementioned sky imager by combining some embodiments of ground-based cloud image-based weather identification methods. Those skilled in the art will understand that these embodiments of ground-based cloud image-based weather identification methods are merely non-limiting implementations provided by the present invention, intended to clearly demonstrate the main concepts of the invention and provide specific solutions convenient for public implementation, rather than limiting all functions or all operating methods of the ground-based cloud image-based weather identification system and the sky imager. Similarly, the ground-based cloud image-based weather identification system and the sky imager are also merely non-limiting implementations provided by the present invention, and do not constitute a limitation on the executing entities and execution order of the steps in these ground-based cloud image-based weather identification methods.

[0078] In some embodiments, the weather recognition method based on ground-based cloud images provided in the first aspect of the present invention can be implemented in three parts: an offline training stage, an online analysis stage, and an online learning and optimization stage.

[0079] Please refer to Figure 4 , Figure 4 A flowchart of a weather identification method based on ground-based cloud images, according to some embodiments of the present invention, is shown.

[0080] like Figure 4 As shown, the weather recognition method 400 based on ground-based cloud images may include step S410 during the online analysis phase: acquiring ground-based cloud images.

[0081] Ground-based cloud images can be acquired by the camera module of a sky imager and then transmitted in real time to a weather recognition system based on ground-based cloud images for batch image preprocessing. In some embodiments, the ground-based cloud image acquired in step S410 may include a ground-based cloud image formed by data preprocessing operations. Here, data preprocessing operations may include data augmentation (e.g., scaling, rotating, flipping, or cropping the image), normalization, and converting the data into tensors. Data preprocessing operations can improve the data quality of the ground-based cloud images used to train the weather recognition model, ensuring that the trained weather recognition model can handle diverse inputs.

[0082] exist Figure 3 In the illustrated embodiment, the weather recognition system can acquire data collected by the sky imager, such as... Figure 3 The original ground-based cloud image shown in the first column 301. The sky imager transmits the acquired original image to the weather recognition system in real time. After acquiring the ground-based cloud image, the weather recognition system can perform a series of image preprocessing operations on the data to obtain, as shown in the image. Figure 3 The second column 302 shows the randomly rotated foundation cloud map, the third column 303 shows the offset foundation cloud map, the fourth column 304 shows the randomly scaled foundation cloud map, the fifth column 305 shows the radially transformed foundation cloud map, the sixth column 306 shows the horizontally flipped foundation cloud map, and the seventh column 307 shows the vertically flipped foundation cloud map.

[0083] Afterward, the weather recognition system can store the acquired ground-based cloud images and preprocessed image data to form a weather dataset.

[0084] like Figure 4 As shown, the weather recognition method 400 based on ground-based cloud images may include step S420: inputting the ground-based cloud image into a weather recognition model to recognize the weather in the ground-based cloud image. The weather recognition model is constructed based on a residual network and a multi-head attention mechanism, and is trained using a combination of knowledge distillation and adversarial training.

[0085] Please refer to Figure 5 , Figure 5 A schematic diagram of a weather recognition model provided according to some embodiments of the present invention is shown.

[0086] like Figure 5As shown, the weather recognition model 500 may include residual blocks 510, an adaptive dynamic adjustment attention mechanism 520, and residual blocks 530. The weather recognition system inputs a ground-based cloud image into the weather recognition model 500 and identifies the weather in the input ground-based cloud image based on the output 540 of the weather recognition model 500. The output 540 may include pre-defined weather categories, such as sunny, rainy, cloudy, and overcast.

[0087] Unlike traditional static attention mechanisms, the weather recognition model 500 is an improved residual network model (Multi-Head Attention Mechanism + ResNet50*Block, a multi-head attention mechanism plus residual blocks using ResNet50). The introduced attention mechanism 520 can be a multi-head attention mechanism. By introducing attention mechanism 520, the structure of the residual blocks is improved, and attention mechanism 520 can automatically adjust the attention weights of the residual network in the residual blocks according to the characteristics of the ground-based cloud image, thereby enhancing the weather recognition model's ability to recognize different weather features. Furthermore, compared to the traditional ResNet50 model, the weather recognition model 500 with the introduction of the attention mechanism can also improve its performance in complex cloud image recognition tasks, enabling it to better handle changing weather conditions.

[0088] In this way, the weather recognition model improves the residual block by introducing an attention mechanism, realizes feature learning of ground-based cloud images, and enhances the parallel processing capability of the weather recognition model for different image regions, improves the recognition capability of key areas of different weather types, thereby realizing the recognition of different weather features and being able to cope with complex cloud image recognition tasks.

[0089] Weather recognition methods based on ground-based cloud images also include an offline training phase for the weather recognition model. The weather recognition model is trained using a combination of knowledge distillation and adversarial training.

[0090] First, during the offline training phase, the weather recognition system can acquire a ground-based cloud image dataset for training using a sky imager. In one instance, the ground-based cloud image dataset used in the training phase can be a pre-collected dataset including various weather conditions, and the dataset may also include weather labels corresponding to various ground-based cloud images. In some embodiments, the ground-based cloud image dataset acquired by the weather recognition system can be formed through data preprocessing operations. Data preprocessing may include tensor transformation of the ground-based cloud image dataset to ensure that the ground-based cloud image dataset is suitable for deep learning frameworks.

[0091] The ground-based cloud image dataset can be divided into a training set, a validation set, and a test set. In some embodiments, the weather recognition system can divide the ground-based cloud image dataset into training, validation, and test sets in an 8:1:1 ratio for subsequent training and evaluation of the weather recognition model.

[0092] Subsequently, the weather recognition system can train a weather recognition model using a training set based on a combination of knowledge distillation and adversarial training. The weather recognition model trained using this combined approach can maintain high performance while significantly reducing computational overhead, further accelerating inference speed, enhancing robustness to anomalous data, and maintaining high inference efficiency and accuracy.

[0093] Please refer to Figure 6 , Figure 6 A schematic diagram illustrating the training of a weather recognition model provided according to some embodiments of the present invention is shown.

[0094] like Figure 6 As shown, in training a weather recognition model, the input data can include adversarial examples in addition to the ground-based cloud image dataset. The weather recognition system can generate adversarial examples using algorithms based on the ground-based cloud image dataset. These added adversarial examples can enhance the robustness of the trained weather recognition model to input noise.

[0095] A weather recognition system can be designed with a relatively small neural network structure as a student model S to mimic the teacher model T, thereby reducing computational complexity and the number of parameters. The trained student model S can then serve as the weather recognition model. In some embodiments, the weather recognition model can adopt a lightweight architecture tailored to the characteristics of weather data to ensure rapid inference during training and to enable operation on restrictive devices.

[0096] The teacher model T can be a weather recognition model with strong expressive power. For example, the teacher model T can be a deep convolutional neural network (CNN) or a trend recognition model with high accuracy. In other embodiments, the teacher model T can also be a large, pre-trained model capable of capturing more complex features.

[0097] In some embodiments, the teacher model T can be trained using a complete ground-based cloud image dataset provided by a sky imager, thereby ensuring that the teacher model T has good accuracy and robustness. Training techniques such as data augmentation, regularization, and early stopping can be used during training.

[0098] Next, the trained teacher model T is transferred to the training process of the weather recognition model. Based on adversarial examples, the knowledge output by the teacher model T is used to train the student model S through forward and backward propagation steps. During forward propagation, the input data x+δ is input into both the student model S and the teacher model T, where δ can be random noise. The student model S processes the input ground-based cloud image to obtain the output S(x+δ); the teacher model T processes the input ground-based cloud image to obtain the output T(x+δ).

[0099] Subsequently, during backpropagation, the Kullback-Leibler divergence (KL) between the output S(x+δ) of the student model S and the output T(x+δ) of the teacher model T is calculated: KL(S(x+δ)‖T(x+δ)). This KL divergence is then used as the loss signal for backpropagation to update the parameters of the student model, enabling it to learn from the teacher model's knowledge.

[0100] In other embodiments, in addition to KL divergence, a cross-entropy loss function can be incorporated into the backpropagation process. The weather recognition system can determine the cross-entropy based on the outputs of the teacher model T and the student model S, and backpropagate using KL divergence and / or cross-entropy as loss signals to train the weather recognition model. By incorporating the cross-entropy loss function, the trained weather recognition model can learn effective information from both real samples (ground-based cloud image dataset) and adversarial examples.

[0101] In summary, knowledge distillation and adversarial training techniques can further lightweight the inference stage by training a smaller model (student model) to mimic the behavior of a larger model (teacher model), thus providing an effective approach for training weather recognition models. This reduces resource consumption while improving the performance of weather recognition models, which is particularly important for large-scale weather recognition applications and resource-constrained scenarios. Furthermore, by introducing adversarial examples into the training process, the robustness of weather recognition models can be improved, enabling them to maintain high accuracy even when faced with abnormal or noisy data.

[0102] During the training phase, the weather recognition system can employ a dynamic learning rate adjustment mechanism, dynamically adjusting the learning rate based on the training progress and performance of the weather recognition model. The definition and adjustment of the learning rate can utilize the Adam (Adaptive Moment Estimation) optimization algorithm. Furthermore, due to the adaptive nature of the Adam optimization algorithm, it can help the weather recognition model learn more effectively and improve performance. Simultaneously, the weather recognition system can define the number of training epochs and set an appropriate loss function; in some embodiments, the loss function can be the cross-entropy loss function.

[0103] Thus, during the training phase, the weather recognition model can be optimized with faster convergence and a more stable training process.

[0104] Next, the weather recognition system can use the validation set to comprehensively evaluate the trained weather recognition model and adjust its parameters based on the evaluation results to ensure its performance in practical applications. Finally, a comprehensive test of the weather recognition model is conducted on the test set to evaluate its performance under different conditions, ensuring its capability for real-world applications.

[0105] Please refer to Figure 7 and Figure 8 , Figure 7 The graphs show the accuracy and loss rate during the training phase for a weather recognition model with multi-head attention and a residual network model without multi-head attention. Figure 8 The graphs show the accuracy and loss rate of the weather recognition model with multi-head attention mechanism and the residual network model without multi-head attention mechanism during the testing phase.

[0106] like Figure 7 As shown in Figure 711, the accuracy curve of the weather recognition model incorporating a multi-head attention mechanism during the training phase is close to 100%, indicating a high accuracy rate. However, the accuracy curve of the residual network model without a multi-head attention mechanism during the training phase (Figure 712) shows that its accuracy is between 95% and 99%, lower than that of the weather recognition model. The loss rate can be used to measure the difference between the model's predictions and the actual values. The loss rate curve of the weather recognition model incorporating a multi-head attention mechanism during the training phase (Figure 721) is close to 0%, indicating a small difference between the predicted and actual values. Conversely, the loss rate curve of the residual network model without a multi-head attention mechanism during the training phase (Figure 722) shows that its loss rate is between 0% and 0.2%, higher than that of the weather recognition model.

[0107] like Figure 8As shown, the accuracy curve 811 of the weather recognition model incorporating the multi-head attention mechanism is close to 100% in the testing phase, indicating that the accuracy of the weather recognition model after training is very high. The accuracy curve 812 of the residual network model without the multi-head attention mechanism shows that the accuracy of the residual network model improved after training, but it is still lower than the accuracy of the weather recognition model. The loss rate curve 821 of the weather recognition model incorporating the multi-head attention mechanism is close to 0% in the testing phase, indicating that the difference between the predicted and actual values ​​of the weather recognition model after training is very small. The loss rate curve 822 of the residual network model without the multi-head attention mechanism shows that the loss rate of the residual network model after training fluctuates more and is higher than the loss rate of the weather recognition model.

[0108] Thus, the weather recognition model undergoes adversarial training and learning through an improved residual network, and is further optimized for inference using knowledge distillation techniques. The offline-trained weather recognition model can be deployed in real-world weather recognition systems and used for weather identification on real-time captured ground-based cloud images during the online analysis phase. In some embodiments, the weather recognition method can identify sunny, cloudy, partly cloudy, slightly cloudy, and rainy weather conditions.

[0109] In practical applications, weather recognition methods can also include an online learning and optimization phase. Sky imagers continuously capture ground-based cloud images and store them to form a ground-based cloud image dataset. In some embodiments, the weather recognition system can store the acquired ground-based cloud images and pre-processed images together in real time into the weather dataset. Based on the continuously updated weather dataset, the weather recognition system can continuously optimize the weather dataset using online learning optimization algorithms to maintain its adaptability to new data; and through an online learning mechanism, it continuously learns new weather data, adjusting its own structure as needed through a dynamic learning rate adjustment mechanism to maintain a high level of weather recognition accuracy.

[0110] In this way, through the online learning and optimization phase, the weather recognition method can achieve a closed-loop online weather recognition process and ensure that the weather recognition model can maintain optimal performance in practical applications. Through online learning, the performance of the weather recognition model is continuously optimized, enabling the weather recognition model to adapt to new weather data and patterns during long-term operation and maintain efficient recognition performance.

[0111] The following is a specific, non-limiting preferred embodiment, which is used to further describe the weather identification system based on ground-based cloud images proposed in this invention.

[0112] Please refer to Figure 9 , Figure 9A flowchart of a weather identification method based on ground-based cloud images according to a preferred embodiment of the present invention is shown.

[0113] exist Figure 9 In the illustrated embodiment, the sky imager can acquire captured ground-based cloud images and transmit them to a weather recognition system via data transmission. The weather recognition system can perform data preprocessing on the received data. Data preprocessing may include data augmentation, data standardization and normalization, and tensor transformation. Data augmentation includes rotation, cropping, or scaling.

[0114] Through data preprocessing operations by the weather recognition system, ground-based cloud images for various weather conditions can be gradually obtained, such as those for sunny, cloudy, partly cloudy, slightly cloudy, or rainy days. The weather recognition system can then store the ground-based cloud images and the preprocessed ground-based cloud image data into a self-made weather dataset.

[0115] The acquired ground-based cloud images of various weather conditions can be used to form a ground-based cloud image dataset for the offline training phase. During the training process, the ground-based cloud image dataset can be further divided into a training set, a test set, and a validation set, which are used to implement training, testing, and validation during the training process, respectively.

[0116] A self-made weather dataset can be used for online analysis in a weather recognition system. After the weather recognition system acquires ground-based cloud images transmitted by a sky imager, weather recognition can be performed based on the weather recognition model deployed within the system. The weather recognition model is constructed based on a dynamic attention mechanism and a residual network, and is formed through knowledge distillation and adversarial training.

[0117] The self-made weather dataset can also be used for online learning and continuous optimization of the weather recognition model. Here, the weather recognition system can adjust and optimize the parameters of the weather recognition model based on a dynamic learning rate adjustment mechanism (such as the Adam algorithm).

[0118] In summary, the weather recognition method based on ground-based cloud images provided by this invention enhances the parallel processing capability of the weather recognition model for different image regions and improves the recognition capability for key regions of different weather types by introducing an attention mechanism to improve residual blocks. Simultaneously, adversarial training methods are incorporated during the training process of the weather recognition model to improve its robustness in the face of noisy data or abnormal weather conditions, thereby increasing the accuracy and efficiency of weather recognition. Furthermore, in terms of optimization, lightweight inference optimization techniques and knowledge distillation techniques are adopted to reduce resource consumption during the training phase of the weather recognition model and improve its performance, effectively addressing large-scale weather recognition applications and resource-constrained scenarios.

[0119] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0120] Those skilled in the art will understand that information, signals, and data can be represented using any of a variety of different techniques and arts. For example, the data, instructions, commands, information, signals, bits, symbols, and chips described throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0121] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0122] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0123] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0124] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0125] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A weather identification method based on ground-based cloud images, characterized in that, include: Obtain foundation cloud image; as well as The ground-based cloud image is input into a weather recognition model to identify the weather in the ground-based cloud image. The weather recognition model is constructed based on a residual network and a multi-head attention mechanism, and is trained by a combination of knowledge distillation and adversarial training.

2. The weather identification method based on ground-based cloud images as described in claim 1, characterized in that, The steps involved in training the weather recognition model using a combination of knowledge distillation and adversarial training include: The ground-based cloud map dataset and adversarial examples are used as input data and divided into training set, validation set and test set; The weather recognition model is trained using the training set based on a combination of knowledge distillation and adversarial training. The parameters of the trained weather recognition model are adjusted using the validation set; and The performance of the weather recognition model was evaluated using the test set.

3. The weather identification method based on ground-based cloud images as described in claim 2, characterized in that, The steps involved in training the weather recognition model using a combination of knowledge distillation and adversarial training include: The learning rate is defined using a dynamic learning rate adjustment mechanism.

4. The weather identification method based on ground-based cloud images as described in claim 2, characterized in that, The steps for training the weather recognition model include: The ground-based cloud map and adversarial examples in the training set are input into the student model and the teacher model to determine the output of the student model and the output of the teacher model; Calculate the KL divergence between the output of the student model and the output of the teacher model; and Backpropagation is performed based on the KL divergence, enabling the student model to learn the knowledge of the teacher model, thereby training the weather recognition model.

5. The weather identification method based on ground-based cloud images as described in claim 4, characterized in that, The steps for training the weather recognition model also include: The cross-entropy loss is determined based on the outputs of the student model and the teacher model; and Backpropagation is performed based on the KL divergence and the cross-entropy loss to train the weather recognition model.

6. The weather identification method based on ground-based cloud images as described in claim 1, characterized in that, It also includes the following steps: The acquired ground-based cloud images are stored in a weather dataset; as well as Based on the weather dataset, the weather recognition model is optimized using an online learning optimization algorithm.

7. The weather identification method based on ground-based cloud images as described in claim 1, characterized in that, The ground cloud map includes a ground cloud map formed by data preprocessing operations, which include data augmentation, normalization, and / or tensor transformation.

8. A weather recognition system based on ground-based cloud images, characterized in that, include: Memory, on which computer instructions are stored; as well as A processor, connected to the memory, and configured to execute computer instructions stored in the memory to implement the weather identification method based on ground-based cloud images as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, the weather identification method based on ground-based cloud images as described in any one of claims 1 to 7 is implemented.

10. A sky imager, characterized in that, include: The camera module is used to capture ground-based cloud images; A microcomputer, wherein the microcomputer is equipped with a weather recognition system based on ground-based cloud images as described in claim 8.

Images