Cloud sky segmentation method and device, terminal equipment and storage medium

Abstract

The application is suitable for the technical field of image processing, and provides a cloud-sky segmentation method and device, terminal equipment and storage medium.The cloud-sky segmentation method comprises: acquiring a ground-based sky image; extracting a brightness coefficient of the ground-based sky image according to the ground-based sky image; if the brightness coefficient is less than a preset threshold, segmenting a cloud cluster and the sky in the ground-based sky image according to three primary colors in the ground-based sky image to obtain a cloud cluster region and a sky region in the ground-based sky image; and if the brightness coefficient is greater than or equal to the preset threshold, inputting the ground-based sky image into a trained cloud-sky segmentation model to output the cloud cluster region and the sky region in the ground-based sky image by the cloud-sky segmentation model.The above method adopts different cloud-sky segmentation modes based on the brightness coefficient, and the accuracy of cloud cluster recognition and cloud amount calculation in the ground-based sky image can be improved through the complementation of the two cloud-sky segmentation modes.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a cloud and sky segmentation method, apparatus, terminal equipment and storage medium. Background Technology

[0002] Accurate prediction of photovoltaic power is of great significance for the stable dispatch of the power grid, the production of power plants, and the safe operation of photovoltaic systems. Surface solar irradiance is the main factor affecting photovoltaic power output, and many factors influence surface solar irradiance. Cloud distribution, cloud thickness, movement and changes, and differences in cloud radiation attenuation are the main reasons for the uncertainty of surface solar irradiance. Therefore, accurately identifying cloud clusters in the sky and accurately calculating cloud cover based on the identified cloud clusters is the foundation for achieving refined power prediction.

[0003] However, due to weather changes, the shape, outline, position, and thickness of clouds are not fixed and are prone to change over time. This increases the difficulty of segmenting clouds and sky in ground-based sky images, resulting in low accuracy in cloud identification and cloud cover calculation in ground-based sky images. Summary of the Invention

[0004] This application provides a cloud-sky segmentation method, apparatus, terminal device, and storage medium, which can improve the accuracy of cloud cluster identification and cloud cover calculation in ground-based sky images.

[0005] A first aspect of this application provides a cloud-sky segmentation method, the cloud-sky segmentation method comprising:

[0006] Acquire ground-based sky images;

[0007] Based on the ground-based sky image, extract the brightness coefficient of the ground-based sky image;

[0008] If the brightness coefficient is less than a preset threshold, then the clouds and sky in the ground-based sky image are segmented according to the three primary colors in the ground-based sky image to obtain the cloud region and sky region in the ground-based sky image.

[0009] If the brightness coefficient is greater than or equal to a preset threshold, the ground-based sky image is input into a trained cloud-sky segmentation model, and the cloud-sky segmentation model outputs the cloud region and sky region in the ground-based sky image.

[0010] A second aspect of this application provides a cloud-sky segmentation device, the cloud-sky segmentation device comprising:

[0011] The image acquisition module is used to acquire ground-based sky images;

[0012] The coefficient acquisition module is used to extract the brightness coefficient of the ground-based sky image based on the ground-based sky image;

[0013] The first judgment module is used to divide the clouds and sky in the ground-based sky image according to the three primary colors in the ground-based sky image if the brightness coefficient is less than a preset threshold, so as to obtain the cloud area and sky area in the ground-based sky image.

[0014] The second judgment module is used to input the ground-based sky image into the trained cloud-sky segmentation model if the brightness coefficient is greater than or equal to a preset threshold, and the cloud-sky segmentation model outputs the cloud region and sky region in the ground-based sky image.

[0015] A third aspect of this application provides a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the cloud-sky segmentation method described in the first aspect above.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the cloud-sky segmentation method described in the first aspect.

[0017] The fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the cloud-sky segmentation method described in the first aspect.

[0018] The beneficial effects of the embodiments in this application compared with the prior art are:

[0019] In this embodiment, a ground-based sky image can be acquired, and its brightness coefficient can be extracted. When the brightness coefficient is less than a preset threshold, the clouds and sky in the ground-based sky image are segmented according to the three primary colors in the image to obtain cloud and sky regions. When the brightness coefficient is greater than or equal to the preset threshold, the ground-based sky image is input into a trained cloud-sky segmentation model, which outputs the cloud and sky regions. The above method uses different cloud-sky segmentation methods based on the brightness coefficient. The complementarity of these two methods can improve the accuracy of cloud identification and cloud cover calculation in ground-based sky images. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a cloud-sky segmentation method provided in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the process for obtaining the brightness coefficient provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the process for determining cloud and sky regions provided in the embodiments of this application. Figure 1 ;

[0024] Figure 4 This is a flowchart. Figure 1 Corresponding segmentation results illustration Figure 1 ;

[0025] Figure 5 This is a flowchart. Figure 1 Corresponding segmentation results illustration Figure 2 ;

[0026] Figure 6 This is a flowchart of the training process for the cloud-sky segmentation model provided in this application embodiment;

[0027] Figure 7 These are labeled ground-based sky images from the training dataset;

[0028] Figure 8 This is a schematic diagram of the segmentation results obtained using a trained cloud and sky segmentation model. Figure 1 ;

[0029] Figure 9 This is a schematic diagram of the segmentation results obtained using a trained cloud and sky segmentation model. Figure 2 ;

[0030] Figure 10 This is a schematic diagram of the structure of a cloud-sky segmentation device provided in an embodiment of this application;

[0031] Figure 11 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0032] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0033] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0034] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0035] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0036] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0037] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0038] It should be understood that the sequence number of each step in this embodiment does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0039] With the rapid development of photovoltaic power generation, accurate prediction of photovoltaic power is of great significance for the stable dispatch of the power grid, the production of power plants, and the safe operation of photovoltaic systems. Surface solar irradiance is the main factor affecting photovoltaic power output, and many factors influence surface solar irradiance. Cloud distribution, cloud thickness, movement and changes, and differences in cloud radiation attenuation are the main reasons for the uncertainty of surface solar irradiance. Therefore, accurately identifying cloud clusters in the sky and accurately calculating cloud cover based on the identified cloud clusters is the foundation for achieving refined power prediction.

[0040] Due to weather changes, the shape, outline, position, and thickness of clouds are not fixed and are prone to change over time. This increases the difficulty of segmenting clouds and the sky in ground-based sky images, resulting in low accuracy in cloud identification and cloud cover calculation in ground-based sky images.

[0041] In existing technologies, threshold segmentation is commonly used to segment clouds and sky in ground-based sky images. This method can segment clouds and sky and calculate cloud cover to a certain extent, but it has significant limitations. It is difficult to accurately segment clouds and sky in areas of strong light around the sun and along the edges of cloud clusters.

[0042] Therefore, to improve the accuracy of cloud identification and cloud cover calculation in ground-based sky images, this application provides a cloud-sky segmentation method. This method involves acquiring a ground-based sky image and extracting its brightness coefficient. When the brightness coefficient is less than a preset threshold, the cloud and sky regions in the ground-based sky image are segmented based on the three primary colors in the image. When the brightness coefficient is greater than or equal to the preset threshold, the ground-based sky image is input into a trained cloud-sky segmentation model, which outputs the cloud and sky regions. This method employs different cloud-sky segmentation methods based on the brightness coefficient, and the complementarity of these two methods can improve the accuracy of cloud identification and cloud cover calculation in ground-based sky images.

[0043] To illustrate the technical solution of this application, specific embodiments are described below.

[0044] Reference Figure 1 The diagram illustrates a flowchart of a cloud-sky segmentation method provided in Embodiment 1 of this application. Figure 1As shown, the cloud-sky segmentation method may include the following steps:

[0045] Step 101: Obtain the ground-based sky image.

[0046] It should be noted that the cloud-sky segmentation method of this application embodiment can be executed by the cloud-sky segmentation device of this application embodiment. The cloud-sky segmentation device of this application embodiment can be configured in any terminal device to execute the cloud-sky segmentation method of this application embodiment. For example, the device of this application embodiment can be configured in the monitoring terminal of the cloud computing system, and this application embodiment does not limit this.

[0047] Ground-based sky images refer to sky images acquired using ground-based observation equipment. It should be noted that ground-based observation equipment can refer to a Whole Sky Imager (WSI), a Total Sky Imager (TSI), or an Infrared Cloud Imager (ICI).

[0048] Step 102: Extract the brightness coefficient of the ground-based sky image based on the ground-based sky image.

[0049] In this embodiment of the application, the brightness coefficient of the ground sky can be extracted by first obtaining the brightness of the ground sky image, and then extracting the brightness coefficient of the ground sky image based on the brightness of the ground sky image.

[0050] Among them, the brightness of the ground-based sky image refers to the brightness of the image. According to the principle of three primary colors imaging, the brightness intensity of monochromatic light is different for each color.

[0051] In the embodiments of this application, see Figure 2 To better measure the brightness of the ground-based sky image, the brightness of the ground-based image can be normalized to obtain the brightness coefficient of the ground-based sky image. That is, step 102 above can specifically include the following steps:

[0052] Step 201: Obtain the pixel values ​​corresponding to the three primary color channels of the ground-based sky image.

[0053] The three primary color channels refer to the color channels corresponding to red, green, and blue, respectively. It's important to note that an image has three color channels, meaning all colors are composed of these three primary colors. Therefore, the pixel values ​​corresponding to the three primary color channels of the ground-sky image can be obtained separately; that is, the average of the RGB values ​​of all pixels can be used as the pixel values ​​corresponding to the three primary color channels of the image.

[0054] For example, the mean value R of the red pixel values ​​corresponding to all pixels in the ground-based sky image is obtained, the mean value G of the green pixel values ​​corresponding to all pixels in the ground-based sky image is obtained, and the mean value B of the blue pixel values ​​corresponding to all pixels in the ground-based sky image is obtained, where R is the pixel value corresponding to the red channel of the ground-based sky image, G is the pixel value corresponding to the green channel of the ground-based sky image, and B is the pixel value corresponding to the blue channel of the ground-based sky image.

[0055] Step 202: Determine the brightness of the ground-based sky image based on the pixel values ​​corresponding to the three primary color channels and the preset conversion coefficients corresponding to the three pixel values.

[0056] In this embodiment, the preset conversion coefficient is used to convert the pixel values ​​corresponding to the three primary colors into image brightness. Therefore, the brightness of the ground-based sky image can be determined based on the pixel values ​​corresponding to the three primary color channels and their respective preset conversion coefficients.

[0057] For example, the brightness of a ground-based sky image can be calculated using the following formula:

[0058] Y = R * 0.299 + G * 0.587 + B * 0.114

[0059] Where Y is the brightness of the ground-based sky image, 0.299 is the preset conversion coefficient corresponding to the pixel value of the red channel, 0.587 is the preset conversion coefficient corresponding to the pixel value of the green channel, and 0.114 is the preset conversion coefficient corresponding to the pixel value of the blue channel.

[0060] Step 203: Normalize the brightness of the ground-based sky image to determine the brightness coefficient of the ground-based sky image.

[0061] In this embodiment of the application, in order to better measure the brightness of the ground-based sky image and make the brightness coefficient better reflect the brightness characteristics of the ground-based sky image, the brightness of the ground-based sky image can be normalized to obtain a dimensionless brightness coefficient.

[0062] For example, the brightness coefficient of a ground-based sky image can be solved using the following formula:

[0063]

[0064] Where brighe_sacle is the brightness coefficient of the ground-based sky image.

[0065] It should be understood that the brightness of ground-based sky images can significantly affect the implementation of cloud-sky segmentation. For example, when the brightness of ground-based sky images is high, the threshold segmentation method exhibits poor segmentation results. Therefore, the threshold segmentation method is not suitable when the brightness of ground-based sky images is high.

[0066] Step 103: If the brightness coefficient is less than the preset threshold, the cloud and sky regions in the ground sky are segmented according to the three primary colors in the ground sky image to obtain the cloud region and sky region in the ground sky image.

[0067] The preset threshold can refer to a threshold set based on the segmentation effect, such as 0.7. When the brightness coefficient is less than the preset threshold, it indicates that the brightness of the currently acquired ground-based sky image is low. The pixel values ​​of the three primary colors corresponding to each pixel in the ground-based sky image are relatively accurate. At this time, the clouds and sky in the ground-based sky image can be segmented based on the three primary colors of the ground-based sky image.

[0068] In this embodiment of the application, the clouds and sky in the ground sky are segmented according to the three primary colors in the ground sky image. The threshold segmentation method can be used based on the color difference between the clouds and the sky, that is, the pixel values ​​corresponding to the pixels in the cloud area and the sky area are different.

[0069] In this embodiment of the application, by performing three-primary-color feature analysis on the ground-based sky image, it can be determined that: if the pixel value of a pixel is (0,0,255), then the pixel represents the sky; and if the pixel value of a pixel is (255,255,255) or (192,192,192), then the pixel represents a cloud. Based on the above feature analysis, see [link to relevant documentation]. Figure 3 Step 103 above may specifically include the following steps:

[0070] Step 301: Determine the blue channel value, red channel value, and green channel value of each pixel based on the pixel values ​​corresponding to the three primary colors of each pixel in the ground-based sky image;

[0071] In this embodiment of the application, the red pixel value of a pixel can be determined as the red channel value of that pixel, the green pixel value as the green channel value of that pixel, and the blue pixel value as the blue channel value of that pixel.

[0072] For example, suppose a pixel has a value of (0, 0, 255), then the red channel value, green channel value, and blue channel value of that pixel are 0, 0, 255 respectively.

[0073] Step 302: Determine the normalized red-blue ratio value of each pixel based on the blue channel value and the red channel value of each pixel;

[0074] In the embodiments of this application, it was found through research that when the brightness coefficient is less than the preset threshold of 0.7, the cloud and sky segmentation based on the red-blue ratio threshold shows good performance. Therefore, it is first necessary to determine the normalized red-blue ratio value of each pixel based on the blue channel value and the red channel value of each pixel.

[0075] For example, the normalized red-blue ratio is calculated as follows:

[0076]

[0077] Wherein, NRBR is the normalized red-blue ratio value corresponding to the pixel, B is the blue channel value of the pixel, and R is the red channel value of the pixel.

[0078] Step 303: Determine the average value of the three channels for each pixel based on the blue channel value, red channel value, and green channel value of each pixel;

[0079] In this embodiment, the cloud and sky segmentation method based on the red-blue ratio threshold can only segment the sky and clouds. However, clouds can include thin white clouds and opaque gray clouds. The cloud and sky segmentation method based on the red-blue ratio threshold cannot segment the two types of clouds. Therefore, the three channel values ​​corresponding to the pixels of thin white clouds are close to saturation, while the three channel values ​​of opaque gray clouds are relatively small. The average value of the three channels of each pixel can be obtained.

[0080] Step 304: For each pixel, based on the corresponding normalized red-blue ratio and preset red-blue ratio threshold, as well as the corresponding three-channel average value and preset three-channel average value threshold, determine the category of each pixel to obtain the cloud region and sky region in the ground-based sky image.

[0081] In this embodiment, the cloud region includes thin cloud regions and opaque cloud regions. To better distinguish between the cloud region and the sky region, the category of each pixel is determined twice: based on the normalized red-blue ratio and preset red-blue ratio threshold of each pixel, as well as the corresponding three-channel average value and preset three-channel average value threshold, the category of each pixel is determined.

[0082] In one possible implementation, a specific embodiment of step 304 above may include the following steps:

[0083] If the normalized red-blue ratio value corresponding to a pixel is greater than the preset red-blue ratio threshold, then the pixel is determined to belong to the category of sky.

[0084] If the normalized red-blue ratio value corresponding to a pixel is less than or equal to the preset red-blue ratio threshold, and the three-channel average value corresponding to the pixel is greater than the preset three-channel average value threshold, then the pixel is determined to belong to the category of thin cloud.

[0085] If the normalized red-blue ratio value corresponding to a pixel is less than or equal to the preset red-blue ratio threshold, and the three-channel average value corresponding to the pixel is less than or equal to the preset three-channel average value threshold, then the pixel is determined to belong to the category of opaque cloud.

[0086] Based on the category of each pixel in the ground-based sky image, determine the cloud region and sky region in the ground-based sky image.

[0087] In this embodiment, the cloud-sky segmentation method based on the red-blue ratio threshold and the three-channel average threshold is determined by the following formula:

[0088]

[0089] Where C represents the category of the pixel, and L represents the average value of the three channels corresponding to the pixel.

[0090] In this embodiment, the ground-based cloud and sky image is segmented according to the above-mentioned cloud and sky segmentation method based on the red-blue ratio threshold and the three-channel average value threshold, resulting in the following: Figure 4 and Figure 5 The segmentation result is shown. The white area on the right side of the image represents clouds, and the black area represents the sky.

[0091] Step 104: If the brightness coefficient is greater than or equal to the preset threshold, the ground-based sky image is input into the trained cloud-sky segmentation model, and the cloud-sky segmentation model outputs the cloud region and sky region in the ground-based sky image.

[0092] In this embodiment, when the brightness coefficient is greater than or equal to a preset threshold, it indicates that the brightness of the ground-based sky image is relatively high, making it unsuitable for threshold-based segmentation methods. In this case, a cloud and sky segmentation model that requires only a small dataset for training can be used to accurately segment the ground-based sky image, effectively compensating for the shortcoming of threshold-based cloud and sky segmentation methods that perform poorly when the image brightness is too high.

[0093] The cloud-sky segmentation model can refer to a neural network model based on an improved Unet structure. The cloud-sky segmentation model used in this embodiment adds a self-attention module between the output results, which can autonomously select target features from the ground-based sky image and analyze those features.

[0094] As one possible implementation method, such as Figure 6 As shown, the training process of the above-mentioned cloud-sky segmentation model can specifically include the following steps:

[0095] Step 601: Obtain the training dataset.

[0096] The training dataset includes N ground-based sky images with a first label, which is used to indicate the target cloud region in the ground-based sky image. The N ground-based sky images include the original ground-based sky image and the ground-based sky image enhanced from the original ground-based sky image.

[0097] In this embodiment, the ground-based sky images in the training dataset are all labeled ground-based sky images, that is, the target cloud region in the ground-based sky image is labeled using a first label. It should be noted that when labeling, the first label needs to mark the contour points of the target, i.e., as shown... Figure 7 The outline of the cloud cluster is shown in the diagram.

[0098] In this embodiment of the application, when the number of original ground-based sky images is limited and the amount of data may not be sufficient to meet the training needs of the neural network (which may be a Unet network), the original ground-based sky images and the original ground-based sky images can be enhanced (e.g., brightness and contrast enhancement). The enhanced labeled images can also be used as training datasets, which can greatly enrich the scale of cloud and sky segmentation data and improve the segmentation effect.

[0099] Step 602: Based on the first label, extract the target cloud region from the ground-based sky image contained in the training dataset, and input the training dataset into the cloud-sky segmentation model. Before output, use the self-attention module to select target features in the ground-based sky image, and analyze the target features to obtain the predicted cloud region.

[0100] In this embodiment of the application, since the target cloud region in the image contained in the training dataset has a first label, the target cloud region can be extracted from the image contained in the training dataset based on the first label.

[0101] In this embodiment, assuming the cloud-sky segmentation model is a Unet network structure, a transformer self-attention module can be added before the softmax input of the Unet network. The output of the transformer self-attention module is used as the input of the softmax to achieve cloud-sky segmentation. The self-attention module can adaptively select salient features (i.e. target features) in the ground-based sky image to achieve cloud-sky segmentation and obtain the predicted cloud area, which can effectively improve the segmentation results.

[0102] Step 603: Using an overlap-based loss function, obtain the difference information between the target cloud region and the predicted cloud region;

[0103] In this embodiment of the application, the loss function based on overlap can refer to the loss function based on IOU, such as the Lovasz-Softmax loss function. According to this loss function, the target cloud region and the predicted cloud region are input into the loss function to calculate the difference information between the target cloud region and the predicted cloud region.

[0104] Step 604: Based on the difference information, perform backpropagation on the cloud-sky segmentation model to obtain the trained cloud-sky segmentation model.

[0105] In this embodiment, the calculated difference information is compared with the set loss function threshold. If the difference information is greater than the loss function threshold, the cloud-sky segmentation model is backpropagated to optimize the model parameters in the neural network. The model parameters of the neural network are backpropagated until the difference information output by the neural network model is less than the loss function threshold, that is, the training network converges, indicating that the training is complete and the trained cloud-sky segmentation model can be obtained.

[0106] It should be noted that this application uses a trained cloud-sky segmentation model to segment the cloud data from a ground-based image. The resulting image can be found in [reference needed]. Figure 8 and Figure 9 The white area on the right side of both images refers to the cloud formations in the left image.

[0107] In one possible implementation, the cloud-sky segmentation method also includes:

[0108] The trained cloud-sky segmentation model is evaluated based on the number of overlapping pixels between the target cloud region and the predicted cloud region, as well as the total number of pixels in the target cloud region and the predicted cloud region.

[0109] In this embodiment of the application, the trained cloud segmentation model can be evaluated using the Dice coefficient. The Dice coefficient is a set similarity measurement function, which is usually used to calculate the similarity between two samples. Its value ranges from [0,1]. It is commonly used in the field of image segmentation as an evaluation index to evaluate the quality of the model. The closer it is to 1, the better the model performance.

[0110] Its definition is as follows:

[0111]

[0112] Where A∩B represents the number of overlapping pixels between the target cloud region and the predicted cloud region, the absolute value of A + the absolute value of B represents the total number of pixels in the target cloud region and the predicted cloud region, and Dice is used to evaluate the trained cloud-sky segmentation model.

[0113] It should be noted that, in this embodiment of the application, after obtaining the training dataset and designing the loss function and evaluation function, Unet can be trained on a GPU server until the cloud-sky segmentation model is trained.

[0114] Verification shows that the cloud-sky segmentation model in this application achieves a Dice value of 92.4% after training, which is nearly 10% higher than the standard Unet segmentation using the cross-entropy loss function.

[0115] In one possible implementation, the cloud-sky segmentation method also includes:

[0116] The cloud cover corresponding to the cloud cluster area is determined based on the total number of pixels in the cloud cluster area and the total number of pixels in the ground-based sky image.

[0117] In this embodiment of the application, the formula for calculating cloud cover is as follows:

[0118]

[0119] Cloud cover can be used for power prediction of photovoltaic power plants.

[0120] In this embodiment, a ground-based sky image can be acquired, and its brightness coefficient can be extracted. When the brightness coefficient is less than a preset threshold, the clouds and sky in the ground-based sky image are segmented according to the three primary colors in the image to obtain cloud and sky regions. When the brightness coefficient is greater than or equal to the preset threshold, the ground-based sky image is input into a trained cloud-sky segmentation model, which outputs the cloud and sky regions. The above method uses different cloud-sky segmentation methods based on the brightness coefficient. The complementarity of these two methods can improve the accuracy of cloud identification and cloud cover calculation in ground-based sky images.

[0121] See Figure 10 The diagram shows a schematic of a cloud-sky segmentation device provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0122] The cloud-sky segmentation device may specifically include the following modules:

[0123] Image acquisition module 1001 is used to acquire ground-based sky images;

[0124] The coefficient acquisition module 1002 is used to extract the brightness coefficient of the ground-based sky image based on the ground-based sky image;

[0125] The first judgment module 1003 is used to divide the clouds and sky in the ground sky image according to the three primary colors in the ground sky image if the brightness coefficient is less than the preset threshold, so as to obtain the cloud area and sky area in the ground sky image.

[0126] The second judgment module 1004 is used to input the ground-based sky image into the trained cloud-sky segmentation model if the brightness coefficient is greater than or equal to a preset threshold, and the cloud-sky segmentation model outputs the cloud region and sky region in the ground-based sky image.

[0127] In this embodiment of the application, the cloud-sky segmentation device may further include the following modules:

[0128] The cloud cover determination module is used to determine the cloud cover corresponding to a cloud cluster area based on the total number of pixels in the cloud cluster area and the total number of pixels in the ground-based sky image.

[0129] In this embodiment of the application, the coefficient acquisition module 1002 may further include the following sub-modules:

[0130] The pixel value acquisition submodule is used to acquire the pixel values ​​corresponding to the three primary color channels of the ground-based sky image.

[0131] The brightness determination submodule is used to determine the brightness of the ground-based sky image based on the pixel values ​​corresponding to the three primary color channels and the preset conversion coefficients corresponding to the three pixel values. The preset conversion coefficients are used to convert the pixel values ​​corresponding to the three primary colors into image brightness.

[0132] The normalization submodule is used to normalize the brightness of the ground-based sky image and determine the brightness coefficient of the ground-based sky image.

[0133] In this embodiment of the application, the first determination module 1003 may specifically include the following sub-modules:

[0134] The channel value determination submodule is used to determine the blue channel value, red channel value, and green channel value of each pixel based on the pixel values ​​corresponding to the three primary colors of each pixel in the ground-based sky image.

[0135] The ratio determination submodule is used to determine the normalized red-blue ratio of each pixel based on the blue channel value and the red channel value of each pixel;

[0136] The average value determination submodule is used to determine the three-channel average value of each pixel based on the blue channel value, red channel value, and green channel value of each pixel.

[0137] The first region determination submodule is used to determine the category of each pixel based on the corresponding normalized red-blue ratio and preset red-blue ratio threshold, as well as the corresponding three-channel average value and preset three-channel average value threshold, so as to obtain the cloud region and sky region in the ground-based sky image.

[0138] In this embodiment of the application, when the sky region includes thin cloud regions and opaque cloud regions, the first region determination submodule may specifically include the following units:

[0139] The sky determination unit is used to determine the category of a pixel as sky if the normalized red-blue ratio value corresponding to the pixel is greater than the preset red-blue ratio threshold.

[0140] The thin cloud determination unit is used to determine the category of a pixel as thin cloud if the normalized red-blue ratio value corresponding to the pixel is less than or equal to a preset red-blue ratio threshold and the three-channel average value corresponding to the pixel is greater than a preset three-channel average value threshold.

[0141] The opaque cloud determination unit is used to determine the category of the pixel as an opaque cloud if the normalized red-blue ratio value corresponding to the pixel is less than or equal to a preset red-blue ratio threshold and the three-channel average value corresponding to the pixel is less than or equal to a preset three-channel average value threshold.

[0142] The category determination unit is used to determine the cloud region and sky region in the ground-based sky image based on the category to which each pixel belongs.

[0143] In this embodiment, the trained cloud-sky segmentation model includes a self-attention module; the training process of the cloud-sky segmentation model includes the following sub-modules:

[0144] The dataset acquisition submodule is used to acquire the training dataset, which includes N ground-based sky images with a first label. The first label is used to indicate the target cloud region in the ground-based sky image. The N ground-based sky images include the original ground-based sky image and the ground-based sky image after enhancement of the original ground-based sky image.

[0145] The prediction submodule is used to extract the target cloud region from the ground-based sky image contained in the training dataset based on the first label, and input the training dataset into the cloud-sky segmentation model. Before output, the self-attention module is used to select the target features in the ground-based sky image and analyze the target features to obtain the predicted cloud region.

[0146] The comparison submodule is used to obtain the difference information between the target cloud region and the predicted cloud region by using an overlap-based loss function;

[0147] The model determination submodule is used to perform backpropagation on the cloud-sky segmentation model based on the difference information to obtain the trained cloud-sky segmentation model.

[0148] In this embodiment of the application, the cloud-sky segmentation device may further include the following modules:

[0149] The evaluation module is used to evaluate the trained cloud-sky segmentation model based on the number of overlapping pixels between the target cloud region and the predicted cloud region, as well as the total number of pixels in the target cloud region and the predicted cloud region.

[0150] The cloud-sky segmentation device provided in this application embodiment can be applied in the foregoing method embodiments. For details, please refer to the description of the above method embodiments, which will not be repeated here.

[0151] Figure 11 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 11 As shown, the terminal device 1100 of this embodiment includes: at least one processor 1110 ( Figure 11 (Only one is shown) a processor, a memory 1120, and a computer program 1121 stored in the memory 1120 and executable on the at least one processor 1110, wherein the processor 1110 executes the computer program 1121 to implement the steps in the above-described cloud-sky segmentation method embodiment.

[0152] The terminal device 1100 may be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor 1110 and a memory 1120. Those skilled in the art will understand that... Figure 11 This is merely an example of terminal device 1100 and does not constitute a limitation on terminal device 1100. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0153] The processor 1110 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0154] In some embodiments, the memory 1120 may be an internal storage unit of the terminal device 1100, such as a hard disk or memory of the terminal device 1100. In other embodiments, the memory 1120 may be an external storage device of the terminal device 1100, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 1100. Furthermore, the memory 1120 may include both internal and external storage units of the terminal device 1100. The memory 1120 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 1120 can also be used to temporarily store data that has been output or will be output.

[0155] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0156] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0157] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0158] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0159] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0161] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0162] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the various method embodiments described above.

[0163] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A cloud sky segmentation method, characterized in that, The cloud-sky segmentation method includes: Acquire ground-based sky images; Based on the ground-based sky image, extract the brightness coefficient of the ground-based sky image; If the brightness coefficient is less than a preset threshold, then the clouds and sky in the ground-based sky image are segmented according to the three primary colors in the ground-based sky image to obtain the cloud region and sky region in the ground-based sky image. If the brightness coefficient is greater than or equal to a preset threshold, the ground-based sky image is input into a trained cloud-sky segmentation model, and the cloud-sky segmentation model outputs the cloud region and sky region in the ground-based sky image. The step of segmenting the clouds and sky in the ground-based sky image according to the three primary colors to obtain the cloud region and sky region in the ground-based sky image includes: Based on the pixel values ​​corresponding to the three primary colors of each pixel in the ground-based sky image, determine the blue channel value, red channel value, and green channel value of each pixel; Based on the blue channel value and red channel value of each pixel, determine the normalized red-blue ratio value of each pixel; The average value of the three channels for each pixel is determined based on the blue channel value, red channel value, and green channel value of each pixel. For each pixel, based on the corresponding normalized red-blue ratio and preset red-blue ratio threshold, as well as the corresponding three-channel average value and preset three-channel average value threshold, the category of each pixel is determined, thus obtaining the cloud region and sky region in the ground-based sky image.

2. The cloud-sky segmentation method as described in claim 1, characterized in that, The cloud-sky segmentation method also includes: The cloud cover corresponding to the cloud cluster region is determined based on the total number of pixels in the cloud cluster region and the total number of pixels in the ground-based sky image.

3. The cloud-sky segmentation method as described in claim 1, characterized in that, The step of extracting the brightness coefficient of the ground-based sky image based on the ground-based sky image includes: Obtain the pixel values ​​corresponding to the three primary color channels of the ground-based sky image; The brightness of the ground-based sky image is determined based on the pixel values ​​corresponding to the three primary color channels and the preset conversion coefficients corresponding to the three pixel values, wherein the preset conversion coefficients are used to convert the pixel values ​​corresponding to the three primary colors into image brightness. The brightness of the ground-based sky image is normalized to determine the brightness coefficient of the ground-based sky image.

4. The cloud-sky segmentation method as described in claim 1, characterized in that, The sky region includes thin cloud regions and opaque cloud regions; for each pixel, based on the corresponding normalized red-blue ratio and a preset red-blue ratio threshold, as well as the corresponding three-channel average value and a preset three-channel average value threshold, the category of each pixel is determined, resulting in the cloud region and sky region in the ground-based sky image, including: If the normalized red-blue ratio value corresponding to the pixel is greater than the preset red-blue ratio threshold, then the pixel is determined to belong to the category of sky. If the normalized red-blue ratio value corresponding to the pixel is less than or equal to the preset red-blue ratio threshold, and the three-channel average value corresponding to the pixel is greater than the preset three-channel average value threshold, then the pixel is determined to belong to the category of thin cloud. If the normalized red-blue ratio value corresponding to the pixel is less than or equal to the preset red-blue ratio threshold, and the three-channel average value corresponding to the pixel is less than or equal to the preset three-channel average value threshold, then the pixel is determined to belong to the category of opaque cloud. Based on the category of each pixel in the ground-based sky image, determine the cloud region and sky region in the ground-based sky image.

5. The cloud-sky segmentation method as described in claim 1, characterized in that, The trained cloud-sky segmentation model includes a self-attention module; The training process of the cloud-sky segmentation model includes: Obtain a training dataset, which includes N ground-based sky images with a first label, the first label being used to indicate target cloud regions in the ground-based sky images, and the N ground-based sky images including original ground-based sky images and ground-based sky images enhanced from the original ground-based sky images; Based on the first label, the target cloud region is extracted from the ground-based sky image contained in the training dataset, and the training dataset is input into the cloud-sky segmentation model. Before output, the self-attention module is used to select target features in the ground-based sky image, and the target features are analyzed to obtain the predicted cloud region. A loss function based on overlap is used to obtain the difference information between the target cloud region and the predicted cloud region; Based on the difference information, backpropagation is performed on the cloud-sky segmentation model to obtain the trained cloud-sky segmentation model.

6. The cloud-sky segmentation method as described in claim 5, characterized in that, The cloud-sky segmentation method also includes: The trained cloud-sky segmentation model is evaluated based on the number of overlapping pixels between the target cloud region and the predicted cloud region, as well as the total number of pixels in the target cloud region and the predicted cloud region.

7. A cloud-sky segmentation device, characterized in that, include: The image acquisition module is used to acquire ground-based sky images; The coefficient acquisition module is used to extract the brightness coefficient of the ground-based sky image based on the ground-based sky image; The first judgment module is used to divide the clouds and sky in the ground-based sky image according to the three primary colors in the ground-based sky image if the brightness coefficient is less than a preset threshold, so as to obtain the cloud area and sky area in the ground-based sky image. The second judgment module is used to input the ground-based sky image into the trained cloud-sky segmentation model if the brightness coefficient is greater than or equal to a preset threshold, and the cloud-sky segmentation model outputs the cloud region and sky region in the ground-based sky image. The first determination module further includes: The channel value determination submodule is used to determine the blue channel value, red channel value, and green channel value of each pixel based on the pixel values ​​corresponding to the three primary colors of each pixel in the ground-based sky image. The ratio determination submodule is used to determine the normalized red-blue ratio of each pixel based on the blue channel value and the red channel value of each pixel; The average value determination submodule is used to determine the three-channel average value of each pixel based on the blue channel value, red channel value, and green channel value of each pixel. The first region determination submodule is used to determine the category of each pixel based on the corresponding normalized red-blue ratio and preset red-blue ratio threshold, as well as the corresponding three-channel average value and preset three-channel average value threshold, so as to obtain the cloud region and sky region in the ground-based sky image.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images