A ground-based cloud image repairing method based on an improved Criminisi algorithm

Abstract

The application relates to a ground-based cloud image repairing method based on an improved Criminisi algorithm, which comprises the following steps: determining the position of the sun through the shooting time of the ground-based cloud image and the longitude and latitude of the position where the equipment is located, so as to determine the position of the light-shading band; pre-processing the ground-based cloud image and marking a region to be repaired; extracting the boundary of the region to be repaired; calculating a priority function of each point of the boundary of the region to be repaired, and selecting a to-be-repaired block with the maximum priority; calculating the red-blue ratio average value of the to-be-repaired block with the maximum priority; selecting different repairing strategies according to the size of the red-blue ratio average value of the to-be-repaired block, and obtaining a ground-based cloud image repairing result. The method optimizes the ground-based cloud image repairing strategy by introducing the red-blue ratio feature of the ground-based cloud image, improves the ground-based cloud image repairing effect, and reduces the time complexity of the image repairing.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of digital image processing, and relates to a ground-based cloud image repairing method based on an improved Criminisi algorithm. BACKGROUND

[0002] Solar energy is a natural clean energy. With the rapid development of solar power generation technology and large-scale commercial application in recent years, solar energy will have the potential to become the main energy source for human beings in the future. Meanwhile, using clean energy such as solar energy to generate electricity is one of the main ways to achieve the carbon peak target in China. Sky cloud conditions are one of the important factors affecting the output power of solar photovoltaic panels. Using ground-based cloud images to observe sky cloud conditions has low cost and high resolution, and can provide accurate local sky cloud condition information. The all-sky imager is the main equipment for ground-based cloud image observation.

[0003] However, when the all-sky imager is working, it needs to rely on a light-shielding belt to eliminate the adverse effects of the sun on the camera shooting work, resulting in incomplete cloud image information, missing key part information, and affecting cloud detection, classification and cloud amount calculation, etc. Therefore, it is necessary to repair the ground-based cloud image.

[0004] The purpose of image repair is to reconstruct the missing or damaged parts of the picture in a way that the observer cannot perceive, and restore the integrity of the picture. With the development of digital image processing technology, the detection and restoration of missing or damaged parts of the image using digital technology has developed from manual processing to fully automatic process. The image repair algorithm obtains the best matching block of the to-be-repaired area from the effective information area through a certain repair sequence, so as to complete the image repair. How to design the optimal to-be-repaired sequence and quickly and accurately find the best matching block is the key and difficulty of the image repair algorithm. Based on this, the application combines the characteristics of ground-based cloud images, improves the Criminisi algorithm through red-blue ratio, and proposes a ground-based cloud image repairing method based on the improved Criminisi algorithm. SUMMARY

[0005] In order to solve the shortcomings and deficiencies in the prior art, the application proposes a ground-based cloud image repairing method based on an improved Criminisi algorithm, which improves the Criminisi algorithm by using the red-blue ratio feature of the ground-based cloud image to realize the repair of the ground-based cloud image.

[0006] The application is realized by the following technical solutions, and the application is specifically:

[0007] Step 1: Determine the light-shielding belt position and lens arm position through the ground-based cloud image shooting time and the latitude and longitude of the location of the equipment, including the following steps:

[0008] 1) Calculate the solar declination angle δ and the solar hour angle ω;

[0009] 2) Calculate the solar azimuth angle γ s and the zenith angle θ z ;

[0010] 3) Obtain the solar position according to the solar azimuth angle γ s and the zenith angle θ z , and determine the position of the shading band;

[0011] Step 2: Pre-process the photographed ground-based cloud image, remove the interference of surrounding buildings and the like in the image, set the shading band area and the lens arm area to black, and mark as a to-be-repaired area;

[0012] Step 3: Determine whether the to-be-repaired area obtained in step 2 is empty; if yes, output the repair result, otherwise extract the to-be-repaired area boundary;

[0013] Step 4: Calculate the priority of all to-be-repaired blocks on the boundary of the to-be-repaired area and the area containing valid information, select the to-be-repaired block with the highest priority, and repair it preferentially;

[0014] Step 5: Calculate the red-blue ratio mean value of the to-be-repaired block with the highest priority;

[0015] Step 6: Determine whether the red-blue ratio mean value obtained in step 5 is greater than a threshold value; if yes, calculate the sum of squared differences (SSD) of pixels between each matching block in the area containing valid information and the to-be-repaired block, and select the matching block with the smallest SSD as the best matching block; if no, use the nearest sky pixel point for copying as the best matching block;

[0016] Step 7: Assign the best matching block to the to-be-repaired block, update the to-be-repaired area, and return to step 3.

[0017] In step 1, when calculating the solar declination angle δ and the solar hour angle ω, the formulas are as follows:

[0018]

[0019] ω = 15 (t-2)

[0020] Where N is the date order number in a year, starting from the first day of the year (January 1), and the value range of N is 1-365, and t represents the true solar time information recorded based on 24h.

[0021] In step 1, when calculating the solar azimuth angle γ s and the zenith angle θ z , the formulas are as follows:

[0022]

[0023] wherein, is the dimension of the device at the time of taking the ground cloud picture, δ is the solar declination angle, and ω is the solar hour angle.

[0024] In step 2, the image size is normalized to 415x415 after image preprocessing.

[0025] In step 4, the priority of each block to be repaired is calculated, and the improved priority function formula is described as follows:

[0026] P(p) = RB(p) · (1 + C(p) · D(p))

[0027] wherein, RB(p) represents the red-blue ratio confidence term of the block to be repaired, C(p) represents the confidence term, mainly including pixel color information, and D(p) is the data term, representing the strength of the isophote of the current block, and the formula is described as follows:

[0028]

[0029] wherein, q is a pixel point in the image to be repaired, Ψ p is the block to be repaired, p is the center pixel point of the block to be repaired, Φ is the region containing effective information in the image, rb(q) represents the normalized red-blue ratio value at pixel point q, n p is the normal vector perpendicular to the boundary line, represents the isophote direction, a is a normalization parameter, a takes the value of 255, and the formula of rb(q) is described as follows:

[0030]

[0031] wherein, R(q) is the red channel value of pixel point q, and B(q) is the red channel value of pixel point q.

[0032] In step 5, the red-blue ratio mean of the block to be repaired with the maximum priority is calculated, and the formula is described as follows:

[0033]

[0034] In step 6, the SSD of each matching block and the block to be repaired in the region containing effective information is calculated, and the formula is described as follows:

[0035]

[0036] wherein, R, G, and B respectively represent the three channel pixel values of pixel point (i, j) in the block to be repaired, respectively represent the three channel pixel values of pixel point (i, j) in the matching block, and M is the size of the block to be repaired.

[0037] Compared with the prior art, the present application has the beneficial effects that:

[0038] The present application proposes a ground-based cloud image repairing method based on improved Criminisi algorithm, the image repairing algorithm Criminisi algorithm is improved, the unique red blue ratio feature of ground-based cloud image is introduced into the priority function, the ground-based cloud image repairing sequence and repairing strategy are improved, whether the to-be-repaired block is a sky area without structure information is judged through the red blue ratio feature, different repairing strategies are used for different areas. Compared with the existing popular image repairing method, the method disclosed by the present application optimizes the image repairing sequence, improves the ground-based cloud image repairing effect, reduces the time complexity of ground-based cloud image repairing, and has good application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The method flowchart is described in the present application;

[0040] Figure 2 The display of the solar azimuth angle, the solar zenith angle and the solar position in the ground-based cloud image is described;

[0041] Figure 3 The to-be-repaired area and the area containing effective information are marked;

[0042] Figure 4 The red blue ratio gray scale is described;

[0043] Figure 5 The ground-based cloud image repairing effect diagram of the present application is described. DETAILED DESCRIPTION

[0044] In order to make the structure and advantages of the present application clearer, the technical solutions of the present application will be further described below in combination with the drawings and examples.

[0045] Figure 1 The method flowchart described in the present application specifically includes the following steps:

[0046] Step 1: Determine the shading band position and the lens arm position through the ground-based cloud image shooting time and the longitude and latitude of the device position, including the following steps:

[0047] 1) Calculate the solar declination angle δ and the solar hour angle ω, the formula description is as follows:

[0048]

[0049] ω = 15 (t-2)

[0050] Wherein, N is the date order sequence number in a year, starting from the first day (January 1) in a year, the value range is 1-365. t represents the true solar time information recorded based on 24h.

[0051] 2) Calculate the solar azimuth angle γ s and the zenith angle θ z , whose formula is described as follows:

[0052]

[0053] wherein, is the latitude of the location where the device is located when taking the ground cloud image, δ is the solar declination angle, and ω is the solar hour angle.

[0054] 3) Obtain the position of the sun according to the solar azimuth angle γ s and the zenith angle θ z , so as to determine the position of the shading band.

[0055] Step 2: Image preprocessing:

[0056] The following preprocessing operations are sequentially performed on the ground cloud image:

[0057] 1) Image cropping: uniformly crop the image size to 415x415.

[0058] 2) Image denoising: remove the surrounding environment area contained in the image taken by the all-sky imager, and only keep the cloud image area.

[0059] 3) Mark the area to be repaired: set the pixel point values of the shading band area and the lens support arm area to 0.

[0060] Step 3: Boundary extraction of the area to be repaired:

[0061] The boundary of the area to be repaired and the area containing effective information is extracted by performing boundary extraction operation on the ground cloud image obtained in step 2.

[0062] Step 4: Calculation of priority function:

[0063] The priority of each repair block on the boundary of the area to be repaired and the area containing effective information obtained in step 3 is calculated, and its formula is described as follows:

[0064] P(p)=RB(p)·(1+C(p)·D(p))

[0065] Wherein, RB(p) represents the red-blue ratio confidence item of the repair block, C(p) represents the confidence item, mainly including pixel color information, and D(p) is the data item, representing the strength of the isophote of the current block, and its formula is described as follows:

[0066]

[0067] Wherein, q is a pixel point in the image repair area, Ψ pFor the block to be repaired, is the area containing effective information in the image, rb(q) represents the normalized red-blue ratio value at pixel point q, n p is the normal vector perpendicular to the boundary line, represents the direction of the isophote line, a takes the value of 255, and the formula of rb(q) is described as follows:

[0068]

[0069] wherein R(q) is the red channel value of pixel point q, and B(q) is the red channel value of pixel point q.

[0070] Step 5: Calculation of the red-blue ratio mean value of the block to be repaired with the highest priority:

[0071] In order to distinguish whether the block to be repaired is a cloud area containing more texture information or a sky area without texture information, the red-blue ratio mean value thereof is calculated, and the formula is described as follows:

[0072]

[0073] Step 6: Calculation of whether the block to be repaired is a cloud area through a threshold value, and the threshold value of the red-blue ratio mean value is 0.4. If the red-blue ratio mean value of the block to be repaired is greater than 0.4, the SSD of each matching block and the block to be repaired in the area containing effective information is calculated, and the matching block with the smallest SSD is selected to assign the missing information in the block to be repaired; if the red-blue ratio mean value of the block to be repaired is less than or equal to 0.4, the nearest sky pixel point is directly used for assignment. The formula of SSD is described as follows:

[0074]

[0075] wherein R, G and B respectively represent the three channel pixel values of pixel point (i, j) in the block to be repaired, respectively represent the three channel pixel values of pixel point (i, j) in the matching block, and M is the size of the block to be repaired.

[0076] Step 7: The block to be repaired is filled by using the optimal matching block obtained in step 6, and the image repairing process is completed.

[0077] As can be seen from the image repairing result obtained in step 7 above, the repairing result of the method has a good visual effect, the ground-based cloud image repairing sequence is optimized, the problems such as structure distortion and discontinuity between the repairing area and the area containing effective information are overcome, and the fast repairing of the ground-based cloud image can be realized. In the test result, the average time required for image repairing is 42.1363s, and compared with other methods, the ground-based cloud repairing method based on the improved Criminisi algorithm has achieved a better repairing performance.

Claims

1. A method for repairing foundation cloud maps based on an improved Criminisi algorithm, characterized in that, The method includes: Step 1: Determine the location of the sunshade and lens arm using the time of the ground-based cloud image capture and the latitude and longitude of the equipment's location. This includes the following steps: 1) Calculate the solar declination angle And calculating solar hour angle ; 2) Calculate the solar azimuth angle and zenith ; 3) Based on the solar azimuth angle and zenith The position of the sun is obtained, and the location of the shading zone is determined accordingly; Step 2: Preprocess the captured foundation cloud image to remove interference from surrounding buildings, black out the shading area and lens arm area, and mark them as areas to be repaired; Step 3: Determine if the area to be repaired obtained in Step 2 is empty; if so, output the repair result; otherwise, extract the boundary of the area to be repaired. Step 4: Calculate the priority of all blocks to be repaired on the boundary between the area to be repaired and the area containing valid information, select the block with the highest priority, and repair it first; Step 5: Calculate the average red-blue ratio of the block to be repaired with the highest priority; Step 6: Determine whether the average red-blue ratio obtained in Step 5 is greater than the threshold; if it is greater than the threshold, calculate the pixel squared difference and SSD between each matching block and the block to be repaired in the region containing valid information, and select the matching block with the smallest SSD as the best matching block; if it is less than the threshold, use the nearest sky pixel to copy as the best matching block. Step 7: Assign values ​​to the block to be repaired using the best matching block, update the area to be repaired, and return to Step 3.

2. The method for repairing foundation cloud maps based on the improved Criminisi algorithm as described in claim 1, characterized in that: Calculate the solar declination angle in step 1. With solar hour angle When the formula is used, it is described as follows: ， ， in, N is the sequential number of dates in a year, calculated starting from the first day of the year, with a value ranging from 1 to 365. This indicates true solar time information based on 24-hour records.

3. The method for repairing foundation cloud maps based on the improved Criminisi algorithm as described in claim 1, characterized in that: Calculate the solar azimuth angle in step 1. and zenith When the formula is used, it is described as follows: , ,in, This refers to the latitude of the equipment's location when the ground-based cloud image was captured. The solar declination angle, It is the solar hour angle.

4. The method for repairing foundation cloud maps based on the improved Criminisi algorithm as described in claim 1, characterized in that: In step 2, after image preprocessing, the image size is uniformly normalized to 415×415.

5. The method for repairing foundation cloud maps based on the improved Criminisi algorithm as described in claim 1, characterized in that: In step 4, the priority of each block to be repaired is calculated. The improved priority function formula is described as follows: ,in, This represents the confidence level of the red-blue ratio of the block to be repaired. This represents the confidence level, which mainly includes pixel color information. This is a data item representing the intensity of the isolines of the current block, and its formula is described as follows: , , ,in, For the pixels in the area of ​​the image to be repaired, The block to be repaired. The center pixel of the block to be repaired. The region in the image that contains valid information. Represents pixels The normalized red-blue ratio at the point, It is the normal vector perpendicular to the boundary line. Indicates the direction of isolux lines. For normalization parameters, The value is 255. The formula is described as follows: ， in, For pixels The red channel value, For pixels The blue channel value.

6. The method for repairing foundation cloud maps based on the improved Criminisi algorithm as described in claim 5, characterized in that: In step 5, the average red-to-blue ratio of the block to be repaired with the highest priority is calculated. The formula is described as follows: 。 7. The method for repairing foundation cloud maps based on the improved Criminisi algorithm as described in claim 1, characterized in that: Step 6 calculates the SSD values ​​of each matching block and the block to be repaired within the region containing valid information. The formula is described below: ,in, , , These represent the pixels in the block to be repaired. The three channel pixel values, , , These represent the pixels in the matching block. The three channel pixel values, The size of the block to be repaired.

Images