Large-scale image color adjustment method, system and device and medium

Abstract

The invention relates to a large-scale image color adjustment method, system and device and a medium. The method comprises the steps that a to-be-processed area on a to-be-processed first image is manually selected; carrying out zooming and fuzzy processing on the area to respectively obtain a second image, a third image and a fourth image; calculating three channel values of red, green and blue of each pixel in the second image and a color mean value, and combining results to obtain a first matrix; calculating each pixel value of the third image and each pixel value of the fourth pixels to obtain a second matrix; and performing color compensation on the second image by a difference value obtained by subtracting the first matrix from the second matrix to obtain a color unified image, and then performing conversion to obtain a grey-scale image. According to the invention, the time for manual treatment of the sanitary film by art is reduced to a great extent, the treatment efficiency is improved, and the actual use requirements are met.

Description

Technical Field

[0001] The present invention belongs to the field of satellite image technology, and in particular relates to a large-scale image color adjustment method, system, device and medium. Background Art

[0002] As a training system, the flight simulator has extremely high requirements for the authenticity and continuity of terrain images. However, there are some urgent problems to be solved in the existing satellite image resources. First, the splicing traces are obvious. This is because the satellite shoots the ground at different times and seasons. Due to reasons such as lighting conditions and vegetation changes, the images have large differences in tone and content. Simply splicing these images will lead to discontinuous terrain and affect the pilot's judgment. Secondly, although the method of artificial post-PS processing can improve the image effect, the workload is large and it may also change the original terrain information. The existing satellite image processing technology solutions mainly include the following: Artificial PS processing method: Engineers use software such as Photoshop to adjust the overall color of the image, and use tools such as layer fusion and stamps to repair the splicing edges and areas with large color differences, so as to make the image more coordinated and unified. Image selection and comprehensive method: By comparing multiple satellite image sources, select images with better quality in the same area for comprehensive processing. Histogram matching method: Using histogram matching technology, the image histogram characteristics of the area to be processed are adjusted to match the histogram of the target area. Figure 1 The above methods have the following disadvantages: the manual PS processing method has a large workload and low efficiency, and may also cause local distortion of terrain information. The image selection and comprehensive method requires more manual comparison and analysis work, and it cannot completely eliminate the color difference and splicing marks between images. Different sources need to be purchased separately, which increases the cost. The histogram matching method has high requirements for region selection and determination of target histograms, and the processing effect also has certain limitations.

[0003] Therefore, it is urgent to develop an automated color adjustment method for large-resolution satellite images that can eliminate stitching traces and maintain the accuracy of terrain information to meet the needs of flight simulation training. This automated large-resolution satellite image processing method can not only solve the existing image quality problems, but also greatly improve processing efficiency, providing a more realistic and reliable terrain environment for flight simulation training. Summary of the invention

[0004] In order to overcome the problems existing in the prior art, the present invention provides a large-scale image color adjustment method, system, device and medium to overcome the existing defects.

[0005] A large-scale image color adjustment method, the method comprising the steps of: S1. Manually select an area to be processed on the first image to be processed; S2. Scaling the area to be processed to obtain a second image; S3. Blurring the second image to obtain a third image and a fourth image; S4. Calculate the red, green and blue values ​​of each pixel in the second image and their respective color means, and combine the obtained results to obtain a first matrix; S5. Calculate each pixel value of the third image and each pixel value of the fourth image together to obtain a second matrix; S6. Using the difference obtained by subtracting the first matrix from the second matrix as a compensation value, color compensation is performed on the second image to obtain a color uniform image, which is then converted to obtain a grayscale image.

[0006] According to the aspects described above and any possible implementation method, an implementation method is further provided, wherein the area to be processed includes different illumination maps of regular areas, different time maps of irregular areas, maps where the illumination of the entire image is inconsistent with other images, cloud shadow maps of irregular areas, or fused maps of the original images of irregular areas.

[0007] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S2 includes: reducing the first image to be processed by a certain multiple to obtain a second image.

[0008] According to the aspects described above and any possible implementation method, an implementation method is further provided, wherein S3 includes: using a first convolution kernel and a second convolution kernel to blur the second image respectively to obtain a third image and a fourth image, and the first convolution kernel is different from the second convolution kernel.

[0009] According to the above aspects and any possible implementation, an implementation is further provided, wherein S4 includes: S41. calculating the mean values ​​of the three colors of red, green and blue in the second image, respectively denoted as Ravge, Gavge and Bavge; S42. Subtract Ravge, Gavge and Bavge from the red, green and blue values ​​of each pixel of the second image respectively, perform the above calculation on all pixels, and record the result as the first matrix.

[0010] According to the aspects described above and any possible implementation manner, an implementation manner is further provided, where S5 includes performing a subtraction calculation on each pixel value of the third image and each pixel value of the fourth pixel to obtain the second matrix.

[0011] According to the aspects and any possible implementation methods described above, an implementation method is further provided, which also includes S7. The grayscale image is trained using an LBFGS style transfer neural network to obtain a style transfer map of the neural network, and the style transfer map is enlarged using a super-resolution network.

[0012] The present invention also provides a large-scale image color adjustment system, wherein the system implements the method described, and comprises the following modules: A selection module, used for manually selecting a region to be processed on the first image to be processed; A scaling module, used for scaling the area to be processed to obtain a second image; A blur processing module, used for blurring the second image to obtain a third image and a fourth image; A first calculation module, used for calculating the red, green and blue three channel values ​​and the color mean of each pixel in the second image, and combining the obtained results to obtain a first matrix; A second calculation module, used for calculating each pixel value of the third image and each pixel value of the fourth image to obtain a second matrix; The compensation module is used to use the difference obtained by subtracting the first matrix from the second matrix as a compensation value, perform color compensation on the second image to obtain a color uniform image, and then convert it to obtain a grayscale image.

[0013] The present invention further provides an electronic device, comprising: A memory storing executable instructions; A processor is used to execute the executable instructions in the memory to implement the method.

[0014] The present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the method described.

[0015] Beneficial effects of the present invention The large-scale image color adjustment method of the present invention comprises: manually selecting a to-be-processed area on a to-be-processed first image; scaling the to-be-processed area to obtain a second image; blurring the second image to obtain a third image and a fourth image; calculating the red, green and blue three channel values ​​and the color mean of each pixel in the second image, and combining the obtained results to obtain a first matrix; calculating each pixel value of the third image and each pixel value of the fourth pixel to obtain a second matrix; using the difference obtained by subtracting the first matrix from the second matrix as a compensation value, color compensation is performed on the second image to obtain a color uniform map, and then converted to obtain a grayscale map. The present invention greatly reduces the time for artists to manually process satellite images and improves the processing efficiency. The method can perform well in different processing tasks such as different illumination in regular areas, different times in irregular areas, inconsistent illumination of the entire image with other images, cloud shadows in irregular areas, and fusion of original images in irregular areas. The processing results are unified in style and meet the actual use requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is the unprocessed original image - a schematic diagram of different lighting in a regular area; Figure 2 This is the unprocessed original image - the image of different time periods in irregular areas; Figure 3 This is a schematic diagram of the original unprocessed image - the lighting of the entire image is inconsistent with other images; Figure 4 This is the unprocessed original image - schematic diagram of cloud shadows in irregular areas; Figure 5 This is a schematic diagram of the fusion of the unprocessed original image and the irregular area original image; Figure 6 Schematic diagram of the second image B after 3*3 convolution kernel mean blur; Figure 7 Schematic diagram of the second image B after the 51*51 convolution kernel mean blur; Figure 8 This is a schematic diagram after taking the absolute value of the first matrix E; Fig. 9 This is a schematic diagram after taking the absolute value of the second matrix F; Fig.10 is a schematic diagram of the second image B being converted into grayscale after color compensation; Fig.11 This is a schematic diagram of the style transfer neural network training results (2048*2048); Fig.12 This is the image super-resolution output result (16384*16384). DETAILED DESCRIPTION

[0017] In order to better understand the technical solution of the present invention, the content of the present invention includes but is not limited to the specific implementation methods described below, and similar technologies and methods should be considered to be within the scope of protection of the present invention. In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0018] It should be clear that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The large-scale image color adjustment method provided by the present invention comprises the steps of: S1. Manually select an area to be processed on the first image to be processed; S2. Scaling the area to be processed to obtain a second image; S3. Blurring the second image to obtain a third image and a fourth image; S4. Calculate the red, green and blue values ​​of each pixel in the second image and the mean values ​​corresponding to each color, and combine the obtained results to obtain a first matrix; S5. Calculate each pixel value of the third image and each pixel value of the fourth image to obtain a second matrix; S6. Using the difference obtained by subtracting the first matrix from the second matrix as a compensation value, performing color compensation on the second image to obtain a color uniform image, and then converting it to obtain a grayscale image; S7. The grayscale image is trained using an LBFGS style transfer neural network to obtain a style transfer map of the neural network, and the style transfer map is enlarged using a super-resolution network.

[0020] When processing satellite images, Figure 1 , Figure 4 , Figure 5 For the obvious color boundary part, S1 and S2 can also be implemented by the following steps: First, we manually use the image processing software PS to use the copy tool identification, content recognition filling, layer overlay, filter blur and other methods to eliminate the Figure 1 , Figure 4 , Figure 5 The stitching seam area in the figure is used to obtain the image to be processed; Then all the images to be processed are converted into grayscale images, and their size is reduced to 2048*2048, and the overall brightness of the image is adjusted and optimized to obtain a grayscale image.

[0021] Preferably, the area to be processed includes different illumination maps of regular areas, different time maps of irregular areas, maps of inconsistent illumination of the entire image and other images, cloud shadow maps of irregular areas, or fused maps of the original images of irregular areas.

[0022] Specifically, the specific process of the present invention is as follows: The present invention classifies all satellite images to be processed to obtain original images, which are divided into the following five situations: There are regular areas with different illumination in the original images, such as Figure 1 As shown in the figure, there are two regular areas. The two areas were taken by a satellite in different lighting environments in a short period of time (such as a few days) and then stitched together. The content of this image is only different in the main color, and the content is not much different. Among them, the regular area refers to a small number of points that can form a simple concave and convex polygon.

[0023] There are irregular areas in the original image at different times, such as Figure 2 As shown in the figure, the area after stitching is irregular when the satellite takes photos in a similar lighting environment but at a long time (such as more than a few weeks). A regular area refers to a simple concave and convex polygon formed by a small number of points. Among them, an irregular area refers to an area that cannot be selected by a simple polygon.

[0024] The original image has a lighting that is inconsistent with other images, such as Figure 3 As shown, the entire image is taken by the satellite under non-ideal lighting conditions, seasons, time and other conditions. There is no obvious difference in color tone in the image, and the color tone of the entire image is uniform, so the color of the entire image needs to be adjusted.

[0025] There are cloud shadows in irregular areas in the original image, such as Figure 4 As shown, there were thin clouds below the satellite when it took the photo, which appear as shadowed irregular areas or whitish irregular areas on the satellite image.

[0026] There are irregular areas in the original image. The original image has been fused, such as Figure 5 As shown, the above-mentioned situations exist that the images have been modified and adjusted by others when they were acquired. In view of the above-mentioned situations, the present invention proposes a large-scale image color adjustment method based on style transfer and super-resolution, which realizes efficient and automatic image color adjustment. The large scale here refers to the situation where the resolution of a single satellite image to be processed is very large, such as an image with a size of 16384*16384.

[0027] The present invention includes the following operation process: 1. Processing the original image, reducing it by a certain multiple to obtain a new image, retaining the high-frequency texture information in the original image, processing the low-frequency color information, and adjusting the color of the satellite image to make the colors of different areas uniform.

[0028] 2. The processed grayscale image meets the input requirements of the style transfer neural network and performs style transfer.

[0029] 3. Send the style-transferred image to the super-resolution neural network, and enlarge the image back to its original size without losing accuracy, thereby improving the image's clarity and details.

[0030] Compared with traditional methods, this invention makes full use of the advantages of deep learning to achieve automation and intelligence of image processing. At the same time, combined with steps such as manual selection and color adjustment, it effectively avoids the limitations that may arise from a single algorithm and greatly improves the quality and efficiency of image processing. The more specific process is as follows: 1. Color Processing The work done by the present invention in this step is to unify the colors of the original image on a smaller size (2048*2048) to form a grayscale image that can be processed by a neural network. The present invention uses an unprocessed satellite image as the original image for processing, which is recorded as Figure A. The resolution of this image includes but is not limited to 16384*16384. The original image A can be divided into the above five categories due to different image contents. Different processing methods are performed for different situations in this step, as follows: For the case where the color of the original image A is inconsistent with the expected effect, that is, the first case, such as Figure 3 As shown, the processing method is: reduce the original image A to a size of (2048*2048), and convert it into a grayscale image, recorded as image G. For example, if the size of the original image A is 16384*16384, it is reduced by 8 times to obtain an image of size 2048*2048, and then converted into a grayscale G image. The reduction multiple of the present invention is not limited, and scaling is performed according to actual conditions.

[0031] For the case where the original image A has inconsistent colors, that is, the second case, such as Figure 1 , Figure 2 , Figure 4 and Figure 5 As shown, the processing steps include the following process, which is Figure 4 As an example: 1). Reduce the size of the original image A including the area to be processed to (2048*2048) to obtain a reduced image, recorded as image B. The original image A is used as the first image to be processed and is obtained by manual selection; image B is used as the second image.

[0032] 2). The second image B is blurred using the mean blur method, using two different first convolution kernels and second convolution kernels, and the first convolution kernel is (3,3), and the second convolution kernel is (51,51). The second image B is blurred using the first convolution kernel to obtain the third image C, as shown in Figure 6As shown; the second image B is blurred by the second convolution kernel to obtain the fourth image D, as shown Figure 7 As shown. The purpose of this step is to use mean blur to eliminate high-frequency texture information in the second image B. Using convolution kernels of two scales means eliminating texture information of two scales. This step is to blur the second image B to different degrees, wherein the convolution kernel of the fourth image D is larger than that of the third image C, so its image is more blurred than the third image C. The present invention can also use other convolution kernels, or multiple convolution kernels, to blur the second image B, which is selected according to actual requirements, and the present invention is not limited thereto.

[0033] 3). Calculate the mean of the three colors of red, green and blue for each pixel in the second image B, and record them as Ravge, Gavge and Bavge. The purpose of this step is to obtain a color average value as a benchmark, which will be used as a benchmark for calculation and comparison in subsequent steps.

[0034] 4). Subtract Ravge, Gavge and Bavge from the three values ​​of red, green and blue (RGB) of each pixel of the second image B, that is, the R value of this pixel is subtracted from Ravge, the G value is subtracted from Gavge, and the B value is subtracted from Bavge. Perform the above calculation operation on all pixels of the second image B and form a matrix with the results, which is recorded as the first matrix E, as shown in Figure 8 The purpose of this step is to obtain all the texture information of the second image B after removing the basic colors represented by Ravge, Gavge and Bavge, represented by the second matrix E.

[0035] 5). Subtract the pixel value of the corresponding position of the fourth image D from each pixel value of the third image C, and record the result as the second matrix F, as follows: Fig. 9 The purpose of this step is to obtain the texture information that is eliminated between the convolution kernel 3*3 and the convolution kernel 51*51 blur.

[0036] 6). Subtract the first matrix E from the second matrix F, and the result is a color compensation matrix. The resolution of the color compensation matrix is ​​the same as that of the second image B, and it also has three RGB channels. The meaning of the color compensation matrix is ​​to calculate how much each pixel on the original image A should be increased or decreased so that the color of the entire image is unified after compensation. The present invention adds the color compensation matrix to the pixel value of each corresponding position in the second image B. For example, for the pixel in the jth row and i column, the value of the jth row and i column of the color compensation matrix is , the value of the jth row and ith column of the second image B is , then the value obtained after adding is , i, j are integers greater than or equal to 1. Perform color compensation on the second image B to obtain a color uniform image, and convert the color uniform image into a grayscale image, denoted as image G, as shown in Fig.10 shown.

[0037] 2. Style Transfer The present invention will recolor the grayscale image G (the first case mentioned above) obtained when the color of the original image is inconsistent with the expected effect, or the grayscale image G (the second case mentioned above) obtained when the color of the original image A is inconsistent. The method used is the LBFGS style transfer neural network. The grayscale image G is used as the content map, and another style map is selected as the target style map of the neural network for training. The number of training rounds is 520 rounds, but the present invention is not limited to this number, and multiple rounds of training are possible. The grayscale image G is trained using the LBFGS style transfer neural network. The training is stopped after a certain number of times or the coloring evaluation of the content map is close to the target style map, and the output result is as follows: Fig.11 As shown, it is recorded as style transfer graph H. This step includes but is not limited to using LBFGS neural network for style transfer training, and the rounds are not limited to 520 times. Style transfer neural networks with other network structures can also be used. The present invention uses this style transfer neural network, which is a prior art and will not be repeated here.

[0038] 3. Super-resolution processing Since the size of the satellite image is 16384*16384 in actual use, directly enlarging the result image H after style transfer will cause a 64-fold loss of accuracy, so a super-resolution network is needed to improve the resolution of the image. This method is implemented using the SRGAN network, which is implemented by the SRGAN super-resolution generative adversarial network, and has the ability to significantly improve the clarity of low-resolution images. Its core consists of two connected modules, the generator and the discriminator, which are jointly trained through the generative adversarial network framework. The generator is responsible for converting the input low-resolution result image H into a high-resolution image, while the discriminator is used to distinguish between the high-resolution image generated by the conversion and the real high-resolution image. During the training process of image H, the generator and the discriminator compete with each other: the generator continuously optimizes itself to generate realistic images in an attempt to deceive the discriminator; the discriminator continuously improves its identification ability to better distinguish between real images and converted images. By adopting this characteristic of the generator and the discriminator, SRGAN can not only make the generated images more detailed at the pixel level, but also more natural and realistic in high-level features, thereby retaining and restoring more details. Therefore, SRGAN significantly improves the quality of the converted images, making the edges of the converted images obtained after the low-resolution image H is processed by SRGAN clearer, the texture is more realistic, and the overall visual experience is greatly improved. Therefore, the present invention uses SRGAN to super-resolution the result image H to obtain a high-resolution image with rich details and realistic visual effects, such as Fig.12 As shown, the specific implementation process of the SRGAN network is a known mature technology and will not be described in detail in the present invention.

[0039] As an embodiment disclosed by the present invention, the present invention further provides a large-scale image color adjustment system, wherein the system implements the method described, and comprises the following modules: A selection module, used for manually selecting a region to be processed on the first image to be processed; A scaling module, used for scaling the area to be processed to obtain a second image; A blur processing module, used for blurring the second image to obtain a third image and a fourth image; A first calculation module, used for calculating the red, green and blue three channel values ​​and the color mean of each pixel in the second image, and combining the obtained results to obtain a first matrix; A second calculation module, used for calculating each pixel value of the third image and each pixel value of the fourth image to obtain a second matrix; The compensation module is used to use the difference obtained by subtracting the first matrix from the second matrix as a compensation value, perform color compensation on the second image to obtain a color uniform image, and then convert it to obtain a grayscale image.

[0040] As an embodiment disclosed in the present invention, the present invention further discloses an electronic device, the electronic device comprising: A memory storing executable instructions; A processor is used to execute the executable instructions in the memory to implement the method described in the present invention.

[0041] As an embodiment disclosed in the present invention, the present invention further discloses a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the method described in the present invention.

[0042] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0043] The above description shows and describes several preferred embodiments of the present invention, but as mentioned above, it should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the application concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not depart from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A large-scale image color adjustment method, characterized in that: The method comprises the steps of: S1. Manually select an area to be processed on the first image to be processed; S2. Scaling the area to be processed to obtain a second image; S3. Blurring the second image to obtain a third image and a fourth image; S4. Calculate the red, green and blue values ​​of each pixel in the second image and their respective color means, and combine the obtained results to obtain a first matrix; S5. Calculate each pixel value of the third image and each pixel value of the fourth image together to obtain a second matrix; S6. Using the difference obtained by subtracting the first matrix from the second matrix as a compensation value, color compensation is performed on the second image to obtain a color uniform image, which is then converted to obtain a grayscale image.

2. The large-scale image color adjustment method according to claim 1, characterized in that: The area to be processed includes different illumination images of regular areas, different time images of irregular areas, images of inconsistent illumination of the entire image and other images, cloud shadow images of irregular areas, or images of fused original images of irregular areas.

3. The large-scale image color adjustment method according to claim 1, characterized in that: The step S2 includes: reducing the first image to be processed by a certain multiple to obtain a second image.

4. The large-scale image color adjustment method according to claim 1, characterized in that: The S3 includes: using a first convolution kernel and a second convolution kernel to blur the second image respectively to obtain a third image and a fourth image, wherein the first convolution kernel is different from the second convolution kernel.

5. The large-scale image color adjustment method according to claim 4, characterized in that: The S4 includes: S41. Calculate the mean values ​​of the three colors of red, green and blue in the second image, which are respectively recorded as Ravge, Gavge and Bavge; S42. Subtract Ravge, Gavge and Bavge from the red, green and blue values ​​of each pixel of the second image respectively, perform the above calculation on all pixels, and record the result as the first matrix.

6. The large-scale image color adjustment method according to claim 5, characterized in that: S5 includes performing a subtraction calculation on each pixel value of the third image and each pixel value of the fourth pixel to obtain the second matrix.

7. The large-scale image color adjustment method according to claim 1, characterized in that: It also includes S7. training the grayscale image using an LBFGS style transfer neural network to obtain a style transfer map of the neural network, and amplifying it using a super-resolution network.

8. A large-scale image color adjustment system, characterized in that: The system implements the method described in any one of claims 1 to 7, and includes the following modules: A selection module, used for manually selecting a region to be processed on the first image to be processed; A scaling module, used for scaling the area to be processed to obtain a second image; A blur processing module, used for blurring the second image to obtain a third image and a fourth image; A first calculation module, used for calculating the red, green and blue three channel values ​​and the color mean of each pixel in the second image, and combining the obtained results to obtain a first matrix; A second calculation module, used for calculating each pixel value of the third image and each pixel value of the fourth image to obtain a second matrix; The compensation module is used to use the difference obtained by subtracting the first matrix from the second matrix as a compensation value, perform color compensation on the second image to obtain a color uniform image, and then convert it to obtain a grayscale image.

9. An electronic device, characterized in that: The electronic device comprises: A memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images