A large-scale image color adjustment method, system, device and medium

Abstract

The present invention relates to a large-scale image color adjustment method, system, device and medium. The method includes: manually selecting a region to be processed on the image to be processed; using a histogram to adjust the color of the region to be processed and eliminating the splicing edges between the region with adjusted color and other regions to obtain a grayscale image; reducing the size of the grayscale image and performing style transfer to obtain a transferred image; using a sliding window on the transferred image to achieve mapping between local color gamuts; and fusing all the mapping results to obtain a fused and adjusted image. The present invention greatly reduces the time for manual processing of satellite images by artists and improves the processing efficiency; it can perform well in different processing tasks such as different illuminations in regular regions, different times in irregular regions, inconsistent illuminations of the whole image with other images, cloud shadows in irregular regions, and different fusions of the original images in irregular regions. The processing results have a unified style and meet the actual usage requirements.

Description

Technical Field

[0001] The present invention belongs to the field of satellite image technology, and in particular relates to a large-scale image color adjustment method, system, device and medium. Background Art

[0002] As a training system, the flight simulator has extremely high requirements for the authenticity and continuity of terrain images. However, there are some urgent problems with existing satellite image resources. First, there are obvious splicing traces. This is because the satellite shoots the ground at different times and seasons. Due to lighting conditions and vegetation changes, the images have large differences in tone and content. Simply splicing these images will lead to discontinuous terrain and affect the pilot's judgment. Secondly, although the method of artificial post-PS processing can improve the image effect, the workload is large and it may also change the original terrain information. The existing satellite image processing technology solutions mainly include the following: Artificial PS processing method: Engineers use software such as Photoshop to adjust the overall color of the image, and use tools such as layer fusion and stamps to repair the splicing edges and areas with large color differences, so as to make the image more coordinated and unified. Image selection and comprehensive method: By comparing multiple satellite image sources, select images with better quality in the same area for comprehensive processing. Histogram matching method: Using histogram matching technology, the image histogram characteristics of the area to be processed are adjusted to the same as the histogram of the target area. Figure 1 The above methods have the following disadvantages: the manual PS processing method has a large workload and low efficiency, and may also cause local distortion of terrain information. The image selection and comprehensive method requires more manual comparison and analysis work, and it cannot completely eliminate the color difference and splicing marks between images. Different sources need to be purchased separately, which increases the cost. The histogram matching method has high requirements for region selection and determination of target histograms, and the processing effect also has certain limitations.

[0003] Therefore, it is urgent to develop an automated color adjustment method for large-resolution satellite images that can eliminate stitching traces and maintain the accuracy of terrain information to meet the needs of flight simulation training. This automated large-resolution satellite image processing method can not only solve the existing image quality problems, but also greatly improve processing efficiency, providing a more realistic and reliable terrain environment for flight simulation training. Summary of the invention

[0004] In order to overcome the problems existing in the prior art, the present invention provides a large-scale image color adjustment method, system, device and medium to overcome the existing defects.

[0005] A large-scale image color adjustment method, the method comprising the steps of:

[0006] S1. Manually select the area to be processed on the image to be processed;

[0007] S2. Use a histogram to adjust the color of the area to be processed, and eliminate the splicing edges between the area with adjusted color and other areas to obtain a grayscale image;

[0008] S3. Reduce the size of the grayscale image and perform style transfer to obtain the transferred image;

[0009] S4. Use a sliding window on the transferred image to achieve mapping between local color gamuts;

[0010] S5. Integrate all the mapping results to obtain the integrated and adjusted image.

[0011] In the above-mentioned aspects and any possible implementation manners, a further implementation manner is provided, where the area to be processed includes different illumination maps of regular areas, different time maps of irregular areas, maps with inconsistent illumination between the whole image and other maps, cloud shadow maps of irregular areas, or maps that have been integrated in the original maps of irregular areas.

[0012] In the above-mentioned aspects and any possible implementation manners, a further implementation manner is provided, where S1 includes:

[0013] S11. Reduce the satellite image to be processed by a certain multiple to obtain a second image, and use the satellite image to be processed as the original image, where the original image includes negative samples;

[0014] S12. Select a point (xb, yb) in the boundary area of a negative sample of the second image, and then intercept the image with coordinates in the range of xb*8 - 100 to xb*8 + 100, yb*8 - 100 to yb*8 + 100 in the original image as the third image for display;

[0015] S13. Select a point (xc, yc) in the third image, and the coordinates of this point restored to the original image are (xb*8 + xc - 100, yb*8 + yc - 100);

[0016] S14. Repeat steps S12 and S13 until the negative sample area of the second image is completely selected;

[0017] S15. If there are multiple negative sample areas in the original image, after repeating S12 - S14, finally obtain a polygon, and process this polygon to obtain positive and negative sample images.

[0018] In the above-mentioned aspects and any possible implementation manners, a further implementation manner is provided, where S2 includes:

[0019] S21. Encode the positive and negative sample images with their corresponding masks to obtain the encoded positive and negative samples. According to the histogram of the encoded positive samples, perform histogram matching on the encoded negative samples;

[0020] S22. Decode the result after histogram matching to obtain the decoded image;

[0021] S23. Perform weighted calculation processing on the decoded image to obtain the weight image;

[0022] S24. Multiply the negative sample image by the weight image, then add it to the positive sample image to obtain the fused image. Shrink the fused image and perform grayscale conversion to obtain the grayscale image.

[0023] In the aspects and any possible implementation manners as described above, a further implementation manner is provided. The S3 includes: training the grayscale image as the content image using the LBFGS-style transfer neural network to obtain the target style image of the neural network.

[0024] In the aspects and any possible implementation manners as described above, a further implementation manner is provided. S4 includes:

[0025] S41. Set the size of the sliding window Gs on the style transfer map G to 64*64, and the initial coordinates (xg, yg) = (0, 0). The size of the sliding window FS on the first grayscale image F1 is 64*64, and the initial coordinates (xg, yg) = (32*i, 32*j). The sliding window FL corresponding to this Fs window on the second grayscale image F2 has a size of 512*512, and the initial coordinates (xf, yf) = (256*i, 256*j). Here, i is the number of horizontal sliding times, j is the number of vertical sliding times, and the value ranges of i and j are 0 to 62;

[0026] S42. Create an empty color gamut mapping space: The point value in the space with the coordinate axes (x, y, z) is (0, 0, 0). Traverse Gs in a loop. Gs and Fs have the same window size, and there is a one-to-one mapping of pixel values at the same position. Use the pixel values in Fs as the coordinates of the color gamut mapping space, and use the pixel values in Gs as the values under this coordinate in the color gamut mapping space. After completing 64*64 mapping relationships, perform interpolation to obtain the mapping relationship of the complete color gamut space;

[0027] S43. For each of the 512*512 pixels in the sliding window FL, use each pixel as the coordinate value in the color gamut space to obtain a new pixel value after mapping. After all these pixels are mapped, save the mapped image;

[0028] S44. Set the sliding step size to 32 pixels. Slide horizontally and vertically i and j times respectively. Then, (xg, yg) = (32 * i, 32 * j), and (xf, yf) = (256 * i, 256 * j); Execute steps S42 - S44 again each time of sliding.

[0029] For the aspects and any possible implementation manners as described above, a further implementation manner is provided. S5 includes: setting a completely black image and a sliding window located therein, traversing the sliding window with the mapped image according to the set sliding step size, multiplying the mapped image by the set weight value and then adding it to the image within the sliding window in the completely black image to obtain the final result picture.

[0030] The present invention also provides a large - scale image color adjustment system. The system implements the method described above and includes the following modules:

[0031] A selection module, used for manually selecting a region to be processed on the image to be processed;

[0032] An adjustment module, used for adjusting the color of the region to be processed by using a histogram and eliminating the splicing edge between the region with adjusted color and other regions to obtain a grayscale image;

[0033] A style transfer module, used for reducing the size of the grayscale image and performing style transfer to obtain a transferred image;

[0034] A mapping module, used for implementing the mapping between local color gamuts on the transferred image by using a sliding window;

[0035] A fusion module, used for fusing all the mapping results to obtain a fused and adjusted image.

[0036] The present invention also provides an electronic device, which includes:

[0037] A memory, storing executable instructions;

[0038] A processor, which runs the executable instructions in the memory to implement the method described above.

[0039] The present invention also provides a computer storage medium, on which a computer program is stored. The computer program is executed by a processor to implement the method described above.

[0040] Advantages of the present invention

[0041] The large-scale image color adjustment method of the present invention includes: manually selecting a processing area on the image to be processed; using a histogram to adjust the color of the processing area and eliminating the splicing edges between the color-adjusted area and other areas to obtain a grayscale image; reducing the size of the grayscale image and performing style transfer to obtain a transferred image; using a sliding window on the transferred image to achieve mapping between local color gamuts; and fusing all the mapping results to obtain a fused and adjusted image. The present invention greatly reduces the time for manual processing of satellite images by artists and improves the processing efficiency. This method can perform well in different processing tasks such as different illuminations in regular areas, different times in irregular areas, inconsistent full-image illumination with other images, cloud shadows in irregular areas, and different fusions of original images in irregular areas. The processing results have a unified style and meet the actual usage requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of an original unprocessed image - different illuminations in a regular area;

[0043] Figure 2 It is a schematic diagram of an original unprocessed image - different times in an irregular area;

[0044] Figure 3 It is a schematic diagram of an original unprocessed image - inconsistent full-image illumination with other images;

[0045] Figure 4 It is a schematic diagram of an original unprocessed image - cloud shadows in an irregular area;

[0046] Figure 5 It is a schematic diagram of an original unprocessed image - original image in an irregular area has been fused;

[0047] Figure 6 It is a schematic diagram of a negative sample mask;

[0048] Figure 7 It is a schematic diagram of a negative sample image;

[0049] Figure 8 It is a schematic diagram of a positive sample mask;

[0050] Figure 9 It is a schematic diagram of a positive sample image;

[0051] Figure 10 It is an image after histogram matching and edge fusion;

[0052] Figure 11 It is a schematic diagram of the training result of a style transfer neural network;

[0053] Figure 12 It is a weight map;

[0054] Figure 13 It is a weighted fusion map. Detailed implementation manners

[0055] For a better understanding of the technical solution of the present invention, the content of the present invention includes but is not limited to the specific implementation manners hereinafter, and similar technologies and methods should be regarded as within the scope of protection of the present invention. To make the technical problems to be solved, technical solutions and advantages of the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0056] It should be clear that the embodiments described in the present invention are only a part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0057] The large-scale image color adjustment method provided by the present invention, the method includes the steps of:

[0058] S1. Manually select the area to be processed on the image to be processed;

[0059] S2. Use a histogram to adjust the color of the area to be processed, and eliminate the splicing edges between the area with adjusted color and other areas to obtain a grayscale image;

[0060] S3. Reduce the size of the grayscale image and perform style transfer to obtain a transferred image;

[0061] S4. Use a sliding window on the transferred image to achieve mapping between local color gamuts;

[0062] S5. Fuse all the mapping results to obtain a fused and adjusted image.

[0063] Preferably, the area to be processed includes regular areas with different illumination maps, irregular areas with different time maps, full map illumination inconsistent with other maps, irregular area cloud shadow maps, or irregular area original maps that have been fused.

[0064] Preferably, the S1 includes:

[0065] S11. Reduce the satellite image to be processed by a certain multiple to obtain a second image, and use the satellite image to be processed as the original image, and the original image includes negative samples;

[0066] S12. Select a point (xb, yb) in the boundary area of a negative sample of the second image, and then intercept the image with coordinates in the range of xb*8 - 100 to xb*8 + 100, yb*8 - 100 to yb*8 + 100 in the original image as the third image for display;

[0067] S13. Select a point (xc, yc) in the third image, then the coordinates of this point restored on the original image are (xb * 8 + xc - 100, yb * 8 + yc - 100);

[0068] S14. Repeat steps S12 and S13 until the negative sample region of the second image is completely selected;

[0069] S15. If there are multiple negative sample regions in the original image, after repeating S12 - S14, a polygon is finally obtained. Process this polygon to obtain positive and negative sample images.

[0070] Preferably, S2 includes:

[0071] S21. Encode the positive and negative sample images with their corresponding masks to obtain the encoded positive and negative samples. According to the histogram of the encoded positive samples, perform histogram matching on the encoded negative samples;

[0072] S22. Decode the result after histogram matching to obtain a decoded image;

[0073] S23. Perform weighted calculation processing on the decoded image to obtain a weight image;

[0074] S24. Multiply the negative sample image by the weight image, then add it to the positive sample image to obtain a fused image. Shrink the fused image and perform grayscale conversion to obtain a grayscale image.

[0075] Preferably, S3 includes: Training the grayscale image as the content image using an LBFGS-style transfer neural network to obtain the target style image of the neural network.

[0076] Preferably, S4 includes:

[0077] S41. Let the size of the sliding window Gs on the style transfer map G be 64 * 64, and the initial coordinates (xg, yg) = (0, 0). The size of the sliding window FS on the first grayscale image F1 is 64 * 64, and the initial coordinates (xg, yg) = (32 * i, 32 * j). The sliding window FL on the second grayscale image F2 corresponding to this Fs window has a size of 512 * 512, and the initial coordinates (xf, yf) = (256 * i, 256 * j), where the value ranges of i and j are 0 to 62;

[0078] S42. Create a new empty color gamut mapping space: The point value in the space with axes (x, y, z) is (0, 0, 0). Traverse Gs and Fs in a loop. Gs and Fs have the same window size, and there is a one-to-one mapping of pixel values at the same positions. Use the pixel values in Fs as the coordinates of the color gamut mapping space, and use the pixel values in Gs as the values at these coordinates in the color gamut mapping space. After completing 64 * 64 mapping relationships, perform interpolation to obtain the mapping relationship of the complete color gamut space;

[0079] S43. For the 512 * 512 pixels in the sliding window FL, each pixel is used as the coordinate value of the color gamut space to obtain a new pixel value after mapping. After all these pixels are mapped, save the mapped image;

[0080] S44. Set the sliding step to 32 pixels and perform horizontal and vertical sliding respectively. Let the horizontal sliding be i times and the vertical sliding be j times, then (xg, yg) = (32 * i, 32 * j), (xf, yf) = (256 * i, 256 * j); Re - execute steps S42 - S44 every time it slides.

[0081] Preferably, S5 includes: Set a completely black image and a sliding window located therein. Traverse the sliding window according to the set sliding step for the mapped image, multiply the mapped image by the set weight value and then add it to the image within the sliding window of the completely black image to obtain the final result image.

[0082] Specifically, the specific process of the present invention is as follows:

[0083] The present invention classifies all satellite pictures to be processed. The whole satellite picture has the following five situations: Regular regions with different illuminations, such as Figure 1 As shown, there are two regular regions, and two regions are taken by the satellite in different illumination environments within a relatively short time (such as within a few days) and then stitched together. The content of this kind of image only has differences in the main colors, and the content has no large differences. Among them, the regular region refers to a simple concave - convex polygon that can be composed of a small number of points.

[0084] Irregular regions at different times, such as Figure 2 As shown, the satellite takes pictures in an approximately same illumination environment but at a relatively long time (such as more than a few weeks), and the stitched - together region is irregular. The regular region refers to a simple concave - convex polygon that can be composed of a small number of points. Among them, the irregular region refers to a region that cannot be selected by a simple polygon.

[0085] The illumination of the whole picture is inconsistent with other pictures, such as Figure 3As shown, the entire image is an image taken by a satellite under non-ideal lighting conditions, seasons, time, etc. There are no obvious color tone differences in this image, and the color tone of the entire image is unified, so the color of the entire image needs to be adjusted.

[0086] Irregular area cloud shadow, such as Figure 4 As shown, there is a thin cloud below the satellite during shooting, which appears as an irregular area with a shadow or an irregular area that is whitish on the satellite image.

[0087] The original image of the irregular area has been fused. As Figure 5 shown, that is, the above-mentioned several situations that exist have been modified and adjusted by others when the image was obtained. In view of the above situations, the present invention proposes a large-scale image color adjustment method based on style transfer and super-resolution, which realizes efficient and automated image color adjustment. Here, the large scale refers to the situation where the resolution of a single satellite picture to be processed is very large, such as an image with a size of 16384*16384. The specific steps are as follows:

[0088] 1. Manually select the area to be processed on the image. This step can perform targeted processing for specific geographical areas or image defects, improving the pertinence of the method.

[0089] 2. Use histogram matching technology to adjust the overall color. Histogram matching can effectively eliminate the color differences between images and achieve a unified color tone effect. The iterative color fusion method is used to eliminate the stitching edges. This method realizes natural transition and eliminates obvious stitching traces by applying a gradient color weight at the stitching edge. This step can greatly improve the visual effect of the image.

[0090] 3. Reduce the size of the processed grayscale image to meet the input requirements of the style transfer neural network and perform style transfer.

[0091] 4. Use a sliding window to realize the mapping between local color gamuts and achieve high-resolution color reconstruction.

[0092] Compared with traditional methods, the present invention makes full use of the advantages of deep learning to realize the automation and intelligence of image processing. At the same time, by combining steps such as manual selection and color adjustment, the limitations that may be generated by a single algorithm are effectively avoided, and the quality and efficiency of image processing are greatly improved.

[0093] Due to the limitations of GPU video memory and computing power hardware devices, the style transfer neural network cannot process images with a large scale of 16384*16384. On the RTX4080 Super, the maximum processing size is 2048*2048. If the image is split for processing, it will result in obvious splicing marks, inconsistent hues between different slices, and consume a large amount of time. For this reason, the present invention proposes a method capable of processing such large-scale (size) images, and it only takes a very short time in the test environment, such as 15 minutes to complete. The more specific process is as follows:

[0094] I. Selection of the image area to be corrected

[0095] For the five different image processing tasks mentioned in the foregoing of the present invention, there may be 0 ( Figure 3 ), 1 ( Figure 1 ), or more splicing marks ( Figure 3 ) on the unprocessed original image. This is because the satellite takes pictures of the ground at different times and in different seasons. Due to reasons such as lighting conditions and vegetation changes, there are significant differences in hue and content among the images. Moreover, the current satellite images are processed and not the original ones taken. Therefore, when processing, such images with significant differences in hue and content are simply spliced together, forming splicing marks. In this step of the present invention, the method of manual selection is used to select the area to be processed. Among them, the area to be processed is called the negative sample. The area on the original image that is not the area to be processed and has the target color style is called the positive sample. The process is as follows:

[0096] 1. The satellite image to be processed used in the present invention is used as the original image with a resolution of 16384*16384. However, the present invention is not limited to this size and is only used for illustrative purposes. The original image is denoted as Figure A. One original image contains both positive and negative samples, and it must contain positive samples. The number of negative samples can be 0. After reading this image, the image size is reduced to 2048*2048, but not limited to this size, and it is denoted as the second Figure B and displayed. The reduction ratio is 16384 / 2048 = 8 times. After compression, the second Figure B also contains the same number of positive and negative samples as the original image A. However, due to the too large resolution of the original image A, not all pixels can be fully displayed on the display screen. Therefore, this method uses a rough selection - fine selection method to obtain the boundary coordinates of the relatively accurate negative samples. Select the coordinate of a point on the boundary area of the negative sample on the second Figure B as (xb, yb), regarded as the rough selection coordinate. Correspondingly, in the original image A, the image in the interval of xb*8 - 100 to xb*8 + 100, yb*8 - 100 to yb*8 + 100 is intercepted as the third image C for display.

[0097] 2. Select the coordinates of a point in the third image C as (xc, yc). Then the coordinates of this point restored to the original image or on the original image are (xb * 8 + xc - 100, yb * 8 + yc - 100). The coordinates of this point are the boundary coordinates of the selected negative sample obtained after two selections;

[0098] 3. Repeat steps 2 and 3 until a negative sample area is completely selected.

[0099] 4. If there are multiple negative sample areas in an original image A, repeat steps 1, 2, and 3. Finally, a polygon is obtained. Process this polygon to obtain positive and negative sample images. After completing the above manual selection of areas, for all the points selected for negative samples, that is, all the points on the boundary area of the selected negative samples on the original image A form a polygon. Mark the inside of the polygon as 255 and the outside as 0 to obtain a negative sample mask. The inside of the polygon serves as the negative sample mask, as Figure 6 shown, where the white part is the internal mask, and a negative sample image AN, that is, the black part. The black part retains the image of the original image A corresponding to the position of the internal area of the negative sample polygon, that is, Figure 7 as shown in the graphic area in. Take the non-operation of the sum of all negative sample masks to obtain a positive sample mask, as Figure 8 shown. The white part is the positive sample mask. The positive sample image AP is the image obtained by cropping the satellite image to be processed according to the positive sample mask, as Figure 9 shown as the part with an image.

[0100] II. Histogram Matching and Iterative Color Fusion Method

[0101] Since the positive and negative sample images are of irregular regions and there are large black areas in the images, the effect of directly performing histogram matching is very poor. In this method, the sample images are processed by conversion according to the masks, then histogram matching is performed, and finally the result is inversely converted to generate a normal image. The specific steps are as follows:

[0102] 1. Create a new empty array. The positive and negative sample images and the positive and negative sample masks have the same size, that is, 16384 * 16384. Traverse the pixels of the positive and negative sample images and their corresponding masks simultaneously. When the mask pixel value traversed is 255, add the pixel at the corresponding position to the empty array, and when the mask pixel value traversed is 0, skip it. Then the array finally generated by the empty array is the result of the image conversion. After the above processing, the negative sample image is converted to the converted negative sample, and the positive sample image is converted to the converted positive sample.

[0103] 2. Calculate the positive sample histogram based on the converted positive samples, perform histogram matching on the converted negative samples, and obtain the result after histogram matching. The method for calculating the histogram is a mature existing technology and will not be elaborated here. The data format of the result after matching is the same as that of the converted negative samples. Perform inverse transformation on the result after histogram matching, that is, decoding processing. The specific inverse transformation method is as follows: Traverse the negative sample mask. When the mask value at a certain position (xd, yd) is 255, take out the first pixel of the result after histogram matching and place it at this position (xd, yd) of the negative sample mask, so as to finally obtain a newly created all-black image D, that is, the decoded image D.

[0104] 3. Create a weight image E with all pixel values being 1 and a size of 16384 * 16384. Calculate the brightness ratio of each pixel at the boundary of the all-black image D corresponding to the positive samples. The specific method is as follows: Assume that the pixel coordinates of a certain point in the all-black image D are (xe, ye). First, calculate the average value of all non-zero positive sample pixels within a square with a side length of 100 centered on this pixel point, denoted as AverP. At the same time, calculate the average value of all non-zero negative sample pixels within a square with a side length of 100 centered on this pixel point, denoted as AverN. Then the brightness ratio is R = AverP / AverN, which is also called the weight. Thus, assign the value of R to the point with pixel coordinates (xe, ye) in the all-black image D, and then obtain the weight image E.

[0105] 4. Traverse the weight image E. If there is a calculated brightness ratio (not 1) within a 3 * 3 area near a certain pixel point inside this image E, then the brightness ratio value of this point is the average of the brightness ratios (not 1) existing within the 3 * 3 area nearby; otherwise, it is 1. Repeat step 4 1000 times. Finally, a gray-scale brightness weight transition zone with a width of 1000 and adjacent to the positive and negative sample regions can be obtained. After multiplying this weight transition zone by the negative samples, the gray-scale difference at the adjacent contact boundary between the positive and negative samples can be eliminated, achieving smooth image transition. This step can be selected according to the situation. If the gap is too small, it can be not executed; if the gap is too large, this step can be adopted. The size of the gap can be specified in advance according to requirements.

[0106] 5. Multiply the negative sample image AN by the weight image E, and then add it to the positive sample image AP to obtain the fused image F. Reduce the size of the image F to 2048 * 2048 and convert it to a grayscale image at the same time to obtain the first grayscale image F1. Then further convert the first grayscale image F1 to obtain the converted second grayscale image F2, as Figure 10 shown. The conversion process can use conventional methods and will not be elaborated here.

[0107] III. Style Transfer

[0108] The present invention re - colors the first grayscale image F1, which is achieved by using an LBFGS style - transfer neural network. The first grayscale image F1 is used as the content image, and another selected style image is used as the target style image of the neural network for training. The number of training rounds is 520 rounds. It is tested that this number is sufficient, but it is not limited to this number. The first grayscale image F1 is trained using the LBFGS style - transfer neural network. Training stops after reaching a certain number of times or when the coloring evaluation of the content image is similar to that of the target style image, and the output result is as Figure 11 shown, denoted as the style - transfer image G. This step includes but is not limited to using the LBFGS neural network for style - transfer training, and the number of rounds is not limited to 520 times. It can be a style - transfer neural network with other network structures. The use of this kind of style - transfer neural network in the present invention is prior art, and the specific training process will not be elaborated here.

[0109] IV. Sliding - window color - gamut mapping

[0110] The maximum image size that style transfer can handle is 2048 * 2048. Therefore, this method uses a sliding - window - based color - gamut mapping method to achieve image reconstruction from a resolution of 2048 * 2048 to 16384 * 16384. The specific steps are as follows:

[0111] 1. Let the size of the sliding window Gs on the style - transfer image G be 64 * 64, and the initial coordinates (xg, yg)=(32 * i, 32 * j), where the value ranges of i and j are 0 to 62. The size of the sliding window Fs on the first grayscale image F1 is 64 * 64, and the initial coordinates (xg, yg)=(32 * i, 32 * j), where the value ranges of i and j are 0 to 62. The sliding window FL on the second grayscale image F2 corresponding to the Fs window has a size of 512 * 512, and the initial coordinates (xf, yf)=(256 * i, 256 * j), where the value ranges of i and j are 0 to 62.

[0112] 2. Create an empty color - gamut mapping space: Set the point value in the space with coordinate axes (x, y, z) to (0, 0, 0), and loop through Gs. Since the window sizes of Gs and Fs are the same, there is a one - to - one mapping of pixel values at the same positions. Therefore, use the pixel values in Fs as the coordinates of the color - gamut mapping space, and use the pixel values in Gs as the values at these coordinates in the color - gamut mapping space. After writing 64 * 64 mapping relationships, use the griddata method for interpolation to obtain the mapping relationship of the complete color - gamut space. Among them, the griddata method uses an existing open - source algorithm, and the specific interpolation process will not be elaborated here.

[0113] 3. For the 512 * 512 pixels in the sliding window FL, each pixel serves as the coordinate value in the color gamut space, and a new pixel value after mapping can be obtained. After mapping all these pixels, save the mapped image, denoted as Hij, where the value ranges of both i and j are from 0 to 62.

[0114] 4. Set the sliding step size to 32 pixels and slide horizontally and vertically respectively. Suppose it slides i times horizontally and j times vertically, then (xg, yg) = (32 * i, 32 * j), (xf, yf) = (256 * i, 256 * j), and the value ranges of both i and j are from 0 to 62. Each time it slides, re - execute the aforementioned steps 2, 3, and 4 in sequence.

[0115] 5. Based on the mapping result Hij for fusion. Since the step size in each of steps 1 - 4 is half of the size of one picture, in the 16384 * 16384 area, most areas have multiple pictures overlapping. Therefore, in this part, this method uses a weighted way to fuse the images, and the preset weights are as Figure 12 shown, where black represents a value of 0 and white represents a value of 1. The specific steps are as follows:

[0116] (1) Create a completely black image K with a size of 16384 * 16384;

[0117] (2) Traverse all images Hij, specifically as follows: If the step size i + j is even, multiply Hij by the weight 1, and then add it to the image within the sliding window with the coordinate range (256 * i ~ 256 * i + 512, 256 * j ~ 256 * j + 512) in the completely black image K; if the step size i + j is odd, multiply Hij by the weight 2, and then add it to the image within the coordinate range (256 * i ~ 256 * i + 512, 256 * j ~ 256 * j + 512) in the completely black image K. Each Hij is multiplied by different weights according to different positions and then superimposed, which can eliminate the sudden change at the edges between different Hij. The purpose of executing this step is to fuse the non - edge area images of Hij into the completely black image K. Executing steps (2) - (7) is to fuse all Hij into one picture.

[0118] (3) Traverse all Hij where i equals 0 and j is not equal to 0 and j is not equal to 62. If j is even, multiply Hij by the weight 8, and then add it to the image within the sliding window with the coordinate range (256 * i ~ 256 * i + 512, 256 * j ~ 256 * j + 512) in the completely black image K; if the step size j is odd, multiply Hij by the weight 4, and then add it to the image within the coordinate range (256 * i ~ 256 * i + 512, 256 * j ~ 256 * j + 512) in the completely black image K. The purpose is to fuse the left - edge area images of Hij into the completely black image K.

[0119] (4) Traverse all Hij where i equals 62, j is not equal to 0, and j is not equal to 62. If j is even, multiply Hij by the weight value 7, and then add it to the image within the sliding window of the all-black image K with the coordinate range (256*i to 256*i + 512, 256*j to 256*j + 512); if the step size j is odd, multiply Hij by the weight value 3, and then add it to the image within the sliding window of the all-black image K with the coordinate range (256*i to 256*i + 512, 256*j to 256*j + 512). The purpose is to fuse the image in the right edge region of Hij into the all-black image K.

[0120] (5) Traverse all Hij where j equals 0, i is not equal to 0, and i is not equal to 62. If i is even, multiply Hij by the weight value 9, and then add it to the image within the coordinate range (256*i to 256*i + 512, 256*j to 256*j + 512) of the all-black image K; if the step size i is odd, multiply Hij by the weight value 5, and then add it to the image within the coordinate range (256*i to 256*i + 512, 256*j to 256*j + 512) of the all-black image K. The purpose is to fuse the image in the upper edge region of Hij into the all-black image K.

[0121] (6) Traverse all Hij where j equals 62, i is not equal to 0, and i is not equal to 62. If i is even, multiply Hij by the weight value 10, and then add it to the image within the coordinate range (256*i to 256*i + 512, 256*j to 256*j + 512) of the all-black image K; if the step size i is odd, multiply Hij by the weight value 5, and then add it to the image within the coordinate range (256*i to 256*i + 512, 256*j to 256*j + 512) of the all-black image K. The purpose is to fuse the image in the lower edge region of Hij into the all-black image K.

[0122] (7) Multiply the image Hij with i = 0 and j = 0 by the weight 11, and then add it to the image in the all - black image K within the coordinate range (256 * i ~ 256 * i + 512, 256 * j ~ 256 * j + 512); Multiply the image Hij with i = 62 and j = 62 by the weight 12, and then add it to the image in the all - black image K within the coordinate range (256 * i ~ 256 * i + 512, 256 * j ~ 256 * j + 512); Multiply the image Hij with i = 62 and j = 0 by the weight 13, and then add it to the image in the all - black image K within the coordinate range (256 * i ~ 256 * i + 512, 256 * j ~ 256 * j + 512); Multiply the image Hij with i = 0 and j = 62 by the weight 14, and then add it to the image in the all - black image K within the coordinate range (256 * i ~ 256 * i + 512, 256 * j ~ 256 * j + 512). The purpose is to fuse the images in the four - corner regions of Hij into the all - black image K.

[0123] (8) After the processing in steps (2) - (7), the final result image M is obtained. As Figure 13 shown, without changing the content information of the satellite image, this image can unify the tone and other information of the whole image, eliminate the splicing traces caused by the satellite image being taken at different times and seasons of the ground due to reasons such as lighting conditions and vegetation changes, and has the characteristics of less manual interference, automation and intelligence. At the same time, the processed effect meets the requirements of actual production use.

[0124] As an embodiment of the present invention, the present invention also discloses a large - scale image color adjustment system. The system implements the method described above and includes the following modules:

[0125] A selection module for manually selecting the area to be processed on the image to be processed;

[0126] An adjustment module for adjusting the color of the area to be processed using a histogram and eliminating the splicing edge between the area with adjusted color and other areas to obtain a grayscale image;

[0127] A style transfer module for reducing the size of the grayscale image and performing style transfer to obtain a transferred image;

[0128] A mapping module for implementing the mapping between local color gamuts on the transferred image using a sliding window;

[0129] A fusion module for fusing all the mapping results to obtain a fused and adjusted image.

[0130] As an embodiment of the present invention, the present invention also discloses an electronic device, which includes:

[0131] A memory storing executable instructions;

[0132] A processor that runs the executable instructions in the memory to implement the method of the present invention.

[0133] As an embodiment disclosed by the present invention, the present invention also discloses a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the method of the present invention.

[0134] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "said" used in the embodiments of the present invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0135] The above description shows and describes several preferred embodiments of the present invention. However, as mentioned above, it should be understood that the present invention is not limited to the forms disclosed herein, should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be changed within the scope of the application concept described herein through the above teachings or the technology or knowledge in the relevant field. And the changes and modifications made by those skilled in the art that do not depart from the spirit and scope of the present invention shall fall within the protection scope of the appended claims of the present invention.

Claims

1. A large-scale image color adjustment method, characterized in that: The method comprises the steps of: S1. Manually selecting a region to be processed on the image to be processed, wherein S1 comprises: S11. Reduce the satellite image to be processed by a certain multiple to obtain a second image, and use the satellite image to be processed as the original image, wherein the original image includes a negative sample; S12. Select a point (xb, yb) in the boundary area of ​​a negative sample in the second image, intercept an image with coordinates in the range of xb*8-100 to xb*8+100, yb*8-100 to yb*8+100 in the original image according to the point, and display it as the third image; S13. Select a point (xc, yc) in the third image, and the coordinates of the point restored to the original image are (xb*8+xc-100, yb*8+yc-100); S14. Repeat steps S12 and S13 until the negative sample area of ​​the second image is selected; S15. If there are multiple negative sample areas in the original image, then after repeatedly executing S12-S14, a polygon is finally obtained, and the polygon is processed to obtain positive and negative sample images; S2. Using a histogram to adjust the color of the area to be processed, and eliminating the splicing edges between the area with the adjusted color and other areas to obtain a grayscale image, wherein S2 includes: S21. Encode the positive and negative sample images and their corresponding masks to obtain encoded positive and negative samples, and perform histogram matching on the encoded negative samples according to the encoded positive sample histogram; S22. Decoding the result after the histogram matching to obtain a decoded image; S23. Performing weighted calculation on the decoded image to obtain a weighted image; S24. multiplying the negative sample image by the weight image, and then adding the negative sample image to the positive sample image to obtain a fused image, reducing the fused image and performing grayscale conversion to obtain a first grayscale image, and further performing grayscale conversion on the first grayscale image to obtain a second grayscale image; S3. reducing the size of the first grayscale image and performing style migration to obtain a migrated image; S4. Use a sliding window to achieve mapping between local color gamuts for the migrated image; S5. Fuse all mapping results to obtain a fused and adjusted image.

2. The large-scale image color adjustment method according to claim 1, characterized in that: The areas to be processed include: different lighting in regular areas, different time in irregular areas, lighting in the whole image inconsistent with other images, cloud shadows in irregular areas, or the original images in irregular areas have been fused.

3. The large-scale image color adjustment method according to claim 1, characterized in that: The S3 includes: training the first grayscale image as a content image using an LBFGS style transfer neural network to obtain a style transfer image of the neural network.

4. The large-scale image color adjustment method according to claim 3, characterized in that S4 include: S41. Assume that the size of the sliding window Gs on the style transfer map G is 64*64, the initial coordinates (xg, yg) = (0, 0), the size of the sliding window Fs on the first grayscale image F1 is 64*64, the initial coordinates (xg, yg) = (32*i, 32*j), the size of the sliding window FL on the second grayscale image F2 corresponding to the Fs window is 512*512, the initial coordinates (xf, yf) = (256*i, 256*j), i is the number of horizontal sliding times, j is the number of vertical sliding times, and the value range of i, j is 0 to 62; S42. Create a new empty color gamut mapping space: the point value on the space with coordinate axes (x, y, z) is (0, 0, 0), loop through Gs, the Gs and Fs windows have the same size, and there is a one-to-one mapping between pixel values ​​at the same position. The pixel value in Fs is used as the coordinate of the color gamut mapping space, and the pixel value in Gs is used as the value of the color gamut mapping space at the coordinate. After completing 64*64 mapping relationships, interpolation is performed to obtain the mapping relationship of the complete color gamut space; S43. For the 512*512 pixels in the sliding window FL, each pixel is used as the coordinate value of the color gamut space to obtain a mapped new pixel value. After all the pixels are mapped, the mapped image is saved; S44. Assume that the sliding step is 32 pixels, and slide horizontally and vertically respectively. Assume that the horizontal sliding is i times and the vertical sliding is j times, then (xg, yg) = (32*i, 32*j), (xf, yf) = (256*i, 256*j); re-execute steps S42-S44 for each sliding.

5. The large-scale image color adjustment method according to claim 4, characterized in that: S5 includes: Set a completely black image and a sliding window in it, traverse the mapped image through the sliding window according to the set sliding step size, multiply the mapped image with the set weight and then add it to the image in the sliding window in the completely black image to obtain the final result image.

6. A large-scale image color adjustment system, characterized in that: The system implements the method described in any one of claims 1 to 5, and includes the following modules: A selection module, used for manually selecting a region to be processed on the image to be processed; An adjustment module, used for adjusting the color of the area to be processed by using a histogram, and eliminating the splicing edge between the area with the adjusted color and other areas to obtain a grayscale image; A style transfer module, used to reduce the size of the grayscale image and perform style transfer to obtain a transferred image; A mapping module, used for using a sliding window to achieve mapping between local color gamuts for the migrated image; The fusion module is used to fuse all the mapping results to obtain a fused and adjusted image.

7. An electronic device, characterized in that: The electronic device comprises: A memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the method according to any one of claims 1 to 5.

8. A computer storage medium, characterized in that: The medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 5.

Images