Large-scale image color adjustment method, system and device and medium

Abstract

The invention relates to a large-scale image color adjustment method, system and device and a medium. The method comprises the steps that a to-be-processed area on a to-be-processed image is manually selected; adjusting the color of the to-be-processed region by adopting a histogram, and eliminating the splicing edges of the region with the adjusted color and other regions to obtain a gray level image; reducing the size of the grayscale image and performing style migration to obtain a migrated image; mapping between local color gamuts of the migrated image is realized by using a sliding window; and fusing all the mapping results to obtain a fused and adjusted image. According to the invention, the time for manually processing the sanitary film by art is reduced to a great extent, and the processing efficiency is improved; the method has good performance when the regular area has different illumination, the irregular area has different time, the total image illumination is inconsistent with other images, the irregular area cloud shadow and the irregular area original image have been fused with different processing tasks, the processing result style is unified, and the actual use requirement is met.

Description

Technical Field

[0001] The present invention belongs to the field of satellite image technology, and in particular relates to a large-scale image color adjustment method, system, device and medium. Background Art

[0002] As a training system, the flight simulator has extremely high requirements for the authenticity and continuity of terrain images. However, there are some urgent problems with existing satellite image resources. First, there are obvious splicing traces. This is because the satellite shoots the ground at different times and seasons. Due to lighting conditions and vegetation changes, the images have large differences in tone and content. Simply splicing these images will lead to discontinuous terrain and affect the pilot's judgment. Secondly, although the method of artificial post-PS processing can improve the image effect, the workload is large and it may also change the original terrain information. The existing satellite image processing technology solutions mainly include the following: Artificial PS processing method: Engineers use software such as Photoshop to adjust the overall color of the image, and use tools such as layer fusion and stamps to repair the splicing edges and areas with large color differences, so as to make the image more coordinated and unified. Image selection and comprehensive method: By comparing multiple satellite image sources, select images with better quality in the same area for comprehensive processing. Histogram matching method: Using histogram matching technology, the image histogram characteristics of the area to be processed are adjusted to the same as the histogram of the target area. Figure 1 The above methods have the following disadvantages: the manual PS processing method has a large workload and low efficiency, and may also cause local distortion of terrain information. The image selection and comprehensive method requires more manual comparison and analysis work, and it cannot completely eliminate the color difference and splicing marks between images. Different sources need to be purchased separately, which increases the cost. The histogram matching method has high requirements for region selection and determination of target histograms, and the processing effect also has certain limitations.

[0003] Therefore, it is urgent to develop an automated color adjustment method for large-resolution satellite images that can eliminate stitching traces and maintain the accuracy of terrain information to meet the needs of flight simulation training. This automated large-resolution satellite image processing method can not only solve the existing image quality problems, but also greatly improve processing efficiency, providing a more realistic and reliable terrain environment for flight simulation training. Summary of the invention

[0004] In order to overcome the problems existing in the prior art, the present invention provides a large-scale image color adjustment method, system, device and medium to overcome the existing defects.

[0005] A large-scale image color adjustment method, the method comprising the steps of: S1. Manually select the area to be processed on the image to be processed; S2. Using a histogram to adjust the color of the area to be processed, and eliminating the splicing edges between the area where the color is adjusted and other areas to obtain a grayscale image; S3. reducing the size of the grayscale image and performing style migration to obtain a migrated image; S4. Use a sliding window to achieve mapping between local color gamuts for the migrated image; S5. Fuse all mapping results to obtain a fused and adjusted image.

[0006] According to the aspects described above and any possible implementation method, an implementation method is further provided, wherein the area to be processed includes different illumination maps of regular areas, different time maps of irregular areas, maps where the illumination of the entire image is inconsistent with other images, cloud shadow maps of irregular areas, or fused maps of the original images of irregular areas.

[0007] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S1 includes: S11. Reduce the satellite image to be processed by a certain multiple to obtain a second image, and use the satellite image to be processed as the original image, wherein the original image includes a negative sample; S12. Select a point (xb, yb) in the boundary area of ​​a negative sample of the second image, and then intercept the image with coordinates in the range of xb*8-100 to xb*8+100, yb*8-100 to yb*8+100 in the original image according to the point, and display it as the third image; S13. Select a point (xc, yc) in the third image, and the coordinates of the point restored to the original image are (xb*8+xc-100, yb*8+yc-100); S14. Repeat steps S12 and S13 until the negative sample area of ​​the second image is selected; S15. If there are multiple negative sample areas in the original image, then after repeatedly executing S12-S14, a polygon is finally obtained, and the polygon is processed to obtain positive and negative sample images.

[0008] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S2 includes: S21. Encode the positive and negative sample images and their corresponding masks to obtain encoded positive and negative samples, and perform histogram matching on the encoded negative samples according to the encoded positive sample histogram; S22. Decoding the result after the histogram matching to obtain a decoded image; S23. Performing weighted calculation on the decoded image to obtain a weighted image; S24. Multiply the negative sample image by the weight image, and then add the negative sample image to the positive sample image to obtain a fused image, reduce the fused image and perform grayscale conversion to obtain a grayscale image.

[0009] According to the above aspects and any possible implementation methods, an implementation method is further provided, wherein S3 includes: training the grayscale image as a content map using an LBFGS style transfer neural network to obtain a target style map of the neural network.

[0010] According to the above aspects and any possible implementation, an implementation is further provided, where S4 includes: S41. Assume that the size of the sliding window Gs on the style transfer map G is 64*64, the initial coordinates (xg, yg) = (0, 0), the size of the sliding window FS on the first grayscale image F1 is 64*64, the initial coordinates (xg, yg) = (32*i, 32*j), the size of the sliding window FL on the second grayscale image F2 corresponding to the Fs window is 512*512, the initial coordinates (xf, yf) = (256*i, 256*j), i is the number of horizontal sliding times, j is the number of vertical sliding times, and the value range of i, j is 0 to 62; S42. Create a new empty color gamut mapping space: the point value on the space with coordinate axes (x, y, z) is (0, 0, 0), loop through Gs, the Gs and Fs windows have the same size, and there is a one-to-one mapping between pixel values ​​at the same position. The pixel value in Fs is used as the coordinate of the color gamut mapping space, and the pixel value in Gs is used as the value of the color gamut mapping space at the coordinate. After completing 64*64 mapping relationships, interpolation is performed to obtain the mapping relationship of the complete color gamut space; S43. For the 512*512 pixels in the sliding window FL, each pixel is used as the coordinate value of the color gamut space to obtain a mapped new pixel value. After all the pixels are mapped, the mapped image is saved; S44. Assume that the sliding step is 32 pixels, and slide horizontally and vertically i and j times respectively, then (xg, yg) = (32*i, 32*j), (xf, yf) = (256*i, 256*j); re-execute steps S42-S44 for each slide.

[0011] As described above and any possible implementation method, an implementation method is further provided, S5 includes: setting a completely black image and a sliding window located therein, traversing the mapped image through the sliding window according to the set sliding step size, multiplying the mapped image with the set weight and then adding it to the image in the sliding window in the completely black image to obtain a final result image.

[0012] The present invention also provides a large-scale image color adjustment system, wherein the system implements the method described, and comprises the following modules: A selection module, used for manually selecting a region to be processed on the image to be processed; An adjustment module, used for adjusting the color of the area to be processed by using a histogram, and eliminating the splicing edge between the area with the adjusted color and other areas to obtain a grayscale image; A style transfer module, used to reduce the size of the grayscale image and perform style transfer to obtain a transferred image; A mapping module, used for using a sliding window to achieve mapping between local color gamuts for the migrated image; The fusion module is used to fuse all the mapping results to obtain a fused and adjusted image.

[0013] The present invention further provides an electronic device, comprising: A memory storing executable instructions; A processor is used to execute the executable instructions in the memory to implement the method.

[0014] The present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the method described.

[0015] Beneficial Effects of the Invention The large-scale image color adjustment method of the present invention comprises: manually selecting a region to be processed on the image to be processed; using a histogram to adjust the color of the region to be processed, and eliminating the splicing edges between the region where the color is adjusted and other regions to obtain a grayscale image; reducing the size of the grayscale image and performing style migration to obtain a migrated image; using a sliding window to achieve mapping between local color domains for the migrated image; and fusing all mapping results to obtain a fused and adjusted image. The present invention greatly reduces the time for artists to manually process satellite images and improves processing efficiency. The method can perform well in processing tasks with different lighting in regular areas, different times in irregular areas, inconsistent lighting in the entire image with other images, cloud shadows in irregular areas, and different fusions of the original image in irregular areas. The processing results have a unified style and meet actual use requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is the unprocessed original image - a schematic diagram of different lighting in a regular area; Figure 2 This is the unprocessed original image - the image of different time periods in irregular areas; Figure 3 This is a schematic diagram of the original unprocessed image - the lighting of the entire image is inconsistent with other images; Figure 4 This is the unprocessed original image - schematic diagram of cloud shadows in irregular areas; Figure 5 This is a schematic diagram of the fusion of the unprocessed original image and the irregular area original image; Figure 6 Schematic diagram of negative sample mask; Figure 7 Schematic diagram of negative sample image; Figure 8 Schematic diagram of positive sample mask; Fig. 9 Schematic diagram of a positive sample image; Fig.10 The image after histogram matching and edge fusion; Fig.11 This is a schematic diagram of the style transfer neural network training results; Fig.12 is the weight graph; Fig.13 It is a weighted fusion graph. DETAILED DESCRIPTION

[0017] In order to better understand the technical solution of the present invention, the content of the present invention includes but is not limited to the specific implementation methods described below, and similar technologies and methods should be considered to be within the scope of protection of the present invention. In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0018] It should be clear that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The large-scale image color adjustment method provided by the present invention comprises the steps of: S1. Manually select the area to be processed on the image to be processed; S2. Using a histogram to adjust the color of the area to be processed, and eliminating the splicing edges between the area where the color is adjusted and other areas to obtain a grayscale image; S3. reducing the size of the grayscale image and performing style migration to obtain a migrated image; S4. Use a sliding window to achieve mapping between local color gamuts for the migrated image; S5. Fuse all mapping results to obtain a fused and adjusted image.

[0020] Preferably, the area to be processed includes different illumination maps of regular areas, different time maps of irregular areas, maps of inconsistent illumination of the entire image and other images, cloud shadow maps of irregular areas, or fused maps of the original images of irregular areas.

[0021] Preferably, the S1 includes: S11. Reduce the satellite image to be processed by a certain multiple to obtain a second image, and use the satellite image to be processed as the original image, wherein the original image includes a negative sample; S12. Select a point (xb, yb) in the boundary area of ​​a negative sample of the second image, and then intercept an image with coordinates in the range of xb*8-100 to xb*8+100, yb*8-100 to yb*8+100 in the original image according to the point, and display it as the third image; S13. Select a point (xc, yc) in the third image, and the coordinates of the point restored on the original image are (xb*8+xc-100, yb*8+yc-100); S14. Repeat steps S12 and S13 until the negative sample area of ​​the second image is selected; S15. If there are multiple negative sample areas in the original image, then after repeatedly executing S12-S14, a polygon is finally obtained, and the polygon is processed to obtain positive and negative sample images.

[0022] Preferably, the S2 includes: S21. Encode the positive and negative sample images and their corresponding masks to obtain encoded positive and negative samples, and perform histogram matching on the encoded negative samples according to the encoded positive sample histogram; S22. Decoding the result after the histogram matching to obtain a decoded image; S23. Performing weighted calculation on the decoded image to obtain a weighted image; S24. Multiply the negative sample image by the weight image, and then add the negative sample image to the positive sample image to obtain a fused image, reduce the fused image and perform grayscale conversion to obtain a grayscale image.

[0023] Preferably, S3 includes: training the grayscale image as a content map using an LBFGS style transfer neural network to obtain a target style map of the neural network.

[0024] Preferably, S4 includes: S41. Assume that the size of the sliding window Gs on the style transfer map G is 64*64, the initial coordinates (xg, yg) = (0, 0), the size of the sliding window FS on the first grayscale image F1 is 64*64, the initial coordinates (xg, yg) = (32*i, 32*j), the size of the sliding window FL on the second grayscale image F2 corresponding to the Fs window is 512*512, the initial coordinates (xf, yf) = (256*i, 256*j), and the value range of i, j is 0~62; S42. Create a new empty color gamut mapping space: the point value on the space with coordinate axes (x, y, z) is (0, 0, 0), loop through Gs, the Gs and Fs windows have the same size, and there is a one-to-one mapping between pixel values ​​at the same position. The pixel value in Fs is used as the coordinate of the color gamut mapping space, and the pixel value in Gs is used as the value of the color gamut mapping space at the coordinate. After completing 64*64 mapping relationships, interpolation is performed to obtain the mapping relationship of the complete color gamut space; S43. For the 512*512 pixels in the sliding window FL, each pixel is used as the coordinate value of the color gamut space to obtain a mapped new pixel value. After all the pixels are mapped, the mapped image is saved; S44. Assume that the sliding step is 32 pixels, and slide horizontally and vertically respectively. Assume that the horizontal sliding is i times and the vertical sliding is j times, then (xg, yg) = (32*i, 32*j), (xf, yf) = (256*i, 256*j); re-execute steps S42-S44 for each sliding.

[0025] Preferably, S5 includes: setting a completely black image and a sliding window therein, traversing the mapped image through the sliding window according to a set sliding step, multiplying the mapped image with a set weight and then adding the image with the image in the sliding window in the completely black image to obtain a final result image.

[0026] Specifically, the specific process of the present invention is as follows: The present invention classifies all satellite images to be processed. The entire satellite image has the following five conditions: different illumination in regular areas, such as Figure 1 As shown in the figure, there are two regular areas. The two areas were taken by a satellite in different lighting environments in a short period of time (such as a few days) and then stitched together. The content of this image is only different in the main color, and the content is not much different. Among them, the regular area refers to a small number of points that can form a simple concave and convex polygon.

[0027] Different times in irregular areas, e.g. Figure 2 As shown in the figure, the area after stitching is irregular when the satellite takes photos in a similar lighting environment but at a long time (such as more than a few weeks). A regular area refers to a simple concave and convex polygon formed by a small number of points. Among them, an irregular area refers to an area that cannot be selected by a simple polygon.

[0028] The lighting of the whole image is inconsistent with other images, such as Figure 3 As shown, the entire image is taken by the satellite under non-ideal lighting conditions, seasons, time and other conditions. There is no obvious difference in color tone in the image, and the color tone of the entire image is uniform, so the color of the entire image needs to be adjusted.

[0029] Irregular area cloud shadows, such as Figure 4 As shown, there were thin clouds below the satellite when it took the photo, which appear as shadowed irregular areas or whitish irregular areas on the satellite image.

[0030] The original image of the irregular area has been fused, such as Figure 5 As shown, the above-mentioned situations exist that the images have been modified and adjusted by others when they were acquired. In view of the above-mentioned situations, the present invention proposes a large-scale image color adjustment method based on style transfer and super-resolution, which realizes efficient and automatic image color adjustment. The large scale here refers to the situation where the resolution of a single satellite image to be processed is very large, such as an image with a size of 16384*16384. Specifically, the following steps are included: 1. Manually select the area to be processed on the image. This step can be used to target specific geographical areas or image defects, thus improving the pertinence of the method.

[0031] 2. Use histogram matching technology to adjust the overall color. Histogram matching can effectively eliminate color differences between images and achieve a unified tone effect. Use iterative color fusion to eliminate splicing edges. This method achieves a natural transition and eliminates obvious splicing marks by applying a gradient color weight to the splicing edge. This step can greatly improve the visual effect of the image.

[0032] 3. Reduce the size of the processed grayscale image to meet the input requirements of the style transfer neural network and perform style transfer.

[0033] 4. Use sliding windows to achieve mapping between local color gamuts and realize high-resolution color reconstruction.

[0034] Compared with traditional methods, the present invention makes full use of the advantages of deep learning to achieve automation and intelligence in image processing. At the same time, combined with steps such as manual selection and color adjustment, it effectively avoids the limitations that may arise from a single algorithm and greatly improves the quality and efficiency of image processing.

[0035] Due to the limitations of GPU video memory and computing power hardware, the style transfer neural network cannot process large-scale images of size 16384*16384. The maximum processing size on RTX4080 Super is 2048*2048. If the image is split and processed separately, it will result in obvious splicing marks and inconsistent tones between different slices, and it will take a lot of time. To this end, the present invention proposes a method that can process such large-scale (size) images, and it only takes a short time in the test environment, such as 15 minutes. The more specific process is as follows: 1. Selection of image area to be corrected For the five different image processing tasks mentioned in the above description of the present invention, there may be 0 ( Figure 3 )、1 place( Figure 1 ) and above splicing traces ( Figure 3 ). This is because satellites photograph the ground at different times and seasons. Due to lighting conditions and vegetation changes, the images have large differences in tone and content, and the current satellite images are processed, not originally taken. Therefore, during processing, these images with large differences in tone and content are simply spliced ​​together to form splicing traces. The present invention uses a manual selection method to select the area to be processed in this step. Among them, the area to be processed is called a negative sample. The non-to-be-processed area on the original image is called a positive sample as the area of ​​the target color style. The process is as follows: 1. The satellite image to be processed used in the present invention is used as the original image, and the resolution is 16384*16384, but the present invention is not limited to this size. It is only used for illustration. The original image is recorded as Figure A. An original image contains positive and negative samples, which must contain positive samples, and the number of negative samples can be 0. After reading the image, the image size is reduced to 2048*2048, but not limited to this size, and recorded as the second image B for display. The reduction rate is 16384 / 2048=8 times. After compression, the second image B also includes positive and negative samples, and the number is the same as that of the original image A. However, since the resolution of the original image A is too large, not all pixels are displayed on the display screen, so this method adopts a coarse selection-fine selection method to obtain more accurate boundary coordinates of negative samples. The coordinates of a point on the boundary area of ​​the negative sample on the second image B are selected as (xb, yb), which are regarded as the coarse selected coordinates. Accordingly, the image with coordinates in the range of xb*8-100 to xb*8+100, yb*8-100 to yb*8+100 is intercepted in the original image A and displayed as the third image C.

[0036] 2. Select the coordinates of a point in the third image C as (xc, yc), then the coordinates of the point restored to the original image or on the original image are (xb*8+xc-100, yb*8+yc-100), and the coordinates of this point are the boundary coordinates of the selected negative sample obtained after selecting twice; 3. Repeat steps 2 and 3 until a negative sample area is selected.

[0037] 4. If there are multiple negative sample areas in an original image A, repeat steps 1, 2, and 3 to finally obtain a polygon, process the polygon, and obtain positive and negative sample images. After completing the above manual selection of areas, all points selected for negative samples, that is, all points on the boundary area of ​​the selected negative samples on the original image A, form a polygon, mark the inside of the polygon as 255, and the outside as 0, and you can get a negative sample mask. The inside of the polygon is used as the negative sample mask, as shown in Figure 6 As shown, the white part is the internal mask, and a negative sample image AN, that is, the black part, which retains the image of the original image A corresponding to the position of the internal area of ​​the negative sample polygon, that is Figure 7 The positive sample mask is obtained by negating the sum of all negative sample masks, as shown in the graphic area of ​​. Figure 8 As shown in , the white part is the positive sample mask, and the positive sample image AP is the image obtained by cropping the satellite image to be processed according to the positive sample mask, as shown in Fig. 9 The portion with the image is shown.

[0038] 2. Histogram matching and iterative color fusion method Since the positive and negative sample images are irregular areas, there are large black areas in the images. The effect of directly performing histogram matching is very poor. This method converts the sample image according to the mask, then performs histogram matching, and finally inversely converts the result to generate a normal image. The specific steps are as follows: 1. Create a new empty array. The positive and negative sample images and the positive and negative sample masks have the same size, that is, 16384*16384. At the same time, traverse the pixels of the positive and negative sample images and their corresponding masks. When the traversed mask pixel value is 255, add the pixel at the corresponding position to the empty array, and when the traversed mask pixel value is 0, skip it. The array finally generated by the empty array is the result of image conversion. After the above processing, the negative sample image is converted to obtain the converted negative sample, and the positive sample image is converted to obtain the converted positive sample.

[0039] 2. According to the converted positive sample, calculate the positive sample histogram, perform histogram matching on the converted negative sample, and obtain the result after histogram matching. The method for calculating the histogram is an existing mature technology and will not be repeated. The data format of the matched result is the same as the data format of the converted negative sample. The result after histogram matching is reverse converted, that is, decoded. The specific reverse conversion method is: traverse the negative sample mask, when the mask value of a certain position (xd, yd) is 255, take out the first pixel of the result after the histogram matching, and put it into the position (xd, yd) of the negative sample mask, so as to finally obtain a new all-black image D, that is, the decoded image D.

[0040] 3. Create a weighted image E with all pixel values ​​1 and a size of 16384*16384. Calculate the brightness ratio of each pixel at the boundary of the all-black image D to the positive sample. The specific method is: Assume that the pixel coordinates of a point in the all-black image D are (xe, ye), first calculate the average value of all non-zero positive sample pixels with a side length of 100 centered on the pixel of the point, recorded as AverP, and at the same time calculate the average value of all non-zero negative sample pixels with a side length of 100 centered on the pixel of the point, recorded as AverN, then the brightness ratio is R = AverP / AverN, also known as the weight, so that the point value with pixel coordinates (xe, ye) in the all-black image D is assigned to R, and then the weighted image E is obtained.

[0041] 4. Traverse the weighted image E. If the calculated brightness ratio (non-1) exists in the 3*3 area near a certain pixel point in the image E, the brightness ratio of the point is the average of the brightness ratios (non-1) in the nearby 3*3 area, otherwise it is 1. Repeat step 4 1000 times, and finally get the grayscale brightness weight transition band with a width of 1000 at the adjacent contact boundary of the positive sample area and the negative sample area. After multiplying this weight transition band with the negative sample, the difference in grayscale between the adjacent contact boundaries of the positive and negative samples can be eliminated, so as to achieve smooth transition of the image. This step can be selected according to the situation. If the gap is too small, it can be omitted. If the gap is too large, this step is adopted. The size of the gap can be specified in advance according to the requirements.

[0042] 5. Multiply the negative sample image AN with the weight image E, and then add it to the positive sample image AP to obtain the fused image F. Reduce the size of image F to 2048*2048, and convert it to a grayscale image to obtain a first grayscale image F1. Then further convert the first grayscale image F1 to obtain a converted second grayscale image F2, as shown in Fig.10 As shown, the conversion process can be carried out using conventional methods, which will not be described in detail here.

[0043] 3. Style Transfer The present invention recolors the first grayscale image F1 and uses the LBFGS style transfer neural network to achieve this. The first grayscale image F1 is used as the content image, and another style image is selected as the target style image of the neural network for training. The training rounds are 520 rounds. The test shows that this number is sufficient, but it is not limited to this number. The first grayscale image F1 is trained using the LBFGS style transfer neural network. The training is stopped after reaching a certain number of times or the coloring evaluation of the content image is close to the target style image, and the output result is as follows Fig.11As shown, it is recorded as style transfer graph G. This step includes but is not limited to using LBFGS neural network for style transfer training, and the rounds are not limited to 520 times. It can be a style transfer neural network with other network structures. The present invention uses this style transfer neural network, which is a prior art, and the specific training process is not repeated here.

[0044] 4. Sliding window color gamut mapping The maximum image size that style transfer can process is 2048*2048, so this method uses a sliding window-based color gamut mapping method to achieve image reconstruction from 2048*2048 to 16384*16384 resolution. The specific steps are as follows: 1. Assume that the size of the sliding window Gs on the style transfer map G is 64*64, the initial coordinates (xg, yg) = (32*i, 32*j), and the value range of i, j is 0 to 62. The size of the sliding window Fs on the first grayscale image F1 is 64*64, the initial coordinates (xg, yg) = (32*i, 32*j), and the value range of i, j is 0 to 62. The sliding window FL on the second grayscale image F2 corresponding to the Fs window is 512*512, the initial coordinates (xf, yf) = (256*i, 256*j), and the value range of i, j is 0 to 62.

[0045] 2. Create a new empty color gamut mapping space: Set the point value on the space with coordinate axes (x, y, z) to (0, 0, 0), and loop through Gs. Because the Gs and Fs windows have the same size, there is a one-to-one mapping of pixel values ​​at the same position. Therefore, the pixel value in Fs is used as the coordinate of the color gamut mapping space, and the pixel value in Gs is used as the value at the coordinate of the color gamut mapping space. After writing 64*64 mapping relationships, the griddata method is used for interpolation to obtain the mapping relationship of the complete color gamut space. Among them, the griddata method uses an existing open source algorithm, and the specific interpolation process is not repeated here.

[0046] 3. For the 512*512 pixels in the sliding window FL, each pixel is used as the coordinate value of the color gamut space, and a mapped new pixel value can be obtained. After all the pixels are mapped, the mapped image is saved and recorded as Hij, where the value range of i and j is 0 to 62.

[0047] 4. Assume the sliding step length is 32 pixels, sliding horizontally and vertically respectively. Assume sliding horizontally i times and vertically j times, then (xg, yg) = (32*i, 32*j), (xf, yf) = (256*i, 256*j), the value range of i and j is 0-62, and the above steps 2, 3 and 4 are repeated in sequence for each sliding.

[0048] 5. Fusion is performed based on the mapping result Hij. Since the step length of each step in steps 1-4 is half the size of an image, and there are multiple images overlapping in most areas of the 16384*16384 area, this method uses a weighted method to fuse images in this part. The preset weights are as follows: Fig.12 As shown, black represents a value of 0 and white represents a value of 1. The specific steps are as follows: (1) Create a completely black image K of size 16384*16384; (2) Traverse all images Hij, as follows: If the step size i+j is an even number, multiply Hij by the weight 1, and then add it to the image in the sliding window with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K; if the step size i+j is an odd number, multiply Hij by the weight 2, and then add it to the image in the sliding window with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K. Each Hij is multiplied by different weights according to different positions and then superimposed, so that the edge mutation between different Hij can be eliminated. The purpose of executing this step is to merge the image of the non-edge area of ​​Hij into the all-black image K. The purpose of executing steps (2)-(7) is to merge all Hij into one image.

[0049] (3) Traverse all Hij where i is equal to 0 and j is not equal to 0 and j is not equal to 62. If j is an even number, multiply Hij by the weight 8 and then add it to the image in the sliding window with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K; if the step size j is an odd number, multiply Hij by the weight 4 and then add it to the image in the sliding window with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K. The purpose is to fuse the image in the left edge area of ​​Hij into the all-black image K.

[0050] (4) Traverse all Hij where i is equal to 62 and j is not equal to 0 and j is not equal to 62. If j is an even number, multiply Hij by the weight 7 and then add it to the image in the sliding window with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K; if the step size j is an odd number, multiply Hij by the weight 3 and then add it to the image in the sliding window with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K. The purpose is to fuse the image in the right edge area of ​​Hij into the all-black image K.

[0051] (5) Traverse all Hij where j is equal to 0 and i is not equal to 0 and i is not equal to 62. If i is an even number, multiply Hij by the weight 9 and then add it to the image in the coordinate range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K; if the step size i is an odd number, multiply Hij by the weight 5 and then add it to the image in the coordinate range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K. The purpose is to fuse the image in the upper edge area of ​​Hij into the all-black image K.

[0052] (6) Traverse all Hij where j is equal to 62 and i is not equal to 0 and i is not equal to 62. If i is an even number, multiply Hij by the weight 10 and then add it to the image in the coordinate range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K; if the step size i is an odd number, multiply Hij by the weight 5 and then add it to the image in the coordinate range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K. The purpose is to fuse the image in the edge area below Hij into the all-black image K.

[0053] (7) Multiply the image Hij with i=0 and j=0 by the weight 11, and then add it to the image with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K; multiply the image Hij with i=62 and j=62 by the weight 12, and then add it to the image with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K; multiply the image Hij with i=62 and j=0 by the weight 12, and then add it to the image with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K; The image Hij is multiplied by the weight 13, and then added to the image with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K; the image Hij with i=0 and j=62 is multiplied by the weight 14, and then added to the image with coordinates in the range of (256*i~256*i+512, 256*j~256*j+512) in the all-black image K, in order to fuse the images of the four corners of Hij into the all-black image K; (8) After processing steps (2)-(7), the final result image M is obtained, as shown in Fig.13 As shown in the figure, this image can unify the color tone and other information of the entire image without changing the content information of the satellite image, and eliminate the stitching marks caused by the satellite image shooting the ground at different times and seasons, due to lighting conditions and vegetation changes. It has the characteristics of less human intervention, automation and intelligence, and the processed effect meets the needs of actual production use.

[0054] As an embodiment disclosed by the present invention, the present invention also discloses a large-scale image color adjustment system, wherein the system implements the method described, and comprises the following modules: A selection module, used for manually selecting a region to be processed on the image to be processed; An adjustment module, used for adjusting the color of the area to be processed by using a histogram, and eliminating the splicing edge between the area with the adjusted color and other areas to obtain a grayscale image; A style transfer module, used to reduce the size of the grayscale image and perform style transfer to obtain a transferred image; A mapping module, used for using a sliding window to achieve mapping between local color gamuts for the migrated image; The fusion module is used to fuse all the mapping results to obtain a fused and adjusted image.

[0055] As an embodiment disclosed in the present invention, the present invention further discloses an electronic device, the electronic device comprising: A memory storing executable instructions; A processor is used to execute the executable instructions in the memory to implement the method described in the present invention.

[0056] As an embodiment disclosed in the present invention, the present invention further discloses a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the method described in the present invention.

[0057] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0058] The above description shows and describes several preferred embodiments of the present invention, but as mentioned above, it should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the application concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not depart from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A large-scale image color adjustment method, characterized in that: The method comprises the steps of: S1. Manually select the area to be processed on the image to be processed; S2. Using a histogram to adjust the color of the area to be processed, and eliminating the splicing edges between the area where the color is adjusted and other areas to obtain a grayscale image; S3. reducing the size of the grayscale image and performing style migration to obtain a migrated image; S4. Use a sliding window to achieve mapping between local color gamuts for the migrated image; S5. Fuse all mapping results to obtain a fused and adjusted image.

2. The large-scale image color adjustment method according to claim 1, characterized in that: The areas to be processed include: different lighting in regular areas, different time in irregular areas, lighting in the whole image inconsistent with other images, cloud shadows in irregular areas, or the original images in irregular areas have been fused.

3. The large-scale image color adjustment method according to claim 1, characterized in that: The S1 includes: S11. Reduce the satellite image to be processed by a certain multiple to obtain a second image, and use the satellite image to be processed as the original image, wherein the original image includes a negative sample; S12. Select a point (xb, yb) in the boundary area of ​​a negative sample in the second image, intercept an image with coordinates in the range of xb*8-100 to xb*8+100, yb*8-100 to yb*8+100 in the original image according to the point, and display it as the third image; S13. Select a point (xc, yc) in the third image, and the coordinates of the point restored to the original image are (xb*8+xc-100, yb*8+yc-100); S14. Repeat steps S12 and S13 until the negative sample area of ​​the second image is selected; S15. If there are multiple negative sample areas in the original image, then after repeatedly executing S12-S14, a polygon is finally obtained, and the polygon is processed to obtain positive and negative sample images.

4. The large-scale image color adjustment method according to claim 3, characterized in that: The S2 includes: S21. Encode the positive and negative sample images and their corresponding masks to obtain encoded positive and negative samples, and perform histogram matching on the encoded negative samples according to the encoded positive sample histogram; S22. Decoding the result after the histogram matching to obtain a decoded image; S23. Performing weighted calculation on the decoded image to obtain a weighted image; S24. Multiply the negative sample image with the weight image, and then add it to the positive sample image to obtain a fused image, reduce the fused image and perform grayscale conversion to obtain a first grayscale image, and further perform grayscale conversion on the first grayscale image to obtain a second grayscale image.

5. The large-scale image color adjustment method according to claim 1, characterized in that: The S3 includes: training the first grayscale image as a content image using an LBFGS style transfer neural network to obtain a style transfer image of the neural network.

6. The large-scale image color adjustment method according to claim 5, characterized in that: S4 includes: S41. Assume that the size of the sliding window Gs on the style transfer map G is 64*64, the initial coordinates (xg, yg) = (0, 0), the size of the sliding window Fs on the first grayscale image F1 is 64*64, the initial coordinates (xg, yg) = (32*i, 32*j), the size of the sliding window FL on the second grayscale image F2 corresponding to the Fs window is 512*512, the initial coordinates (xf, yf) = (256*i, 256*j), i is the number of horizontal sliding times, j is the number of vertical sliding times, and the value range of i, j is 0 to 62; S42. Create a new empty color gamut mapping space: the point value on the space with coordinate axes (x, y, z) is (0, 0, 0), loop through Gs, the Gs and Fs windows have the same size, and there is a one-to-one mapping between pixel values ​​at the same position. The pixel value in Fs is used as the coordinate of the color gamut mapping space, and the pixel value in Gs is used as the value of the color gamut mapping space at the coordinate. After completing 64*64 mapping relationships, interpolation is performed to obtain the mapping relationship of the complete color gamut space; S43. For the 512*512 pixels in the sliding window FL, each pixel is used as the coordinate value of the color gamut space to obtain a mapped new pixel value. After all the pixels are mapped, the mapped image is saved; S44. Assume that the sliding step is 32 pixels, and slide horizontally and vertically respectively. Assume that the horizontal sliding is i times and the vertical sliding is j times, then (xg, yg) = (32*i, 32*j), (xf, yf) = (256*i, 256*j); re-execute steps S42-S44 for each sliding.

7. The large-scale image color adjustment method according to claim 6, characterized in that: S5 includes: Set a completely black image and a sliding window in it, traverse the mapped image through the sliding window according to the set sliding step size, multiply the mapped image with the set weight and then add it to the image in the sliding window in the completely black image to obtain the final result image.

8. A large-scale image color adjustment system, characterized in that: The system implements the method described in any one of claims 1 to 7, and includes the following modules: A selection module, used for manually selecting a region to be processed on the image to be processed; An adjustment module, used for adjusting the color of the area to be processed by using a histogram, and eliminating the splicing edge between the area with the adjusted color and other areas to obtain a grayscale image; A style transfer module, used to reduce the size of the grayscale image and perform style transfer to obtain a transferred image; A mapping module, used for using a sliding window to achieve mapping between local color gamuts for the migrated image; The fusion module is used to fuse all the mapping results to obtain a fused and adjusted image.

9. An electronic device, characterized in that: The electronic device comprises: A memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images